I did some more research. Some hardware monitoring softwares polled idle cores. Also, some softwares like like HW Monitor may apparently only report the last known temperature of the cpu core, so they weren't very accurate. All this only applies to the latest Ryzen 3000 generation/series so may also apply to other future Ryzen CPU's
Ryzen Master is a software that is made by amd, which gives me an accurate reading as far as I'm aware, because they're not polling or showing the showing the last temperature of idle cores. My temperature now is idling at a good temperature now using this software, but still fluctuates between 44°C and 46°C. This is apparently common with the Ryzen 3000 gen/series though