Freenas/esxi and 10th gen intel CPUs - 10100/10400 /F for power usage at idle?

[paddy-stone]

Hi, 

Just wondered if anyone here is running a 10100/10400 with or without F SKU, and can comment on idle power draw using esxi and/or freenas?

 

ATM I am running a Ryzen 1700 in my esxi homelab, but power consumption is a bit high for it being unused 90-95% of the time. At the moment I am really only running 2 cores 8GB RAM for my freenas VM (but might increase to 4 cores if I run more services/jails like nextcloud), and 1 core 2GB RAM for an ubuntu VM (running it mostly for instant file manager operations and such when remote connecting). Those are running 24/7, mostly idle, with me spinning up other VMs as and when needed throughout the day for testing and whatnot.

It's using approx 120W at the wall, which is a little over powered for me, and I know that Ryzen CPUs aren't the best for power saving.

 

If I were only running freenas and 1 or 2 VMs then I'd just swap the 2400GE Athlon in there, but I like having my homelab for other usage, and it saves me having to run a whole other machine for the purpose, so that's why I'm possibly considering swapping to an intel 4/8 or 6/12 cores/threads if it'll save me some power usage while not limiting me too much for having spare resources I think. I am open to other suggestions hardware wise, but the hardware needs to be available at a retailer like Amazon.co.uk or similar, I was considering a -T CPU, but I'd have better luck trying to find rocking horse crap  

I am running 6x 4TB HDDs  so they're using approx 30W there as a conservative estimate. The HBA uses around 6-14W IIRC, then there's the RAM 2x16GB, and the GPU (wouldn't need it with the iGPU of the intel. Even if I could shave 20-40W off the usage that'd be great, that's still £26 - £52 a year saved in energy usage, and the ROI would pay for itself in 6 - 12 years.... but that's not even a needed factor really, it's just a bonus.

I have older hardware that I could test with I guess, have a 6700K/b250 that'd give me 4/8 cores and see what the idle power draw is like. But the b250 only has 1 pcie slot and no intel NIC to use with esxi, so wouldn't be able to test properly, would only be able to see idle usage with freenas on bare metal.

 

I'm just riffing ideas here really I guess, £26 a year power saving isn't that much, but I hate waste. If I can spend £300 approx and still have a decent amount of cores to use whilst saving some energy to boot, then that'd be sweet IMO

 

[leadeater]

58 minutes ago, paddy-stone said:

It's using approx 120W at the wall, which is a little over powered for me, and I know that Ryzen CPUs aren't the best for power saving.

Part of that is ESXi power management, mainly idle just isn't that good. Have you tried changing the ESXi host power profile to low power? I also think Intel idle power is slightly better as it is, certainly with ESXi just for how long it has been optimized for it unlike Zen. Still not too bad, my ESXi host with 1x E5-2670 192GB RAM 6 SSD 3 HDD idle usage is around 200W.

 

Try that 6700K just to see what the idle usage is, that'll give you a decent idea if changing hardware is even worth it. I suspect not as you'll probably only drop to something like 80W a best then how long will that take to pay off what you had tp spend to get it that low.

[paddy-stone]

8 minutes ago, leadeater said:

Part of that is ESXi power management, mainly idle just isn't that good. Have you tried changing the ESXi host power profile to low power? I also think Intel idle power is slightly better as it is, certainly with ESXi just for how long it has been optimized for it unlike Zen. Still not too bad, my ESXi host with 1x E5-2670 192GB RAM 6 SSD 3 HDD idle usage is around 200W.

 

Try that 6700K just to see what the idle usage is, that'll give you a decent idea if changing hardware is even worth it. I suspect not as you'll probably only drop to something like 80W a best then how long will that take to pay off what you had tp spend to get it that low.

Yeah, I'll give it a shot, I can plug in an intel x1 or x4 NIC that I have just for testing.

 

I'm not really that bothered by how long the ROI is as such, it was just a thought more than anything, I'd need to use something after all anyway. So even if it's 12 years to even out what I pay it's no big deal when we're only talking £300 approx. And it might live a lot longer than that maybe... I'd only really consider upgrading in the future again, if the power saving was significant or my needs change where I'd need even more cores, different I/O etc. It's still better than running say 2-3 machines to do the same job anyway, say running a pfsense box, freenas, a raspberry pi or 2, you get the drift. I've also cut back on the amount of time my main desktop is running, which pulls around 91W all-in when idle, including my monitor, according to my UPS. So that's saving a bit too... I'm mostly using my low power laptop for interacting with my VMs ATM.

 

If I can get the power usage to 80W that's still a significant reduction over 120W, that's approx £52 a year saving at the current average energy rate of 15p/KWh, so about 6 years at that rate to pay off the £300 outlay on a 10400 for example with a half decent b460 board. Ignore the fans and cooler in the list below, I already have those (or equivalents)

PCPartPicker Part List

Type	Item	Price
CPU	Intel Core i5-10400 2.9 GHz 6-Core Processor	£154.97 @ CCL Computers
CPU Cooler	Noctua NH-D9L 46.44 CFM CPU Cooler	£44.99 @ Amazon UK
Motherboard	Gigabyte B460 AORUS PRO AC ATX LGA1200 Motherboard	£139.59 @ SmartTeck.co.uk
Memory	Corsair Vengeance LPX 32 GB (2 x 16 GB) DDR4-2133 CL13 Memory	Purchased For £0.00
Storage	Crucial M500 M.2 240 GB M.2-2280 Solid State Drive	Purchased For £0.00
Storage	Toshiba OCZ TR200 480 GB 2.5" Solid State Drive	Purchased For £0.00
Storage	Seagate IronWolf NAS 4 TB 3.5" 5900RPM Internal Hard Drive	Purchased For £0.00
Storage	Seagate IronWolf NAS 4 TB 3.5" 5900RPM Internal Hard Drive	Purchased For £0.00
Storage	Seagate IronWolf NAS 4 TB 3.5" 5900RPM Internal Hard Drive	Purchased For £0.00
Storage	Seagate IronWolf NAS 4 TB 3.5" 5900RPM Internal Hard Drive	Purchased For £0.00
Storage	Seagate IronWolf NAS 4 TB 3.5" 5900RPM Internal Hard Drive	Purchased For £0.00
Storage	Seagate IronWolf NAS 4 TB 3.5" 5900RPM Internal Hard Drive	Purchased For £0.00
Case	Silverstone GD08B HTPC Case	Purchased For £0.00
Power Supply	EVGA G2 650 W 80+ Gold Certified Fully Modular ATX Power Supply	Purchased For £0.00
Case Fan	be quiet! Pure Wings 2 65.51 CFM 120 mm Fan	£9.49 @ Amazon UK
Case Fan	be quiet! Pure Wings 2 65.51 CFM 120 mm Fan	£9.49 @ Amazon UK
Case Fan	be quiet! Pure Wings 2 65.51 CFM 120 mm Fan	£9.49 @ Amazon UK
Case Fan	be quiet! Pure Wings 2 65.51 CFM 120 mm Fan	£9.49 @ Amazon UK
Custom	PRO/1000 Ptquadport Bulk	Purchased For £0.00
Custom	10Gtek® LSI 9211-8I HBA Card, 8Port 6Gb/s Per Port SATA+SAS PCIe 2.0	Purchased For £0.00
Custom	DEEPCOOL FH-10 Integrated Fan Hub, Powering up to 10 fans (3-pin or 4pin) with PWM, Occupying only One 4-pin Motherboard Header	Purchased For £0.00
	Prices include shipping, taxes, rebates, and discounts
	Total	£377.51
	Generated by PCPartPicker 2020-12-13 12:07 GMT+0000

 

[paddy-stone]

29 minutes ago, leadeater said:

Part of that is ESXi power management, mainly idle just isn't that good. Have you tried changing the ESXi host power profile to low power?

Just done that, I'll see if it makes any worthwhile saving, thank you

[leadeater]

55 minutes ago, paddy-stone said:

that's approx £52 a year saving at the current average energy rate of 15p/KWh

Geez your energy unit prices sucks, I thought mine was bad but that's really not great at all.

[paddy-stone]

2 hours ago, leadeater said:

Geez your energy unit prices sucks, I thought mine was bad but that's really not great at all.

Yeah, actually we are on a rate that closely follows the base cost (can't think what that's called right now), so it changes every 30 mins. So it's not as bad as it sounds, at least it goes down at night, and if there is a good influx of green energy such as solar or wind. It even goes negative at times which means they are paying US to use electricity. Just lately it's been averaging higher at around 12-15p per unit, the actual rates are fluctuating between around 7-35p per unit. The 35p per unit is a max, and only usually happens between 4-7pm each day, it's to try and encourage people to shift their energy usage where possible from the peak hours... this then means they don't have to power on nuclear/gas etc so much or often, if they can keep a more even usage pattern.

 

I just try to save energy where possible, like doing laundry overnight when much cheaper, using a slow cooker for 1 pot meals etc... money's better in my pocket than theirs

 

[Electronics Wizardy]

7 hours ago, paddy-stone said:

It's using approx 120W at the wall, which is a little over powered for me, and I know that Ryzen CPUs aren't the best for power saving.

That seems kinda high to me. I used a 1700 in my main server, and saw about 50w idle, and about 80w under medium load. And that was with 8 drives and a gt 710, and a hba.

 

Try running anouther hypervisor if you can, see if that will help effiency.

 

 

What drives do you have? Those really add up power wise, id try to cut down on the number of drives.

 

The 1700 is a pretty efficient chip, so you probably aren't saving much power. Id focus more on getting rid of hbas, drives and other things in the system.

[paddy-stone]

23 hours ago, Electronics Wizardy said:

That seems kinda high to me. I used a 1700 in my main server, and saw about 50w idle, and about 80w under medium load. And that was with 8 drives and a gt 710, and a hba.

 

Try running anouther hypervisor if you can, see if that will help effiency.

 

 

What drives do you have? Those really add up power wise, id try to cut down on the number of drives.

 

The 1700 is a pretty efficient chip, so you probably aren't saving much power. Id focus more on getting rid of hbas, drives and other things in the system.

So is ESXi not good for efficiency?  I'm using Seagate Ironwolf 4TB drives, I can't cut down the number of drives as I'm using Raid Z2 to get 16TB usable. And using the HBA for ease of use mostly in the case I'm using, but also because I can then have up to 8 drives in the future when needed. 

 

How the hell did you get 50W idle? what hypervisor were you using, and what VMs? IIRC even my microserver running 3x 3TB reds uses around 35W and that's only running a celeron dual core, no HBA etc.... and that's only running freenas on bare metal.

[Electronics Wizardy]

30 minutes ago, paddy-stone said:

So is ESXi not good for efficiency?  I'm using Seagate Ironwolf 4TB drives, I can't cut down the number of drives as I'm using Raid Z2 to get 16TB usable. And using the HBA for ease of use mostly in the case I'm using, but also because I can then have up to 8 drives in the future when needed. 

 

How the hell did you get 50W idle? what hypervisor were you using, and what VMs? IIRC even my microserver running 3x 3TB reds uses around 35W and that's only running a celeron dual core, no HBA etc.... and that's only running freenas on bare metal.

I think those drives and hba are really hurting your idle power usage. Try removing them to see how much power there usage. If power is a big issue id probably replace those 4tb drives with a mirror of 16tb drives.

 

I was running proxmox to get that idle power usage. Those chips are pretty efficient, I had a gt 710 which doesn't use much power either.