Ultimate Low Idle Power HPC Server

[MorseCode]

I run Finite Element Analysis simulation at home both for my day job and for my consulting business. The HPC computers that I have are immensely capable for my applications but they only see single digit utilization most of the time. Does anyone have any ideas on how to build extremely low idle power HPC computers? Just for reference, I have a Lenovo 3850 x6 with 4x 24 core Xeon processors and 1.8TB of RAM and a new dell R7625 with 2x EPYC 9954 processors and 1.8TB of RAM. When both systems are churning away on models my electric bill jumps up to $1000 or more per month, if they are on but mostly idle it goes down to around $700/month and off but with the iDRAC still on it goes down to around $500-600/month. I run a separate low power NAS and Home Assistant server on my network but would love to combine that into an idling low power HPC server to have less hardware overall.

[jaslion]

A intel n100 may be a nice way to cobine the nas and ha?

 

I just run a 5600g in eco mode with 6 hdds and 2 sata ssds all on a lsi raid card and its like 22w idle or so mostly because I got fans goin a bit much

 

 

[MorseCode]

8 minutes ago, jaslion said:

A intel n100 may be a nice way to cobine the nas and ha?

 

I just run a 5600g in eco mode with 6 hdds and 2 sata ssds all on a lsi raid card and its like 22w idle or so mostly because I got fans goin a bit much

 

 

I used a Qotom Q20332G9-S10 Mini PC running Proxmox 8 to combine HA, NAS, and Adguard.

[jaslion]

10 minutes ago, MorseCode said:

I used a Qotom Q20332G9-S10 Mini PC running Proxmox 8 to combine HA, NAS, and Adguard.

Oh an atom c3758 system. Yeah the n100 is a bit more efficient but well changing wont save anything noticeable. The n100 is heaps more powerfull but unless you are strugglin for cpu power its not worth a switch

[Needfuldoer]

Where are you located and how much do you pay per kilowatt-hour? That's a ton of hungry compute power, but at least the PowerEdge should only draw about 20 watts if it's "off" and only iDRAC is awake. (EPYC isn't exactly known for its low idle power consumption.)

 

Do you have a power meter like a Kill-a-Watt that shows you how much each machine is drawing from the wall? (I know iDRAC can tell you as well, not sure about the Lenovo.)

 

Would a fast "regular" desktop be able to handle an average workload that only hits the big servers at single digit percentages? If so, you should be able to leave them off most of the time and only fire them up as needed.

  

31 minutes ago, jaslion said:

Oh an atom c3758 system. Yeah the n100 is a bit more efficient but well changing wont save anything noticeable. The n100 is heaps more powerfull but unless you are strugglin for cpu power its not worth a switch

Did you see the specs on his other machines? Performance per watt on the NAS is splitting hairs!

[MorseCode]

38 minutes ago, Needfuldoer said:

Where are you located and how much do you pay per kilowatt-hour? That's a ton of hungry compute power, but at least the PowerEdge should only draw about 20 watts if it's "off" and only iDRAC is awake. (EPYC isn't exactly known for its low idle power consumption.)

 

Do you have a power meter like a Kill-a-Watt that shows you how much each machine is drawing from the wall? (I know iDRAC can tell you as well, not sure about the Lenovo.)

 

Would a fast "regular" desktop be able to handle an average workload that only hits the big servers at single digit percentages? If so, you should be able to leave them off most of the time and only fire them up as needed.

  

Did you see the specs on his other machines? Performance per watt on the NAS is splitting hairs!

I live in the southwest and pay about $0.15 per kWh. When running simulations I will easily use 300-900 GB of RAM and will peg as many CPU cores at 100% as I can give the simulations for anywhere between a few hours to a few days. The low percentage utilization is over a month and running the simulations takes a lot less time than setup and post run data analysis that happens with the systems just idling. I thought about building up a new rig to replace my Lenovo and that is why I started this post. There isn't much I can do to the Dell server as I don't own it, the company I work for does. I was just thinking that with a budget of say $6-10k I could probably build a rig comparable to the Lenovo rig that I could keep on at all times that just sips power when not solving models. I can more easily expense hardware for consulting on my taxes than I can expense a power bill I share between consulting, home, and work sources.

 

I do all sorts of power monitoring, I have several power monitors that feed back into my Home Assistant server and then I graph everything over time. The large rackmount 240V UPS in my rack also displays power usage and the Dell system uses around 500-600W at idle and the Lenovo is around the same with maybe 150W more than the Dell. I am assuming the torrent of fans are not helping the power consumption. Windows does not like to idle as low as Ubuntu server does and I had to switch my workflow over from Ubuntu to Windows for insurance data safety compliance.

[Needfuldoer]

Is your place suitable for solar?

[MorseCode]

45 minutes ago, Needfuldoer said:

Is your place suitable for solar?

Yes it is, and I am installing a large array to help offset the costs.

[jaslion]

1 hour ago, MorseCode said:

Yes it is, and I am installing a large array to help offset the costs.

Honestly I think just investing into that and a battery (check salt batteries they are supposed to just last forever with salt top ups) will do more than any other power savings.

 

 

[jaslion]

4 hours ago, Needfuldoer said:

Did you see the specs on his other machines? Performance per watt on the NAS is splitting hairs!

Yup misunderstood the question asked!

[Needfuldoer]

2 hours ago, MorseCode said:

Yes it is, and I am installing a large array to help offset the costs.

That’s probably all you can do, except maybe replace the Lenovo with another R7625. It wouldn’t save power, but at least you’d get more performance per watt than with quad-socket Broadwell.