Unraid Server High Power Consumption

[derWinky]

Hello,

I built an Unraid Home-Server for myself and it´s running fine mostly. The only thing that really bothers me is the extremely high power consumption.

 

My Config:
MoBo: X99 MSI Gaming Pro Carbon
CPU: Intel i7 6900K (No OC, set to defautlt settings of the Motherboard)
GPU: Nvidia GT710 (6900k has no iGPU, PC won´t start without a GPU)
PSU: Corsair RMi 750W
Case: Fractal 7XL
Cooler: Alpenföhn Brocken 3
RAM: 4x 8GB 2400MHz
HDDs: 2x Parity 3TB and about 10 or 11 drives with a mix of 3 and 2TB adding up to 27TB of total storage
SSD: 1TB Corsair MP510 Cache
HBA: Dell PERC H310 RAID Controller
NIC: Intel X550-T2 2x10G
 

Hope I didnt forget anything important. I will provide screenshots when I get home today.

The Server is a mix of parts I specifically bought new and stuff I got for free or reaaaaaly cheap.
The HDDs were all for free, old drives from our companys server. Thats why it´s a bunch of low capacity drives instead of like 3 16TB drives.
I started out with an i7 4790 and 32GB of RAM which were also for free and I just had to get a z97Board for it. 
I later got an offer from a friend of mine to buy his set of x99 board, 6900K and 32GB RAM for just 250€ so I sold the old i7 4790 set and bought the 6900K set and put that in my server.

A few months ago I discovered the Corsair PSU plugin and was shocked by how much power my Homeserver draws even when in idle doing basically nothing.

It´s consuming 115-130 Watts when doing pretty much nothing, and thats just the number the PSU shows. Powerdraw from the wall should be even more.
It´s about 100W or more on the 12V Rail, depending on how many drives are spun up about 10-35W on 5V and I dont remember how much, but very little on 3V.
Again, I will provide a screenshot as soon as I get home today.

I expected my server to be not exactly power efficient with a 6900k and somewhere around 13 Hard drives. But 120W in idle doing nothing is way too much.

Spun down harddrives shouldn´t take that much power, and the CPU is mostyl sitting there doing nothing since I dont run any VMs right now and I only use a Plex Container so I have no idea where the 100 Watts are going.

 

Until now I also didnt find a tool that shows me which part of the PC consumes how much power, kinda how HW Monitor on windows shows you how much power your CPU and GPU are consuming.

 

I was thinking about replacing my current server with something like a Z570 board with an integrated 10G Nic, 10700T CPU for performance while also being way more energy efficient and not needing a dedicated GPU unless I want to pass it through to a Windows VM replacing my living room PC and get like 3 16TB drives and call it a day. Unfortunately that setup is way too expensive for me right now.

 

Hope some people here can help me out.

[cacoe]

An idle HDD can use around 5w so you're doing pretty well at that power usage at idle. Can you try booting with only one drive physically connected to power to see the base power usage?

[RollinLower]

How are you so sure the drives are spinning down? I believe the H310 needs to be flashed to IT mode for spindown to work.

anyway, the x99 platform was not made for low power operation, so i guess 100 watts is still pretty good.

[derWinky]

4 minutes ago, RollinLower said:

How are you so sure the drives are spinning down? I believe the H310 needs to be flashed to IT mode for spindown to work.

anyway, the x99 platform was not made for low power operation, so i guess 100 watts is still pretty good.

It is flashed to IT Mode, the unraid dashboard says they´re spun down and I hear them spinning down.
I know the platform wasnt made to be as power efficient as a celeron. It still shouldn´t be wasting 100Watts in idle. My 10900k in my Main PC uses about 10 Watts in idle.

 

9 minutes ago, cacoe said:

An idle HDD can use around 5w so you're doing pretty well at that power usage at idle. Can you try booting with only one drive physically connected to power to see the base power usage?

I could try disconnecting all the drives from the PSU and boot to an external windows SSD or something but that would be a really great hassle and if possible i´d like to avoid that.
My Server is in a very small room with my switches raspi and FireWall on top of it. I really dont want to disconnect and reconnect everything to be able to disconnect the drives.
That would be an enormous pain in the ass.

[cacoe]

4 minutes ago, derWinky said:

It is flashed to IT Mode, the unraid dashboard says they´re spun down and I hear them spinning down.
I know the platform wasnt made to be as power efficient as a celeron. It still shouldn´t be wasting 100Watts in idle. My 10900k in my Main PC uses about 10 Watts in idle.

 

I could try disconnecting all the drives from the PSU and boot to an external windows SSD or something but that would be a really great hassle and if possible i´d like to avoid that.
My Server is in a very small room with my switches raspi and FireWall on top of it. I really dont want to disconnect and reconnect everything to be able to disconnect the drives.
That would be an enormous pain in the ass.

Sounds like you're not willing to go through troubleshooting steps then? Don't think you'll get a silver bullet without some trial and error.

[RollinLower]

9 minutes ago, derWinky said:

It still shouldn´t be wasting 100Watts in idle. My 10900k in my Main PC uses about 10 Watts in idle

well, there are a lot of components to this system. Like that 10G NIC, it uses 8Watts on it's own. add another 6 or so watts from the H310 and the watts from all your drives and you'd be a long way to 100 already.

then there are still things like the CPU, RAM and motherboard functions. 100 watts really isn't all that bad.

[derWinky]

2 minutes ago, cacoe said:

Sounds like you're not willing to go through troubleshooting steps then? Don't think you'll get a silver bullet without some trial and error.

I am, I´d just like to avoid those that will cost me multiple hours and might be unneccessary. I also don´t have time for going through that today.
 

[derWinky]

2 minutes ago, RollinLower said:

well, there are a lot of components to this system. Like that 10G NIC, it uses 8Watts on it's own. add another 6 or so watts from the H310 and the watts from your drives and you'd be a long way to 100 already.

then there are still things like the CPU, RAM and motherboard functions.

Do spun down drives that don´t do anything really still use that much power? Isn´t the whole point of spinning them down to make them not waste energy and be silent?
I know, all that stuff adds up to more than average for a NAS, but about 130Watts in idle still seems very high to me.
I´ve read stuff from people also having Unraid servers with 10G nics and something like an i5 10400 and somewhere around 6 drives still being below 30W.

Is there a simple way to monitor per device powerconsumption in unraid/linux? My PSU unfortunately only tells me the consumption of 12v 5v and 3v.
I´d really like to find out without having to pull everything out of my networking room.

[Sprawlie]

first: what drives? can you provide us the models? you said mostly are used and smaller. I think this is where you are goig to find the bulk of your power draw. Even if drives spin down, they still will draw power to maintain data and wake capabilities. they're not "off", the heads just get parked and the spinning platters stop. 

 

And as you said, you used a lot of older drives in mixed fashion instead of a few largers ones.

 

I think you'll find that if you look at the power requirements from some of the drives you'll find your culprits. Older spinning drives probably are showing a higher idle power draw than many of the newer ones. Especially if you are comparing to a computer using SSD's which will have ridiculously miniscule power draws at idle.

 

Just basic "averages" of hard drive idle power draw is around 5.5w per drive (if modern). with 10 of them (get an exact count if you want more precision), you're using 55w when idle just for your hard drives alone.

 

so if your unRaid server is using about 100w when idle, that would be 55-60w for your drives, and 40ish whats for everything else. Which given the rest of your parts seems in line. 

 

the i7-6900k will idle at arond 30-35w 

RAM is likely drawing 12-15w (doesn't matter if it's idle or not, it's not Lowe power ram)

The GPU will draw about 5w

Some quick findings on the PERC controller puts it around 30w idle in IT mode.

 

your primary components are idling around 140 as you said? Your already pretty much at your floor with your setup. if you want to dramatically reduce your idle power usage, get rid of the PERC controller and the large quantity of hard drives, and move to a more efficient setup

 

at this point, there's really nothing else that can be said to help you further. things for your system absolutely appear to be running nominally. you juts have massive inefficiency in your hard drive configurations.

 

 

 

 

 

 

Edited August 19, 2021 by Sprawlie
update with some math

[Sprawlie]

5 hours ago, derWinky said:

Do spun down drives that don´t do anything really still use that much power? Isn´t the whole point of spinning them down to make them not waste energy and be silent?

A spun down drive does save power and wear and tear. as well as heat energy and not wasting it.

 

But the idle wattage of a hard drive is still around 5-10w per drive depending on age, status of the mechanics etc. A 7200 RPM hard drive will use up somewhere between 15 to 35w while in use and spinning, So even the decrease to 5 or 10w is a power savings.

 

But this is where SSD's are also so beneficial in power reduction. Even Samsung's 980pro gen 4 idles as low as 5(or 40 depending on state) mW (microwatts) and run active ~10w

 

 

[derWinky]

Thanks a lot for your insight @Sprawlie. Seems I dramatically underestimated the powerdraw of those HDDs. Obviously the powerdraw of that many drives is high when all are used, but most of them are spun down most of the time and only 2 or 3 spin up depending on what share I´m trying to access, so the peak consumption isnt´t the problem. However I thought those drives might consume maybe 2 Watts or less in idle because they are doing absolutely nothing at all, why would they take that much energy. Very interesting.

Buying other components would probably be the best solution but again, atm and probably for the next year I won´t have that much money to spend on a new powerful and also efficient system like I listed above with a 10G NIC integrated z590 board, 10700T CPU and 3 16TB Drives (1 parity 2 data) and the rest of the system. That would easyly exceed 2000€. The reason why I put the system together the way it is now is because I got the parts either for free or for an insanely good deal, was just kinda hopeing to get the cost for the power it draws down a little. But as you said apparently for what it is the power draw is already as minimal as possible.

 

Do you by any chance know how good CPUs from Intels T series work for stuff like that? I barely find any usable data for T CPUs of the 10 and 11 Intel series on how good they actually perform in benchmarks, games, how far and how long they exceed their official 35W TDP etc.
My Homeserver sits in idle by far most of the time, but if I upgrade it at some point i´d like to use a windows VM with a passed through GPU to replace my livingroom PC and use some more docker containers. Still no really heavy duty tasks but I´d hope something like a 11700T would provide enough power when needed while also being way more efficient the rest of the time. 

Thanks again for your help. Much appreciated.

 

 

[Sprawlie]

No idea about the "T" models sorry.

 

Quote

 why would they take that much energy.

Because that's what they use. this isn't something out of the ordinary, your devices appear to be working exactly within their spec.

 

it's just the nature of Hard drive technology that even when spun down and Idle, they still require ~5w of power draw in order to maintain functionality.  You will not be able to reduce this since it's operating nominally unless you replace the hard drives with fewer drives r

 

 

[Electronics Wizardy]

1 hour ago, derWinky said:

 

Do you by any chance know how good CPUs from Intels T series work for stuff like that? I barely find any usable data for T CPUs of the 10 and 11 Intel series on how good they actually perform in benchmarks, games, how far and how long they exceed their official 35W TDP etc.
My Homeserver sits in idle by far most of the time, but if I upgrade it at some point i´d like to use a windows VM with a passed through GPU to replace my livingroom PC and use some more docker containers. Still no really heavy duty tasks but I´d hope something like a 11700T would provide enough power when needed while also being way more efficient the rest of the time. 

They won't really make a different here, as idle power is about the same. The big different for T skus is a lower tdp max power.

 

Id get the normal version of the cpu here.

 

If you want a save a couple watts drop to 16gb of ram, and take some dims out.

 

The high end sockers normally idle at a good amount power power as there is more stuff running, the consumer sockets like am4 and 1200 use a good amount less idle power normally.

 

Id get a lower end board if you can, the high end parts just use extra power, id go something like b560 if you can.

 

The h310 and 10gbe nics are pretty power hungry, sometimes upto about 10w each, or a couple watts idle.

 

But I think the drives are your big power draw here, you can test it easily but unplugging drives. This will also tell you if spindown is working right.

[derWinky]

1 hour ago, Electronics Wizardy said:

They won't really make a different here, as idle power is about the same. The big different for T skus is a lower tdp max power.

 

Id get the normal version of the cpu here.

 

If you want a save a couple watts drop to 16gb of ram, and take some dims out.

 

The high end sockers normally idle at a good amount power power as there is more stuff running, the consumer sockets like am4 and 1200 use a good amount less idle power normally.

 

Id get a lower end board if you can, the high end parts just use extra power, id go something like b560 if you can.

 

The h310 and 10gbe nics are pretty power hungry, sometimes upto about 10w each, or a couple watts idle.

 

But I think the drives are your big power draw here, you can test it easily but unplugging drives. This will also tell you if spindown is working right.

Thx for the advice.

A lower end board would obviously be nice for power consumption and it would be way cheaper. But if I'm at the point of buying into a new platform i'd prefer to take a higher end board so I get PCIe Gen 4 on the top PCIe and nvme slot. Also I think it would be better to just buy a board with integrated 10G Networking than having the extra PCIe card. Also that means one more open slot for future expansion. If I just get a better board with features like 10G integrated there's no need for me to get an hedt platform for more PCIe Lanes. And if it's Gen4 I can populate the first 2 PCIe x16 Slots making them both electricaly x8 and still have the the same bandwidth as Gen3 x16 in the top slot. 

 

But I'm not exactly in a rush to get a new system. Way to expensive one way or another. Also maybe alder lake may turn out as s good option.

[Pixel5]

what exactly are you running on that server that requires so much computing power?

 

not really surprising that this CPU idles very high and also your GPU doesnt help you either.

 

Having disks spinning also can be a major power drain, my system has an i3 10100, 10G NIC and an HBA and idles at around 40W putting light load on the CPU gets this to about 60W and spinning up my 4x 8TB disks puts the consumption to a bit over 100W during something like a parity check.

[derWinky]

13 minutes ago, Pixel5 said:

what exactly are you running on that server that requires so much computing power?

 

not really surprising that this CPU idles very high and also your GPU doesnt help you either.

 

Having disks spinning also can be a major power drain, my system has an i3 10100, 10G NIC and an HBA and idles at around 40W putting light load on the CPU gets this to about 60W and spinning up my 4x 8TB disks puts the consumption to a bit over 100W during something like a parity check.

I´m not running a bunch of stuff at the moment but I plan on using 2 or 3 Linux VMs and a Windows VM with passed through GPU to replace my livingroom PC. Maybe also a MacOS VM but this one won´t be running a lot.

Other than that I´m running Plex and just use it as a regular NAS.

[Pixel5]

55 minutes ago, derWinky said:

I´m not running a bunch of stuff at the moment but I plan on using 2 or 3 Linux VMs and a Windows VM with passed through GPU to replace my livingroom PC. Maybe also a MacOS VM but this one won´t be running a lot.

Other than that I´m running Plex and just use it as a regular NAS.

it looks like you have to decide then what you really wanna run and how much it is worth it for you.

 

You could serve that basic plex and NAS function easily with an i3 10100 which alone will idle below 20W and run everything else on another system and fire up the VM´s on demand.

 

personally over 100W idle would be a huge no go for me as 1W of electricity being consumed 24/7 for an entire year costs me about 2,50€ so idling your system would be 250€ a year while its doing absolutely nothing at all.

[derWinky]

15 minutes ago, Pixel5 said:

it looks like you have to decide then what you really wanna run and how much it is worth it for you.

 

You could serve that basic plex and NAS function easily with an i3 10100 which alone will idle below 20W and run everything else on another system and fire up the VM´s on demand.

 

personally over 100W idle would be a huge no go for me as 1W of electricity being consumed 24/7 for an entire year costs me about 2,50€ so idling your system would be 250€ a year while its doing absolutely nothing at all.

Agree, but again. I can´t just go out and spend 2k€ on a new homeserver when I already have one thats working perfectly fine.

I also dont have it turned on 24/7. My system doesnt need to run when I sleep or work. 

 

In an ideal world I´d just get a z590 aorus master with integrated 10G NIC, an i7 11700, 3x 16TB drives and whatever GPU i´d be able to get for gaming in my livingroom.
That would be efficient enough I´d probably have it running 24/7
 

But I dont just have 2000€ lieing around so as long as my current server works I´m probably not buying a new system.
I was just hopeing to find a way to further optimize my current system, without spending thousand of euros.

Don´t get me wrong, I´m thankfull for your insight, but replacing my system is just not viable for me right now.
The reason why the system is the way it is right now is because the parts were dirt cheap.
Especially since I spent most of the money I had set aside on retro stuff over the last few weeks. Retrotink 5X, Analogue Mega SG, Everdrives, and also a Mikrotik 10G Switch.

[Pixel5]

2 minutes ago, derWinky said:

Agree, but again. I can´t just go out and spend 2k€ on a new homeserver when I already have one thats working perfectly fine.

I also dont have it turned on 24/7. My system doesnt need to run when I sleep or work. 

 

In an ideal world I´d just get a z590 aorus master with integrated 10G NIC, an i7 11700, 3x 16TB drives and whatever GPU i´d be able to get for gaming in my livingroom.
That would be efficient enough I´d probably have it running 24/7
 

But I dont just have 2000€ lieing around so as long as my current server works I´m probably not buying a new system.
I was just hopeing to find a way to further optimize my current system, without spending thousand of euros.

you dont really need 2000€ when you build a purpose made system instead of trying to do everything with one inefficient system.

 

an i3 10100 cost less then 100€ a motherboard also less then 100€ 16GB RAM can be had for 50€ and a case is somewhere between 50 - 100€ depending on what you want.

 

This system would easily handle plex, all normal NAS functions and even one VM easily.

That would mean you can use the current and inefficient system for everything else and only start it when you need it.

[derWinky]

17 minutes ago, Pixel5 said:

you dont really need 2000€ when you build a purpose made system instead of trying to do everything with one inefficient system.

 

an i3 10100 cost less then 100€ a motherboard also less then 100€ 16GB RAM can be had for 50€ and a case is somewhere between 50 - 100€ depending on what you want.

 

This system would easily handle plex, all normal NAS functions and even one VM easily.

That would mean you can use the current and inefficient system for everything else and only start it when you need it.

That is true but the whole point of buying a new efficient system would be to have it and the services it provides besides being a NAS running 24/7. I´d rather sell the other stuff later and integrate stuff like a livingroom PC into the unraid server.
And again, for all the stuff I want to set up a 10100 wont do. A cheapo 100€ board is also no use because then i dont have an integrated 10G NIC and need to use my Intel x550-T2 again wasting another PCIe x16 Slot for no reason and limit future expansion. At that point selling the card and buying a better board with integrated 10G would be the better way.
Consumer Platform is already limited by how few PCIe lanes it has. Really dont want to waste them for an extra NIC when I switch back to that platform. Also the total system cost when integrating the livingroom PC into the unraid server would be way lower since I dont need 2 cases PSU Coolers Cpus bla bla. Basically all it takes is to put the GPU into the unraid server and buy an i7 instead of i5. Also no heat and noise in the livingroom.

And if you add up all the things I´d need to buy for a new better server. The 3 16Tb drives alone take up more than 1k€ on their own. Its no use to just buy a new MoBo and CPU to maybe safe like 30W or whatever and still have 13drives running.

If I have to replace stuff, I´d basically replace everything with the exception of the 750 Watt PSU, CPU cooler, SSD and RAM.
 

Here´s a screenshot of the parts and their cost without a new case. The SSDs wouldnt be neccessary from the start, i´d add them later.

 

Even without the 2 SSDs and without a spare hard drive thats still about 1.7k€

[CerealExperimentsLain]

On 8/19/2021 at 4:47 AM, derWinky said:

It´s consuming 115-130 Watts when doing pretty much nothing, and thats just the number the PSU shows. Powerdraw from the wall should be even more.
It´s about 100W or more on the 12V Rail, depending on how many drives are spun up about 10-35W on 5V and I dont remember how much, but very little on 3V.

...

 

My 10900k in my Main PC uses about 10 Watts in idle.

Uhh... I don't think you have a main line desktop PC idling at 10 watts.  I have a pfSense box running a j4005 board that only idles at 13.5w.

I think you should check your measurement methods.  You don't seem to be using a Kill-A-Watt or any other similar device, you're relying on PSU reports?  I'd not trust that.

[derWinky]

1 hour ago, CerealExperimentsLain said:

Uhh... I don't think you have a main line desktop PC idling at 10 watts.  I have a pfSense box running a j4005 board that only idles at 13.5w.

I think you should check your measurement methods.  You don't seem to be using a Kill-A-Watt or any other similar device, you're relying on PSU reports?  I'd not trust that.

1. That is obviously the number it reports in tools like hwmonitor or MSI afterburner and talking about just the CPU itself, not the complete system. 
No I dont have a kill a watt, I have a proper UPS that can tell me the usage too

2. Nice Username, genuinely made me laugh ^^

[Mihle]

On 8/20/2021 at 6:26 PM, derWinky said:

1. That is obviously the number it reports in tools like hwmonitor or MSI afterburner and talking about just the CPU itself, not the complete system. 
No I dont have a kill a watt, I have a proper UPS that can tell me the usage too

2. Nice Username, genuinely made me laugh ^^

If your UPS days a 10900k PC use 10w, idle, unless it's in sleep I think your UPS is then a little inaccurate. My NAS is a G5400, and it uses about 30-35w idle, I have measured, subtract HDD and it would use about 20w. (I spin my HDDs all the time). (No GPU, mATX mobo.)

[CerealExperimentsLain]

30 minutes ago, Mihle said:

If your UPS days a 10900k PC use 10w, idle, unless it's in sleep I think your UPS is then a little inaccurate. My NAS is a G5400, and it uses about 30-35w idle, I have measured, subtract HDD and it would use about 20w. (I spin my HDDs all the time). (No GPU, mATX mobo.)

He was apparently being inconsistent. His first set of numbers is what the PSU reported as whole system draw.  The second number, 10w, was him citing ONLY the CPU draw based on diagnostic tools.

[Mihle]

12 hours ago, CerealExperimentsLain said:

He was apparently being inconsistent. His first set of numbers is what the PSU reported as whole system draw.  The second number, 10w, was him citing ONLY the CPU draw based on diagnostic tools.

ah, re reading it I see that now, I must have been blind when i wrote it.