Fans slowly ramp up to 100% (hp proliant ml370G6)

[corruptedButterlfy]

Hello guys, 

 

Recently i bought a server hp proliant ml370G6.
When in the bios the fans spin up to 18% and just stay there.
When the os is loaded (openmediavault 5.5.4), the fans slowly ramp up to 100% when doing nothing.

 

Information:
system ROM: P63  05/21/2018
ILo 2 Firmware: 2.33   03/20/2018
OS : openmediavault 5.5.4

CPU: 2x Intel® Xeon® Processor X5650

Memory: 9*8GB 1333MHz (72GB)

Fans: 5 x HP 615641-001

power supply: 2x HP HSTNS-PL 18

PCI Express cards: nothing installed in the PCI Express slot.

storage: raid 0 with 7 * (250gb) hdds. Will probably change it to raid5 when the fans are fixed. Currently I have 1 ssd (128GB) and 3HDD's(500GB each)  in the system. The ssd is in raid 0, and the 3 are in raid5.

Spoiler

 

 

Issue

All fans spin to 100%

Spoiler

Sorry no foto.

 Temps

Spoiler

 

Tests: 

 

Test1, Dont boot to os just stay in the bios for 30-60min

All fans stay at 18%

Spoiler

 All temps stay relativly low (one temp is 0 what should be possible i think this is the temp of the hard drives but not sure)

Spoiler

 

Test2, Dont boot to os just stay in the bios for 30-60min

All fans stay at 18%-19%

Spoiler

 All temps stay relativly low (one temp is 0 what should be possible i think this is the temp of the hard drives but not sure)

Spoiler

 

Test 3: with only 1 hard drive in the system and booted in debian-10.4

Fans stay at 18%

Spoiler

 All temps stay relativly low and temp zone 25 is now reading 35C when all drives where insurted the temp was 50C

Spoiler

 

Test 4: with only 1 hard drive in the system and booted in debian-10.4

Fans stay at 18%

Spoiler

 All temps stay relativly low and temp zone 25 is still reading 35C when all drives where insurted the temp was 50C

Spoiler

 

Test 5: This time i switched drives in and out until i found all the drives that stayed around 35C. I had 4 drives that reached 50C and 3 that didn't. So if only the drives are insurted that stayed around 35C the fans stay at 18%.

Fans stay at 18%

Spoiler

All temps stay relativly low and temp zone 25 (the zone that probably represents the temp of the storage drives) stays at 35C

Spoiler

 

Things to try  :
1) Updated the system ROM. (changed nothing, fans still ramped up to 100%)
2) Updated ILO 2 Firmware. (changed nothing, fans still ramped up to 100%)

3) Changed the power regulator for proliant to OS Control mode, HP Dynamic Power Saving Mode, HP Static low Power mode, HP static High Performance Mode. Non of these changed anything. (changed nothing, fans still ramped up to 100%)

4) Pulled out all the drives but one. (made a huge difference, fans ramped up to 18% and stayed there)

5) Inserted 1 drive at a time. Left only the drives insurted when temp 25 didn't go to 50C. (Same result as for but now with more storage, fans ramped up to 18% and stayed there)

6) Change option in bios to increased cooling/optimal cooling /acoustic cooling. (Still need to try this myself)

7) Install windows server with its drivers. (Still need to try this myself)

 

The Solution to the slowly ramping up fans:

It seems that the HP proliant servers can be a bit picky when it comes to hard drives. So the best solution is to pick hard drives that the server likes. The best way to know if the server will like the drive is to take a look at this site. This is no guarantee that these will work in your server but its better than nothing.

 

Special thanks to:

@svmlegacy, @Maticks and @kompetenzbolzen

 

 

Questions:
1) Ideal i would want the fans to stay between 20%-30% when doing nothing. Is there some way that i can configure the fans by myself.
2) One of the temp zones (temp 10) has status "n/a". Could this be the problem, where is this in the server mayby there is a cable loose, could i disable this reading.
3) Temp zone 25 is relatively hot. Could this be the problem, where in the server is this located.

4) Is temp zone 25 indeed for the drive bays ?

5) is there an option to disable a temp sensor ?

6) Why would the fans spin up when the drives are running hot because non of the fans are able to reache the drives as how the server is desinged.

 

Thanks in advance

 

Edited July 29, 2020 by corruptedButterlfy
Made it easer if people would have the same problem and added final solution.

[kompetenzbolzen]

Does your ILO show any errors or warnings?

Have you checked for failed fans? If one is below spec, the server compensates by running all other fans at 100%

[corruptedButterlfy]

First of all thanks for your reaction

 

The ILO shows no warnings and/or errors (see image)

Spoiler

 

I think that all fans are working properly (I can feel them all blowing wind), system health shows "Fans: ok; Fully Redundant"(see image). Is there some other way to test If one is below spec ? 

Spoiler

 

[kompetenzbolzen]

2 minutes ago, corruptedButterlfy said:

I think that all fans are working properly (I can feel them all blowing wind), system health shows "Fans: ok; Fully Redundant"(see image). Is there some other way to test If one is below spec

If the ILO says they're OK, then that's not the issue.

And if it doesn't complain about temps, that missing zone is also probably not it.

This is indeed very strange. Have you tried another OS?

[corruptedButterlfy]

Tried to install "debian-10.4.0-amd64-dvd-1.iso" fans are still loud. Tommorow i will try to install windows. I will also try when only 1 hdd is in the system because they are not the same i read somewhere that this can give issues. Other tips are always welcome.

[corruptedButterlfy]

I think the problem is fixed. It seems that the issue had to do with the hard drives. Will do some more tests and update this topic if needed.

[Maticks]

few other reasons servers run fans at 100%.

All power supplies are not plugged into power, some have 2 PSU's and only have two slots for PSU's plugging both in will lower fan speeds.

if you have 4 PSU slots and only 2 PSU's that could be part of the reason depending on the hardware.

 

remove any PCI Express Cards some servers will bump up the fan speed because something is installed in the PCI Express Slot.

Then there is the obvious things any fans that are showing up dead or without speeds being recorded in the bios, any hard drives that are third party that might also cause this type of problem.

Even SD Cards if your unit has those installed that are not official ones.

 

One thing i did see online was

first 

 press f9 at start up
--> Advanced Options
--> Thermal Configuration
--> Choose "increased cooling"

F10 to confirm.....exit

fans noises decreased but not to the normal state

Then

press f9 at start up
--> Advanced Options
--> Thermal Configuration
--> Choose "Optimal cooling"

F10 to confirm.....exit

all fans speed went to normal.

 

Few things online saying a bios change to this version helps.

https://community.hpe.com/t5/proliant-servers-ml-dl-sl/disappointed-with-new-ml350-g6-fan-noise/td-p/4416470/page/2#.Xx4XZZ4zb-i

 

realise this thread is more than three years old, but I too am facing this. I wanted to share my findings to hopefully help someone out there as I've had no luck. What I'm seeing is the server will either post fine, no loud fans, boot into Windows then ramp the fans up 5% at a time every 15 seconds or so, or POST fine, boot into Windows then randomly spike all fans to 80% and leave it there.
 

1. Updated BIOS to latest (June 2018)
2. Updated iLO to latest (2.33)
3. Removing ALL PCIe cards (including quad port NIC)
4. Setting cooling profile to optimal cooling, reduced acoustics etc (both with no PCIe cards and with PCIe cards)
5. Populating ports 3, 6 and 8 only with PCIe cards and having it in reduced acoustics
6. Disconnecting 1 SAS backplane from power and data when the RAID card is removed
7. Installing Windows drivers for the devices that reported with no driver (IPMI interface and Base System Device)
8. Single PSU in slot 1, slot 2, dual PSUs.
9. Removing 1 DIMM to have only slots A, D and G populated. (Had A, B D and G populated before)

I've resorted to purchasing an additional two fans to populate fan ports 1 and 3. I'm hoping the reduced acoustic mode will behave better with all fans populated and PCIe ports 3 6 and 8 populated only.

I've also purchased some I2C cables to go from the SAS backplanes to the motherboard. This is kind of a shot in the dark, not holding my breath for this one.

Also whilst writing this it's managed to boot with the following:

3x DIMMs
1 PROC
1 RAID card in port 6
1 quad port NIC in port 10
Cooling profile: Optimal cooling
3 fans, (Fan 2, 4 and 5)
Both backplanes connected
2 PSUs

Fans were as:

Fan 1: System Zone Not Installed n/a
Fan 2: System Zone Ok 18%
Fan 3: System Zone Not Installed n/a
Fan 4: System Zone Ok 43%
Fan 5: System Zone Ok 18%

Removing the quad port NIC made them go to the below with fans 2 and 5 increasing every so often.

Fan 1: System Zone Not Installed n/a
Fan 2: System Zone Ok 25%
Fan 3: System Zone Not Installed n/a
Fan 4: System Zone Ok 43%
Fan 5: System Zone Ok 25%

Rebooted the system and the fans have stayed like the below:

Fan 1: System Zone Not Installed n/a
Fan 2: System Zone Ok 18%
Fan 3: System Zone Not Installed n/a
Fan 4: System Zone Ok 43%
Fan 5: System Zone Ok 18%

Seems to me that on hardware removal and the next boot the system assumes something has failed and throws the fans up in small increments.

I shall test the same with the additional fans installed and in different modes, then with removing RAID cards but not without attaching the SAS connectors to something the other end, don't want to add the possibility of the server kicking the fans up because it suspects a RAID card has failed as it detects it is providing power to a SAS backplane but then not having a device the other end for data.

Long post, apologies. TL;DR
 

1. On reboots fans go nuts
2. Acoustic mode may not work without all fans populated
3. Fans remain the same in optimal cooing when quad port NIC connected too.

[Maticks]

from what i can read online there is high fan noise till the windows server drivers load that then quiet then thing right up.

But it appears the drivers are only for windows.. so running the server for anything else seems to mean higher fan noise.

 

[corruptedButterlfy]

In my server there is nothing installed in the PCI Express slot.
All my drives are from a third party (edited my orignal post to show the drives that i'm using).
I will take a look at the increased cooling/optimal cooling /acoustic cooling options in the bios but i will probaly not be able to see a difference because the server is now working without any problems.
I will also try to install windows servers with its drivers. Do you have a link as where i could download these drivers?

What I'm still wondering is why the fans would spin up when the drives are running hot because non of the fans are able to reache the drives as how the server is desinged.

[corruptedButterlfy]

In an another topic that I made about which hard drives to buy for my system the user @svmlegacy gave a very useful link. The link is a whitelist and blacklist for which hard drives to use in a lot of hp Proliant servers. This is no guarantee that these will work in your server but its better than nothing.