Cooling Dual CPU System in a 2U Chassis

[Weatherlight]

So have a tendency to collect old server hardware, and lets say that I end up with some pretty unique builds. Right now I am running into an issue where the second CPU in a dual CPU server is quite a bit warmer than the first. I am using a Machinist x99-D8 Max and this is in a 2U server chassis. As you may have noticed the CPU sockets are inline with each other which is causing my discomfort. In turn I am looking to put some kind of air baffling or something to help move air around. Anyone have some ideas on what I should be looking for?

[freeagent]

Gonna need some zesty 120x38s blowing through there 

[Weatherlight]

1 minute ago, freeagent said:

Gonna need some zesty 120x38s blowing through there 

Lol, if only. 2U means the best I can do is 80mm. I am running a pair of Supermicro SNK-P0048AP4 coolers, so they move some air but that has become an issue as it sounds like an aircraft taking off all the time. Swapping out the fans for a nice pair of Noctua NF-A6x25 seems to work for the front CPU but not so much for the back.

 

[Blue4130]

17 minutes ago, Weatherlight said:

Lol, if only. 2U means the best I can do is 80mm. I am running a pair of Supermicro SNK-P0048AP4 coolers, so they move some air but that has become an issue as it sounds like an aircraft taking off all the time. Swapping out the fans for a nice pair of Noctua NF-A6x25 seems to work for the front CPU but not so much for the back.

 

And her in lies the problem of 2u cases. They are not easy to cool while maintaining silence. Are you really space limited? No way to swap the 2u for a 4u?

[Weatherlight]

16 minutes ago, Blue4130 said:

And her in lies the problem of 2u cases. They are not easy to cool while maintaining silence. Are you really space limited? No way to swap the 2u for a 4u?

More like I have multiple 2u cases available to work with, and wanted to make at least 3 identical servers when I started down this path. Since then I scaled back to a pair of matching servers, but made the jump to a dual socket system. So I could, but then I would need to find a new case, drive bays, rails, ect... 

So going to a 4u chassis almost seems like I would be admitting defeat. 

[freeagent]

80mm fans, I don’t have any of those any more, but I do still have a case that accepts them. The smallest fan I have right now is a little 92x35 4000rpm knuckle remover. Funnily enough, it is one of the strongest fans that I own

[thrasher_565]

15 hours ago, Weatherlight said:

 sounds like an aircraft taking off all the time.

your not going to have good cooling and quiet dose not go together in a small space. if you want that you need to water cool or a bigger case.... so find the biggest fan you can fit and thats that. as for best cooler i dont no. w/e you do its sounding like a sever...

 

posably if you put the server up to each other might be able to mount bigger fans on the front outside... but the gills probly block alot of the air.

 

Edited March 3 by thrasher_565

[Weatherlight]

Going ahead using the higher speed or larger fans does not really address the issue. The exhaust from CPU 0/1 is being forced through CPU 1/2 is resulting in a ~10°C difference in temperatures between the CPU's. I could see where dropping a AIO on one of the CPUs may help, but that likely would require irreversible modifications to the case. I am not looking for great/good cooling, I am looking for adequate. With noise it does not need to be silent, it just needs to not upset the wife.

To me the simple solution would be creating two different cooling zones. How to make these would be the question.

In case you want to see the temps, and possible cooling zone diagram:

Spoiler

Package id 0:  +50.0°C  (high = +82.0°C, crit = +92.0°C)
Core 0:        +43.0°C  (high = +82.0°C, crit = +92.0°C)
Core 1:        +43.0°C  (high = +82.0°C, crit = +92.0°C)
Core 2:        +43.0°C  (high = +82.0°C, crit = +92.0°C)
Core 3:        +46.0°C  (high = +82.0°C, crit = +92.0°C)
Core 4:        +47.0°C  (high = +82.0°C, crit = +92.0°C)
Core 5:        +49.0°C  (high = +82.0°C, crit = +92.0°C)
Core 6:        +47.0°C  (high = +82.0°C, crit = +92.0°C)
Core 8:        +44.0°C  (high = +82.0°C, crit = +92.0°C)
Core 9:        +43.0°C  (high = +82.0°C, crit = +92.0°C)
Core 10:       +45.0°C  (high = +82.0°C, crit = +92.0°C)
Core 11:       +43.0°C  (high = +82.0°C, crit = +92.0°C)
Core 12:       +43.0°C  (high = +82.0°C, crit = +92.0°C)
Core 13:       +43.0°C  (high = +82.0°C, crit = +92.0°C)
Core 14:       +42.0°C  (high = +82.0°C, crit = +92.0°C)

Package id 1:  +61.0°C  (high = +82.0°C, crit = +92.0°C)
Core 0:        +54.0°C  (high = +82.0°C, crit = +92.0°C)
Core 1:        +55.0°C  (high = +82.0°C, crit = +92.0°C)
Core 2:        +54.0°C  (high = +82.0°C, crit = +92.0°C)
Core 3:        +53.0°C  (high = +82.0°C, crit = +92.0°C)
Core 4:        +54.0°C  (high = +82.0°C, crit = +92.0°C)
Core 5:        +55.0°C  (high = +82.0°C, crit = +92.0°C)
Core 6:        +55.0°C  (high = +82.0°C, crit = +92.0°C)
Core 8:        +57.0°C  (high = +82.0°C, crit = +92.0°C)
Core 9:        +59.0°C  (high = +82.0°C, crit = +92.0°C)
Core 10:       +54.0°C  (high = +82.0°C, crit = +92.0°C)
Core 11:       +54.0°C  (high = +82.0°C, crit = +92.0°C)
Core 12:       +54.0°C  (high = +82.0°C, crit = +92.0°C)
Core 13:       +54.0°C  (high = +82.0°C, crit = +92.0°C)
Core 14:       +53.0°C  (high = +82.0°C, crit = +92.0°C)

As a bit of a side note, swapping to the lower RPM fans caused a slight increase in temps ~5°C on the leading CPU and ~7°C on the trailing

 

[Blue4130]

2 hours ago, Weatherlight said:

Going ahead using the higher speed or larger fans does not really address the issue. The exhaust from CPU 0/1 is being forced through CPU 1/2 is resulting in a ~10°C difference in temperatures between the CPU's. I could see where dropping a AIO on one of the CPUs may help, but that likely would require irreversible modifications to the case. I am not looking for great/good cooling, I am looking for adequate. With noise it does not need to be silent, it just needs to not upset the wife.

To me the simple solution would be creating two different cooling zones. How to make these would be the question.

In case you want to see the temps, and possible cooling zone diagram:

I really doubt that you will have enough seperation between the coolers to get good "dual zone" setup working.  This is just the nature of dual cpu setups, one CPU will always be sucking the air from the first. The way to combat that is though brute air speed, as you have found out. The downside is noise and unhappy wives.

[thrasher_565]

thats why there are off set socket mbs...

[RevGAM]

Install a muffler. One from a diesel truck should be good.  

[Weatherlight]

So I took some inspiration from an 1U supermicro chassis and purchased some Mylar sheeting to make an air baffle. This is my first attempt, I am going to watch the temperatures for a little bit to see if it makes a difference. Two things I am looking at is the delta between the two and the temps of CPU 0/1.

Wondering about breaking the PSU into its own zone, adding another to separate out CPU 1/2 and the add-on cards, lastly adding exhaust fans. 

[RevGAM]

Sounds interesting! Please keep us posted! Separation of airflow sounds great!