(FINAL PICTURES) Custom Loop in a Dual Xeon (+780Ti) Dell 2U Server (8K Realtime Render Server)

[wpirobotbuilder]

How much RAM will you be expanding into? You presumably have a 12-DIMM motherboard, finding compatible 16GB modules to increase it might be difficult depending on the board.

 

Total props, btw. LGA1366 dual socket servers are fantastic.

[clarkmcc]

15 minutes ago, wpirobotbuilder said:

How much RAM will you be expanding into? You presumably have a 12-DIMM motherboard, finding compatible 16GB modules to increase it might be difficult depending on the board.

 

Total props, btw. LGA1366 dual socket servers are fantastic.

I have 18-dimm on the board so most likely 6x 8GB sticks which will put my at 144GB of RAM. Most my video projects are under 138 so that should work for me. 

 

Yeah I use it as my daily desktop and the thing is a total beast. Love these dual socket rigs. 

[leadeater]

7 hours ago, Principis said:

Do you know how annoying it is to go to a 3d movie when you have glasses so you have to put the stupid 3d glasses on top of your normal glasses....

Not personally no, but I have a good idea. If I had to actually wear glasses I'd pay for some proper optical lenses that were for 3D, no idea if they exist but I'd try and make them become a thing somehow.

 

3D will always be rather sub par while we have to wear $1 plastic things on our faces. They make the picture look worse, make it darker and they just don't fit properly ARGH!!! Most of the time if I bother going to a cinema I take them off.

 

What the hell was wrong with what we had, 2D was fine damn it. Do it right or just don't do it, I shouldn't have to pay a price premium for a useless gimmick with no other option.

 

"HULK SMASH!!!!", Rant over .

[Principis]

1 hour ago, leadeater said:

Not personally no, but I have a good idea. If I had to actually wear glasses I'd pay for some proper optical lenses that were for 3D, no idea if they exist but I'd try and make them become a thing somehow.

 

3D will always be rather sub par while we have to wear $1 plastic things on our faces. They make the picture look worse, make it darker and they just don't fit properly ARGH!!! Most of the time if I bother going to a cinema I take them off.

 

What the hell was wrong with what we had, 2D was fine damn it. Do it right or just don't do it, I shouldn't have to pay a price premium for a useless gimmick with no other option.

 

"HULK SMASH!!!!", Rant over .

Why optical lenses for 3D? you can just use normal lenses and put the stupid glasses on

 

And yeah I hate it the only reason I went to see 3D is because the 2D version wasn't playing at that time.

[leadeater]

1 minute ago, Principis said:

Why optical lenses for 3D? you can just use normal lenses and put the stupid glasses on

 

And yeah I hate it the only reason I went to see 3D is because the 2D version wasn't playing at that time.

All the new stuff recently hasn't had a 2D option anymore where I am, much disappointment. I'd rather wait for blu ray copy than have to use the stupid 3D glasses, they really do annoy me that much.

[clarkmcc]

2 hours ago, Principis said:

Why optical lenses for 3D? you can just use normal lenses and put the stupid glasses on

 

And yeah I hate it the only reason I went to see 3D is because the 2D version wasn't playing at that time.

 

2 hours ago, leadeater said:

All the new stuff recently hasn't had a 2D option anymore where I am, much disappointment. I'd rather wait for blu ray copy than have to use the stupid 3D glasses, they really do annoy me that much.

I've got to say though... nvidias 3D glasses are so nice  perfect sync, super high quality... and green

[Principis]

5 hours ago, clarkmcc said:

 

I've got to say though... nvidias 3D glasses are so nice  perfect sync, super high quality... and green

Try putting them on while wearing normal glasses

[clarkmcc]

4/8/2016

Just snapped some pictures of the server up to this point. Here's what I have added.

1. 2x SSDs
2. 780Ti 
3. External (messy) power supply
4. External fan controller

My goal is to get rid of the external fan controller and get better temps without blowing my house down with these delta fans. 

 

 

 

 

As you can see that 780Ti is pretty squashed so my ambient temps are around 50-58C, and that SAS card above it gets HOT. In order to keep them both cool I need keep that delta in the back at 50% at least. That's not quite blowing my house down but it did blow a hole in my wall...

[leadeater]

SAS cards generally run very hot as it is, they assume you'll have it in a server with very strong airflow which is annoying if you want to use one in a water cooled desktop. Also keep an eye on the ram temps, I had issues when I converted one of my tower servers to water cooled with H55's and had to remove the plastic shroud/duct. Without enough airflow they get rather hot too.

 

 

[clarkmcc]

On 4/8/2016 at 5:50 PM, leadeater said:

SAS cards generally run very hot as it is, they assume you'll have it in a server with very strong airflow which is annoying if you want to use one in a water cooled desktop. Also keep an eye on the ram temps, I had issues when I converted one of my tower servers to water cooled with H55's and had to remove the plastic shroud/duct. Without enough airflow they get rather hot too.

 

 

I noticed that. It gets crazy hot! I'll have some little 80mm fans blowing over the RAM and SAS card. The radiator intake will pull from inside the case a blow out. Details to come 

[clarkmcc]

Update 4/11/2016

I have 2x LGA1366 water blocks coming tomorrow along with a couple enzo 90 degree compression fittings. The 780Ti block is already here. 

[clarkmcc]

4/13/2016

The water cooling components are all here. The thermal pads for the 780Ti waterblock are on their way. I left the block at the shop to clean which I'll do tomorrow. These are about the cheapest blocks ever. I'll run a leak test and make sure everything is in working condition. 

 

 

 

 

 

 

 

 

 

[clarkmcc]

4/15/2016

We have completed the loop with lots of leaks. Currently leak testing... high res pics tomorrow.

 

Measuring cuts

 

 

Waterblock attached with swiftech and bitspower fittings

 

 

 

Empty case ready for drilling

 

 

Forgot the goggles...

 

 

Finished holes

 

 

Loop building

 

 

 

Fully assembled

 

 

 

[SoloTurkey]

This is awesome!

[clarkmcc]

9 minutes ago, Hungryturkey12 said:

This is awesome!

Thanks! Leak test passed! More details tomorrow.

[Snooli]

Great build! Wouldn't go for a rackmount tho. However, if you already have a rack or are planning on expanding it, it makes a lot of sense. I have been actually looking at quad socket Opterons, as they have even more multithreaded raw power.

 

Regarding the resolution question, look at some LTT videos regarding 4K youtube. The thing is, resolution is just a number. What matters more is compression quality and bitrate. LTT, for example, shoots 2160p, edits 1080p, but uploads 2160p because youtube allows for higher bitrate with higher resolution.

TL;DR: Lossless 4:4:4 chroma 720p will propably look better than 4:2:0 5Mb/s h.264 2160p.

[leadeater]

48 minutes ago, Snooli said:

Great build! Wouldn't go for a rackmount tho. However, if you already have a rack or are planning on expanding it, it makes a lot of sense. I have been actually looking at quad socket Opterons, as they have even more multithreaded raw power.

 

Regarding the resolution question, look at some LTT videos regarding 4K youtube. The thing is, resolution is just a number. What matters more is compression quality and bitrate. LTT, for example, shoots 2160p, edits 1080p, but uploads 2160p because youtube allows for higher bitrate with higher resolution.

TL;DR: Lossless 4:4:4 chroma 720p will propably look better than 4:2:0 5Mb/s h.264 2160p.

@clarkmcc Is already well aware of this.

[clarkmcc]

Just now, Snooli said:

Great build! Wouldn't go for a rackmount tho. However, if you already have a rack or are planning on expanding it, it makes a lot of sense. I have been actually looking at quad socket Opterons, as they have even more multithreaded raw power.

 

Regarding the resolution question, look at some LTT videos regarding 4K youtube. The thing is, resolution is just a number. What matters more is compression quality and bitrate. LTT, for example, shoots 2160p, edits 1080p, but uploads 2160p because youtube allows for higher bitrate with higher resolution.

TL;DR: Lossless 4:4:4 chroma 720p will propably look better than 4:2:0 5Mb/s h.264 2160p.

I wouldn't either but I already had the server. I think that's what makes it unique though! We see lots of custom rackmount watercooled cases but how many stock dell 2u servers have custom loops in them  

 

You're exactly right! Most DSLRs shoot 4:2:0 @1080 which translates to 420p 4:4:4. SO the way linus does it is great because you'r getting a smaller compression ratio while getting the same resolution! As for the title, 8k @ 4:2:0 still takes impressive hardware to run, mostly because the bitrate is so high. While resolution doesn't directly effect performance, generally resolution is proportional to bitrate so you can expect higher resolutions to have higher bitrates. 

[clarkmcc]

4/15/2016

RESULTS ARE HERE! Okay so I just played a round of battlefront with vsync and ULTRA settings and got my temp results back. Unfortunately I don't have the old temp graphs but the averages were

• CPU - 50C
• GPU - 70C

Which was not a surprise with all that stuck in a 2U case. Oh btw I took out the super loud delta fans so there was very little airflow in there. 

 

Now the moment of truth. Did the water cooling make a difference? HECK YEAH

 

P.s. I'm going to get around to taking some pics with my red... sorry you got the iphone ones last night

 

Socket 1

Socket 2

GPU

 

[Snooli]

27 minutes ago, clarkmcc said:

I wouldn't either but I already had the server. I think that's what makes it unique though! We see lots of custom rackmount watercooled cases but how many stock dell 2u servers have custom loops in them  

 

You're exactly right! Most DSLRs shoot 4:2:0 @1080 which translates to 420p 4:4:4. SO the way linus does it is great because you'r getting a smaller compression ratio while getting the same resolution! As for the title, 8k @ 4:2:0 still takes impressive hardware to run, mostly because the bitrate is so high. While resolution doesn't directly effect performance, generally resolution is proportional to bitrate so you can expect higher resolutions to have higher bitrates. 

I am confident yours is the first one. Because if you are building a fully watercooled server, you might as well make your own cases. The design price will get lost in the budget for a ton of Xeons.

[clarkmcc]

Just now, Snooli said:

I am confident yours is the first one. Because if you are building a fully watercooled server, you might as well make your own cases. The design price will get lost in the budget for a ton of Xeons.

It's true... When I bought the server this wasn't my intention, however, Im glad I did it. It runs smoothly and quietly while giving my all the benefits of an actual server board and keeping the small(er) 2u form factor. Although with the watercooling it takes up 3U now.

[clarkmcc]

Teaser pictures of the final. More to come 

 

[leadeater]

@clarkmcc Really like GPU temps on water, personally I think if someone can only pick one thing to do this is actually the best option. CPU temperature difference between air and water is nothing like GPU.

 

It is a shame about having to cut a hole in the side for the GPU block, really useful to know for me since I want to keep mine in a rack when I do it so the rails must still fit and slide in and out (If I have GPUs).

[clarkmcc]

39 minutes ago, leadeater said:

@clarkmcc Really like GPU temps on water, personally I think if someone can only pick one thing to do this is actually the best option. CPU temperature difference between air and water is nothing like GPU.

 

It is a shame about having to cut a hole in the side for the GPU block, really useful to know for me since I want to keep mine in a rack when I do it so the rails must still fit and slide in and out (If I have GPUs).

Totally. Night and day difference! Yeah I wish I would've known too  What case do you have?

[leadeater]

3 hours ago, clarkmcc said:

Totally. Night and day difference! Yeah I wish I would've known too  What case do you have?

Not sure what I'll use atm. Currently got a IBM x3500 M4 (Main) and a modified Intel SC5650BRP/S5520HC which was extremely annoying just to get two H55's in to it since it's tiny, great system and I use it as a portable server.

 

 

I'm more than likely going to buy 3 new server cases and two more Intel S5520HC (ebay) motherboards and leave the IBM alone. Haven't seen any 4U/3U/2U chassis yet that I've instantly liked but I'm not looking that hard. The other option is do what Linus did and get a custom cases made but I'm hopeless at CAD.

 

Also got a LD PC-V8 for my desktop which I'll likely take the 3x 480 XTX's out and use those externally to cool all the servers either in it's own top of rack case or semi outside, depends what I can afford when building my house.