(FINAL PICTURES) Custom Loop in a Dual Xeon (+780Ti) Dell 2U Server (8K Realtime Render Server)

[clarkmcc]

UPDATE 4/20/2016 -

 

 

Hello LTT!

 

It's been ages since I was on here, however I'm in the middle of a cool project in my opinion. I just parted out my custom loop PC and invested $500 in a dual Xeon L5640  2U server w/96GB of ECC RAM. I plan to turn it into a render server/file server/ gaming rig. I pulled the reference 780Ti out of my last build and put the blower back on and managed to fit it in a PCI slot under the SAS card. I set up the drive array with a friend and am not totally sure what specific RAID we used, its 8 drives, 4x 2 drive mirrors striped between 2 blocks of 4 drives (120GB boot SSD and 500GB caching SSD). I'll post some pictures and clarification soon!

 

As for the modifications, it's not on a rack so I'm thinking about really putting some design into making it look attractive and cool at the same time. Here's what I have in mind:

1. Liquid cooling the CPUs, RAM, and VRMs with 5x 120s worth of radiator space and 10 SP120s in push pull config
2. Expanded RAM for 4K realtime editing on RAM drives
3. Upgraded CPU to X5650
4. Mounted Radiator

Specs

1. 2x Xeon L5640 @ 2.7GHz
2. GTX 780Ti
3. LSI SAS Card supporting 8 drives in RAID 10 
4. 2.25TB of redundant, striped storage
5. Triple 120MM Radiator
6. 96GB Ram
7. Dual Load-Balancing 750W PSU

 

FOR THOSE OF YOU THAT RENDER VIDEOS you'll want to keep an eye on this. I've been able to speed up my REDCODE 4k/5k/6k workflow drastically by getting the footage on RAM drives which have a couple GB/s transfer speed into premiere, drastically. I can now EDIT 6K REALTIME.

 

Also, I've been able to transcode REDCODE to ProRes 4:4:4:4 in realtime as well, which according to the folks at REDUSER, is very rare without a couple titans or a red-rocket card. 

 

(P.S. let me know if liquid cooling a server is a stupid idea)

 

 

[laquine]

Sounds sick! Loving these server style build logs. As for liquid cooling CPU's in a server, totally doable. Asetek prides themselves on building custom liquid cooled arrays for massive server racks. They're also a major source for most consumer level all-in-one liquid coolers. This build looks like its gonna be good!

[leadeater]

The SAS drive RAID level would be RAID 10 from your description. Water cooling should work fine if the case as the room etc, rackmount cases are much harder than a tower obviously. That is something I intend to do also but I'm going to be using an external RAD/Res/Pump solution with quick disconnects on the servers so I can still de-rack them and move things around when needed. Reason for external setup is that I have more than 1 server and everything will be rack mounted.

 

Don't bother striping the boot disk, not necessary and won't add any performance (not usable performance). Mirroring it is a good option just for reliability though.

[clarkmcc]

7 minutes ago, leadeater said:

The SAS drive RAID level would be RAID 10 from your description. Water cooling should work fine if the case as the room etc, rackmount cases are much harder than a tower obviously. That is something I intend to do also but I'm going to be using an external RAD/Res/Pump solution with quick disconnects on the servers so I can still de-rack them and move things around when needed. Reason for external setup is that I have more than 1 server and everything will be rack mounted.

 

Don't bother striping the boot disk, not necessary and won't add any performance (not usable performance). Mirroring it is a good option just for reliability though.

The boot disk won't be striped  especially since it takes 2+ minutes to boot a server with an SAS card anyways. I'm still trying to decide but I think external is the way to go. Since mine wont be racked I think I'll just build a 120mm mounting fram onto the outside of the server that the rads can mount onto. What do you think?

[IsaacDaGrazin]

6k exists?

[leadeater]

2 minutes ago, clarkmcc said:

The boot disk won't be striped  especially since it takes 2+ minutes to boot a server with an SAS card anyways. I'm still trying to decide but I think external is the way to go. Since mine wont be racked I think I'll just build a 120mm mounting fram onto the outside of the server that the rads can mount onto. What do you think?

You got another cheap rackmount case with nothing in it? That's what I'll use, you could do the same and sit them on top of each other.

[leadeater]

7 minutes ago, IsaacDaGrazin said:

6k exists?

You can use what ever resolution you want, video rendering is not limited to display resolutions. The editing resolution also is not the final export resolution, it can be but there is a difference.

 

You can capture video in 8K, edit it at 6k and do the final render in 240p, 480p, 720p, 1080p, 2160p. Typically you want to do all editing in the highest or native resolution you can but this isn't always possible, high resolution editing is extremely demanding.

 

Edit: If you trans-code your raw video to a lower resolution you can't get the missing information back, editing 4K footage in 1080p then exporting back to 4K is not a 4K quality video. 

[IsaacDaGrazin]

7 minutes ago, leadeater said:

You can use what ever resolution you want, video rendering is not limited to display resolutions. The editing resolution also is not the final export resolution, it can be but there is a difference.

 

You can capture video in 8K, edit it at 6k and do the final render in 240p, 480p, 720p, 1080p, 2160p. Typically you want to do all editing in the highest or native resolution you can but this isn't always possible, high resolution editing is extremely demanding.

 

Edit: If you trans-code your raw video to a lower resolution you can't get the missing information back, editing 4K footage in 1080p then exporting back to 4K is not a 4K quality video. 

how will you watch it though?

[leadeater]

1 minute ago, IsaacDaGrazin said:

how will you watch it though?

Not an issue since your output resolution to the monitor is not effected at all by a video's resolution. Open a 1080p video in VLC then shrink the window down to half size, the video is still 1080p but the video size is not 1080p.

 

Everything is scaled to your output resolution, viewing a 6K video on a 1080p monitor 'should' be the same quality as the same video at 1080p.

 

This is the difference between when a game is set to fullscreen or windowed full screen. Fullscreen sets your output resolution so picking a lower resolution than your native screen resolution looks fuzzy since the pixels don't line up between the image and the physical screen pixels. Windowed fullscreen does not change the output resolution, it only changes the game's resolution and then this is scaled to your output resolution. This is better since you are always running native resolution and the picture doesn't become fuzzy.

 

Viewing a video in fullscreen on a computer is actually windowed fullscreen.

[clarkmcc]

23 minutes ago, leadeater said:

You can use what ever resolution you want, video rendering is not limited to display resolutions. The editing resolution also is not the final export resolution, it can be but there is a difference.

 

You can capture video in 8K, edit it at 6k and do the final render in 240p, 480p, 720p, 1080p, 2160p. Typically you want to do all editing in the highest or native resolution you can but this isn't always possible, high resolution editing is extremely demanding.

 

Edit: If you trans-code your raw video to a lower resolution you can't get the missing information back, editing 4K footage in 1080p then exporting back to 4K is not a 4K quality video. 

 

15 minutes ago, IsaacDaGrazin said:

how will you watch it though?

 

1 minute ago, leadeater said:

Not an issue since your output resolution to the monitor is not effected at all by a video's resolution. Open a 1080p video in VLC then shrink the window down to half size, the video is still 1080p but the video size is not 1080p.

 

Everything is scaled to your output resolution, viewing a 6K video on a 1080p monitor 'should' be the same quality as the same video at 1080p.

 

This is the difference between when a game is set to fullscreen or windowed full screen. Fullscreen sets your output resolution so picking a lower resolution than your native screen resolution looks fuzzy since the pixels don't line up between the image and the physical screen pixels. Windowed fullscreen does not change the output resolution, it only changes the game's resolution and then this is scaled to your output resolution. This is better since you are always running native resolution and the picture doesn't become fuzzy.

 

Viewing a video in fullscreen on a computer is actually windowed fullscreen.

It comes down to the sampling of the footage and the bitrate at which you edit. I shoot at 4k usually and deliver at 1080 because 4k to 1080 looks better than 1080 to 1080 (due to chroma subampling). 

[clarkmcc]

31 minutes ago, leadeater said:

You got another cheap rackmount case with nothing in it? That's what I'll use, you could do the same and sit them on top of each other.

That's not a bad idea. Discount servers has them for $20.00 I think?

[IsaacDaGrazin]

27 minutes ago, leadeater said:

You can use what ever resolution you want, video rendering is not limited to display resolutions. The editing resolution also is not the final export resolution, it can be but there is a difference.

 

You can capture video in 8K, edit it at 6k and do the final render in 240p, 480p, 720p, 1080p, 2160p. Typically you want to do all editing in the highest or native resolution you can but this isn't always possible, high resolution editing is extremely demanding.

 

Edit: If you trans-code your raw video to a lower resolution you can't get the missing information back, editing 4K footage in 1080p then exporting back to 4K is not a 4K quality video. 

3 minutes ago, leadeater said:

Not an issue since your output resolution to the monitor is not effected at all by a video's resolution. Open a 1080p video in VLC then shrink the window down to half size, the video is still 1080p but the video size is not 1080p.

 

Everything is scaled to your output resolution, viewing a 6K video on a 1080p monitor 'should' be the same quality as the same video at 1080p.

 

This is the difference between when a game is set to fullscreen or windowed full screen. Fullscreen sets your output resolution so picking a lower resolution than your native screen resolution looks fuzzy since the pixels don't line up between the image and the physical screen pixels. Windowed fullscreen does not change the output resolution, it only changes the game's resolution and then this is scaled to your output resolution. This is better since you are always running native resolution and the picture doesn't become fuzzy.

 

Viewing a video in fullscreen on a computer is actually windowed fullscreen.

thanks for the info! i never knew all of this

 

wow that’s very informative, how do you know so much?

and another quick question

if i were to watch 4k on my 1080p monitor it will look near the same? like good quality like 4k?

[clarkmcc]

Just now, IsaacDaGrazin said:

thanks for the info! i never knew all of this

 

wow that’s very informative, how do you know so much?

and another quick question

if i were to watch 4k on my 1080p monitor it will look near the same? like good quality like 4k?

Here's how it works. If its fully un sampled 1080p (4:4:4:4 chroma subsampling) it will fill every pixel and look great. DSLRs do not do this, they average out pixels to compress data and save space. The Arri Alexa ($60k camera) could only shoot 1080p up until a year ago. That's because it was fully uncompressed and unsampled. So no technically speaking, 4k on a 1080p screen will not look better than 1080p as long as the 1080p footage is unsampled and uncompressed. 

 

Unfortunately most 1080p footage is...

[IsaacDaGrazin]

2 minutes ago, clarkmcc said:

Here's how it works. If its fully un sampled 1080p (4:4:4:4 chroma subsampling) it will fill every pixel and look great. DSLRs do not do this, they average out pixels to compress data and save space. The Arri Alexa ($60k camera) could only shoot 1080p up until a year ago. That's because it was fully uncompressed and unsampled. So no technically speaking, 4k on a 1080p screen will not look better than 1080p as long as the 1080p footage is unsampled and uncompressed. 

 

Unfortunately most 1080p footage is...

wow! thanks man for all that info!

are you two like qualified in this type of stuff?

12 minutes ago, leadeater said:

snip

 

[clarkmcc]

Btw, think 6k is cool? Check out this soon to be 8K RAW camera

41 minutes ago, IsaacDaGrazin said:

6k exists?

 

[IsaacDaGrazin]

Just now, clarkmcc said:

Btw, think 6k is cool? Check out this soon to be 8K RAW camera

 

[clarkmcc]

Just now, IsaacDaGrazin said:

wow! thanks man for all that info!

are you two like qualified in this type of stuff?

 

I make videos on RED cameras and they need lots of space and lots of processing power! That's really the only reason and also the reason why I would even want a fast render server! 

 

P.S. forgot the link http://www.red.com/products/weapon-dragon#overview

[leadeater]

2 minutes ago, IsaacDaGrazin said:

wow! thanks man for all that info!

are you two like qualified in this type of stuff?

 

Nope, I have no working experience in video editing. I am an audio visual enthusiast however and have a very very expensive home cinema setup, proper projector not a TV screen.

 

I also work in IT as a Systems Engineer so I have to deal with the server side of things that may be for a video production company etc, not very often though. Knowledge by proxy and interest basically.

[IsaacDaGrazin]

8 minutes ago, clarkmcc said:

I make videos on RED cameras and they need lots of space and lots of processing power! That's really the only reason and also the reason why I would even want a fast render server! 

 

P.S. forgot the link http://www.red.com/products/weapon-dragon#overview

oh wow o-o

i can see why you need all them cores and threads

4 minutes ago, leadeater said:

Nope, I have no working experience in video editing. I am an audio visual enthusiast however and have a very very expensive home cinema setup, proper projector not a TV screen.

 

I also work in IT as a Systems Engineer so I have to deal with the server side of things that may be for a video production company etc, not very often though. Knowledge by proxy and interest basically.

ooh

[clarkmcc]

5 minutes ago, leadeater said:

Nope, I have no working experience in video editing. I am an audio visual enthusiast however and have a very very expensive home cinema setup, proper projector not a TV screen.

 

I also work in IT as a Systems Engineer so I have to deal with the server side of things that may be for a video production company etc, not very often though. Knowledge by proxy and interest basically.

Ah man I would kill for a full home theater system 

[leadeater]

2 minutes ago, clarkmcc said:

Ah man I would kill for a full home theater system 

Pictures on my profile if you want to have a look. Screen is 92", Panasonic PT-AE7000, Oppo 105D, Plinius 8200 & 8200P, Martin Logan Aeon i, Martin Logan Grotto, Custom 12", Bose 301 Series V

[clarkmcc]

1 minute ago, leadeater said:

Pictures on my profile if you want to have a look. Screen is 92", Panasonic PT-AE7000, Oppo 105D, Plinius 8200 & 8200P, Martin Logan Aeon i, Martin Logan Grotto, Custom 12", Bose 301 Series V

That is legitly sick man. What resolution does it play?

[leadeater]

Just now, clarkmcc said:

That is legitly sick man. What resolution does it play?

It's 1080p, 3D also but I never use that. Personally I hate 3D , no glasses and you'll get me interested.

[clarkmcc]

12 hours ago, leadeater said:

It's 1080p, 3D also but I never use that. Personally I hate 3D , no glasses and you'll get me interested.

Hmmm.... http://www.forbes.com/sites/johnarcher/2015/01/13/hands-on-with-samsungs-110-inch-8k-glasses-free-3d-tv-aka-my-head-hurts/#792b3fba5434

 

You'd need a crazy high transfer rate from the media server to the display to run 8k in 3D though... I doubt that's cheap

[Principis]

12 hours ago, leadeater said:

It's 1080p, 3D also but I never use that. Personally I hate 3D , no glasses and you'll get me interested.

Do you know how annoying it is to go to a 3d movie when you have glasses so you have to put the stupid 3d glasses on top of your normal glasses....