We FINISHED the $100,000 PC! - 6 Editors 1 CPU Pt. 5

[jakkuh_t]

 

Buy Tyan Motherboards:
On their site: https://lmg.gg/8KVcq

 

Buy EKWB Watercooling Parts:
On their site: https://lmg.gg/8KVcg
On Newegg: https://lmg.gg/8KVcH

 

Buy Kingston RAM:
On Amazon: https://lmg.gg/8KVcG
On Newegg: https://lmg.gg/8KVcx

 

Buy Intel CPUs
On Amazon: https://lmg.gg/8KVcD
On Newegg: https://lmg.gg/8KVcv

 

Buy Intel Optane SSDs
On Amazon: https://lmg.gg/8KVcy
On Newegg: https://lmg.gg/8KVcd

 

Buy Cablemod Cables:
On their site: https://lmg.gg/8KVcs

 

Buy Noctua fans:
On Amazon: https://lmg.gg/8KVc7
On Newegg: https://lmg.gg/8KVct

 

Buy EVGA Power Supplies:
On Amazon: https://lmg.gg/8KVcM
On Newegg: https://lmg.gg/8KVcC

 

Buy WD Hard Drives:
On Amazon: https://lmg.gg/8KVcf
On Newegg: https://lmg.gg/8KVcF

 

Buy OneStopsSystems PCIe Expansion Cards:
On their site: https://lmg.gg/8KVc2

 

Buy Mellanox ConnectX Cards:
On their site: https://lmg.gg/8KVcn

 

Buy ASUS USB Cards: 
On Amazon: https://lmg.gg/8KVck
On Newegg: https://lmg.gg/8KV58

 

Buy NVIDIA Titan GPUs:
On Amazon: https://lmg.gg/8KV5K
On Newegg: https://lmg.gg/8KV5S 

 

[Boinbo]

3 minutes ago, jakkuh_t said:

removed cuz line spam - jakkuh

Awesome!

[Windows7ge]

Just when I give up and assume the project's some sort of abandoned it comes back to life. I'm excited to see this.

 

"...god tier Tyan S7100EX motherboard. Yes it's a little ugly..."

 

Ugly?  I think it's beautiful.

[r2724r16]

Linus is gonna drop it

[lewdicrous]

9 minutes ago, r2724r16 said:

Linus is gonna drop it

He already dropped a $10k CPU, so that's not out of the question lol

[SansVarnic]

At this point its a server with terminals ..... but who am I to question lol.

[greenhorn]

Meanwhile, Brian (the electrician) is getting worried hoping that Linus used properly rated power cables, wall sockets, a properly rated circuit with the correct fuse and that he does not go near the power limit for the whole building unit.

[GDRRiley]

waiting for the epyc overhall  

128 cores for 6 editors is 20 each. 

[Khiss]

I’m still trying to understand what the benefit of this is over 6 standalone PCs?

 

Is it that if only one user is running a resource heavy task, he would have access to the full power of both CPUs and all GPUs?

[scottyseng]

24 minutes ago, Khiss said:

I’m still trying to understand what the benefit of this is over 6 standalone PCs?

 

Is it that if only one user is running a resource heavy task, he would have access to the full power of both CPUs and all GPUs?

Bragging rights.

 

No, unless the whole VM is reconfigured to use all the CPU / GPUs. But generally passthrough limits GPU to said VM unless reconfigured.

[Computernaut]

Finally!

 

"For each of our editor." (9:03)

 

I'm sad that they didn't use Quadros, then that thing would be truly ultimate, but you can't have it all (or can you, when you're Linus Tech Tips?).

 

In the next video I hope to see what the power draw is like (they'll have to get another power-meter) and some glam-footage.

[Chadahc]

Got a pcpartpicker build list?

[Results45]

Comparatively, it's not too shabby considering that for $45,000 worth you get a Mac Pro on wheels with half the Xeon cores, 4 x Titan V-equivalent Vega II Pros, 1.5TB RAM, 4TB SSD storage (shame's on you Apple), and a beefy 1400W power supply.

 

 

[Sauron]

8 hours ago, Computernaut said:

I'm sad that they didn't use Quadros, then that thing would be truly ultimate, but you can't have it all (or can you, when you're Linus Tech Tips?).

I don't think quadros would perform any better, the only gain would be ecc vram but I don't think it's that important for their workload.

[Rohith_Kumar_Sp]

On 1/28/2019 at 1:34 AM, Rohith_Kumar_Sp said:

The least you could do for those who want to catch up is link the old 2-3 videos that covered this topic. its not in the description nor in the tiles.  

here's a fan made playlist : 

https://www.youtube.com/playlist?list=PLEX56sejvHB1cmrydC_fdhSi6GUocuqlKhttps://www.youtube.com/playlist?list=PLEX56sejvHB1cmrydC_fdhSi6GUocuqlK

 

 

[alpha754293]

I'm surprised that you didn't go with Nvidia Tesla V100 and Nvidia Grid like Wendall did.

 

I'm also surprised that you didn't just plug the QSFP28 cable directly from the server to this system and let it run at the full 100 Gbps IB or 100 GbE speeds (so you can skip the Mellanox switch entirely.*

 

(*Although, if you WERE to get a switch, I'd recommend you shelling out for the 100 GbE switch instead of the IB switch. For what you guys are doing, you're going to be using ethernet a LOT more than me using Infiniband more. You have increased latency and overhead, but it'll ultimately likely be more useful for you guys.)

[Derkoli]

20 hours ago, Khiss said:

I’m still trying to understand what the benefit of this is over 6 standalone PCs?

 

Is it that if only one user is running a resource heavy task, he would have access to the full power of both CPUs and all GPUs?

Benefits: Views

Disadvantages: gLitcHeS

[Computernaut]

14 hours ago, Sauron said:

I don't think quadros would perform any better, the only gain would be ecc vram but I don't think it's that important for their workload.

True (although the Quadro GV100 is technically a bit faster than the Titan V). Maybe (just maybe) I wanted them to do it just for the sake of it.

[Dredgy]

On 6/26/2019 at 7:29 AM, Khiss said:

I’m still trying to understand what the benefit of this is over 6 standalone PCs?

 

Is it that if only one user is running a resource heavy task, he would have access to the full power of both CPUs and all GPUs?

 

Linus has said they don’t actually intend to make their editors use it full time, it’s more of a passion project to see if it’d possible.

[bestbattle]

Correct that noob mistake and move all the fans so that they blow heat away from the radiators, not blow are through then.

You lose all the pressure by trying to push air. And if you look at all industrial application (even your AC internal unit) they all blow air away from the heat exchanger and are always installed in front of them.

Please correct this and keep this aspect in mind always.

[NiMiZ]

Linus - which OS / Hypervisor did you used for this project?

I have Xeon on Asus MB and I can't find OS that will allow me to give dedicated GPU for every VM

(I added the fact that I have Asus MB so EXSI didn't work for me...)

 

[jakkuh_t]

On 6/27/2019 at 12:06 PM, NiMiZ said:

Linus - which OS / Hypervisor did you used for this project?

I have Xeon on Asus MB and I can't find OS that will allow me to give dedicated GPU for every VM

(I added the fact that I have Asus MB so EXSI didn't work for me...)

 

We used Unraid, but Proxmox or any KVM based hypervisor will work fine (fedora w/ KVM/libvirt installed is solid)

[Luscious]

Very nicely done, but I am skeptical that the cooling for those 8 blocks is even adequate for 9x5 use. I have a CaseLabs S8 + pedestal with four GTX 980Ti's and a 5960X being cooled by FIVE massive 80mm thick rads. Right now the room ambient is 30C, my water temp at 47C and the GPU's and CPU hovering around 55C under full load with the fans at maybe 70% - impressive numbers and quiet enough to barely notice above any ambient noise going on.

 

Also, I always make sure to keep the water temp under 55C, since the D5 is only rated to handle up to 60C. Put it another way, I could run my rig in a much warmer ambient location and still keep the hardware cooled - the limiting factor is my water temp, dictated by the coolant volume and the number of radiators. I have a gallon of coolant in my system, and while it does absorb the heat incredibly well, having more would bring my temperatures down even closer to room ambient. Ramping up the 22 fans I have to 100% can bring the water temp down by 2-3 degrees, but at the expense of significantly more noise - something that I'm sure you don't want in a shared space with 5 other guys

 

I think if you were to do this build today the TT WP200 would be the (better) case to go with - it's got the room for four power supplies, a ton of 3.5 inch storage and can handle 80mm thick rads in the pedestal. I wish CaseLabs were still around because I would be eager to slap together a Dominus Extreme build using their TX10... with a Tyan S7105 shoehorned in the back

[Diazonium]

Hmm, what a nice Single Point of Failure you have here...it would be a shame if something happened to it.

Seriously, I think this is a bad idea reliability-wise. So many components, hard to troubleshoot. Dual socket systems, HW passthrough, PCI-E port multipliers and liquid cooling are all temperamental beasts that might end up causing headaches at the worst possible times.

Packing all of them into a single point of failure, especially since you need it for your core business activities just does not seem like a wise move.

[Diazonium]

12 hours ago, VegetableStu said:

that's just for LTX if I remember right ._. or for flexing anyway. the builds for 2 Gamers, 7 Gamers or 10 Gamers never were permanent 

(unless linus mentioned using this longterm, which he hasn't)

Huh, for some reason I thought this will be the permanent solution for their editing room, replacing all their current editing rigs.