Help with gou render farm

[Muaddib12]

Hi guys!

 

I´m working on a 3d studio and we are planning on making our own render farm, in this case, gpu based, but its quite difficult to choose all suitable components without going very high prices. Our idea its to buy the base computer (motherboard,psu, cpu, ram and 1 or 2 graphics card) and then scalate when we have the money to do so.

 

We absolutely could use some tech tips on this, so I´ll try to number the specifications we need and see if some caritative soul could give us a hand:

 

Motherboard:  Need to have 8 pci e lanes at least x8 (render engine needs this bandwitch to render fast)

CPU: Any i7 can do the job, according to engine developers, but I guess any more-tan-4-pcie-lane-motherboard its pointing to 2011-3 cpus...

Ram: At least 32gb capable, 64 desirable.

Graphics card: This are meant to be use at 100% load for long periods of time, so we definitly thought on water cooling. Unfortunately that requires maintenance knowledge, which none on the studio have, so we´ve decided to go with the nzxt g10 bracket and some of the compatible closed loop coolers. On the specific model, needs to be Nvidia, and the much ram and cuda cores, the better, we thought on 980 ti or if we manage to populate the entire 8 pcie slots, the 970.Case: It does not need to be any gpu expensive rack or any superperformance case, we even thought on bitcoin mining case like the  riggit revision 9.

PSU: On linus video he recomend to use at least 2 different psu every 3 cards, so I guess its a matter of choosing the right brand (80 plus gold, of course). Any insigths?

 

Well I think that´s all, I will be very grateful if someone throws us a hand...

 

[Moonzy]

6 minutes ago, Muaddib12 said:

populate the entire 8 pcie slots

means 8 gpu or 4 dual slot gpu?

i dont think theres a single processor out there capable of driving 8 nvidia gpus, each with 8x pcie (that would require 64 lanes and the 5960x only have 40)

[brob]

While an i7 will do the job, I suspect one would want ecc memory. I7 do not support ecc. Which leaves Xeon, likely E5.

 

A prosumer board like the Asus X99-E WS provides 7 PCIe slots and support for running them all at x8. But gpu of any power are double slot width (or more), so one has to consider some method of moving the cards off-board. Extending PCIe any distance is not realistic. 

 

[Muaddib12]

6 hours ago, Moonzy said:

means 8 gpu or 4 dual slot gpu?

i dont think theres a single processor out there capable of driving 8 nvidia gpus, each with 8x pcie (that would require 64 lanes and the 5960x only have 40)

I mean 8 per node (motherboard). What about trentor backplanes? Or someting by supermicro?

[Muaddib12]

5 hours ago, brob said:

While an i7 will do the job, I suspect one would want ecc memory. I7 do not support ecc. Which leaves Xeon, likely E5.

 

A prosumer board like the Asus X99-E WS provides 7 PCIe slots and support for running them all at x8. But gpu of any power are double slot width (or more), so one has to consider some method of moving the cards off-board. Extending PCIe any distance is not realistic. 

 

The pcie risers don´t provide with the neccesary bandwitch? I saw examples of using those in bitcoin mining a lot

[brob]

12 minutes ago, Muaddib12 said:

The pcie risers don´t provide with the neccesary bandwitch? I saw examples of using those in bitcoin mining a lot

Sure, they are a possibility.

[Muaddib12]

After a lot of R&D, we decided to go with this board: https://www.asus.com/es/Motherboards/Z9PED8_WS/specifications/

 and a couple of xeon. Thtas the only way to have that many cards working at x8.

 

Thanks for the help.