quad gpu build

[TFProtoform157]

I am looking to upgrade my PC to  run Octane render, and I am planning on sticking several more gpus in my build. at this point, I am salvaging parts from my current PC and building a new one. I am bringing over my Asus strix 980 ti and all of my storage and disc drives (yes i still use discs). My biggest concern (besides price) has been thermals. As stated many times in the compensator build videos, stacking gpus without water cooling is not the best idea. I am willing to go the extra mile and get hybrid cooled cards, even thinking about a custom loop (which I know nothing about). My first question is what would be the best way to have 4 gpus and have good cooling? 

My new PC will have these parts:

Corsair 750D

i7 7700k

Corsair h100i v2

Asrock z270 supercarrier

corsair vengence 32gb (2x 16gb) ddr4-3000

evga supernova 1300 gold psu

bluray drive

dvd drive

2x 480gb ssds & 1x 3tb hdd

 

second question:

here are my original thoughts: I was thinking of getting a 1080 ti FE for the first slot, 2x msi 1070 sea hawk hybrid gpus, and stick my 980 ti on the bottom, there it would have a extra slot to breathe (b/c the 750d has 9 exp slots). I thought this was good until i realized that all of the fan, usb, and i/o headers were on the bottom of the board, and that my 980 ti could interfere with them, which could be a deal breaker. You would think that the manufacturers would make sure that they wouldn't interfere, right? So, my second question is (again referring to compensator), do the bottom ports interfere with the gpu, because if yes, than I'd have to think of a new gpu configuration.

 

lastly, just two quick points. this is not an sli set up, as octane doesn't use sli. Also, the parts listed above are not finalized (except for the case, and i'd like to keep my 980 ti). I am willing to look at a different motherboard, cpu, etc. layout if there is a better option. Thanks!

[TheCherryKing]

You will likely need two processors in order to have enough PCIe lanes.

[Lurick]

You'll need a 5930K, 6900K or at least one Xeon CPU to provide at least 32 PCIe lanes. The 7700K can't provide enough PCIe lanes to support all the GPUs. The chipset on the x270 or x170 boards might be able to supply the needed lanes but they will be PCIe 2.0 if I remember correctly so you'll be much more limited in that regard. I could be wrong though so I'll let people chime in

[Droidbot]

Just now, TheCherryKing said:

You will likely need two processors in order to have enough PCIe lanes.

this, the only CPUs that can supply 4 x8s are 6850K/6900K/6950X

 

 

[grimreeper132]

3 minutes ago, TheCherryKing said:

You will likely need two processors in order to have enough PCIe lanes.

you can do it with one, if you get an intel extreme CPU, but the cards might be running off of 8X speeds rather than 16X, same is true with Ryzen, but 2 CPUs probably the better idea.

[TheCherryKing]

12 Core Xeons can be purchased used on eBay for about $300 each.

[Starelementpoke]

8 minutes ago, TFProtoform157 said:

I am looking to upgrade my PC to  run Octane render, and I am planning on sticking several more gpus in my build. at this point, I am salvaging parts from my current PC and building a new one. I am bringing over my Asus strix 980 ti and all of my storage and disc drives (yes i still use discs). My biggest concern (besides price) has been thermals. As stated many times in the compensator build videos, stacking gpus without water cooling is not the best idea. I am willing to go the extra mile and get hybrid cooled cards, even thinking about a custom loop (which I know nothing about). My first question is what would be the best way to have 4 gpus and have good cooling? 

My new PC will have these parts:

Corsair 750D

i7 7700k

Corsair h100i v2

Asrock z270 supercarrier

corsair vengence 32gb (2x 16gb) ddr4-3000

evga supernova 1300 gold psu

bluray drive

dvd drive

2x 480gb ssds & 1x 3tb hdd

 

second question:

here are my original thoughts: I was thinking of getting a 1080 ti FE for the first slot, 2x msi 1070 sea hawk hybrid gpus, and stick my 980 ti on the bottom, there it would have a extra slot to breathe (b/c the 750d has 9 exp slots). I thought this was good until i realized that all of the fan, usb, and i/o headers were on the bottom of the board, and that my 980 ti could interfere with them, which could be a deal breaker. You would think that the manufacturers would make sure that they wouldn't interfere, right? So, my second question is (again referring to compensator), do the bottom ports interfere with the gpu, because if yes, than I'd have to think of a new gpu configuration.

 

lastly, just two quick points. this is not an sli set up, as octane doesn't use sli. Also, the parts listed above are not finalized (except for the case, and i'd like to keep my 980 ti). I am willing to look at a different motherboard, cpu, etc. layout if there is a better option. Thanks!

If it's not an SLI setup, might want to try and save yourself a bit of trouble and get a Fury X? Since they're already water cooled?

[grimreeper132]

also what's the budget, and what have you currently got so that can be removed from the parts list?

[TheCherryKing]

Does the Corsair 750D even have enough slots for PCIe devices?

[Droidbot]

Just now, Lurick said:

You'll need a 5930K, 6900K or at least one Xeon CPU to provide at least 32 PCIe lanes. The 7700K can't provide enough PCIe lanes to support all the GPUs. The chipset on the x270 or x170 boards might be able to supply the needed lanes but they will be PCIe 2.0 if I remember correctly so you'll be much more limited in that regard. I could be wrong though so I'll let people chime in

Close, but no cigar. They're 3.0s, and there's 24 lanes on the PCH, but many of them are used for special stuff like NVMe etc.

Just now, TheCherryKing said:

12 Core Xeons can be purchased used on eBay for about $300 each.

ES chips. Not something I'd say is good for normal day-to-day use, especially in a freaking workstation. 

Not to mention owning one is technically illegal as they're property of Intel even after sold

Just now, TheCherryKing said:

Does the Corsair 750D even have enough slots for PCIe devices?

Yes, it does. 9 PCI-E back port things, assuming 2x for each GPU, it has enough

[grimreeper132]

2 minutes ago, TheCherryKing said:

Does the Corsair 750D even have enough slots for PCIe devices?

only room for three I think.

 

 

going by this image

 

scrap that each cards taking up three slots

[TheCherryKing]

Just now, grimreeper132 said:

only room for three I think.

 

 

going by this image

Those are triple slot cards.

[grimreeper132]

Just now, TheCherryKing said:

Those are triple slot cards.

I literally just noticed

 

I just edited it

[TheCherryKing]

The 8 Core Xeon E5 2670 processors are easy to find for less than $100 and they are not engineering samples.

[Droidbot]

Just now, TheCherryKing said:

The 8 Core Xeon E5 2670 processors are easy to find for less than $100 and they are not engineering samples.

Have fun finding a <$350 mobo for them and they've been flogged to death in virtualization rigs at Amazon and Microsoft Azure..

[grimreeper132]

1 minute ago, TheCherryKing said:

The 8 Core Xeon E5 2670 processors are easy to find for less than $100 and they are not engineering samples.

Still wondering what the budget is, and what the OP has so we can recommend shit better

[TheCherryKing]

Just now, Droidbot said:

Have fun finding a <$350 mobo for them and they've been flogged to death in virtualization rigs at Amazon and Microsoft Azure..

There is a few under $350 but I haven't found any with lots of PCIe slots.

[Droidbot]

Just now, TheCherryKing said:

There is a few under $350 but I haven't found any with lots of PCIe slots.

Also, it's a server architecture - the compatibility with consumer PCI-E devices is hit or miss

[TFProtoform157]

8 minutes ago, Lurick said:

You'll need a 5930K, 6900K or at least one Xeon CPU to provide at least 32 PCIe lanes. The 7700K can't provide enough PCIe lanes to support all the GPUs. The chipset on the x270 or x170 boards might be able to supply the needed lanes but they will be PCIe 2.0 if I remember correctly so you'll be much more limited in that regard. I could be wrong though so I'll let people chime in

From what I researched, there isn't much of a noticeable difference between x16 and x8 speeds. i decided to get the supercarrier  because it could run all 4 pci lanes at x8 instead of some boards runing some lanes at x4. 

as far as amd cards, octane uses the cuda cores in nvida cards, so i have to go with nvida (and i'd prefer it anyway).

The parts I do have are the 750d, the 980 ti, the storage, disc drives, and fans. pretty much everything else is up for debate. 

Id hesitate to say my budget is $2800

[grimreeper132]

Just now, TFProtoform157 said:

From what I researched, there isn't much of a noticeable difference between x16 and x8 speeds. i decided to get the supercarrier  because it could run all 4 pci lanes at x8 instead of some boards runing some lanes at x4. 

as far as amd cards, octane uses the cuda cores in nvida cards, so i have to go with nvida (and i'd prefer it anyway).

The parts I do have are the 750d, the 980 ti, the storage, disc drives, and fans. pretty much everything else is up for debate. 

Id hesitate to say my budget is $2800

that is true to a point when all the GPUs are trying to communicate with each other even with SLI bridges, that isn't always the same and 16X will helo in those situations, I believe, could be wrong though

[TheCherryKing]

Even if the graphics cards are running at PCIe 8x the bottleneck will be the single quad-core CPU.

[TFProtoform157]

2 minutes ago, TheCherryKing said:

Even if the graphics cards are running at PCIe 8x the bottleneck will be the single quad-core CPU.

so pretty much get a 6-or more core cpu. ok. Im still wondering what I should do about cooling all the gpus, and i guess which ones to get

[grimreeper132]

1 minute ago, TheCherryKing said:

Even if the graphics cards are running at PCIe 8x the bottleneck will be the single quad-core CPU.

that is also true, I mean I can get a server which is currently on  ebay as an auction with 12 cores with HT for about £80, but it's auction and probably go for £150-£175

 

1 minute ago, TFProtoform157 said:

so pretty much get a 6-or more core cpu. ok. Im still wondering what I should do about cooling all the gpus, and i guess which ones to get

get an old server with 2 6 cores min probably as that way there will be enough cores to process shit, especially in multi-threaded tasks which I guess you will be doing if you need 4 GPUs

[TFProtoform157]

I'm still a little lost. so, getting a server motherboard with maybe two cpus, ok. should i have everything watercooled? hybrid?

[grimreeper132]

14 minutes ago, TFProtoform157 said:

I'm still a little lost. so, getting a server motherboard with maybe two cpus, ok. should i have everything watercooled? hybrid?

your choice on that if the GPUs are running at 100% 24/7, possibly, cause it is possible to have a good air solution which have thermals which are acceptable, if you water cool however the temps will probably be better, but in the end it's your choice on what you want to do, we can only advise we can not make a decisions for you. that being said we can call you an idiot for not doing it the way we would do it, even if your solution is easier and better than ours, welcome to the internet, fuck you we're all arseholes here