build Help building a multi-GPU-pc for ML Tasks

[Moin Moti]

Budget (including currency): 50,000 HKD

Country: Hong Kong

Games, programs or workloads that it will be used for: Machine Learning, CUDA

Other details (existing parts lists, whether any peripherals are needed, what you're upgrading from, when you're going to buy, what resolution and refresh rate you want to play at, etc): 

 

Hey fellas, I am a student in a computer science lab and we want to build a pc for machine learning tasks. We don’t have much experience in setting up a heavy processing pc, hence I would like your help.

A shop recommended us a build-up (attached in the post)

 

The shopkeeper also recommended us to go with a higher power supply (mentioned in the end as optional) so that we may upgrade it to 4 GPU(s) in the future. However, we are skeptical if the current setup can fit 4 GPU(s). Are we right?

 

We would like to have the option to add more GPU(s) in the future, given it doesn’t exceed the budget too much. Can you please suggest any required changes to achieve the same?

 

[adm0n]

I would have some doubt about it as well.

The RTX 2080ti's can draw around 270W sustained and 350 peak (for under 50ms) alone.
So 4 * 300w  = 1200w already.

 

Another thing you have to consider is that your 2080ti's will be right next to each other. Meaning They will have very little air to work with. I'd assume that they won't be able to work at maximum clocks for very long. 

 

So having only two cards would work better.

 

[SupaKomputa]

They really love RGB.

That motherboard support 4 GPU, no problem.

32gb memory is kinda small for the task, you can max it out to 128gb or 64gb at least.

1200w is minimum for 4 2080, just buy the 1600 just to be safe.

[Gorgon]

Tim Dettmers has a post about Hardware requirements for Deep Learning

 

Puget Systems also have some articles about DL including this one by Dr. Donald Klinghorn.

 

You could also look at using a mining frame with dual power supplies.

 

Heres a six GPU system I built for Folding@home:

 

[Moin Moti]

27 minutes ago, adm0n said:

I would have some doubt about it as well.

The RTX 2080ti's can draw around 270W sustained and 350 peak (for under 50ms) alone.
So 4 * 300w  = 1200w already.

 

Another thing you have to consider is that your 2080ti's will be right next to each other. Meaning They will have very little air to work with. I'd assume that they won't be able to work at maximum clocks for very long. 

 

So having only two cards would work better.

 

Thanks for confirming our doubts. What do you recommended as an alternative GPU and motherboard combination so that it can fit up to 4 GPU(s) in the future?

[HenrySalayne]

The mainboard's ATX format is basically as big as it gets.The spacing is standardized. The E-ATX format only adds width, but doesn't increase the amount of slots.You might want to get a case with enough airflow to keep your chunk of graphic cards cool or use custom water cooling.

[adm0n]

5 hours ago, Moin Moti said:

Thanks for confirming our doubts. What do you recommended as an alternative GPU and motherboard combination so that it can fit up to 4 GPU(s) in the future?

You could either use a doughter board, but I don't know if you can get them cheaply enough. There are some servers that use them though.

Alternativly you could look into using PCIe Riser to mount them someplace else. But I haven't really dealt with that before.

 

The easiest way by far, would be just to water cool all the cards. Then they don't depend on air cooling anymore.

 

AS for the power supply, the 1600W unit might be enough, but you might have problems when powering the whole thing on, as the initial current draw will be huge.

 

According to this tool:

https://outervision.com/power-supply-calculator

your recommended PSU wattage for 4 RTX 2080ti's is 1480W

[igormp]

6 hours ago, Moin Moti said:

I'd say to go with more RAM (at least 64gb), the 1600W PSU and blower cards instead of regular ones since you'll be running many of them right next to each other. AFAIK, zotac and pny have blower 2080TIs.

 

Since you're already spending that kind of money, wouldn't it be a good idea to wait for the new Ampere cards, or get a Quadro/Teslas/TItan? Regular Geforce cards are really limited VRAM-wise.

[HenrySalayne]

1 hour ago, igormp said:

Since you're already spending that kind of money, wouldn't it be a good idea to wait for the new Ampere cards, or get a Quadro/Teslas/TItan? Regular Geforce cards are really limited VRAM-wise.

It highly depends on the use case. 2080 Ti and Titan RTX are almost identical except for the RAM, but the Titan costs basically twice as much. IF VRAM is the limiting factor, Titan would be better, if not, get more 2080 Tis.

[igormp]

1 hour ago, HenrySalayne said:

It highly depends on the use case. 2080 Ti and Titan RTX are almost identical except for the RAM, but the Titan costs basically twice as much. IF VRAM is the limiting factor, Titan would be better, if not, get more 2080 Tis.

Yeah, I mentioned the other cards exactly due to the extra VRAM. 11gb is really way too short for anything non-trivial in ML.

[susobhang70]

On 5/28/2020 at 3:45 PM, SupaKomputa said:

They really love RGB.

That motherboard support 4 GPU, no problem.

32gb memory is kinda small for the task, you can max it out to 128gb or 64gb at least.

1200w is minimum for 4 2080, just buy the 1600 just to be safe.

Hi! I'm a friend of the OP.

For the people saying 32 GB is too low, please look at the quantity, it's "2" - so it's 64 GB of RAM.

 

As for @SupaKomputa's response - the motherboard supports 4 GPUs, but these are 2 slot cards. So essentially only two 2080 Tis can be mounted on the motherboard. Tell me if I'm missing something?

[SupaKomputa]

22 hours ago, susobhang70 said:

Hi! I'm a friend of the OP.

For the people saying 32 GB is too low, please look at the quantity, it's "2" - so it's 64 GB of RAM.

 

As for @SupaKomputa's response - the motherboard supports 4 GPUs, but these are 2 slot cards. So essentially only two 2080 Tis can be mounted on the motherboard. Tell me if I'm missing something?

Yes, if you use the slot, you cannot use the 2 pcie. They shared the same channel.

No worries, 1 ssd is enough, just get a big and fast PCIe 4.0 nvme ssd.