Deep Learning Build Planning

[lyinch]

Hey, I plan to build a (multi-GPU) deep learning workstation. It should be upgradable (replace individual components) and extendable (add components). I have no idea about current hardware trends so feel free to take it all apart. I need prices for Switzerland.

 

PCPartPicker Part List

CPU: AMD Threadripper 2920X 3.5 GHz 12-Core Processor  ($449.98 @ Amazon) 
CPU Cooler: Deepcool CASTLE 240EX 64.4 CFM Liquid CPU Cooler  ($120.99 @ Amazon) 
Motherboard: Asus ROG ZENITH EXTREME EATX sTR4 Motherboard 
Memory: G.Skill Ripjaws V Series 64 GB (4 x 16 GB) DDR4-3200 Memory  ($266.99 @ Newegg) 
Storage: ADATA XPG SX8200 Pro 1 TB M.2-2280 NVME Solid State Drive  ($149.99 @ Amazon) 
Storage: Western Digital Red 4 TB 3.5" 5400RPM Internal Hard Drive  ($116.99 @ Adorama) 
Storage: Western Digital Red 4 TB 3.5" 5400RPM Internal Hard Drive  ($116.99 @ Adorama) 
Video Card: Zotac GeForce RTX 2080 Ti 11 GB Blower Video Card (2-Way SLI)  ($1612.98 @ Amazon) 
Video Card: Zotac GeForce RTX 2080 Ti 11 GB Blower Video Card (2-Way SLI)  ($1612.98 @ Amazon) 
Case: Phanteks Evolv X ATX Mid Tower Case  ($177.99 @ Amazon) 
Power Supply: Corsair Professional 1200 W 80+ Platinum Certified Fully Modular ATX Power Supply  ($299.99 @ Amazon) 
Total: $4925.87
Prices include shipping, taxes, and discounts when available
Generated by PCPartPicker 2020-02-06 15:53 EST-0500

 

CPU

For multi-GPU usage I ideally have enough PCIe lanes. Theadripper 2nd Gen is cheap and supports enough lanes compared to the 3950x. While I can't upgrade to 3rd gen, I can always upgrade to the 2990WX which should last me another few years after the upgrade. 

 

MoBo

The same argument as for the CPU is done for the motherboard. The motherboard should fit the sTR4 socket and contain enough PCIe lanes to fully utilise the GPUs. From the specs it says that 4 x PCIe 3.0 x16 (x16, x16/x16, x16/x8/x16, x16/x8/x16/x8) are supported. However, I found an infographic that apparently shows 3 x16 and one x8 GPU support.  The slots should be far enough apart to fit all the big GPUs. The motherboard can also be extended to 128GB which will last for quite some time. Hardware Raid 1 support is also a nice bonus.

 

GPUs

Dual RTX 2080Ti is more than enough to get started. Thermal throttling is a big concern in multi-GPU builds so either I use blower cards or water cooled cards. The free slots can holds two more cards, either 2080Tis, or maybe even an RTX Titan at some point if I want the sweet VRAM.

 

PSU

While 1200W is too much for this build, if I extend it with up to two GPUs I will greatly appreciate this power. I also chose a multirail PSU.

 

Storage & RAM

An M.2 drive for the working data and two HDDs for cold storage that will be put into RAID 1. Maybe even with ZFS file system. (any opinions on ZFS?). 64GB ram is currently plenty. 3200 CAS 16 is apparently a sweet spot for RAM so I chose what seemed a good deal.

 

Cooling

A very important part is cooling. I plan an aio watercooler for the CPU and blower style GPUs. Blower GPUs rather than hybrid cooled GPUs because they are a bit cheaper and I might want to build a custom loop in the future to reduce the noise and cool slightly better.

 

Case

I think that the mid tower case is a bit tight for all the stuff but it should still work. This might be a pain point when I want to upgrade or do a custom water loop.

 

The plan is to start with the above build and then add GPUs as the need arises for the local machine to be more powerful. After thermal issues or noise complaints from the neighbours I'll look into custom water cooling. Any comments or suggestions?

 

This is my first build

 

 

[Radium_Angel]

1 minute ago, lyinch said:

This is my first build

I have done RNN/DL/ML stuff before.

My experience is the nVidia/Intel software is more mature than the ATI/AMD stuff.

Take that for what it's worth. I've built and used it both ways and I got better results going the nVidia/Intel combo.

[lyinch]

5 minutes ago, Radium_Angel said:

I have done RNN/DL/ML stuff before.

My experience is the nVidia/Intel software is more mature than the ATI/AMD stuff.

Take that for what it's worth. I've built and used it both ways and I got better results going the nVidia/Intel combo.

In this case it would be an nVidia/AMD build.

What's the equivalent Intel CPU? Do I need to look into Xeon CPUs for something that supports the equivalent PCIe lanes than the 2nd gen Threadripper? I'm at a loss when looking into the convoluted world of Intel processors.

[Spakes]

6 minutes ago, lyinch said:

In this case it would be an nVidia/AMD build.

What's the equivalent Intel CPU? Do I need to look into Xeon CPUs for something that supports the equivalent PCIe lanes than the 2nd gen Threadripper? I'm at a loss when looking into the convoluted world of Intel processors.

7920x would be an equivalent. Also, did you think of Titan V? It is the best GPU for DL rn.

[Juular]

Overall looks good and you seem to know what you want. Although i'd go for different case, especially if thermal throttling is your concern, i'd recommend Lian-Li PC-O11 with some down mounted fans to blow at GPUs or more conventional cases but with front mesh panel to assist airflow ike Phanteks P400, Phanteks Enthoo Pro M, FD Meshify C, CM H500, hell, even NZXT H700 would have better airflow while having comparable aesthetics and feature set. And i'd go for EVGA CLC 360 AIO almost or the same price, BIG difference with 240mm. And holy fuck, that GPU are overpriced, if you're buying blower-style get the cheapest one, they're identical anyway. Also, there are somewhat cheaper SSD choices.

 

PCPartPicker Part List

CPU: AMD Threadripper 2920X 3.5 GHz 12-Core Processor  ($449.98 @ Amazon)
CPU Cooler: EVGA CLC 360 74.82 CFM Liquid CPU Cooler  ($129.99 @ Newegg)
Motherboard: Asus ROG ZENITH EXTREME EATX sTR4 Motherboard
Memory: G.Skill Ripjaws V Series 64 GB (4 x 16 GB) DDR4-3200 Memory  ($266.99 @ Newegg)
Storage: Mushkin PILOT 1 TB M.2-2280 NVME Solid State Drive  ($128.99 @ Amazon)
Storage: Western Digital Red 4 TB 3.5" 5400RPM Internal Hard Drive  ($116.99 @ Adorama)
Storage: Western Digital Red 4 TB 3.5" 5400RPM Internal Hard Drive  ($116.99 @ Adorama)
Video Card: PNY GeForce RTX 2080 Ti 11 GB Blower Video Card (2-Way SLI)  ($1159.99 @ Walmart)
Video Card: PNY GeForce RTX 2080 Ti 11 GB Blower Video Card (2-Way SLI)  ($1159.99 @ Walmart)
Case: Lian Li PC-O11 Dynamic ATX Full Tower Case  ($139.00 @ Adorama)
Power Supply: Corsair Professional 1200 W 80+ Platinum Certified Fully Modular ATX Power Supply  ($299.99 @ Amazon)
Total: $3968.90
Prices include shipping, taxes, and discounts when available
Generated by PCPartPicker 2020-02-06 16:42 EST-0500

 

4 minutes ago, Spakes said:

Also, did you think of Titan V? It is the best GPU for DL rn.

I mean, it's kinda pricey too.

[Radium_Angel]

14 minutes ago, lyinch said:

I'm at a loss when looking into the convoluted world of Intel processors.

Is this going to be a fun learning experience (the ML part I mean) or are you planning on making money with this?

[lyinch]

8 minutes ago, Spakes said:

7920x would be an equivalent. Also, did you think of Titan V? It is the best GPU for DL rn.

Pricewise the 7920x is twice that of the Threadripper. I could get one for 1.1k but the above mentioned Threadripper for 500. The Intel CPU has "only" 44PCIe lanes. Am I too focused on the 3 x16 + 1 x8 lanes here?

 

I haven't considered the Titan V, that will not fit my budget. I'm already stressing it enough with this build. I want a (very) powerful machine at home, and anything that needs more computational power I'll offload it to the cloud. I did consider the Titan RTX, but I can get two 2080Ti for 2.2k which are blower style, or 2.8k with watercooled aio whereas the Titan costs 3k here and has dual fans which won't cooperate with any future additions.

 

 

[lyinch]

19 minutes ago, Juular said:

Overall looks good and you seem to know what you want. Although i'd go for different case, especially if thermal throttling is your concern, i'd recommend Lian-Li PC-O11 with some down mounted fans to blow at GPUs or more conventional cases but with front mesh panel to assist airflow ike Phanteks P400, Phanteks Enthoo Pro M, FD Meshify C, CM H500, hell, even NZXT H700 would have better airflow while having comparable aesthetics and feature set. And i'd go for EVGA CLC 360 AIO almost or the same price, BIG difference with 240mm. And holy fuck, that GPU are overpriced, if you're buying blower-style get the cheapest one, they're identical anyway. Also, there are somewhat cheaper SSD choices.

 

PCPartPicker Part List

CPU: AMD Threadripper 2920X 3.5 GHz 12-Core Processor  ($449.98 @ Amazon)
CPU Cooler: EVGA CLC 360 74.82 CFM Liquid CPU Cooler  ($129.99 @ Newegg)
Motherboard: Asus ROG ZENITH EXTREME EATX sTR4 Motherboard
Memory: G.Skill Ripjaws V Series 64 GB (4 x 16 GB) DDR4-3200 Memory  ($266.99 @ Newegg)
Storage: Mushkin PILOT 1 TB M.2-2280 NVME Solid State Drive  ($128.99 @ Amazon)
Storage: Western Digital Red 4 TB 3.5" 5400RPM Internal Hard Drive  ($116.99 @ Adorama)
Storage: Western Digital Red 4 TB 3.5" 5400RPM Internal Hard Drive  ($116.99 @ Adorama)
Video Card: PNY GeForce RTX 2080 Ti 11 GB Blower Video Card (2-Way SLI)  ($1159.99 @ Walmart)
Video Card: PNY GeForce RTX 2080 Ti 11 GB Blower Video Card (2-Way SLI)  ($1159.99 @ Walmart)
Case: Lian Li PC-O11 Dynamic ATX Full Tower Case  ($139.00 @ Adorama)
Power Supply: Corsair Professional 1200 W 80+ Platinum Certified Fully Modular ATX Power Supply  ($299.99 @ Amazon)
Total: $3968.90
Prices include shipping, taxes, and discounts when available
Generated by PCPartPicker 2020-02-06 16:42 EST-0500

 

I mean, it's kinda pricey too.

I agree with the case. I didn't put much thought into it and it's the only part that I chose because it looks good  Thank you for the extensive list of cases, I'll take my time tomorrow and go through them. You hit all the pain points, the CPU cooler is the other part that I had on the list and never looked at again. I'll follow your guidance here and chose the EVGA CLC 360.

 

Please ignore the prices of the components on my list, I have to buy them in Switzerland and pcpp doesn't support it. I looked some parts up at local retailers and they differ by a lot (in both directions). I can get a PNY 2080 Ti for 1093CHF and the ZOTAC 2080 Ti (which is in my build) for 1069CHF. That's why I added this specific card to the build.

 

As for the SSD, I don't have any fixed opinion either. If I can save some money there, then I'll probably use the slightly cheaper one.

 

(edit:)

15 minutes ago, Radium_Angel said:

Is this going to be a fun learning experience (the ML part I mean) or are you planning on making money with this?

I'll be working in this field but probably won't make money with it as a side gig. So the machine is strictly personal and not professional. Although it helps me for my studies and job, it's not required for my main income.

 

 

Thank you all for taking the time to give me feedback!

[Radium_Angel]

1 hour ago, lyinch said:

feedback

Ok,

 

The SLI cards, have you determined that's good value for money, I'm not certain which software you will be using, but the stuff I used didn't utilize a 2nd video card, so that might be money wasted.

[igormp]

7 minutes ago, Radium_Angel said:

Ok,

 

The SLI cards, have you determined that's good value for money, I'm not certain which software you will be using, but the stuff I used didn't utilize a 2nd video card, so that might be money wasted.

Most ML frameworks do support multi-gpu out of the box, he won't even need a NVLink bridge.

[lyinch]

7 hours ago, Radium_Angel said:

Ok,

 

The SLI cards, have you determined that's good value for money, I'm not certain which software you will be using, but the stuff I used didn't utilize a 2nd video card, so that might be money wasted.

 

7 hours ago, igormp said:

Most ML frameworks do support multi-gpu out of the box, he won't even need a NVLink bridge.

Correct, looking at the docs for e.g. tensorflow it is almost trivial to add more GPUs. However, it's not always so easy use the full power of both GPUs (might not get a 2x speedup, but rather 1.7x or something) because some tasks have a heavy synchronization overhead. Adding an NVLink bridge might help but I haven't looked into this, yet. I assume it's easy to add once I have the GPUs, or is this something that I need to plan in advance? In general, the 2080Ti and Titan RTX are both amazing cards for this task and I think that I won't suffer by choosing the slightly cheaper setup. 

 

For those interested, the two GPUs don't need to be used in parallel for the exact same model. It is possible to train the model with different hyperparameters on each to "really get the full 2x speedup".

[igormp]

6 hours ago, lyinch said:

 

Adding an NVLink bridge might help but I haven't looked into this, yet. I assume it's easy to add once I have the GPUs, or is this something that I need to plan in advance?

A bridge might not be that useful, specially for regular geforce cards since those fall back to regular SLI. You can read more about that here and here. You also need to remember that some models can actually achieve close to 2x speedup, such as when running ensembles, or models like Alexnet that were made specifically to run in multi gpu.

6 hours ago, lyinch said:

 

For those interested, the two GPUs don't need to be used in parallel for the exact same model. It is possible to train the model with different hyperparameters on each to "really get the full 2x speedup".

Yes! Doing so is amazing while going for a grid search, or even when trying to use some ES instead of relying on regular BP.

[lyinch]

7 hours ago, igormp said:

A bridge might not be that useful, specially for regular geforce cards since those fall back to regular SLI. You can read more about that here and here. You also need to remember that some models can actually achieve close to 2x speedup, such as when running ensembles, or models like Alexnet that were made specifically to run in multi gpu.

Yes! Doing so is amazing while going for a grid search, or even when trying to use some ES instead of relying on regular BP.

That is very interesting to hear, thank you for the two links. I really like the reports of pugetsystems, that's also were I got the confirmation of thermal throttling for multi-GPU setups.

 

To wrap this up, I decided to go with the build.

 

As advised by @Juular I chose a different case, the Phanteks Enthoo Pro. The other changes were the CPU aio cooler as it was not available in Switzerland, the RAM which wasn't available either, and the HDD where I went from the 4TB model to the 3TB WD Red HDD because they are surprisingly expensive here. The major components stayed the same, I chose the cheapest blower GPU that I could find, which was in my case the Zotac RTX 2080Ti for 1098CHF or $1123 and sticked with the motherboard, CPU, PSU and NVME.

 

For anyone interested, if you take the lowest functional subset of components, namely no HDDs and "only" one GPU, the build costs (+shipping etc.) 3074CHF or 2872€ or $3144. Adding the GPU and the hard disks raises the price by 1304CHF to an insane 4.3kCHF. Let's not think about the two free GPU slots... 

 

I'm excited to build this and hope that everything not only fits together but also works well together.

Thank you all for your great support. The next post here will probably be a request for a custom water loop!

[igormp]

32 minutes ago, lyinch said:

That is very interesting to hear, thank you for the two links. I really like the reports of pugetsystems, that's also were I got the confirmation of thermal throttling for multi-GPU setups.

 

To wrap this up, I decided to go with the build.

 

As advised by @Juular I chose a different case, the Phanteks Enthoo Pro. The other changes were the CPU aio cooler as it was not available in Switzerland, the RAM which wasn't available either, and the HDD where I went from the 4TB model to the 3TB WD Red HDD because they are surprisingly expensive here. The major components stayed the same, I chose the cheapest blower GPU that I could find, which was in my case the Zotac RTX 2080Ti for 1098CHF or $1123 and sticked with the motherboard, CPU, PSU and NVME.

 

For anyone interested, if you take the lowest functional subset of components, namely no HDDs and "only" one GPU, the build costs (+shipping etc.) 3074CHF or 2872€ or $3144. Adding the GPU and the hard disks raises the price by 1304CHF to an insane 4.3kCHF. Let's not think about the two free GPU slots... 

 

I'm excited to build this and hope that everything not only fits together but also works well together.

Thank you all for your great support. The next post here will probably be a request for a custom water loop!

Another nice blog that you might like to read is the one from Tim Dettmers, he has nice posts where he talks about blower style gpus being better if you have more than one. You can read a couple posts here and here.

[Juular]

I forgot that almost all coolers\AIOs have smaller coldplate which doesn't go well with Threadrippers \ Intel HEDT (and because of that, your original AIO choice might be better than EVGA CLC since it has slightly larger coldplate, inner chamber are probably still small tho), if you can grab Alphacool Eisbaer for a reasonable price it will be better choice, there are some alternatives from CM and Thermaltake and but they're somewhat hard to find in Europe, just stay away from Enermax AIOs.