GPU Cluster for Deep Learning with Consumer GPUs

erik_s · February 6

Hello everyone,

I've started working in a recently founded lab at a university. As the lab was founded less than a year ago, we are still in the process of setting everything up. This also includes setting up servers/GPU clusters. We have one based on a Supermicro A+ Server 4124 GS-TNR that currently packs 5 RTX A6000 and we would like to set up a second one. However, for this second server we want to go for 8 consumer-grade GPUs, either RTX 3090 or RTX 4090, to save some money. The problem that I've encountered is, that every specification only has room for 2-slot GPUs. I've already thought about using Risers with the GPUs and mounting them outside of the server case, but I'm not sure if that is the most elegant way to go about it. Does anyone have experience with building a server with consumer-grade GPUs? Or do you think it's saving money on the wrong end?

Thanks for your help!

leadeater · February 6

I wouldn't build one out with consumer GPUs myself, you are academic so I'd try contacting a local Nvidia reseller/IT service company and enquire about the academic pricing program for RTX A6000 GPUs.

https://developer.nvidia.com/education-pricing

I can't tell you what we pay for our Nvidia A40's but it's not even close to the MSRP of an RTX A6000

That said we have researchers here using consumer GPUs, usually not in rackmount chassis or not trying to fill the entire chassis.

Anyway if you want a suitable RTX 3090 that is compatible then this is one option: https://www.amazon.com/Gigabyte-NVIDIA-GeForce-Graphics-GV-N3090TURBO-24GD/dp/B08KHKDTSJ

Its true dual slot with the power connector in the end rather than the top.

Edited February 6 by leadeater