Tower server for development and AI training.

[Simon Golbert]

Hi guys,

 

I plan to by a tower server for my home office to use it for development and long AI training. And I would like to get your opinion, tips and suggestions regarding the configuration I'm currently considering.

 

Dell T640 8LFF

2 × Intel Xeon Gold 5120 (14C 19.25M Cache 2.20 GHz)

2 × 32GB DDR4 RDIMM 2400MHz

RAID DELL PCI-e H730p (2GB+FBWC)

iDRAC 9 Express

Power supply Dell 1100w

2 port 1Gb/s (Integrated)

SSD 960GB SAS 2.5" + Tray Caddy

NVIDIA Tesla V100 32GB PCI-e

 

I plan to buy it refubushed (except the GPU) for $4350.

 

What do you think of this purchase? Does it make sense. Would appreciate if you share your insights. Thanks.

[igormp]

IMO you'd be better off with an AM4 or AM5 platform and dual 3090s.

 

More CPU horsepower, more VRAM, and way cheaper.

[Likwid]

8 minutes ago, igormp said:

IMO you'd be better off with an AM4 or AM5 platform and dual 3090s.

 

More CPU horsepower, more VRAM, and way cheaper.

Those cpu's are old and  slow, just get new AM5 system for a lot less

[emosun]

those xeon golds arent very fast for 4,000$

is there a reason you're paying more for this particularly slow machine or?

[ewitte]

Are you planning on only having 1 GPU?  AM4/AM5 can be limited for PCIe lanes depending on what you want.  There is also Epyc.

[Simon Golbert]

That is a problem. For the budget of $4.5k I need a good tower server (possibly refurbished with a warranty) equipped with a descent server-class GPU card optimised for machine learning (I'm not a gamer). And I lack both knowledge and expertise to pick proper offerings.

[Simon Golbert]

11 hours ago, ewitte said:

Are you planning on only having 1 GPU?  AM4/AM5 can be limited for PCIe lanes depending on what you want.  There is also Epyc.

I prefer to get maximum TPFLOPS for the money (nice to have 8/4 bit float support if available). So it practically can be one or two GPUs I suppose.

[ewitte]

On 2/22/2025 at 3:03 AM, Simon Golbert said:

That is a problem. For the budget of $4.5k I need a good tower server (possibly refurbished with a warranty) equipped with a descent server-class GPU card optimised for machine learning (I'm not a gamer). And I lack both knowledge and expertise to pick proper offerings.

There should be something with Epyc that gives you more for less in that price target.  I don't really see something in tower though something could be built maybe a supermicro with most of the components already then you just have ram, etc.  You do NOT want rack for home I've found out over the years the sound isn't something you don't want in a house, although tower may not be all that better.  Maybe a used thread ripper machine???  I see a bew Lenovo P620 workstation for about $3k.  It says 2 GPUs but lists 1000w PSU....

[igormp]

9 hours ago, Simon Golbert said:

nice to have 8/4 bit float support if available

The V100 lacks that. You'll only get FP8 with Ada, and FP4 on blackwell.

9 hours ago, Simon Golbert said:

maximum TPFLOPS for the money

That would be 2~4 used 3090s.

[Simon Golbert]

Yep, but I need server-grade cards to work 24/7.

[igormp]

10 hours ago, Simon Golbert said:

Yep, but I need server-grade cards to work 24/7.

Nothing stops you from using 3090s 24/7, I've been doing so for quite some time now. There's not much benefit going for a V100 honestly, but if that's what you'd feel more comfortable with, more power to you.