Sever for AI around 50K

[tomcer2003]

Hi guys. My school gave me 50K for AI sever (it's my project). And I want to ask, what components should I use. I was thinking about new zen 3 sever chip (7763)64 cores with 2TB ram. And some quadro gpus. Or is there something better? My budget is 50K, ideally 40K. My currency is €. Feel free to any suggestions. This server will be also used as cloud for my school and it will run some virtual machines. And catch is I have to build it. I'm good with normal pc builds( I build more than 25pc, but never server) and how should I configure it. 

 

[brob]

I believe Nvidia offers a PCIe A100 card. Likely limited supply.

 

Consider getting a prebuilt server. Supermicro, Lenovo, Dell, etc. all offer barebone servers that would likely meet needs. The advantage is that they generally have dual psu, support ecc memory and are built for 24x7 operations. 

[tomcer2003]

4 hours ago, brob said:

I believe Nvidia offers a PCIe A100 card. Likely limited supply.

 

Consider getting a prebuilt server. Supermicro, Lenovo, Dell, etc. all offer barebone servers that would likely meet needs. The advantage is that they generally have dual psu, support ecc memory and are built for 24x7 operations. 

So building it by my self is stupid? Because I have to do some physical work on this project. I guess, I will just install gpu if it will not be in the options and setup server. Thank u for your recommendations 

[brob]

Not saying a scratch build is stupid. It's just a lot of work. 

 

With a bare bones server one may still have to acquire and install cpu, memory cooler(s), and storage along with the gpu.

[tomcer2003]

4 hours ago, brob said:

Not saying a scratch build is stupid. It's just a lot of work. 

 

With a bare bones server one may still have to acquire and install cpu, memory cooler(s), and storage along with the gpu.

Good to know. So I should pick server with A100 for school server. And what about cpu. Is it overkill to use epic 7763 and 2 tb of ram? I don't care about storage, we have more some local server with 100TB. Sorry for so many questions, but I never build server and also AI server XD. I still don't understand why they picked me tho. Sever will be also configured for some light video rendering and something with music, idk. But main use will be for AI 

[brob]

I have no idea about a suitable system configuration. You will have to work out the expected workload and resource requirements.

 

I mentioned the A100 in case you were not aware that there is PCIe product designed for ai applications.

[Henry2026]

I don't think building your own would be easy. Servers are very hard and tricky to configure. I think you should get a prebuilt, shove a crap ton of memory in it, and put a few Quadro RTX 8000's in it. I also think an EPYC 7763 is ok. 

[McBUrn]

With such a budget and your need for AI computing I would not consider going the DYI route unless it's your job.

Even though putting the part together remains rather easy (still a CPU in a socket and ram on a slot etc ...), the actual configuration and selection of parts can be a headache.

 

I remember seeing one of ServeTheHome video where the host explained that he was not building the server himself. And you know what, deploying them is he's job!

 

For such workload there is much more than just selecting a CPU, you also need to take into account how you will allocate the PCIE lanes, for which needs, to which components, are you willing to setup tiered storage (Ram -> Optane -> SSD - HDD). What onboard networking will be needed, can this cope with the workload you will put on the server etc etc ... To come back on the CPU, going for the highest core count is not necessarily the best option, certain application prefer higher frequencies, others require you to pay per core/thread used. That's the reason AMD has a large list of Epyc CPU to fit all needs ...

 

I would definitely go to a specialized company for this. If you cannot, pretty sure you can get a rep/technician at dell/HP and others who could assist for such a budget.

Edited April 22, 2021 by McBUrn
Added one more info

[tomcer2003]

12 hours ago, McBUrn said:

With such a budget and your need for AI computing I would not consider going the DYI route unless it's your job.

Even though putting the part together remains rather easy (still a CPU in a socket and ram on a slot etc ...), the actual configuration and selection of parts can be a headache.

 

I remember seeing one of ServeTheHome video where the host explained that he was not building the server himself. And you know what, deploying them is he's job!

 

For such workload there is much more than just selecting a CPU, you also need to take into account how you will allocate the PCIE lanes, for which needs, to which components, are you willing to setup tiered storage (Ram -> Optane -> SSD - HDD). What onboard networking will be needed, can this cope with the workload you will put on the server etc etc ... To come back on the CPU, going for the highest core count is not necessarily the best option, certain application prefer higher frequencies, others require you to pay per core/thread used. That's the reason AMD has a large list of Epyc CPU to fit all needs ...

 

I would definitely go to a specialized company for this. If you cannot, pretty sure you can get a rep/technician at dell/HP and others who could assist for such a budget.

I live in Slovak Republic. So there aren't any good company for this application. So I will go for dell or HP. My classmate have to configure software. I read about AI needs. And. There is a lot of different "AI". But epic 75F3 should be enough (32cores 4GHz). But now I'm lost. Because there is so mutch what my school wants from it. Estimated cost from dell is 35 000€ without A100, that I need to buy (+12 000€)(40GB version) .But I think it should be possible for me to install it and set up PCI lanes for it. So there is 2TB of 3200MHz ram, mentioned epic. 4TB Ssd gen 4 for AI storage and ect. One rtx A6000 for video editing and also AI compilation (cpu will be helping and AI will be running on A100 and cpu) and double PSU. I couldn't opted for one (I wanted only one because of the price, it was +1800€). So if u guys say its good for this application I will send list to my school council. This server will be used for AI learning ( learning for students) . I don't know how in details they will be using it, because I study electrical engineering (semiconductor). They assign me this project only because sponsors wanted only the student to work on this project. If it will go wrong I'm gone from this planet 

[brob]

What you have outlined seems reasonable and should fit budget.