Old socket vs new Socket & Tesla vs RTX

[Adkorpek]

Budget (including currency): 800 (~$900)

Country: Switzerland - Germany

Games, programs or workloads that it will be used for: 1. AI Training and ML (50%) 2. Rendering (45%) 3. Gaming (5 %)

 

I want to build a PC primarily for data science and AI and only a little bit of gaming. I'm on a pretty tight budget.

CPU: I have two options. The first I go with something modern with AM5 or LGA 1700. Or AM4 or LGA 1200.

So when I go with the modern approach I was thinking of the: Intel Core I5-12600FK which is a great value for money 170 CHF (~$185)

Or I save some bucks on the CPU and go with like AM4 and use an old CPU like the AMD Ryzen 7 3700X, which I can get used for 50 CHF (~$55).

 

GPU: Here I also have two options, RTX or Tesla. There, a lot of very good used RTX cards with more than 5000 Cores (30-Series). In comparison, the Tesla P40 only has 3840 cores but 24 GB DDR5 VRAM that good be good for AI training. The main point of the Tesla is that it is now very cheap, for around 100 CHF (~$110).

 

Some advice would be nice!

 

[Needfuldoer]

An older workstation platform like X99 or X299 is also a good option for a ML machine, especially if you want to run a second GPU in the future. They'll take cheap registered ECC server RAM, and they have more available PCIe lanes than consumer platforms.

  

11 minutes ago, Adkorpek said:

GPU: Here I also have two options, RTX or Tesla. There, a lot of very good used RTX cards with more than 5000 Cores (30-Series). In comparison, the Tesla P40 only has 3840 cores but 24 GB DDR5 VRAM that good be good for AI training. The main point of the Tesla is that it is now very cheap, for around 100 CHF (~$110).

Tesla P40 is basically a GTX 1080 ti with 24 GB of VRAM. It can fit larger models than a 12 gig card, but its processing will be much slower. (It has fewer CUDA cores of a much older generation, and it completely lacks Tensor cores.)

[Adkorpek]

So I have already considered a system with even 2 Xeons, but the single core performance was a little bit too low for me because I don't only do AI also gnarl purpose programming. So I'm open to something different, just that it also has ok single core performance. The benefit of Xeon is the four channel memory controller and like you said the cheap memory jut will it be sufficient, I have never used it before. Like, the only thing I can compare it to is my laptop with a Intel Core i5-1135G7 (4 Cores @ boost 4.2 GHz).

1 minute ago, Needfuldoer said:

An older workstation platform like X99 or X299 is also a good option for a ML machine, especially if you want to run a second GPU in the future. They'll take cheap registered ECC server RAM, and they have more available PCIe lanes than consumer platforms.

  

Tesla P40 is basically a GTX 1080 ti with 24 GB of VRAM. It can fit larger models than a 12 gig card, but its processing will be much slower. (It has fewer CUDA cores of a much older generation, and it completely lacks Tensor cores.)

So about the GPU's I'll go with something different, but the main question is AMD or NVIDIA. Basically, what is cheaper and is better with Linux. So like max $400 not more.

[jaslion]

7 minutes ago, Needfuldoer said:

It has fewer CUDA cores of a much older generation, and it completely lacks Tensor cores.)

And like tensor cores arent just a nice to have either. They are a BIIIIIIIGGGGGGGGG deal for a lot of models. Its why a 3060 12gb and such is popular and when the tensor cores are going yeah you'd need multiple p40's to keep up.

 

As for pcie lanes basically a x4 pcir 3.0 link is fine in most cases as the gpu's of this caliber are EASILY overwhelmed with the amount of data that can feed

[Needfuldoer]

5 minutes ago, Adkorpek said:

So about the GPU's I'll go with something different, but the main question is AMD or NVIDIA. Basically, what is cheaper and is better with Linux. So like max $400 not more.

You'll severely limit your options if you go with anything but Nvidia. The big "AI" projects are written for CUDA.

 

5 minutes ago, jaslion said:

And like tensor cores arent just a nice to have either. They are a BIIIIIIIGGGGGGGGG deal for a lot of models. Its why a 3060 12gb and such is popular and when the tensor cores are going yeah you'd need multiple p40's to keep up.

That's exactly why I shoehorned 12 GB 3060s into a couple Dell R730s, instead of buying 24 GB P40s that are designed for pass-through cooling in a rack server. The airflow situation is a little janky, but they process a ton faster. The only limitation is that they "only" have 12 gigs of VRAM per card.

 

Now I'm cramming a 24 GB 3090 into a Precision 5820. It works, but I need to add a PCIe slot exhaust fan and some 8-pin 180 degree adapters if I want to put the side panel back on.

[Adkorpek]

7 minutes ago, jaslion said:

And like tensor cores arent just a nice to have either. They are a BIIIIIIIGGGGGGGGG deal for a lot of models. Its why a 3060 12gb and such is popular and when the tensor cores are going yeah you'd need multiple p40's to keep up.

 

As for pcie lanes basically a x4 pcir 3.0 link is fine in most cases as the gpu's of this caliber are EASILY overwhelmed with the amount of data that can feed

So you're recommending something around a 3060 and I think 12gb will be sufficient. So are some Nvidia cards (in my price range) specifically designed for AI training?

5 minutes ago, Needfuldoer said:

You'll severely limit your options if you go with anything but Nvidia. The big "AI" projects are written for CUDA.

So I personally prefer Nvidia anyway, so ok. But doesn't PyTorch also support AMD??

[jaslion]

8 minutes ago, Needfuldoer said:

Now I'm cramming a 24 GB 3090 into a Precision 5820. It works, but I need to add a PCIe slot exhaust fan and some 8-pin 180 degree adapters if I want to put the side panel back on.

Keep it off the added open air is nice for the cards that and there is NO GOOD 180 adapter that doesnt cause severe issues. Which sucks massively

 

1 minute ago, Adkorpek said:

so ok. But doesn't PyTorch also support AMD??

Theres like 3 models that work on amd natively and they are WAY slower than on cuda also amd has cuda "support" in linux but its still far behind.

 

Basically you are wasting time and moeny for a worse end result going amd

[Adkorpek]

Ok thanks so the GPU's are clare now

[jaslion]

26 minutes ago, Adkorpek said:

Ok thanks so the GPU's are clare now

Yeah you also dont need AI focused cards as all that means is they are focused with their resources on doing the computing tasks that most ai models use but those are also applicable for other things. The rtx cards are gaming focused but they are also good at doing a lot of other things like easily beating a p40

[Adkorpek]

Thank you've been a big help! But do you have some tips on the CPU? Is future proving the build with the AM5 or LGA 1700 socket a good Idea, or should I go with AM4 or LGA 1200 and respectively < 5000 Series or < 11th Gen? Because then I can buy a used CPU, because where I live there aren't much used 7000 Series or 12th or 13th Gen....