Help with PC for LLMs

[Jazmento]

Budget (including currency): £1200

Country: UK

Games, programs or workloads that it will be used for: Compiling Large C++ projects, running LLMs locally, Minecraft

Other details (existing parts lists, whether any peripherals are needed, what you're upgrading from, when you're going to buy, what resolution and refresh rate you want to play at, etc): 

Don't have any parts yet, hoping to buy as soon as possible. Also planning to upgrade to two 3060s (will just get one for now) for LLMs, will that work with my current setup?

 

MB: MSI B760M Gaming plus wifi

CPU: Core i7 12700kf

Cooler: Thermalright Peerless Assassin 120 SE

RAM: Crucial Pro DDR5 2x16GB 6400

GPU: RTX 3060

PSU: Corsair RM850x

Case: Fractal Design Pop Mini Silent

 

[Agall]

5 minutes ago, Jazmento said:

Budget (including currency): £1200

Country: UK

Games, programs or workloads that it will be used for: Compiling Large C++ projects, running LLMs locally, Minecraft

Other details (existing parts lists, whether any peripherals are needed, what you're upgrading from, when you're going to buy, what resolution and refresh rate you want to play at, etc): 

Don't have any parts yet, hoping to buy as soon as possible. Also planning to upgrade to two 3060s (will just get one for now) for LLMs, will that work with my current setup?

 

MB: MSI B760M Gaming plus wifi

CPU: Core i7 12700kf

Cooler: Thermalright Peerless Assassin 120 SE

RAM: Crucial Pro DDR5 2x16GB 6400

GPU: RTX 3060

PSU: Corsair RM850x

Case: Fractal Design Pop Mini Silent

 

If large VRAM is best, you could look at Ryzen AI Max+ 395+^2+ mini PCs that can go up to 96GB of VRAM with shared 128GB of system RAM. There's also discrete motherboards from Framework, but that's not available till Q3/Q4.

[igormp]

I'm a bit confused, you say that:

2 hours ago, Jazmento said:

Don't have any parts yet

But then you mention:

2 hours ago, Jazmento said:

will that work with my current setup?

Is that a setup that you already have, or the one you were planning on building? I'll assume it's the latter.

 

2 hours ago, Jazmento said:

CPU: Core i7 12700kf

I'd say that going with AMD for AVX-512 is a better idea, specially if you can afford to go with the 9000 series.

2 hours ago, Jazmento said:

RAM: Crucial Pro DDR5 2x16GB 6400

Would 32GB really be enough for all your needs? I'd recommend going with at least 2x32GB or even 2x48GB.

2 hours ago, Jazmento said:

GPU: RTX 3060

Why not go for an used 3090?

2 hours ago, Jazmento said:

Also planning to upgrade to two 3060s (will just get one for now) for LLMs, will that work with my current setup?

You'd be better off with a motherboard that has support for x8/x8 GPUs if you plan to go dual GPU.

 

2 hours ago, Agall said:

If large VRAM is best, you could look at Ryzen AI Max+ 395+^2+ mini PCs that can go up to 96GB of VRAM with shared 128GB of system RAM. There's also discrete motherboards from Framework, but that's not available till Q3/Q4.

I don't think there's any 64/128GB option that fits in their budget. The 385 32GB model from framework starts at $1.1k