Dual 4090 build for deep learning 2024 April

[nv-ben]

Budget (including currency): around SGD 8000

Country: Singapore

Games, programs or workloads that it will be used for: deep learning, training

Other details (existing parts lists, whether any peripherals are needed, what you're upgrading from, when you're going to buy, what resolution and refresh rate you want to play at, etc): 

 

Hello LTT community, what are your thoughts about this build for deep learning?

• CPU
  • Intel i7 14700K
    • I'm not sure if an intel build is good here, many discussions I saw online tend to use AMD
• Motherboard
  • MSI Z790 Tomahawk Max Wifi
    • I'm using the 4x lanes here as Tim Dettmer's guide says higher generally isn't needed https://timdettmers.com/2023/01/30/which-gpu-for-deep-learning/#Do_I_need_8x16x_PCIe_lanes
• RAM
  • Kingston Fury Beast Black 5600Mhz CL40 64GB Kit
• SSD
  • WD Black SN770 2TB
• GPU
  • 2x MSI RTX 4090 Ventus 3X https://www.msi.com/Graphics-Card/GeForce-RTX-4090-VENTUS-3X-24G-OC/Specification
    • Dimensions on website is equivalent to their slim variant https://www.msi.com/Graphics-Card/GeForce-RTX-4090-GAMING-X-SLIM-24G/Specification
• PSU
  • Silverstone HELA 2050R
    • for the dual 16 pin connector
• CPU cooler
  • Thermalright Peerless Assassin 120 SE
• Case
  • Fractal Design - Meshify 2 XL or Define 7 XL
• Extra Case fans
  • 2x Coolermaster Sickleflow 120mm

 

Pricing wise, I'm mostly depending on a single vendor in Singapore for the entire setup and assembly. https://dynacoretech.com/pricelist

If you have alternative suggestions, do limit it to vendors in Singapore for the build as well.

 

Thanks for helping out!

Ben

 

[Linuswasright]

19 minutes ago, nv-ben said:

Budget (including currency): around SGD 8000

Country: Singapore

Games, programs or workloads that it will be used for: deep learning, training

Other details (existing parts lists, whether any peripherals are needed, what you're upgrading from, when you're going to buy, what resolution and refresh rate you want to play at, etc): 

 

Hello LTT community, what are your thoughts about this build for deep learning?

• CPU
  • Intel i7 14700K
    • I'm not sure if an intel build is good here, many discussions I saw online tend to use AMD
• Motherboard
  • MSI Z790 Tomahawk Max Wifi
    • I'm using the 4x lanes here as Tim Dettmer's guide says higher generally isn't needed https://timdettmers.com/2023/01/30/which-gpu-for-deep-learning/#Do_I_need_8x16x_PCIe_lanes
• RAM
  • Kingston Fury Beast Black 5600Mhz CL40 64GB Kit
• SSD
  • WD Black SN770 2TB
• GPU
  • 2x MSI RTX 4090 Ventus 3X https://www.msi.com/Graphics-Card/GeForce-RTX-4090-VENTUS-3X-24G-OC/Specification
    • Dimensions on website is equivalent to their slim variant https://www.msi.com/Graphics-Card/GeForce-RTX-4090-GAMING-X-SLIM-24G/Specification
• PSU
  • Silverstone HELA 2050R
    • for the dual 16 pin connector
• CPU cooler
  • Thermalright Peerless Assassin 120 SE
• Case
  • Fractal Design - Meshify 2 XL or Define 7 XL
• Extra Case fans
  • 2x Coolermaster Sickleflow 120mm

 

Pricing wise, I'm mostly depending on a single vendor in Singapore for the entire setup and assembly. https://dynacoretech.com/pricelist

If you have alternative suggestions, do limit it to vendors in Singapore for the build as well.

 

Thanks for helping out!

Ben

 

With a quick look over:

Please pelase pelase get an aio for intel. It runs so so hot

 

If you go and (7950 I assume) it’ll be a bit faster, as it has avx512 

[igormp]

19 minutes ago, nv-ben said:

Intel i7 14700K

• I'm not sure if an intel build is good here, many discussions I saw online tend to use AMD

 

I'd say a 7950x or a 7900 is a better pick due to AVX-512.

19 minutes ago, nv-ben said:

MSI Z790 Tomahawk Max Wifi

• I'm using the 4x lanes here as Tim Dettmer's guide says higher generally isn't needed https://timdettmers.com/2023/01/30/which-gpu-for-deep-learning/#Do_I_need_8x16x_PCIe_lanes

 

It really depends on which kind of models you are planning to train on. If you're distributing your model across the GPUs, then the PCIe lanes becomes a massive bottleneck.

20 minutes ago, nv-ben said:

RAM

• Kingston Fury Beast Black 5600Mhz CL40 64GB Kit

 

Only 64gb for 48gb of vram? Seems quite low, but I guess you should know your workloads better.

 

Can you give more details on what you exactly want to train? Architecture, dataset sizes, model size, etc etc

[Linuswasright]

Just now, igormp said:

I'd say a 7950x or a 7900 is a better pick due to AVX-512.

It really depends on which kind of models you are planning to train on. If you're distributing your model across the GPUs, then the PCIe lanes becomes a massive bottleneck.

Only 64gb for 48gb of vram? Seems quite low, but I guess you should know your workloads better.

 

Can you give more details on what you exactly want to train? Architecture, dataset sizes, model size, etc etc

One thing is, he also need way lower cl. 40s kinda wild

I think there are a couple 2x16x mobos out there, and I know for a fact there a couple good 2x8x that are cheap
 

Also he has a weird amount of storage, which seems like not enough for a deep learning algorithm. Maybe a big hdd is in order

[igormp]

Just now, Linuswasright said:

One thing is, he also need way lower cl. 40s kinda wild

No, that's irrelevant.

Just now, Linuswasright said:

I think there are a couple 2x16x mobos out there

Not with any consumer platform (be either Intel or AMD).

1 minute ago, Linuswasright said:

and I know for a fact there a couple good 2x8x that are cheap

Yeah, there are some x8/x8 capable AM5 and LGA1700 motherboards, but they're pretty high-end and not cheap at all. With AM4 it was easier to find such mobos.

1 minute ago, Linuswasright said:

Also he has a weird amount of storage, which seems like not enough for a deep learning algorithm. Maybe a big hdd is in order

Not really, goes on a case by case, 2tb might be more than enough for them.

[RONOTHAN##]

2 minutes ago, Linuswasright said:

I think there are a couple 2x16x mobos out there

There aren't, 2x16x boards require at least 32 lanes from the CPU, which no consumer platform currently offers. 

 

3 minutes ago, Linuswasright said:

I know for a fact there a couple good 2x8x that are cheap

One thing to remember is that OP is from Singapore, so board selection (and part selection in general) is likely going to be weird. 2x8x boards are rather rare nowadays, and while you can get some for cheap in the US, the pricing probably won't be comparable, especially since it's mostly Z690 boards that Newegg is trying to get rid of. 

[Linuswasright]

29 minutes ago, RONOTHAN## said:

There aren't, 2x16x boards require at least 32 lanes from the CPU, which no consumer platform currently offers. 

 

One thing to remember is that OP is from Singapore, so board selection (and part selection in general) is likely going to be weird. 2x8x boards are rather rare nowadays, and while you can get some for cheap in the US, the pricing probably won't be comparable, especially since it's mostly Z690 boards that Newegg is trying to get rid of. 

That’s fair, but there should be a place to get them

 

ignore 2x 16 i meant 16x and 16x acts as 8x or 8x

[RONOTHAN##]

1 minute ago, Linuswasright said:

That’s fair, but there should be a place to get them

It's not guaranteed though, and I'd say it's more likely you can't. Even throughout a decent portion of Europe, getting an x8/x8 board like the Z690 Ace, Carbon, Taichi, or Hero is well over 300 euro, not exactly cheap. If you go AM5, it's a bit more consistent that you can get a B650-Creator ProArt for 250-300 Euro, still not cheap but a bit cheaper than the other options. 

 

You'd also need to worry about slot spacing though, as unless you want to water cool the 4090s, you'd need a board with quad slot spacing in order for the cards to even fit, and that limits your options down to pretty much only the Crosshair Hero, Z690/Z790 Apex, Z690 Aero D, Z690/Z790 Tachyon, Z790 Dark Hero, or Z790 Formula, all of which are stupid expensive. 

[Linuswasright]

43 minutes ago, RONOTHAN## said:

It's not guaranteed though, and I'd say it's more likely you can't. Even throughout a decent portion of Europe, getting an x8/x8 board like the Z690 Ace, Carbon, Taichi, or Hero is well over 300 euro, not exactly cheap. If you go AM5, it's a bit more consistent that you can get a B650-Creator ProArt for 250-300 Euro, still not cheap but a bit cheaper than the other options. 

 

You'd also need to worry about slot spacing though, as unless you want to water cool the 4090s, you'd need a board with quad slot spacing in order for the cards to even fit, and that limits your options down to pretty much only the Crosshair Hero, Z690/Z790 Apex, Z690 Aero D, Z690/Z790 Tachyon, Z790 Dark Hero, or Z790 Formula, all of which are stupid expensive. 

I didn’t think about that that’s fair. You’re right

[nv-ben]

Thanks Linuswasright, igormp, RONOTHAN## for your comments.

 

17 hours ago, igormp said:

I'd say a 7950x or a 7900 is a better pick due to AVX-512.

It really depends on which kind of models you are planning to train on. If you're distributing your model across the GPUs, then the PCIe lanes becomes a massive bottleneck.

Only 64gb for 48gb of vram? Seems quite low, but I guess you should know your workloads better.

 

Can you give more details on what you exactly want to train? Architecture, dataset sizes, model size, etc etc

I've been tasked to get more compute power for a team of researchers to work with. Previously they have been working with a RTX4080, Ryzen 5 7600 and 32GB RAM but it is now taking them up to 2 weeks to get results.

 

Noted that the RAM is limited, and Puget Systems' guide recommends double the total GPU VRAM.

 

From what I heard, their usage of CPU is rather low, so I did not expect that AVX-512 will be a factor.

I will check with them regarding the training details like dataset sizes, model size etc.

17 hours ago, Linuswasright said:

Also he has a weird amount of storage, which seems like not enough for a deep learning algorithm. Maybe a big hdd is in order

A big HDD will be included, currently the team expects to use a 12TB HDD. 

 

16 hours ago, RONOTHAN## said:

It's not guaranteed though, and I'd say it's more likely you can't. Even throughout a decent portion of Europe, getting an x8/x8 board like the Z690 Ace, Carbon, Taichi, or Hero is well over 300 euro, not exactly cheap. If you go AM5, it's a bit more consistent that you can get a B650-Creator ProArt for 250-300 Euro, still not cheap but a bit cheaper than the other options. 

 

You'd also need to worry about slot spacing though, as unless you want to water cool the 4090s, you'd need a board with quad slot spacing in order for the cards to even fit, and that limits your options down to pretty much only the Crosshair Hero, Z690/Z790 Apex, Z690 Aero D, Z690/Z790 Tachyon, Z790 Dark Hero, or Z790 Formula, all of which are stupid expensive. 

The vendor does carry the Asus Z790 Proart, though as you say the slot spacing is an issue. Which is partly the reason why I was thinking of the MSI Tomahawk if the lanes were not the bottleneck.

 

I'll let the team know if money was less restricted and if it makes sense based on the load, we could go straight for the Asus ROG Maximus Z790 Dark Hero or the Apex Encore for the 16 to x8/x8 lanes.

 

Best Regards

[RONOTHAN##]

1 hour ago, nv-ben said:

The vendor does carry the Asus Z790 Proart, though as you say the slot spacing is an issue. Which is partly the reason why I was thinking of the MSI Tomahawk if the lanes were not the bottleneck.

 

I'll let the team know if money was less restricted and if it makes sense based on the load, we could go straight for the Asus ROG Maximus Z790 Dark Hero or the Apex Encore for the 16 to x8/x8 lanes.

This will be dependent on the exact workload if it will be a bottleneck. A lot of them are however, so if you can get x8/x8 support, I would. One option to try and save money would be to go for the ProArt and get the AIO cooled cards like the 4090 Suprim Liquid as those cards are AIO cooled and therefore are only 2 slots thick and not too much more expensive than their air cooled cousins, though with that case compatibility might be a bit of an issue. 

[igormp]

1 hour ago, nv-ben said:

Noted that the RAM is limited, and Puget Systems' guide recommends double the total GPU VRAM.

 

From what I heard, their usage of CPU is rather low, so I did not expect that AVX-512 will be a factor.

I will check with them regarding the training details like dataset sizes, model size etc.

Both the ram and cpu usage really depend on what kind of preprocessing and training pipeline they're doing. If you could get more info on that, it'd be really useful. 

 

From my personal experience with 2x3090s, even 128gb is sometimes lacking, specially when using pandas and trying to avoid data starvation on the training pipeline.

 

Also, if those systems are already in place, you could just upgrade them with more ram and a mobo that supports 2 GPUs (assuming that the CPU is more than enough), this would save some bucks in the project.