Python Programming / Machine Learning Upgrade

[Koha101]

Budget (including currency): $1500-$2500

Country: USA

Games, programs or workloads that it will be used for: Python programming, primarily using Numpy and some Pandas, with Pytorch for GPU ML

Other details: I am looking to upgrade my gaming build that has a RTX 3080 with a 5950x to either a 7950x or a 14900k. In the future, I would like to upgrade to a 2nd 3080 or 2x 4090 if the 3080 will not be enough. I need at least 64 gb of ram, but would prefer 96-128gb. I think memory speed is important for my use case? If so, 6000 MT/s CL30 would be ideal, from what I understand.

 

The main debate I am currently having is whether to go with a 7950x or a 14900k. The 14900k no longer has AVX-512 support, but this shows the 13900k (proxy for 14900k in my case) has a clear edge over the 7950x in numpy operations. There are a lot of array manipulations that occur to prep the data before it gets sent to the gpu for processing, so this would be/is currently a bottleneck for me.

 

Once I make a decision on a cpu, I'd then like to have a motherboard that has enough room for 2x air-cooled GPUs with some space between them.

 

Also, if I go with a 7950x, do I need to water cool, or will air cooling be sufficient?

 

[filpo]

4 hours ago, Koha101 said:

If so, 6000 MT/s CL30 would be ideal

for am5 that's true, for intel it's more 'try and get a faster speed with cl latency under cl34'

 

4 hours ago, Koha101 said:

Also, if I go with a 7950x, do I need to water cool, or will air cooling be sufficient?

you CAN air cool it and if it's not running at 100% the whole time it should be fine, but I'd rather use an AIO to be safe, such as a 360mm arctic rad

 

4 hours ago, Koha101 said:

Budget (including currency): $1500-$2500

Country: USA

Games, programs or workloads that it will be used for: Python programming, primarily using Numpy and some Pandas, with Pytorch for GPU ML

Other details: I am looking to upgrade my gaming build that has a RTX 3080 with a 5950x to either a 7950x or a 14900k. In the future, I would like to upgrade to a 2nd 3080 or 2x 4090 if the 3080 will not be enough. I need at least 64 gb of ram, but would prefer 96-128gb. I think memory speed is important for my use case? If so, 6000 MT/s CL30 would be ideal, from what I understand.

 

The main debate I am currently having is whether to go with a 7950x or a 14900k. The 14900k no longer has AVX-512 support, but this shows the 13900k (proxy for 14900k in my case) has a clear edge over the 7950x in numpy operations. There are a lot of array manipulations that occur to prep the data before it gets sent to the gpu for processing, so this would be/is currently a bottleneck for me.

 

Once I make a decision on a cpu, I'd then like to have a motherboard that has enough room for 2x air-cooled GPUs with some space between them.

 

Also, if I go with a 7950x, do I need to water cool, or will air cooling be sufficient?

 

example build:

PCPartPicker Part List

CPU: AMD Ryzen 9 7950X 4.5 GHz 16-Core Processor  ($598.99 @ Newegg) 
CPU Cooler: ARCTIC Liquid Freezer II 360 Rev. 2 56.33 CFM Liquid CPU Cooler  ($143.05 @ Amazon) 
Motherboard: Asus ROG STRIX X670E-E GAMING WIFI ATX AM5 Motherboard  ($459.99 @ Amazon) 
Memory: G.Skill Ripjaws S5 96 GB (2 x 48 GB) DDR5-6400 CL32 Memory  ($324.99 @ Amazon) 
Storage: Samsung 980 Pro 2 TB M.2-2280 PCIe 4.0 X4 NVME Solid State Drive  ($119.99 @ Dell Technologies) 
Case: Lian Li LANCOOL III RGB ATX Mid Tower Case  ($139.00 @ Newegg Sellers) 
Power Supply: Thermaltake Toughpower GF3 TT Premium 1650 W 80+ Gold Certified Fully Modular ATX Power Supply  ($263.64 @ Amazon) 
Total: $2049.65
Prices include shipping, taxes, and discounts when available
Generated by PCPartPicker 2024-01-03 05:08 EST-0500

[QuantumSingularity]

+1 for the system from @filpo. The other plus of going AM5 is that you have an open upgradability path and once the new 3nm Ryzen CPUs come out, upgrading to one of them is just a simple BIOS flash away. With Intel this is what you will have on that platform. After that is a new build again, with new mobo and new chipset and probably new, higher price tag, because well... it's intel after all.

[brob]

Intel is of course a viable option. Of interest may be the expected release of the i9-14900KS. Rumour is that the CPU will do 6.2GHz out of the box. Naturally it runs hotter than AMD. But something like the 7950X at stock is only about 50W cooler than the i9-14900K. Significant yes, but not outrageously so.

 

PCPartPicker Part List

CPU: Intel Core i9-14900KF 3.2 GHz 24-Core Processor  ($544.02 @ B&H) 
CPU Cooler: ARCTIC Liquid Freezer II 420 72.8 CFM Liquid CPU Cooler  ($120.50 @ Amazon) 
Motherboard: Asus ProArt Z790-CREATOR WIFI ATX LGA1700 Motherboard  ($439.99 @ Amazon) 
Memory: Corsair Vengeance 96 GB (2 x 48 GB) DDR5-6000 CL30 Memory  ($354.99 @ Amazon) 
Storage: Samsung 980 Pro 2 TB M.2-2280 PCIe 4.0 X4 NVME Solid State Drive  ($119.99 @ Dell Technologies) 
Case: be quiet! Dark Base Pro 901 ATX Full Tower Case  ($288.99 @ Newegg) 
Power Supply: be quiet! Straight Power 12 1500 W 80+ Platinum Certified Fully Modular ATX Power Supply  ($279.90 @ Newegg Sellers) 
Total: $2148.38
Prices include shipping, taxes, and discounts when available
Generated by PCPartPicker 2024-01-03 09:48 EST-0500

[igormp]

11 hours ago, Koha101 said:

I think memory speed is important for my use case? If so, 6000 MT/s CL30 would be ideal, from what I understand.

Not really, you're mostly compute bound. So if you ever to upgrade to 4 sticks of ram, I'd recommend you to go with 5200~5600MHz sticks for  better compatibility instead.

11 hours ago, Koha101 said:

The 14900k no longer has AVX-512 support, but this shows the 13900k (proxy for 14900k in my case) has a clear edge over the 7950x in numpy operations.

Well...

https://openbenchmarking.org/test/pts/numpy&eval=12916313a24be49739ae0974e3763b30474b135e

Make sure to use a proper blas implementation with avx-512 support, then the AM5 options should edge Intel.

11 hours ago, Koha101 said:

I'd then like to have a motherboard that has enough room for 2x air-cooled GPUs with some space between them.

I'd just give up on space. As for the mobo options, have a look at this:

On 12/9/2023 at 12:41 PM, igormp said:

There was a discussion in another thread about this with some options:

 

None of them are really cheap, guess I'll keep going happily with my AM4 setup lol

If you power limit the GPUs there isn't much issue having those close together, I have my 2x3090s running like so without issues so far.

[Koha101]

7 hours ago, igormp said:

Not really, you're mostly compute bound. So if you ever to upgrade to 4 sticks of ram, I'd recommend you to go with 5200~5600MHz sticks for  better compatibility instead.

Well...

https://openbenchmarking.org/test/pts/numpy&eval=12916313a24be49739ae0974e3763b30474b135e

Make sure to use a proper blas implementation with avx-512 support, then the AM5 options should edge Intel.

I'd just give up on space. As for the mobo options, have a look at this:

If you power limit the GPUs there isn't much issue having those close together, I have my 2x3090s running like so without issues so far.

Wow, thanks for the link. I guess that makes the decision for me - I'll go with a 7950x! And yeah, power limiting the GPUs sounds like the right move from other research I've done.

Thanks for the link for the different boards. I looked at it briefly, and it looks very promising. I'll deep dive tonight and order all the stuff I need. Appreciate the help!

[Koha101]

14 hours ago, filpo said:

for am5 that's true, for intel it's more 'try and get a faster speed with cl latency under cl34'

 

you CAN air cool it and if it's not running at 100% the whole time it should be fine, but I'd rather use an AIO to be safe, such as a 360mm arctic rad

 

example build:

PCPartPicker Part List

CPU: AMD Ryzen 9 7950X 4.5 GHz 16-Core Processor  ($598.99 @ Newegg) 
CPU Cooler: ARCTIC Liquid Freezer II 360 Rev. 2 56.33 CFM Liquid CPU Cooler  ($143.05 @ Amazon) 
Motherboard: Asus ROG STRIX X670E-E GAMING WIFI ATX AM5 Motherboard  ($459.99 @ Amazon) 
Memory: G.Skill Ripjaws S5 96 GB (2 x 48 GB) DDR5-6400 CL32 Memory  ($324.99 @ Amazon) 
Storage: Samsung 980 Pro 2 TB M.2-2280 PCIe 4.0 X4 NVME Solid State Drive  ($119.99 @ Dell Technologies) 
Case: Lian Li LANCOOL III RGB ATX Mid Tower Case  ($139.00 @ Newegg Sellers) 
Power Supply: Thermaltake Toughpower GF3 TT Premium 1650 W 80+ Gold Certified Fully Modular ATX Power Supply  ($263.64 @ Amazon) 
Total: $2049.65
Prices include shipping, taxes, and discounts when available
Generated by PCPartPicker 2024-01-03 05:08 EST-0500

Thanks for this. Any reason you chose that memory over a 6000 CL30 mem?

[Koha101]

20 hours ago, filpo said:

for am5 that's true, for intel it's more 'try and get a faster speed with cl latency under cl34'

 

you CAN air cool it and if it's not running at 100% the whole time it should be fine, but I'd rather use an AIO to be safe, such as a 360mm arctic rad

 

example build:

PCPartPicker Part List

CPU: AMD Ryzen 9 7950X 4.5 GHz 16-Core Processor  ($598.99 @ Newegg) 
CPU Cooler: ARCTIC Liquid Freezer II 360 Rev. 2 56.33 CFM Liquid CPU Cooler  ($143.05 @ Amazon) 
Motherboard: Asus ROG STRIX X670E-E GAMING WIFI ATX AM5 Motherboard  ($459.99 @ Amazon) 
Memory: G.Skill Ripjaws S5 96 GB (2 x 48 GB) DDR5-6400 CL32 Memory  ($324.99 @ Amazon) 
Storage: Samsung 980 Pro 2 TB M.2-2280 PCIe 4.0 X4 NVME Solid State Drive  ($119.99 @ Dell Technologies) 
Case: Lian Li LANCOOL III RGB ATX Mid Tower Case  ($139.00 @ Newegg Sellers) 
Power Supply: Thermaltake Toughpower GF3 TT Premium 1650 W 80+ Gold Certified Fully Modular ATX Power Supply  ($263.64 @ Amazon) 
Total: $2049.65
Prices include shipping, taxes, and discounts when available
Generated by PCPartPicker 2024-01-03 05:08 EST-0500

It looks like the motherboard you suggested bifurcates to x8 / x4 if paired with 2 GPUs. Was there a reason you chose this motherboard?

[brob]

4 minutes ago, Koha101 said:

It looks like the motherboard you suggested bifurcates to x8 / x4 if paired with 2 GPUs. 

 

Use the https://pcpartpicker.com/product/mR2WGX/asus-proart-x670e-creator-wifi-atx-am5-motherboard-proart-x670e-creator-wifi., around the same price. It bifurcates x8x8. You will want to verify memory qualification.

 

[filpo]

12 hours ago, Koha101 said:

Thanks for this. Any reason you chose that memory over a 6000 CL30 mem?

Gets lower cl latency is better for am5 so slightly slower speed is better if you get lower cl latency 

 

7 hours ago, Koha101 said:

It looks like the motherboard you suggested bifurcates to x8 / x4 if paired with 2 GPUs. Was there a reason you chose this motherboard?

strange, didn’t see that. Look for one which doesn’t do that then 

[igormp]

16 minutes ago, filpo said:

Gets lower cl latency is better for am5 so slightly slower speed is better if you get lower cl latency 

 

Just wanted to repeat what I said before: for OP's case memory speed/latency isn't that relevant. Going for compatibility and stability is way more important. 5200~5600mhz should do just fine. For some benchmarks, see:

https://www.phoronix.com/review/crucial-ddr5-5200-5600

https://openbenchmarking.org/result/2302279-NE-RYZENDDR533&ppd_RERSNS00ODAwIENydWNpYWw=104&ppd_RERSNS01MjAwIENydWNpYWw=148&ppd_RERSNS01NjAwIENydWNpYWw=148&ppd_RERSNS02MDAwIEdTS0lMTCBGNS02MDAwSjMwMzhGMTZH=169&ppt=D&sgm=1&ppd_RERSNS00ODAwIENydWNpYWw=104&ppd_RERSNS01MjAwIENydWNpYWw=148&ppd_RERSNS01NjAwIENydWNpYWw=148&ppd_RERSNS02MDAwIEdTS0lMTCBGNS02MDAwSjMwMzhGMTZH=169&ppt=D

 

The python/cnn/ml related ones barely show an improvement going from 4800 to 6000mhz.

[brob]

30 minutes ago, filpo said:

Gets lower cl latency is better for am5 so slightly slower speed is better if you get lower cl latency 

 

For an apples to apples comparison of CL latency one must factor in speed. In the case of DDR5 it appears other timings may also need consideration. Simply selecting a kit based on a lower CL number may well lead to selecting a sub-optimal part.

 

There are several articles that explore DDR5 scaling. While there are differences in particular applications, in the general case I don't believe the performance differences are significant.

 

Given the relative immaturity of DDR5 I think choosing known stability far more important than bleeding edge speed.

[Koha101]

Thank you all for your input. I decided to build something that supports only 1 GPU for now, so will only be upgrading the CPU/Mobo/Mem if I can get away with it. It seems like the price skyrockets when thinking about 2 GPUs.

When I get more comfortable using some form of ML, and need the additional HP, Ryzen will hopefully have next gen out and I'll upgrade mobo again (with hopefully a new CPU) to support multiple GPUs.

I put together a build that contains the parts I already have, and the cpu/mem/mobo I'm planning on buying. Let me know what you think: https://pcpartpicker.com/list/3Y4WKX.

 

I checked the memory, and this one seems compatible with the motherboard based on ASRock's Mem QVL. Is there a concern with the pick?

[brob]

19 minutes ago, Koha101 said:

Thank you all for your input. I decided to build something that supports only 1 GPU for now, so will only be upgrading the CPU/Mobo/Mem if I can get away with it. It seems like the price skyrockets when thinking about 2 GPUs.

When I get more comfortable using some form of ML, and need the additional HP, Ryzen will hopefully have next gen out and I'll upgrade mobo again (with hopefully a new CPU) to support multiple GPUs.

I put together a build that contains the parts I already have, and the cpu/mem/mobo I'm planning on buying. Let me know what you think: https://pcpartpicker.com/list/3Y4WKX.

 

I checked the memory, and this one seems compatible with the motherboard based on ASRock's Mem QVL. Is there a concern with the pick?

 

Despite all the AMD cheerleading, the 7950X is not an ice cube. It is rated at 170W TDP. The Hyper 212 Black is simply not adequate. Consider instead an NH-D15 or AK620.

 

Consider replacing the PSU with a Tier A or B model. Ideally a model that is ATX 3.0 conformant.

 

[Koha101]

2 hours ago, brob said:

 

Despite all the AMD cheerleading, the 7950X is not an ice cube. It is rated at 170W TDP. The Hyper 212 Black is simply not adequate. Consider instead an NH-D15 or AK620.

 

Consider replacing the PSU with a Tier A or B model. Ideally a model that is ATX 3.0 conformant.

 

Ah, the Hyper 212 was a placeholder until I figured out what cooler I was using for the 5950x I currently have. I actually have the NH-D15, and thought I updated it in pcpartpicker.

 

wtf, the PSU I have is a Tier F here: https://cultists.network/140/psu-tier-list/. I had no idea there were PSU tiers!

[Koha101]

Since I'm upgrading my PSU, I'd like to find one that I can keep if I do upgrade to 2x GPUs, but can't seem to find a an A-Tier ATX-3 compatible one that's >1500w. This is the setup I would invest in if I was comfortable with using the GPU and better at the ML stuff I'm doing (I'm not there yet, so future build with hopefully 2x 5900 GPUs and 8950x/9950x CPU): https://pcpartpicker.com/list/gnZTjH. Let me know if you recommend a better PSU that would work for this setup.

[brob]

13 minutes ago, Koha101 said:

Since I'm upgrading my PSU, I'd like to find one that I can keep if I do upgrade to 2x GPUs, but can't seem to find a an A-Tier ATX-3 compatible one that's >1500w. This is the setup I would invest in if I was comfortable with using the GPU and better at the ML stuff I'm doing (I'm not there yet, so future build with hopefully 2x 5900 GPUs and 8950x/9950x CPU): https://pcpartpicker.com/list/gnZTjH. Let me know if you recommend a better PSU that would work for this setup.

 

Be Quiet Straight Power 12 1500

Corsair HX1500i (2023)

Be Quiet Dark Power Pro 13 1600

Seasonic Prime PX-1600 ATX 3.0

[igormp]

7 hours ago, Koha101 said:

Since I'm upgrading my PSU, I'd like to find one that I can keep if I do upgrade to 2x GPUs, but can't seem to find a an A-Tier ATX-3 compatible one that's >1500w. This is the setup I would invest in if I was comfortable with using the GPU and better at the ML stuff I'm doing (I'm not there yet, so future build with hopefully 2x 5900 GPUs and 8950x/9950x CPU): https://pcpartpicker.com/list/gnZTjH. Let me know if you recommend a better PSU that would work for this setup.

You can go for a lower wattage if you power limit those GPUs. I'd keep a 4090 at 300~350W tops, so that'd be 700W for the GPUs, give another 200W for the CPU and 100W for other stuff, you end up with 1000W

[Koha101]

Update - I ended up with this setup: https://pcpartpicker.com/list/YMZvN6. CPU/Mobo/Mem/PSU are new. Other stuff are kept from previous build. Hopefully in the future I can squeeze in 2x 3090s in this motherboard. I think 5x4 (similar to 4x8 or 3x16) is still pretty fast for the 2nd PCIe slot.

[brob]

19 minutes ago, Koha101 said:

I think 5x4 (similar to 4x8 or 3x16) is still pretty fast for the 2nd PCIe slot.

 

I suspect you will be fine. But to be clear x4 PCIe 5.0 is only possible if the add-on card supports PCIe 5.0. Otherwise the second slot will run x4 PCIe 4.0 in the case of an RTX 3090.