2990wx ram help

[PepsiBandit]

I am trying to decide on parts for a new workstation to be built in a rack enclosure at my desk. 

It will be running a very specific workload (similar to using a lot of vmware instances, but not actually that). I am running multiple instances of a program that takes 2 - 2.5gb of ram and on a Ryzen 7 1700x, and takes about 2-4% cpu usage. My goal is to scale this up on a system using a 1950x, 2970wx or 2990wx. So far I have just been estimating a similar per core performance to the 1700x. With a 2990wx it should therefore be able to run about 80-90 instances, which would require 200gb ram.

The end goal is to run as many as possible for the cheapest price. I am expecting to use about 3500 - 6500€ on this build.

I have not heard of anyone else using 256gb ram on a threadripper and would like to hear other peoples opinions on it. I think it depends quite a bit on the motherboard and ram that I choose. 

CPU: 1950x - 2990wx
Ram: 128 - 256GB
PSU: RM750x
GPU: 1080 strix 
SSD: 970 Evo m.2 (500GB)
Heatsink: Enermax Liqtech 240mm or Noctua NH-U14
Mobo: ASRock Taichi 

I expect relatively low GPU usage most of the time and may just use a spare 970 to start with. Any other motherboard recommendations would be welcomed. 

[Jurrunio]

Threadripper supports up to 128GB RAM because it doesnt support RDIMMs, i.e. registered/buffered memory found in servers. 16GB is the largest UDIMM (unregistered) memory you can find that's DDR4.

[Phentos]

40 minutes ago, PepsiBandit said:

I am trying to decide on parts for a new workstation to be built in a rack enclosure at my desk. 

It will be running a very specific workload (similar to using a lot of vmware instances, but not actually that). I am running multiple instances of a program that takes 2 - 2.5gb of ram and on a Ryzen 7 1700x, and takes about 2-4% cpu usage. My goal is to scale this up on a system using a 1950x, 2970wx or 2990wx. So far I have just been estimating a similar per core performance to the 1700x. With a 2990wx it should therefore be able to run about 80-90 instances, which would require 200gb ram.

The end goal is to run as many as possible for the cheapest price. I am expecting to use about 3500 - 6500€ on this build.

I have not heard of anyone else using 256gb ram on a threadripper and would like to hear other peoples opinions on it. I think it depends quite a bit on the motherboard and ram that I choose. 

CPU: 1950x - 2990wx
Ram: 128 - 256GB
PSU: RM750x
GPU: 1080 strix 
SSD: 970 Evo m.2 (500GB)
Heatsink: Enermax Liqtech 240mm or Noctua NH-U14
Mobo: ASRock Taichi 

I expect relatively low GPU usage most of the time and may just use a spare 970 to start with. Any other motherboard recommendations would be welcomed. 

As Jurrunio said above, X399 only support up to 128GB of RAM. If you want that much RAM for your workload, I would look into investing in Epyc. This might exceed your budget though.'

[Taf the Ghost]

30 minutes ago, PepsiBandit said:

I am trying to decide on parts for a new workstation to be built in a rack enclosure at my desk. 

I expect relatively low GPU usage most of the time and may just use a spare 970 to start with. Any other motherboard recommendations would be welcomed. 

2970WX won't be out until October, just as a note.

 

It looks like 128 Gb is the max any board can take for X399. You probably want to look into an Epyc-based system.

 

Since speed is far less of an issue than addressable cores, you'd want to be looking into a system around: https://www.newegg.com/Product/Product.aspx?Item=N82E16819113470&cm_re=epyc-_-19-113-470-_-Product   Epyc 7551P is 32c with the full 8 channels. 

https://www.newegg.com/Product/Product.aspx?Item=N82E16813145034&cm_re=socket_sp3-_-13-145-034-_-Product  Seems ~600USD for a motherboard. 

 

Memory is going to be the huge cost, though.

[Taf the Ghost]

4 minutes ago, Phentos said:

As Jurrunio said above, Threadripper CPUs only support up to 128GB of RAM. If you want that much RAM for your workload, I would look into investing in Epyc. This might exceed your budget though.'

 

Edit: Nvm I'm stupid. TR supports up to 2TB theoretically.

The CPU can, but it looks like the boards are the limit for TR. Even Newegg's search function lists 128 Gb as the max memory to filter by.

[Phentos]

Just now, Taf the Ghost said:

The CPU can, but it looks like the boards are the limit for TR. Even Newegg's search function lists 128 Gb as the max memory to filter by.

Yea I got mixed up. Re-edited my post to fix it.

[Streetguru]

52 minutes ago, PepsiBandit said:

 

You most likely have to buy EPYC for over 128GBs of RAM, it has 8-channel memory anyways vs X399's 4 channel.

[Taf the Ghost]

11 minutes ago, Phentos said:

Yea I got mixed up. Re-edited my post to fix it.

All good.

 

@PepsiBandit 

 

With a TR build, you wouldn't be able to run 80 instances. Probably around 50-55 instances would max things out. However, it might be more cost effective to end up running two systems rather than going all the way to Server-class setups. Especially as the larger DIMMs are generally more expensive per Gb than 16 Gb ones.

[PepsiBandit]

6 hours ago, Jurrunio said:

Threadripper supports up to 128GB RAM because it doesnt support RDIMMs, i.e. registered/buffered memory found in servers. 16GB is the largest UDIMM (unregistered) memory you can find that's DDR4.

Thanks, I was hoping there was some (even if it's slightly more expensive) way to get 32gb ram modules for it. 

 

With the other suggestions about using EYPC, I think it would be better to possibly make 2 systems instead then. The program I am using does get affected by core speeds so it would run better with a threadripper I think. I will check though.

[Taf the Ghost]

1 hour ago, PepsiBandit said:

Thanks, I was hoping there was some (even if it's slightly more expensive) way to get 32gb ram modules for it. 

 

With the other suggestions about using EYPC, I think it would be better to possibly make 2 systems instead then. The program I am using does get affected by core speeds so it would run better with a threadripper I think. I will check though.

It's probably 2x 2950X systems you'd want to build. I quick look around suggests you'd save 25-35% on the memory going with 16 Gb sticks rather than 32 Gb sticks. There'd be some redundant costs, but the memory capacity is the limiting issue.

[PepsiBandit]

6 hours ago, Taf the Ghost said:

It's probably 2x 2950X systems you'd want to build. I quick look around suggests you'd save 25-35% on the memory going with 16 Gb sticks rather than 32 Gb sticks. There'd be some redundant costs, but the memory capacity is the limiting issue.

I have decided that this would be the best way to get it done for me. I was hoping to order all the parts today or tomorrow, so I don't know if I will wait for the 2950x.

 

Do you think there is any major advantage with waiting for the 2950x over a 1950x? I assume they will be relatively close to each other (there was as approximate 8% difference between the 1600 and 2600). The cpu - ram balance should already be quite close with this build anyway. 

Do you have any recommendations for reducing the redundant costs? I am mainly thinking in terms of a cheaper motherboard that would still allow a basic stable overclock. I don't know that many different threadripper motherboards. It's going to be connected with ethernet and not doing much file sharing on my local network. 

[brob]

Is each instance running in its own VM?

[Taf the Ghost]

6 hours ago, PepsiBandit said:

I have decided that this would be the best way to get it done for me. I was hoping to order all the parts today or tomorrow, so I don't know if I will wait for the 2950x.

 

Do you think there is any major advantage with waiting for the 2950x over a 1950x? I assume they will be relatively close to each other (there was as approximate 8% difference between the 1600 and 2600). The cpu - ram balance should already be quite close with this build anyway. 

Do you have any recommendations for reducing the redundant costs? I am mainly thinking in terms of a cheaper motherboard that would still allow a basic stable overclock. I don't know that many different threadripper motherboards. It's going to be connected with ethernet and not doing much file sharing on my local network. 

Well, the 2990WX launches Monday, along with the benchmarks for that and the 2950X. At minimum, I'd wait to order until after the news cycle on Monday, just so you know if there is any reason to wait for the 2950X. That part launches August 31st, I believe. I honestly don't think, with your use case, that it'll effect things, but it's the Memory Subsystems that got the biggest boost. Depending on how your application works, The cost difference might be worth it.

 

The other thing is that we're getting a run of new X399 boards, so that review cycle might also be worth waiting for. I know there were a lot of fans of the ASRock X399 Taichi, but there were also issues with X399 boards at the start that got fixed.

[PepsiBandit]

3 hours ago, brob said:

Is each instance running in its own VM?

It is similar to a VMware workload but it is not using it. I may occasionally use vmware on it though. 

 

26 minutes ago, Taf the Ghost said:

Well, the 2990WX launches Monday, along with the benchmarks for that and the 2950X. At minimum, I'd wait to order until after the news cycle on Monday, just so you know if there is any reason to wait for the 2950X. That part launches August 31st, I believe. I honestly don't think, with your use case, that it'll effect things, but it's the Memory Subsystems that got the biggest boost. Depending on how your application works, The cost difference might be worth it.

 

The other thing is that we're getting a run of new X399 boards, so that review cycle might also be worth waiting for. I know there were a lot of fans of the ASRock X399 Taichi, but there were also issues with X399 boards at the start that got fixed.

I didn't realise the benchmarks for all of them came on Monday. If I am going for the 1950x which should be a bit cheaper than the 2950x then I might as well go for the older boards too, since they have already had time to mature. The other board I have been looking at is the Asus Prime, I think it is one of the cheapest but the main differences seem to be in the number of sata and m.2 slots, and then only having gigabit networking. Neither of these things would be an issue. 

[Taf the Ghost]

16 minutes ago, PepsiBandit said:

I didn't realise the benchmarks for all of them came on Monday. If I am going for the 1950x which should be a bit cheaper than the 2950x then I might as well go for the older boards too, since they have already had time to mature. The other board I have been looking at is the Asus Prime, I think it is one of the cheapest but the main differences seem to be in the number of sata and m.2 slots, and then only having gigabit networking. Neither of these things would be an issue. 

Other issue will be RAM speed issues. 8 DIMMS in 4-channel is hard to run, just as a baseline, so a lot of boards/setups just cap out at 2666. The TR2 will have the better microcode for the IMC, which improves a lot of the memory compatibility and speed issues. That actually might be the one reason to wait on the 2950X.

[brob]

Have you determined how much is shared when multiple instances are running, or is the estimate based on a single instance running?

[PepsiBandit]

1 hour ago, brob said:

Have you determined how much is shared when multiple instances are running, or is the estimate based on a single instance running?

My estimates are based on running around 25 instances on a 1700x with 64gb ram. 

1 hour ago, Taf the Ghost said:

Other issue will be RAM speed issues. 8 DIMMS in 4-channel is hard to run, just as a baseline, so a lot of boards/setups just cap out at 2666. The TR2 will have the better microcode for the IMC, which improves a lot of the memory compatibility and speed issues. That actually might be the one reason to wait on the 2950X.

I was planning to just some 2133MHz corsair vengence ram and save some money there. I don't know how much ram speed would really matter. I will certainly wait until monday to see how things are looking anyway.

[Taf the Ghost]

10 minutes ago, PepsiBandit said:

My estimates are based on running around 25 instances on a 1700x with 64gb ram. 

I was planning to just some 2133MHz corsair vengence ram and save some money there. I don't know how much ram speed would really matter. I will certainly wait until monday to see how things are looking anyway.

If your application isn't as sensitive to memory, then 2133 should be fine. You're in both a Use Case that TR makes a lot of sense, and something out there in regards to it as well.

[Taf the Ghost]

@PepsiBandit

 

https://pcpartpicker.com/list/tsMC4q

 

PCPartPicker part list / Price breakdown by merchant

CPU: AMD - Threadripper 1950X 3.4GHz 16-Core Processor  ($775.79 @ OutletPC) 
CPU Cooler: Noctua - NH-U14S TR4-SP3 140.2 CFM CPU Cooler  ($79.90 @ Amazon) 
Motherboard: ASRock - X399 Taichi ATX TR4 Motherboard  ($323.49 @ SuperBiiz) 
Memory: G.Skill - Flare X 128GB (8 x 16GB) DDR4-2400 Memory  ($1180.98 @ Newegg) 
Storage: Samsung - 970 Evo 500GB M.2-2280 Solid State Drive  ($179.99 @ Amazon) 
Video Card: Gigabyte - GeForce GTX 1080 8GB WINDFORCE OC 8G Video Card  ($489.99 @ Amazon) 
Case: Corsair - 750D ATX Full Tower Case  ($129.99 @ Newegg) 
Power Supply: EVGA - SuperNOVA G2 750W 80+ Gold Certified Fully-Modular ATX Power Supply  ($89.99 @ Amazon) 
Total: $3250.12
Prices include shipping, taxes, and discounts when available
Generated by PCPartPicker 2018-08-11 23:00 EDT-0400

 

Can save a little by getting 2x 16 Gb kits rather than a single set of 8 DIMMS, but it shows the general outline.

[brob]

14 hours ago, Taf the Ghost said:

...Can save a little by getting 2x 16 Gb kits rather than a single set of 8 DIMMS, but it shows the general outline.

There is a small chance that two 2x16GB kits will not operate in quad channel.

[PepsiBandit]

17 hours ago, Taf the Ghost said:

@PepsiBandit

 

https://pcpartpicker.com/list/tsMC4q

 

PCPartPicker part list / Price breakdown by merchant

CPU: AMD - Threadripper 1950X 3.4GHz 16-Core Processor  ($775.79 @ OutletPC) 
CPU Cooler: Noctua - NH-U14S TR4-SP3 140.2 CFM CPU Cooler  ($79.90 @ Amazon) 
Motherboard: ASRock - X399 Taichi ATX TR4 Motherboard  ($323.49 @ SuperBiiz) 
Memory: G.Skill - Flare X 128GB (8 x 16GB) DDR4-2400 Memory  ($1180.98 @ Newegg) 
Storage: Samsung - 970 Evo 500GB M.2-2280 Solid State Drive  ($179.99 @ Amazon) 
Video Card: Gigabyte - GeForce GTX 1080 8GB WINDFORCE OC 8G Video Card  ($489.99 @ Amazon) 
Case: Corsair - 750D ATX Full Tower Case  ($129.99 @ Newegg) 
Power Supply: EVGA - SuperNOVA G2 750W 80+ Gold Certified Fully-Modular ATX Power Supply  ($89.99 @ Amazon) 
Total: $3250.12
Prices include shipping, taxes, and discounts when available
Generated by PCPartPicker 2018-08-11 23:00 EDT-0400

 

Can save a little by getting 2x 16 Gb kits rather than a single set of 8 DIMMS, but it shows the general outline.

Thanks. I had already chosen a case since I am planning to move all of my current computers into rack enclosures. I am using a 4u Inter-tech 4129-N, which also means I cannot use the Noctua NH-U14, since it is 165mm high and I would only have about 154mm. 

 

 

2 hours ago, brob said:

There is a small chance that two 2x16GB kits will not operate in quad channel.

I think i will play it safe and get a 128gb set that is verified to work with the Taichi board. There was almost no price difference in the ones I could get where I live.