Work PC for CFD simulation memory bandwidth limited

[bogate]

Budget (including currency): around 3000€

Country: Germany

Games, programs or workloads that it will be used for: Fluid dynamics (CFD) and chemical kinetic simulations + pre-post processing and mesh generation

Other details (existing parts lists, whether any peripherals are needed, what you're upgrading from, when you're going to buy, what resolution and refresh rate you want to play at, etc):

 

To buy next week or beginning of January

 

3x hard drives 4Tb Raid 5: Seagate IronWolf NAS HDD +Rescue 4TB, 24/7, 512e / 3.5" / SATA 6Gb/s ST4000VN006 3x100€ = 300€

1x SSD 1Tb: Samsung SSD 980 PRO 1TB, M.2 2280 / M-Key / PCIe 4.0 x4 91€

1x sound dampened Case: Fractal Design Define XL R2, schwarz, schallgedämmt 120€

1x PSU : Seasonic Focus GX 1000W ATX 3.0   230€

1x Motherboard: Supermicro H12SSL-i retail 572€

1x CPU cooler: Arctic Freezer 4U-M 55€

1x CPU: AMD Epyc 7443P, 24C/48T, 2.85-4.00GHz, tray 756€

8x RAM 64gb (512Gb total): Samsung LRDIMM 64GB, DDR4-2666, CL19-19-19, ECC 8x 75€ = 600€

 

Total price: 2724€ for epyc cpu system

 

We have also looked at an 5955wx threadripper pro system changing only the CPU and motherboard:

1x CPU: AMD Ryzen Threadripper PRO 5955WX, 16C/32T, 4.00-4.50GHz, boxed ohne Kühler 1074€,

1x Motherboard: ASRock WRX80 Creator R2.0 650€

 

Total price: 3120€

 

Hello, we want to build a new computer at my workplace to run CFD and chemistry simulations + pre and post-processing of the cases. We already have an RTX3060 graphics card, which we will put into the system for graphics.

 

We require at least 8 memory channels as we are memory bandwidth limited during simulations, and we are interested primarily in multi-threaded performance. Due to the pc being used also for pre- and postprocessing tasks, we also need good single-threaded performance as a small portion of those workloads are not parallel. We are between either a threadripper 5955WX and the epyc CPU, but we have seen that the epyc system is a bit cheaper. Both systems support 200Gb/s of memory bandwidth and cpubenchmark.net says the epyc system is 10% better in multithreading, and the thread ripper system is 10% better in single threading. For those 2 reasons (price and multi-threading superiority) we decided in favour of the epyc system.

 

 

Connection to the pc will be made over ssh and vncservers for the different users.

 

We are unsure if the amd epyc 7443P CPU is the best value at this price point and wanted to ask for feedback about that.

 

Another unsure point is if 2666MHz memory is sufficient or 3200MHz memory would make a big difference, the AMD Epyc memory guide says LRDIMM runs at 2933MHz when 8 memory sticks are used if we understand the chart correctly. We have seen that LRDIMM 2933 and 3200MHz modules are much more expensive than 2666MHz modules 200€ for 3200MHz vs 75€ for 2666MHz modules both with around 14ns latency. RDIMM 3200MHz modules cost around 150€:

 Could you help us check if the CPU and memory we have chosen are good value or if we should make some changes? Will we get the full 200Gb/s bandwidth with 2666MHz modules or what is the main difference between 2666MHz memory and 3200MHz memory? Are 3200MHz RDIMMs sufficient or do we need the LRDIMMs?

 

Thanks a lot in advance!

 

 

[porina]

24 minutes ago, bogate said:

Will we get the full 200Gb/s bandwidth with 2666MHz modules or what is the main difference between 2666MHz memory and 3200MHz memory?

Peak bandwidth = channels x MT/s x 8 bytes (modules are 64-bit data width)

 

3200 MT/s = 200 GB/s

2666 MT/s = 166 GB/s

 

For max theoretical bandwidth you'll have to aim for the 3200. I'd further add, having a total of 2R or more per channel will typically give higher effective throughput than 1R/channel and is something you can aim for. This can more than offset small rated speed differences! 

 

I can't advise on the rest as it is outside my focus. I do note those are older offerings using DDR4. Are there any DDR5 options that are affordable as they could offer around 2x the bandwidth? Zen 4 also gains AVX-512 and Zen 5 doubles its peak throughput again, if your software can use that. Also consider Intel Sapphire Rapids based workstation or server offerings.

[bogate]

I have checked a ddr5 system with a threadripper 7955wx and the cheapest i managed with 512 GB RAM is 5700€, each RAM module is 275€. This is well beyond our price range. AVX-512 is not required for our application.

 

Thank you for your comment on RAM speed, we found some ddr4 64GB 2933MHz RDIMM modules for 119€ each and leads us to a 3080€ system with the Epyc CPU and 187 GB/s of bandwidth, and this is still isnide our budget. If i understand correctly LRDIMMs are not required for 512 GB RAM 8 stick configuration