Workstation for Computational Fluid Dynamics

[RaB_73]

Budget (including currency): Will depend upon the suggestions to go Intel or AMD. INR 2-2.5 lacs (lesser is very desirable),
Country: India.
Games, programs or workloads that it will be used for: 

I intend to build a new workstation from scratch.

Primary:The target is exclusively for Chemistry coupled 3D CFD simulations for Ansys CFD and AVL Fire with image post-processing and little bit of rendering.

Secondary:I will also need to address Multi-Objective Optimization problems and  GA based Machine Learning.

NO GAMING is intended.

Initially NO OVERCLOCKING is intended.

Please note that the softwares does not enable GPU acceleration (Ansys does but those cards are way out of my budget): Quadro/open GL cards are prescribed.Ansys and FIRE scales very well with cores but for my mesh size (Ansys-50k max) and (AVL -100k) only 6 Gb is utilized on my present i7-2600 dual channel setup.
 

Other details  After a year of survey i have zeroed on the following:
 

INTEL ROUTE:

CPU:I9-9960X or 9980xe (10980xe-Not available). OR 9900KF?
(Considering my mesh size will quad channel have a significant improvement?-though chemistry is involved)
RAM: 4*16 GB 3600 ( or 3200 MHz) in quad channel configuration.
Motherboard: As per availability in order of preference:Asus WS X299 Sage /  Asus Ws-X299-Pro / MSI X299 Gaming Pro Carbon Ac / Asus Rog Strix X299-E/ Asus Prime X299
(The gaming motherboards listed because of good VRM reviews).
Please suggest :
1. GPU (best budget Quadro)
2.CPU Cooler.
3.Power Supply.(very important as there is significant voltage fluctuations in our power line and low voltage till 10 pm)
4.Cabinet.

AMD ROUTE

3950X or 3960X?
Please suggest the MB ,RAM speed (critical for AMD setup, I dont know much about the sensitivity of the frequency)and CPU cooler and PSU accordingly.

Thank you all for dropping in to help me out.

 

[Grabhanem]

27 minutes ago, RaB_73 said:

Budget (including currency): Will depend upon the suggestions to go Intel or AMD. INR 2-2.5 lacs (lesser is very desirable),
Country: India.
Games, programs or workloads that it will be used for: 

I intend to build a new workstation from scratch.

Primary:The target is exclusively for Chemistry coupled 3D CFD simulations for Ansys CFD and AVL Fire with image post-processing and little bit of rendering.

Secondary:I will also need to address Multi-Objective Optimization problems and  GA based Machine Learning.

NO GAMING is intended.

Initially NO OVERCLOCKING is intended.

Please note that the softwares does not enable GPU acceleration (Ansys does but those cards are way out of my budget): Quadro/open GL cards are prescribed.Ansys and FIRE scales very well with cores but for my mesh size (Ansys-50k max) and (AVL -100k) only 6 Gb is utilized on my present i7-2600 dual channel setup.
 

Other details  After a year of survey i have zeroed on the following:
 

INTEL ROUTE:

CPU:I9-9960X or 9980xe (10980xe-Not available). OR 9900KF?
(Considering my mesh size will quad channel have a significant improvement?-though chemistry is involved)
RAM: 4*16 GB 3600 ( or 3200 MHz) in quad channel configuration.
Motherboard: As per availability in order of preference:Asus WS X299 Sage /  Asus Ws-X299-Pro / MSI X299 Gaming Pro Carbon Ac / Asus Rog Strix X299-E/ Asus Prime X299
(The gaming motherboards listed because of good VRM reviews).
Please suggest :
1. GPU (best budget Quadro)
2.CPU Cooler.
3.Power Supply.(very important as there is significant voltage fluctuations in our power line and low voltage till 10 pm)
4.Cabinet.

AMD ROUTE

3950X or 3960X?
Please suggest the MB ,RAM speed (critical for AMD setup, I dont know much about the sensitivity of the frequency)and CPU cooler and PSU accordingly.

Thank you all for dropping in to help me out.

 

A good budget compute GPU is the Tesla K40 as it has excellent FP64 compute (the main distinction between compute cards vs. consumer cards) and can be found used/refurbished for relatively cheap. It has two main caveats though- being a Tesla it has no display outputs so you'd need a small graphics card as a display adapter, and some server models don't even include a fan, so you'd have to rig up a case fan to cool it.

 

If you're looking for CPU performance comparisons, the 3950X is pretty comparable to the 9980XE at a much lower platform cost, but without quad channel memory or high PCIe lane count support. Threadripper is leagues ahead of either, but also more expensive.

 

If you don't have stable power, I'd probably recommend a dedicated UPS to smooth the signal. They can get quite pricey but a basic deskside one shouldn't be crazy in the grand scheme of the build.

 

For cases, I personally really like Fractal Design-- they're a joy to build in, and have excellent airflow and low noise.

 

Here's a starting point for a list-- I left out storage as I don't really know what your needs are, and the GPU is a pretty basic Quadro that you could run in the 3rd x4 slot with a pair of Teslas getting the x8 slots on #1 and 2.

PCPartPicker Part List

Type	Item	Price
CPU	AMD Ryzen 9 3950X 3.5 GHz 16-Core Processor	₹67423.00 @ Newegg India
CPU Cooler	Noctua NH-D15 82.5 CFM CPU Cooler	₹13499.00 @ Amazon India
Motherboard	Gigabyte X570 AORUS PRO ATX AM4 Motherboard	₹28200.00 @ Amazon India
Memory	Corsair Vengeance LPX 64 GB (2 x 32 GB) DDR4-3600 CL18 Memory	₹32252.00 @ Newegg India
Video Card	PNY Quadro P620 2 GB Video Card	₹13699.00 @ Amazon India
Case	Fractal Design Meshify S2 ATX Mid Tower Case	₹24712.00 @ Newegg India
Power Supply	Corsair RMi 1000 W 80+ Gold Certified Fully Modular ATX Power Supply	₹19490.00 @ Amazon India
	Prices include shipping, taxes, rebates, and discounts
	Total	₹199275.00
	Generated by PCPartPicker 2020-06-05 22:57 IST+0530

 

[RaB_73]

Thank you so much for dropping in and taking the time to reach out.Thank you.

Considering my mesh size so you think I can go about with the dual channel setup? If so it is a huge shift in my thinking and i will align everything else accordingly. Also i forgot to mention is that the ANSYS code takes advantage of intel MKL so perhaps also the AVX512 instruction set. Now as AMD threadripper as compared to i9-99xx do not have this, does it make any difference at all? that was another factor which was making me lean towards Intel.

but thank you again.

[Grabhanem]

13 minutes ago, RaB_73 said:

Thank you so much for dropping in and taking the time to reach out.Thank you.

Considering my mesh size so you think I can go about with the dual channel setup? If so it is a huge shift in my thinking and i will align everything else accordingly. Also i forgot to mention is that the ANSYS code takes advantage of intel MKL so perhaps also the AVX512 instruction set. Now as AMD threadripper as compared to i9-99xx do not have this, does it make any difference at all? that was another factor which was making me lean towards Intel.

but thank you again.

If you need AVX512 then skylake X/X299 is basically the only affordable option. From what I'm reading though, memory bandwidth matters a lot more than AVX, so that rules out the 3950X. It might be worth looking at Threadripper if you have the budget, but I'm not sure it justifies the price bump over X299 as memory bandwidth will be pretty similar. You do get a lot better options for future expansion, though, with 32 and 64 core CPUs available as an upgrade path and possibly even more down the line, while X299 is a complete dead end at 18 cores.

 

BTW- if you quote my posts I'll get a notif and be more likely to see it

[RaB_73]

6 minutes ago, Grabhanem said:

If you need AVX512 then skylake X/X299 is basically the only affordable option. From what I'm reading though, memory bandwidth matters a lot more than AVX, so that rules out the 3950X. It might be worth looking at Threadripper if you have the budget, but I'm not sure it justifies the price bump over X299 as memory bandwidth will be pretty similar. You do get a lot better options for future expansion, though, with 32 and 64 core CPUs available as an upgrade path and possibly even more down the line, while X299 is a complete dead end at 18 cores.

 

BTW- if you quote my posts I'll get a notif and be more likely to see it

sorry for the quote overlook. Thank you for going the extra mile to look it up for me.It means a lot to me.

Just the stupid AVX 512 forces  to look the Intel way. But would it  really make a huge jump for such small mesh size.I really doubt.Actually thank you for opening up the 3960x route.Finances are real tight with me.Going with intel is fill it-forget to upgrade it route. Really need to think now. that however is my (one of the ) weakest point when it comes to decision making.But hey, mucha gracias.I will definitely follow up with you in this thread.pls do drop in when you can.

[Grabhanem]

5 hours ago, RaB_73 said:

sorry for the quote overlook. Thank you for going the extra mile to look it up for me.It means a lot to me.

Just the stupid AVX 512 forces  to look the Intel way. But would it  really make a huge jump for such small mesh size.I really doubt.Actually thank you for opening up the 3960x route.Finances are real tight with me.Going with intel is fill it-forget to upgrade it route. Really need to think now. that however is my (one of the ) weakest point when it comes to decision making.But hey, mucha gracias.I will definitely follow up with you in this thread.pls do drop in when you can.

What do prices look like between X299 and Threadripper? If the prices are similar I'd go Threadripper, even if AVX512 is some wonder drug there's no way it'll make up a 50% lead in performance. I haven't found any concrete benchmarks but most people seem to estimate a few % difference at most.

[RaB_73]

9 hours ago, Grabhanem said:

What do prices look like between X299 and Threadripper? If the prices are similar I'd go Threadripper, even if AVX512 is some wonder drug there's no way it'll make up a 50% lead in performance. I haven't found any concrete benchmarks but most people seem to estimate a few % difference at most.

"there's no way it'll make up a 50% lead in performance"- that's the part that changes everything. But here, the Threadripper 3960x has a price which is nearly 1.2- 1.3 times higher than 9980xe.I have requested my contact for 9960 and 9980xe seven days ago still no news.I am requesting you to give me a part picker list as per your perspectives based on 3960x.Thank you so much for reaching out. I had posted in reddit months ago.Nobody cared.But you did. Thank you.

[Grabhanem]

Just now, RaB_73 said:

"there's no way it'll make up a 50% lead in performance"- that's the part that changes everything. But here, the Threadripper 3960x has a price which is nearly 1.2- 1.3 times higher than 9980xe.I have requested my contact for 9960 and 9980xe seven days ago still no news.I am requesting you to give me a part picker list as per your perspectives based on 3960x.Thank you so much for reaching out. I had posted in reddit months ago.Nobody cared.But you did. Thank you.

PCPartPicker Part List

Type	Item	Price
CPU	AMD Threadripper 3960X 3.8 GHz 24-Core Processor	₹151000.00 @ Amazon India
CPU Cooler	Noctua NH-U14S TR4-SP3 82.52 CFM CPU Cooler	₹13299.00 @ Amazon India
Motherboard	ASRock TRX40 Creator ATX sTRX4 Motherboard	₹46521.00 @ Newegg India
Memory	G.Skill Ripjaws V 64 GB (4 x 16 GB) DDR4-3600 CL18 Memory	₹26077.00 @ Newegg India
Video Card	PNY Quadro P620 2 GB Video Card	₹13699.00 @ Amazon India
Case	Fractal Design Meshify S2 ATX Mid Tower Case	₹24712.00 @ Newegg India
Power Supply	Corsair RMi 1000 W 80+ Gold Certified Fully Modular ATX Power Supply	₹19490.00 @ Amazon India
	Prices include shipping, taxes, rebates, and discounts
	Total	₹294798.00
	Generated by PCPartPicker 2020-06-06 19:54 IST+0530

This is about what I'd go with-- it's a bit over budget but a lot more powerful than an AM4 or X299 system. The motherboard isn't the best for VRM, but it's more than enough since you're not overclocking, and it allows you to run up to 3 Teslas+a low profile Quadro if you want. Also be aware that there's still no storage included with this build, but it shouldn't be a major cost increase compared to the rest of the build.

[RaB_73]

2 hours ago, Grabhanem said:

PCPartPicker Part List

Type	Item	Price
CPU	AMD Threadripper 3960X 3.8 GHz 24-Core Processor	₹151000.00 @ Amazon India
CPU Cooler	Noctua NH-U14S TR4-SP3 82.52 CFM CPU Cooler	₹13299.00 @ Amazon India
Motherboard	ASRock TRX40 Creator ATX sTRX4 Motherboard	₹46521.00 @ Newegg India
Memory	G.Skill Ripjaws V 64 GB (4 x 16 GB) DDR4-3600 CL18 Memory	₹26077.00 @ Newegg India
Video Card	PNY Quadro P620 2 GB Video Card	₹13699.00 @ Amazon India
Case	Fractal Design Meshify S2 ATX Mid Tower Case	₹24712.00 @ Newegg India
Power Supply	Corsair RMi 1000 W 80+ Gold Certified Fully Modular ATX Power Supply	₹19490.00 @ Amazon India
	Prices include shipping, taxes, rebates, and discounts
	Total	₹294798.00
	Generated by PCPartPicker 2020-06-06 19:54 IST+0530

This is about what I'd go with-- it's a bit over budget but a lot more powerful than an AM4 or X299 system. The motherboard isn't the best for VRM, but it's more than enough since you're not overclocking, and it allows you to run up to 3 Teslas+a low profile Quadro if you want. Also be aware that there's still no storage included with this build, but it shouldn't be a major cost increase compared to the rest of the build.

WOW

thank you

actually we usually do not go with amazon india and newegg india has the import problems.I will cross check with other online suppliers dedicated to such causes and see where it finally costs. I will keep you updated for sure.Thank you again for the indicators and your concern for my cause.

[RaB_73]

On 6/6/2020 at 5:37 AM, Grabhanem said:

What do prices look like between X299 and Threadripper? If the prices are similar I'd go Threadripper, even if AVX512 is some wonder drug there's no way it'll make up a 50% lead in performance. I haven't found any concrete benchmarks but most people seem to estimate a few % difference at most.

Hi

You were spot on with the AVX512 hype. I fished this out:
https://www.extremetech.com/computing/302546-amd-threadripper-3970x-3960x-intel-10980xe-cascade-lake-cpu-review

in the y cruncher test, the author states:

"We specifically used an AVX-512 binary to give the Cascade Lake platform an opportunity to put its best foot forward.The 10980XE picks up about 5 percent over the 9980XE, and both beat the 2990WX. The 3960X and 3970X crush all other competitors, even without an optimized binary. AVX-512 gives Intel a boost, but it’s not enough to overcome the core count hit in and of itself."
So threadripper shows a rational route. But the guys here are just ripping us out with their old stocks.

[koptr]

For graphics, I don't know if simulation and the other things need much VRAM, but 2gb is not enough in my opinion, especially if you work with textures while simulating.

[RaB_73]

54 minutes ago, koptr said:

For graphics, I don't know if simulation and the other things need much VRAM, but 2gb is not enough in my opinion, especially if you work with textures while simulating.

hi thanks for dropping in.Actually i need raw computational power. I respect your suggestion .At this moment though I will not be rendering or post processing the CFD results to its requisite details. Mainly it will be line plots and some contour plots.If i can build the base model and validate it then only I intend to venture into that domain.But please feel free to suggest.  Actually AVL specifies quadro/open GL and i had asked for the least expensive one and @Grabhanem had kindly suggested.

[Grabhanem]

5 hours ago, RaB_73 said:

hi thanks for dropping in.Actually i need raw computational power. I respect your suggestion .At this moment though I will not be rendering or post processing the CFD results to its requisite details. Mainly it will be line plots and some contour plots.If i can build the base model and validate it then only I intend to venture into that domain.But please feel free to suggest.  Actually AVL specifies quadro/open GL and i had asked for the least expensive one and @Grabhanem had kindly suggested.

The GPU I included is basically just for a display adapter and to do any basic renders/visualizations. The intent is that if you want to add GPU acceleration for your compute processes, you can stack Teslas in the top 3 slots and put the Quadro in the last slot where it'll be out of the way.