Building a NAS and compute node for work

[bogate]

Budget (including currency): if possibly under 3000€, max 5000€

Country: Germany

Games, programs or workloads that it will be used for: NAS and CFD, AI, Machine learning, pre, post-processing.

I have access to a supercomputing cluster with 40-800 CPUs so I need a compute node for workloads that dont quite reach this level + local pre-,post-processing.

Other details: Hello I am looking to build a NAS and compute node for work. Can this be done in the same tower or do i need 2 towers?

 

For the NAS part I was looking at 2 10TB drives in RAID1.

 

For the computation part I was looking for a CPU around 16-20 cores. I also need a GPU to help during pre and post-processing + CAD. i was thinking something around a 3060 or 3060Ti might be enough. I am starting to do some Machine learning with genetic algorithms and projects always like it when some machine learning capability is advertised I was thinking an RTX 3060 or 3060Ti might be enough as a starting point with a possible upgrade path to a 3080/3090 or 4080/4090 down the line when machine learning is truly incorporated into the workflow and that kind of power is actually needed. Storage wise I was thinking about a 500GB nvme drive for the OS and one 1TB nvme ssd to store the simulations while they are running or being post-processed. Two 1TB drives in RAID0/RAID1 were also on my mind but i havent been able to decide. Long term storage would go to the NAS. RAM needs are 128GB DDR4 or DDR5. I have a cloud storage for off-site backup and the NAS should be able to automatically update it.

 

Some questions:

Are intel CPUs good for this? Can I assign E-cores to running the NAS and leave the P-cores with disabled multithreading for Simulations? I have seen the 12900K, 13900K only have 8 P-cores do they not have CPUs with more?

Can I have two different OS on the same computer, one running the NAS and the other running the compute node?

Do I need to have 2 towers for this?

 

Summary:

NAS:

• storage: 2 10TB drives in RAID1. Upgrade path to 3rd and 4th drives
• OS: Dont care, but user friendly/good documentation is a must
• Wall ethernet reads 200 Mbps symmetrical

Compute node:

• 1 500GB OS nvme SSD
• 1 1TB nvme SSD (or 2 drives in RAID0/RAID1), need to be fast for the simulations
• ~16-20 CPU cores (multithreading is worthless for simulations)
• GPU capable of pre-,post-processing, visualization of some million mesh elements 4-6GB VRAM, and some machine learning capability.
• 128 GB RAM DDR4 or DDR5. ECC is not needed.
• Upgrade path to 3080/3090/4090/4090
• OS: Mainly Linux, would like to have the option to use Windows in case some workload is not available on linux

 

 

 

I tried to configure something on alternate.de and came up with this:

CPU Intel® Core™ i9-12900K, Prozessor  644,00 €
Motherboard MSI PRO Z690-A, Mainboard  225,90 €
CPU cooler be quiet! Dark Rock 4, CPU-Kühler  69,90 €
RAM Corsair DIMM 64 GB DDR5-5200 Kit 319.90 € x2= 639,80€
OS SSD SAMSUNG 970 EVO Plus 500 GB, SSD  80,90€
Simulation SSD SAMSUNG 970 EVO Plus 1 TB, SSD 119,90 €
NAS HDD Seagate IronWolf Pro NAS 10 TB CMR, 299,00 € x2 =598,00€
Case Cooler Master Silencio S600 109,90 € (max x4 3.5" Drives)
PSU Thermaltake Toughpower GF1 ARGB 850W, 134,90 €

 

 

Any guidance is much appreciated
 

 

 

[brob]

What would the loading be on the NAS, i.e. # users, transaction volume.

 

[manikyath]

'for work' being 'you do this for a living'

or 'for work' being 'for a business'

 

if the latter, contact a supermicro partner.

also, dont combine network storage and compute if you are able to separate them. tasks that max out the cpu will also choke the network bandwidth handling.

 

supermicro has some interesting options on half-workstation half-server options, and partners usually can get you a good price.

 

if you're set on building it yourself, look into amd epyc, it should fit your budget, you get ipmi, and you have MUCH MANY MORE pcie lanes for multiple GPU's, if you potentially want to go that path, otherwise you can just stuff it with NVME.

[bogate]

1 hour ago, brob said:

What would the loading be on the NAS, i.e. # users, transaction volume.

 

Users around 10, i am guessing max 500GB traffic on the busiest day.

43 minutes ago, manikyath said:

'for work' being 'you do this for a living'

or 'for work' being 'for a business'

 

if the latter, contact a supermicro partner.

also, dont combine network storage and compute if you are able to separate them. tasks that max out the cpu will also choke the network bandwidth handling.

 

supermicro has some interesting options on half-workstation half-server options, and partners usually can get you a good price.

 

if you're set on building it yourself, look into amd epyc, it should fit your budget, you get ipmi, and you have MUCH MANY MORE pcie lanes for multiple GPU's, if you potentially want to go that path, otherwise you can just stuff it with NVME.

Work being a business. I dont know if we have supermicro in germany but i will look into it. Thanks your recommending about splitting the NAS and the compute node. That has been a big discussion point.

Regarding amd epyc, are they not much more expensive and slower than the desktop CPUs? I think that would be overkill for the NAS and for the price of an amd epyc (3500$ for a 32 core) we could just get 2 systems with 7950X. For simulations we would prefer two fast 16 core system vs one slow 32 core system. We do have access to a cluster for high core count work.

[manikyath]

3 minutes ago, bogate said:

Users around 10, i am guessing max 500GB traffic on the busiest day.

Work being a business. I dont know if we have supermicro in germany but i will look into it. Thanks your recommending about splitting the NAS and the compute node. That has been a big discussion point.

Regarding amd epyc, are they not much more expensive and slower than the desktop CPUs? I think that would be overkill for the NAS and for the price of an amd epyc (3500$ for a 32 core) we could just get 2 systems with 7950X. For simulations we would prefer two fast 16 core system vs one slow 32 core system. We do have access to a cluster for high core count work.

for the NAS i'd honestly just get a small off the shelf server (for reliability purposes) or a rackmount synology.

 

as for epyc for the compute side, they're not *that* insane once you price out the entire thing, especially if you add in that the epyc motherboards may include stuff like 10gig lan and ipmi.

 

but the beauty of epyc is the amount of expandability you have access to. you could literally have a douzen M.2 SSD's in there as your fast storage, and still have the lanes for multiple GPU's if you feel so inclined. i guess it does depend on how much reliability matters, and how much GPU acceleration you can do.

 

if reliability doesnt really matter for the compute node (reliability in the meaning that issues do happen occasionally, not that it's gonna set on fire in a week) sure thing get a bunch of enthousiast CPU's and overclock the living heck out of them.

that's another good reason to split NAS and compute too, now i think of it: NAS is 100% about reliability. for a compute node you may as well fly closer to the sun if you can get 10% more performance with 2% more downtime.