computer server Building School Compute Server

[xl3b4n0nx]

Hello,

 

I am being tasked with picking out the parts list for a new compute server we may be getting. I have a budget of about $3000, possibly more. We are planning to use this server to run Vivado/Quartus which are HDL Synthesis programs that potentially a course full of students could be using. Those are very CPU and RAM heavy programs. Core count is key here. We don't need any GPU power for this. I am looking for recommendations for building a rack mount server using either Xeon or Epyc chips.

 

My other idea is getting a group of less powerful desktop of weaker rack mount machines and making a compute cluster where a user logging on gets pushed to their own machine to work with. I was thinking I5 or Ryzen 5 machines.

 

Any suggestions would be greatly appreciated.

[EricMarsi]

I would recommend HP DL360 or DL380 Gen8 Server loaded with memory and some high-end Xeon CPU's. Theyre highly reliable servers and while buying used, they offer great benefits, plus 8GB modules are $14 a piece on ebay for them

[ImNotThere]

epyc isnt truly released yet so i wouldnt bother waiting for it

[xl3b4n0nx]

3 minutes ago, ImNotThere said:

epyc isnt truly released yet so i wouldnt bother waiting for it

Ok Thank you. I will stick to Xeon then.

[dalekphalm]

I have some questions:

1. Is this server cluster supposed to be an ongoing resource for many years?

2. Is it critical to the classroom? (Eg: Bad if it's not working)

 

 

I ask because at $3000, you're likely not going to be able to afford multiple new high core count servers. That means used.

 

That's not necessarily a bad thing. But that means no service, no warranty. That means if a part dies unexpectedly, you have to go out and buy a replacement, wait for it to arrive, and deal with downtime during that.

 

I'd suggest that you look at used Dell or HPE servers - preferably relatively new generation. There will be plenty of spare parts available online. Just get the most amount of cores and RAM per server as you can.

[Essence_of_Darkness]

3 hours ago, xl3b4n0nx said:

Hello,

 

I am being tasked with picking out the parts list for a new compute server we may be getting. I have a budget of about $3000, possibly more. We are planning to use this server to run Vivado/Quartus which are HDL Synthesis programs that potentially a course full of students could be using. Those are very CPU and RAM heavy programs. Core count is key here. We don't need any GPU power for this. I am looking for recommendations for building a rack mount server using either Xeon or Epyc chips.

 

My other idea is getting a group of less powerful desktop of weaker rack mount machines and making a compute cluster where a user logging on gets pushed to their own machine to work with. I was thinking I5 or Ryzen 5 machines.

 

Any suggestions would be greatly appreciated.

What about this ?

PCPartPicker part list / Price breakdown by merchant

CPU: Intel - Xeon E5-2640 V4 2.4GHz 10-Core Processor  ($899.99 @ SuperBiiz) 
CPU: Intel - Xeon E5-2640 V4 2.4GHz 10-Core Processor  ($899.99 @ SuperBiiz) 
CPU Cooler: Noctua - NH-L9x65 33.8 CFM CPU Cooler  ($49.95 @ Amazon) 
CPU Cooler: Noctua - NH-L9x65 33.8 CFM CPU Cooler  ($49.95 @ Amazon) 
Motherboard: Supermicro - MBD-X10DAL-I-O ATX Dual-CPU LGA2011-3 Motherboard  ($286.99 @ Newegg) 
Memory: Crucial - 16GB (1 x 16GB) DDR4-2133 Memory  ($178.99 @ B&H) 
Memory: Crucial - 16GB (1 x 16GB) DDR4-2133 Memory  ($178.99 @ B&H) 
Memory: Crucial - 16GB (1 x 16GB) DDR4-2133 Memory  ($178.99 @ B&H) 
Storage: Samsung - 850 EVO-Series 500GB 2.5" Solid State Drive  ($139.99 @ B&H) 
Case: Corsair - 200R ATX Mid Tower Case  ($39.99 @ Newegg) 
Power Supply: EVGA - SuperNOVA G2 750W 80+ Gold Certified Fully-Modular ATX Power Supply  ($77.98 @ Newegg) 
Total: $2981.80
Prices include shipping, taxes, and discounts when available
Generated by PCPartPicker 2017-11-29 15:50 EST-0500

 

[dalekphalm]

25 minutes ago, Essence_of_Darkness said:

What about this ?

PCPartPicker part list / Price breakdown by merchant

CPU: Intel - Xeon E5-2640 V4 2.4GHz 10-Core Processor  ($899.99 @ SuperBiiz) 
CPU: Intel - Xeon E5-2640 V4 2.4GHz 10-Core Processor  ($899.99 @ SuperBiiz) 
CPU Cooler: Noctua - NH-L9x65 33.8 CFM CPU Cooler  ($49.95 @ Amazon) 
CPU Cooler: Noctua - NH-L9x65 33.8 CFM CPU Cooler  ($49.95 @ Amazon) 
Motherboard: Supermicro - MBD-X10DAL-I-O ATX Dual-CPU LGA2011-3 Motherboard  ($286.99 @ Newegg) 
Memory: Crucial - 16GB (1 x 16GB) DDR4-2133 Memory  ($178.99 @ B&H) 
Memory: Crucial - 16GB (1 x 16GB) DDR4-2133 Memory  ($178.99 @ B&H) 
Memory: Crucial - 16GB (1 x 16GB) DDR4-2133 Memory  ($178.99 @ B&H) 
Storage: Samsung - 850 EVO-Series 500GB 2.5" Solid State Drive  ($139.99 @ B&H) 
Case: Corsair - 200R ATX Mid Tower Case  ($39.99 @ Newegg) 
Power Supply: EVGA - SuperNOVA G2 750W 80+ Gold Certified Fully-Modular ATX Power Supply  ($77.98 @ Newegg) 
Total: $2981.80
Prices include shipping, taxes, and discounts when available
Generated by PCPartPicker 2017-11-29 15:50 EST-0500

 

I would highly recommend changing out the PSU for a dual-redundant PSU. Remember, this is a server after all.

 

Also, you only have 3 RAM sticks. In most Dual-Socket servers, they don't like it when each CPU has different amount of RAM installed. Ideally, match both sockets with the same RAM (Either 1x 16GB each, or 2x 16GB each).

[Essence_of_Darkness]

3 minutes ago, dalekphalm said:

I would highly recommend changing out the PSU for a dual-redundant PSU. Remember, this is a server after all.

 

Also, you only have 3 RAM sticks. In most Dual-Socket servers, they don't like it when each CPU has different amount of RAM installed. Ideally, match both sockets with the same RAM (Either 1x 16GB each, or 2x 16GB each).

Redundant PSU will exceed the budget and this is already high quality. If he can go $3000 to get additional 16 gb's of ram then it will be perfect.

Or go like this : 

PCPartPicker part list / Price breakdown by merchant

CPU: Intel - Xeon E5-2640 V4 2.4GHz 10-Core Processor  ($899.99 @ SuperBiiz) 
CPU: Intel - Xeon E5-2640 V4 2.4GHz 10-Core Processor  ($899.99 @ SuperBiiz) 
CPU Cooler: Noctua - NH-L9x65 33.8 CFM CPU Cooler  ($49.95 @ Amazon) 
CPU Cooler: Noctua - NH-L9x65 33.8 CFM CPU Cooler  ($49.95 @ Amazon) 
Motherboard: Supermicro - MBD-X10DAL-I-O ATX Dual-CPU LGA2011-3 Motherboard  ($286.99 @ Newegg) 
Memory: Crucial - 16GB (2 x 8GB) Registered DDR4-2133 Memory  ($211.99 @ B&H) 
Memory: Crucial - 32GB (2 x 16GB) Registered DDR4-2133 Memory  ($402.99 @ Amazon) 
Storage: Samsung - 850 EVO-Series 500GB 2.5" Solid State Drive  ($139.99 @ B&H) 
Case: Corsair - 200R ATX Mid Tower Case  ($39.99 @ Newegg) 
Power Supply: EVGA - SuperNOVA G2 750W 80+ Gold Certified Fully-Modular ATX Power Supply  ($77.98 @ Newegg) 
Total: $3059.81
Prices include shipping, taxes, and discounts when available
Generated by PCPartPicker 2017-11-29 16:24 EST-0500

[Guest]

4 minutes ago, Essence_of_Darkness said:

Redundant PSU will exceed the budget and this is already high quality. If he can go $3000 to get additional 16 gb's of ram then it will be perfect.

Or go like this : 

PCPartPicker part list / Price breakdown by merchant

CPU: Intel - Xeon E5-2640 V4 2.4GHz 10-Core Processor  ($899.99 @ SuperBiiz) 
CPU: Intel - Xeon E5-2640 V4 2.4GHz 10-Core Processor  ($899.99 @ SuperBiiz) 
CPU Cooler: Noctua - NH-L9x65 33.8 CFM CPU Cooler  ($49.95 @ Amazon) 
CPU Cooler: Noctua - NH-L9x65 33.8 CFM CPU Cooler  ($49.95 @ Amazon) 
Motherboard: Supermicro - MBD-X10DAL-I-O ATX Dual-CPU LGA2011-3 Motherboard  ($286.99 @ Newegg) 
Memory: Crucial - 16GB (2 x 8GB) Registered DDR4-2133 Memory  ($211.99 @ B&H) 
Memory: Crucial - 32GB (2 x 16GB) Registered DDR4-2133 Memory  ($402.99 @ Amazon) 
Storage: Samsung - 850 EVO-Series 500GB 2.5" Solid State Drive  ($139.99 @ B&H) 
Case: Corsair - 200R ATX Mid Tower Case  ($39.99 @ Newegg) 
Power Supply: EVGA - SuperNOVA G2 750W 80+ Gold Certified Fully-Modular ATX Power Supply  ($77.98 @ Newegg) 
Total: $3059.81
Prices include shipping, taxes, and discounts when available
Generated by PCPartPicker 2017-11-29 16:24 EST-0500

System won't work just FYI: SSD has issues with Supermicro boards, CPU coolers won't work with those processors, that case will struggle with the internal temprature.

 

As mentioned, used DL380 G8/9 or DL360 G8/9 will be your best bet. Dell and IBM also have good options but I can't remember the part numbers.

[Essence_of_Darkness]

Just now, Windspeed36 said:

System won't work just FYI: SSD has issues with Supermicro boards, CPU coolers won't work with those processors, that case will struggle with the internal temprature.

 

As mentioned, used DL380 G8/9 or DL360 G8/9 will be your best bet. Dell and IBM also have good options but I can't remember the part numbers.

Ok, thanks for clearing up. I didn't know those potential issues,

[Guest]

2 hours ago, Essence_of_Darkness said:

Ok, thanks for clearing up. I didn't know those potential issues,

Building servers is a pain - spent like 4 years doing it with SuperMicro and Asus systems and still didn’t know every gotcha with them

[leadeater]

Yep far easier to go used server then replace the CPUs and RAM with the configuration you actually want.

[leadeater]

8 hours ago, dalekphalm said:

I would highly recommend changing out the PSU for a dual-redundant PSU. Remember, this is a server after all.

 

Also, you only have 3 RAM sticks. In most Dual-Socket servers, they don't like it when each CPU has different amount of RAM installed. Ideally, match both sockets with the same RAM (Either 1x 16GB each, or 2x 16GB each).

Also it's best to add RAM in the number of channels the CPUs support each for optimum bandwidth, that means in a dual socket configuration E5-2600 (all generations) the minimum optimal number of ram sticks is 8 (4 per CPU).

 

6 hours ago, Windspeed36 said:

Building servers is a pain - spent like 4 years doing it with SuperMicro and Asus systems and still didn’t know every gotcha with them

And just when you think you have to it down and can config them with your eyes closed NVMe came out and messed it all up again.

[Guest]

10 minutes ago, leadeater said:

And just when you think you have to it down and can config them with your eyes closed NVMe came out and messed it all up again.

Oh NVME was easy: Intel had their training thing each year here in Melbourne to which SuperMicro was at - all the help in the world was provided there

[leadeater]

40 minutes ago, Windspeed36 said:

Oh NVME was easy: Intel had their training thing each year here in Melbourne to which SuperMicro was at - all the help in the world was provided there

HPE is annoying because they have different drive bays and PCIe expansion bays depending on config and there is a lot of them.

[mackncheesiest]

On 11/29/2017 at 2:23 PM, Essence_of_Darkness said:

Redundant PSU will exceed the budget and this is already high quality. If he can go $3000 to get additional 16 gb's of ram then it will be perfect.

Or go like this : 

PCPartPicker part list / Price breakdown by merchant

CPU: Intel - Xeon E5-2640 V4 2.4GHz 10-Core Processor  ($899.99 @ SuperBiiz) 
CPU: Intel - Xeon E5-2640 V4 2.4GHz 10-Core Processor  ($899.99 @ SuperBiiz) 
CPU Cooler: Noctua - NH-L9x65 33.8 CFM CPU Cooler  ($49.95 @ Amazon) 
CPU Cooler: Noctua - NH-L9x65 33.8 CFM CPU Cooler  ($49.95 @ Amazon) 
Motherboard: Supermicro - MBD-X10DAL-I-O ATX Dual-CPU LGA2011-3 Motherboard  ($286.99 @ Newegg) 
Memory: Crucial - 16GB (2 x 8GB) Registered DDR4-2133 Memory  ($211.99 @ B&H) 
Memory: Crucial - 32GB (2 x 16GB) Registered DDR4-2133 Memory  ($402.99 @ Amazon) 
Storage: Samsung - 850 EVO-Series 500GB 2.5" Solid State Drive  ($139.99 @ B&H) 
Case: Corsair - 200R ATX Mid Tower Case  ($39.99 @ Newegg) 
Power Supply: EVGA - SuperNOVA G2 750W 80+ Gold Certified Fully-Modular ATX Power Supply  ($77.98 @ Newegg) 
Total: $3059.81
Prices include shipping, taxes, and discounts when available
Generated by PCPartPicker 2017-11-29 16:24 EST-0500

I'd be curious what level of experience these students will be working with. If it's an introductory digital logic class or something, 48GB of RAM might be fine (still might be tricky if there's, say, the hypervisor and multiple virtual OSes to partition that with among like 20 students), but I've had some designs in non-class-based design work that used upwards of 16GB + ~8 threads (I think the limit for `set_param general.maxThreads` was around 8 for the version of Vivado I was on) just synthesizing/placing-and-routing/compiling/launching a single post-implementation simulation. And that's just for me, one user. Even if, after the simulation launches, usage quickly drops to basically a single thread and frees a ton of that memory back up (much to the disdain of the user who wants results quickly ). I guess with some intelligent job queuing that takes advantage of those characteristics, that wouldn't be a problem.

 

However, I mostly have experience with single computer licenses of ISE/Vivado, so I can't speak much to the operational characteristics of Quartus or the pros/cons of having a dedicated, remote "synthesis server".

[dalekphalm]

11 hours ago, leadeater said:

Also it's best to add RAM in the number of channels the CPUs support each for optimum bandwidth, that means in a dual socket configuration E5-2600 (all generations) the minimum optimal number of ram sticks is 8 (4 per CPU).

 

And just when you think you have to it down and can config them with your eyes closed NVMe came out and messed it all up again.

There's also the question of added benefit (or simply requirement) of ECC RAM - this is data simulation after all. Would kind of suck if you spend hours processing the data and the result was inaccurate due to a RAM error.

 

Granted, I don't have any experience with the application at hand so I cannot say how serious a concern that is.