Multiple VMs and NAS - Xeon?

[tsegreto]

So I have a different kind of build going on, and can use some help.

 

I have a friend who owns a design studio (think MythBusters/M5 type place) he has:

 

(1) Laser Cutter

(1) Laser Engraver

(1) UV Printer

(1) CNC

(1) Big industrial printer

(1) 3D Printer

 

Right now he has all these things hooked up to one machine and every week or so he is experiencing compatibility/driver issues. He is currently using 2.5" external HDDs for storage. My solution to this issue was to build a machine capable of running a Windows 7 VM for each separate machine and 1 VM for a NAS.

 

Stability is paramount, and I need a pretty decent amount of USB and ethernet ports. I was thinking about running QEMU. What kind of processor and motherboard would be recommended?

[joshfrog]

first off have you looked at Intel NUCs for each of them rather then building a single large server running VMs?

gigabyte also make a version of the Intel NUC that has 2 network ports on it and come in i3 to i7.

then you could use remote desktop to login to each of the NUCs. 

if you went with NUCs you could have 1 for each machine with anything from a Celeron to i7 between 4 and 32gb or ram. it also means if theres an issue with one of them it doesnt take all of them down with it. 

just a though though.

 

are you able to give some info on the hardware requirements for each of those devices and the software that runs them because if 2cores with 4 threads is enough for each VM to run them and the software then you could get a 12 or 14 core with hyper threading.

you could even look at the 16 core 32 thread, threadripper cpu that way you have few more cores if needed you could assign later.

if you went with a threadripper build you would also get 64 pcie lanes so you could add more network cards or usb ports via add-in cards.

do you know what VM software your wanting to run? theres many good options from free to paid?

going with a single server running VMs does also add complexity to the set up.

[tsegreto]

24 minutes ago, joshfrog said:

first off have you looked at Intel NUCs for each of them rather then building a single large server running VMs?

gigabyte also make a version of the Intel NUC that has 2 network ports on it and come in i3 to i7.

then you could use remote desktop to login to each of the NUCs. 

if you went with NUCs you could have 1 for each machine with anything from a Celeron to i7 between 4 and 32gb or ram. it also means if theres an issue with one of them it doesnt take all of them down with it. 

just a though though.

 

are you able to give some info on the hardware requirements for each of those devices and the software that runs them because if 2cores with 4 threads is enough for each VM to run them and the software then you could get a 12 or 14 core with hyper threading.

you could even look at the 16 core 32 thread, threadripper cpu that way you have few more cores if needed you could assign later.

if you went with a threadripper build you would also get 64 pcie lanes so you could add more network cards or usb ports via add-in cards.

do you know what VM software your wanting to run? theres many good options from free to paid?

going with a single server running VMs does also add complexity to the set up.

Honestly, the system requirement per machine are pretty low and not more than 3 machines would be actually running at a time. 2 cores/4 threads should be more than enough. Last time I dealt with AMD was back in like 2004, but I am open to it. I don't know what VM software I am going to be running, I'm open to suggestions.

[joshfrog]

AMD has come along way in those 13 years.

how confident are you at setting up VMs in this sort of environment.

do you think the client would be able to stop, start the VMs as needed?

 

if only 3 would be running at a time then you could look at an 8 core 16 thread cpu intel has 8 core xeoms and amd has 8 core cpus as well. look up LTT 2 computers 1 system build as you could use the same software he used for your VMs. you just wouldnt need the graphics cards he used.

if you only need 8 cores you could look at consumer gear rather then spending more on enterprise you just dont get stuff like ecc memory and other little things that may not be needed anyway for this sort of set up.

 

before you commit to going with VMs check Intels NUCs it may be a better option for the client and could be as cheap as building a server to run the VMs.

you also avoid all the set up of the VMs and passing the right devices to the right VM by going with a computer per device.
 

 

 

[tsegreto]

6 hours ago, joshfrog said:

AMD has come along way in those 13 years.

how confident are you at setting up VMs in this sort of environment.

do you think the client would be able to stop, start the VMs as needed?

 

if only 3 would be running at a time then you could look at an 8 core 16 thread cpu intel has 8 core xeoms and amd has 8 core cpus as well. look up LTT 2 computers 1 system build as you could use the same software he used for your VMs. you just wouldnt need the graphics cards he used.

if you only need 8 cores you could look at consumer gear rather then spending more on enterprise you just dont get stuff like ecc memory and other little things that may not be needed anyway for this sort of set up.

 

before you commit to going with VMs check Intels NUCs it may be a better option for the client and could be as cheap as building a server to run the VMs.

you also avoid all the set up of the VMs and passing the right devices to the right VM by going with a computer per device.
 

 

 

Well, optimally all would be running at a time but only 3 at a time (max) would be active. Most likely only 1 at a time. At this point he is probably going to be the only person accessing the VMs but I want to leave room in the future for his employees to access them.

 

The only problem with the NUCs is (1) it doesn't solve the storage problem like a NAS RAID 6/10 would and (2) it would make things a lot easier of the NAS and the VM doing the printing/cutting/engraving shared storage for my buddy.

 

What processor/motherboard/RAM combos would you recommend?

Thanks

[joshfrog]

it doesnt solve the problem of the NAS your right but something like a qnap nas could you can get 4 bay qnaps cheap and run raid 10 or a larger one if you really wanted raid 6 or 10. over a gigabit network if the NUCs had a small SSD like 120gb as the boot drive could still leave room for files to be stored on it while print to loser the risk of the network cutting in and out while printing and killing the print. that said for a server build check these links, you may want to start a new thread on here asking people about the builds with some info as to wat it will be used for someone else may know of better combo,

 

https://pcpartpicker.com/list/FmtgkT

 

i put this together for you there are options where you could change parts to save money or may work better i know intel more then AMD but AMD will give you better options price wise at the moment.

one thing i wanted to point out is that professional graphics cards like the 1 i put down support being aple to pass the card though to VMs Nvidias pro line of cards do as well but i think AMDs are cheaper, you could also look at a cheaper card in the pro line it was just the first card i new bit about.

 

https://www.amd.com/Documents/firepro-s9050-datasheet.pdf

 

 one last thing to think about i know you said there would only really be 3 VMs running at once but he would need up to 6 set up for different machines but if he has people working for him they may also need to use a different machine then what he is using so you may want to look at the AMD 1920 cpu so he has more headroom to allow for large print jobs that need more cpu cores or say he forgets to shut down a VM and starts a different one up otherwise he may run out of cores fast with the 8core i have listed.

i also listed 64gb of ram in 16gb sticks, the motherboard i put down supports up to 8 sticks of ram so you would leave half blank incase down the line he needs more ram he has the option to add more while being able to pass 8-16gb per VM depending on the amount he needs to run at once.

 

i also didnt list a case because you could build it in a server case and put it in a server room somewhere or in a computer case if he doesnt have a server room and keep it anywhere in the office. you could also look at water cooling however i wouldnt to avoid the risk of it leaking and killing the server, and AMDs chip i think is able to run on air aslong as its a good enough cooler and has enough fans to move air though the case.

 

storage i will let you work that out as i wouldnt know how many or how big the hard drives he would want and WD or seagate are both fine aslong as you get drives designed for running in a server. the SSD i put there you could also change i would run off an SSD so it doesnt take long to boot the VM and while working its faster and soomer just consider having 2 and running in raid 1 on the SSDs incase it was to die he is still able to work while he waits to replace it.

 

also i didnt put a power supply down as you may want to consider a redundant power supply  in the event something happens to the power supply. if not look at a good 700 or 800w and you should be fine.

[tsegreto]

Thank you! I would be willing to spend a little more ($500 or so) for Intel, would you recommend anything at that price point?

[brob]

This would be an Intel version. I've included a pair of 8TB hdd as placeholders for whatever NAS storage is desired.

 

PCPartPicker part list / Price breakdown by merchant

CPU: Intel - Core i7-7820X 3.6GHz 8-Core Processor  ($559.99 @ Amazon) 
CPU Cooler: NZXT - Kraken X62 Rev 2 98.2 CFM Liquid CPU Cooler  ($149.49 @ SuperBiiz) 
Motherboard: Asus - TUF X299 MARK 2 ATX LGA2066 Motherboard  ($259.99 @ Amazon) 
Memory: Kingston - FURY 64GB (4 x 16GB) DDR4-2400 Memory  ($630.61 @ Newegg Marketplace) 
Storage: Samsung - 850 EVO-Series 500GB 2.5" Solid State Drive  ($149.89 @ OutletPC) 
Storage: Hitachi - Deskstar NAS 8TB 3.5" 7200RPM Internal Hard Drive  ($273.93 @ Amazon) 
Storage: Hitachi - Deskstar NAS 8TB 3.5" 7200RPM Internal Hard Drive  ($273.93 @ Amazon) 
Video Card: PNY - Quadro P2000 5GB Video Card  ($409.99 @ B&H) 
Case: Phanteks - Enthoo Pro ATX Full Tower Case  ($89.99 @ Amazon) 
Power Supply: SeaSonic - FOCUS Plus Gold 750W 80+ Gold Certified Fully-Modular ATX Power Supply  ($83.99 @ SuperBiiz) 
Total: $2881.80
Generated by PCPartPicker 2017-10-31 16:00 EDT-0400

[joshfrog]

the  system above is good there would be a few changes i would make like the cpu cooler not going with liquid, but an air cooler.

consider a higher core cpu just for some over head but the 8core listed would work and let you run the virtualization operating system and have 3 VMs open at once.

look at a 4 port network card so you can pass a network port to each VM.

and run your boot drive in RAID 1 with a 2nd SSD for safety.

the p2000 is a good card and would work for what you need.

 

[tsegreto]

7 hours ago, brob said:

This would be an Intel version. I've included a pair of 8TB hdd as placeholders for whatever NAS storage is desired.

 

PCPartPicker part list / Price breakdown by merchant

CPU: Intel - Core i7-7820X 3.6GHz 8-Core Processor  ($559.99 @ Amazon) 
CPU Cooler: NZXT - Kraken X62 Rev 2 98.2 CFM Liquid CPU Cooler  ($149.49 @ SuperBiiz) 
Motherboard: Asus - TUF X299 MARK 2 ATX LGA2066 Motherboard  ($259.99 @ Amazon) 
Memory: Kingston - FURY 64GB (4 x 16GB) DDR4-2400 Memory  ($630.61 @ Newegg Marketplace) 
Storage: Samsung - 850 EVO-Series 500GB 2.5" Solid State Drive  ($149.89 @ OutletPC) 
Storage: Hitachi - Deskstar NAS 8TB 3.5" 7200RPM Internal Hard Drive  ($273.93 @ Amazon) 
Storage: Hitachi - Deskstar NAS 8TB 3.5" 7200RPM Internal Hard Drive  ($273.93 @ Amazon) 
Video Card: PNY - Quadro P2000 5GB Video Card  ($409.99 @ B&H) 
Case: Phanteks - Enthoo Pro ATX Full Tower Case  ($89.99 @ Amazon) 
Power Supply: SeaSonic - FOCUS Plus Gold 750W 80+ Gold Certified Fully-Modular ATX Power Supply  ($83.99 @ SuperBiiz) 
Total: $2881.80
Generated by PCPartPicker 2017-10-31 16:00 EDT-0400

 

7 hours ago, joshfrog said:

the  system above is good there would be a few changes i would make like the cpu cooler not going with liquid, but an air cooler.

consider a higher core cpu just for some over head but the 8core listed would work and let you run the virtualization operating system and have 3 VMs open at once.

look at a 4 port network card so you can pass a network port to each VM.

and run your boot drive in RAID 1 with a 2nd SSD for safety.

the p2000 is a good card and would work for what you need.

 

Yes that system looks really good with the exception of liquid cooling. I already have a spare 840 Pro laying around and I love the TUF series. I would like more cores if possible, I don't need anywhere near that high of a clock speed - would another processor be more effective?

[joshfrog]

so the pcpartpicker list has a i7-7820k with 8 cores 16 threads but you can get 10 core i9-7900x or the 12 core i9-7920x you could go xeon i just had a look at intels xeon range but i dont understand there new naming so cant help there but they do have 8 10 12 and more core count cpus but you will pay more for a xeon. thats not always bad though there made for servers and virtualization.

they also have lower power needs in most cases so should be little bit lighter on power over an i7 or i9 cpu.

aslong as you find a cpu with the same socket and is supported on the motherboard you should be ok. you may want to make sure if you go with a xeon the cpu is listed as supported with that board because if its not it may work but maybe not 100%.

[brob]

18 minutes ago, tsegreto said:

 

Yes that system looks really good with the exception of liquid cooling. I already have a spare 840 Pro laying around and I love the TUF series. I would like more cores if possible, what would that be?

Ten cores: Intel Core i9-7900X

Twelve cores: Intel Core i9-7920X

Fourteen cores: Intel Core i9-7940X

Sixteen cores:Intel Core i9-7960X

 

Cores are rarely the limiting factor running VM. As physical cores are not typically dedicated to virtual cores. 

 

The various devices that will be controlled through VM are relatively slow and I suspect have quite low data loads. You would have to gather some data from the existing system to make a valid determination, but likely the current machine is not running at 100% during peak periods. That being the case, the new machine simply needs enough additional power to account for additional workload (NAS) and anticipated additional demand.

[joshfrog]

your right it would be low on the cpu so having 2 cores and 4 threads per VM should be fine the problem i foresee being if your using 2 VMs running print jobs on those machines and then start a 3rd or 4th VM to start other print jobs you dont want the resources taken from the already running VMs because it could stop the machines or cause a problem. im sure there smart enough to handle stuff like that i just know when i have printed images on printers if something happens even for a second that it can screw the whole print up and in a business that could mean wasted time, money and product.

 

starting a VM would most likely tax a drive more then cpu so having them start from an SSD should help but its something to keep in mind. 

 

[tsegreto]

Is there any 10 core CPUs that have a lower frequency that I might be able to get for a cheaper price that are suggested?

 

What are thoughts on Intel E5-2670 or Intel Xeon E5-2630? Would those be viable options?

[joshfrog]

i just checked and asus only has that 10 core cpu as supported on there site there are xeons with a lower frequency you can get in the same socket but because its not listed as supported you should contact asus and ask them as they would know better what cpus are supported or can help you with another motherboard that would work with a lower frequency 10 core cpu. or you could ask on here im sure there would be people who know.

[tsegreto]

2 minutes ago, joshfrog said:

i just checked and asus only has that 10 core cpu as supported on there site there are xeons with a lower frequency you can get in the same socket but because its not listed as supported you should contact asus and ask them as they would know better what cpus are supported or can help you with another motherboard that would work with a lower frequency 10 core cpu. or you could ask on here im sure there would be people who know.

I'm sorry I should rephrase, I don't mean for that motherboard - I meant for my application

[joshfrog]

ohhh right sorry yeah i though you wanted for that motherboard.

these are some 10 cores i found on other socks that will be fine

https://ark.intel.com/products/92981/Intel-Xeon-Processor-E5-2630-v4-25M-Cache-2_20-GHz

https://ark.intel.com/products/92984/Intel-Xeon-Processor-E5-2640-v4-25M-Cache-2_40-GHz

https://ark.intel.com/products/81705/Intel-Xeon-Processor-E5-2650-v3-25M-Cache-2_30-GHz

https://ark.intel.com/products/81909/Intel-Xeon-Processor-E5-2687W-v3-25M-Cache-3_10-GHz

 

these will all work for your needs, they are all on the 2011-3 motherboard socket and you should be able to still get motherboards for them such as these.

 

https://www.bhphotovideo.com/c/product/1163658-REG/asus_x99_e_ws_usb_3_1_workstation.html

https://www.bhphotovideo.com/c/product/1279910-REG/asus_x99_e_10g_ws_lga_2011_v3.html

https://www.bhphotovideo.com/c/product/1319012-REG/supermicro_mbd_x10srh_cln4f_o_x10srh_cln4f_o_motherboard.html

https://www.bhphotovideo.com/c/product/1319015-REG/supermicro_mbd_x10drg_q_x10drg_q_motherboard.html

 

just double check your cpu is supported as there is a mix of boards and cpu combos.

[tsegreto]

The above are pretty pricy, I did some searching on NewEgg and I found these processors:

 

AMD RYZEN Threadripper 1920X 12-Core for ~$800
Intel Core i7-7820X Skylake-X 8-Core 3.6 GHz for ~$600
Intel Xeon E5-2670 Sandy Bridge-EP 2.6GHz 8-Core for ~$175 x2 = ~$350
Intel Xeon E5-2690 Sandy Bridge-EP 2.9GHz 8-Core for ~$400 x2 = ~$800
Intel Xeon E5-2680 Sandy Bridge-EP 2.7GHz 8-Core for ~$300 x2 = ~$600
Intel Xeon E5-2670 v2 Ivy Bridge-EP 2.5 GHz 10-Core for ~$320 x2 = ~$640

 

I am under the impression that the Xeon EP line use dual processors, I never dealt with a 2 processor system what are the added performance benefits?

 

What processor would be the best for my application? What is most performance per dollar? Could I get away with the E5-2670?

[joshfrog]

yeah there not cheap i stuck with cpus that are on the 2011-3 socket because it would be easier for you to get a motherboard then maybe something older.

i did list the AMD chip in another comment but you wanted intel so i didnt list any AMD stuff

xeons dont need 2 cpus but there designed so you can you up to 4 or 8 depending on the generation you look at per motherboard. you would be better off sticking with just 1 cpu because once you get 2 the cost per dollar isnt as good. dual socket motherboards are not cheap. to keep things simple yoe better off sticking with just 1 cpu. 

amd cpus will be cheaper then intel when your looking at the 12 core range.

you should be able to use the e5-2670 with no problem but i think it will be hard trying to find a motherboard, that fits your needs given how old it is.

[tsegreto]

2 minutes ago, joshfrog said:

yeah there not cheap i stuck with cpus that are on the 2011-3 socket because it would be easier for you to get a motherboard then maybe something older.

i did list the AMD chip in another comment but you wanted intel so i didnt list any AMD stuff

xeons dont need 2 cpus but there designed so you can you up to 4 or 8 depending on the generation you look at per motherboard. you would be better off sticking with just 1 cpu because once you get 2 the cost per dollar isnt as good. dual socket motherboards are not cheap. to keep things simple yoe better off sticking with just 1 cpu. 

amd cpus will be cheaper then intel when your looking at the 12 core range.

you should be able to use the e5-2670 with no problem but i think it will be hard trying to find a motherboard, that fits your needs given how old it is.

Thanks, the one thing I excluded in that list was socket type, that list was just CPUs that were in my budget that looked like they fit the bill. Let me go back to NewEgg.

 

 

[joshfrog]

its 2011 socket not to be confused with 2011-3 the chips WILL NOT work on the wrong socket.

i dont know the availability of those motherboards.

[tsegreto]

What are your thought on the Intel Xeon E5-2620 V4 Broadwell-EP 2.1 GHz 8 x 256KB L2 Cache 20MB L3 Cache LGA 2011-3 85W BX80660E52620V4 Server Processor for $420? It's 8 cores, Socket 2011-V3. Would the E5-2630 v4 be that much better for the additional $250?

[brob]

23 minutes ago, tsegreto said:

The above are pretty pricy, I did some searching on NewEgg and I found these processors:

 

AMD RYZEN Threadripper 1920X 12-Core for ~$800
Intel Core i7-7820X Skylake-X 8-Core 3.6 GHz for ~$600
Intel Xeon E5-2670 Sandy Bridge-EP 2.6GHz 8-Core for ~$175 x2 = ~$350
Intel Xeon E5-2690 Sandy Bridge-EP 2.9GHz 8-Core for ~$400 x2 = ~$800
Intel Xeon E5-2680 Sandy Bridge-EP 2.7GHz 8-Core for ~$300 x2 = ~$600
Intel Xeon E5-2670 v2 Ivy Bridge-EP 2.5 GHz 10-Core for ~$320 x2 = ~$640

 

I am under the impression that the Xeon EP line use dual processors, I never dealt with a 2 processor system what are the added performance benefits?

 

What processor would be the best for my application? What is most performance per dollar? Could I get away with the E5-2670?

Sandy Bridge is several generations old. There is nothing wrong with it, but if you have driver problems now, why invite even more as support for older hardware is dropped?

 

I'm still trying to puzzle out the need for more cores. Do you have system requirements for the software used to control each of the machines? 

[joshfrog]

that cpu should be good its only bit slower then the E5-2640 v4 listed before. your kinda jusy paying that extra $250 for more cores. if the workload your thinking is right then there difference in speed should be to bad there both going to do what you need.

[joshfrog]

you can see the chips side by side with this link and see the differences 

https://ark.intel.com/compare/92984,92986

its only the clock speed, core count and the power needed but nothing else is different.