Cheapest Way To Run 100+VM's

[DJ_Anuz]

Hey all,

 

I need to be able to run about 150 instances of a program, and there can only be 1 program installed per OS. The nice thing is that this program can run just fine on old dental servers that are running windows 2000.

Because of this we need to build a server that can run about 150+ Instances of windows 10 dedicated to the sole purpose of running this 1 program from 11pm to 4 am MST.

 

I'm familiar with hardware building and virtualization, but I've honestly never run into a use case like this so I was wondering if anyone would have advice on the most cost effective way to do this. The application itself is only about 20 mb, large so the biggest system drain is going to be the OS itself for each instance. The only requirement for the OS that needs to be used is that it runs .net 4.6 or higher.

 

Thanks!

[imreloadin]

2 minutes ago, DJ_Anuz said:

Hey all,

 

I need to be able to run about 150 instances of a program, and there can only be 1 program installed per OS. The nice thing is that this program can run just fine on old dental servers that are running windows 2000.

Because of this we need to build a server that can run about 150+ Instances of windows 10 dedicated to the sole purpose of running this 1 program from 11pm to 4 am MST.

 

I'm familiar with hardware building and virtualization, but I've honestly never run into a use case like this so I was wondering if anyone would have advice on the most cost effective way to do this. The application itself is only about 20 mb, large so the biggest system drain is going to be the OS itself for each instance. The only requirement for the OS that needs to be used is that it runs .net 4.6 or higher.

 

Thanks!

What program is it you want to run?

[Spotty]

1 minute ago, DJ_Anuz said:

I need to be able to run about 150 instances of a program, and there can only be 1 program installed per OS.

If you need 150 VMs, you're doing it wrong. You'd probably be better off modifying the program to remove the 1 instance restriction tbh.

[DJ_Anuz]

It's something written by our devs. It's a bridge program that reaches out to a dental clients cloud database, requests their data, and then transforms the received data and sends it to our marketing server.

I agree with you spotty, but our devs for whatever reason don't want to do it so I'm exploring whether or not it's possible to achieve this through virtualization. Technically speaking each instance runs the program for 8 different clients, so we're actually running about 1000 instances off of those 150+ VM's.

[Spotty]

2 minutes ago, DJ_Anuz said:

I agree with you spotty, but our devs for whatever reason don't want to do it so I'm exploring whether or not it's possible to achieve this through virtualization. Technically speaking each instance runs the program for 8 different clients, so we're actually running about 1000 instances off of those 150+ VM's.

You should speak to the developers about what you want to do with it, and inquire as to why it can't run multiple instances. They might be able to tailor the program to better suit your needs.

[Electronics Wizardy]

Your probably ram limited here the most

 

So if the program really doesn't need much from a system, then probably a vm with 2gb of ram is fine. Cpu wise, you probably won't need too much.

 

You want about 256gb+ of ram here

 

Im assuming you don't mind some power consumption and don't mine buying used servers.

 

Id get a dell r820 or similar. Cheap ram, lots of cpu power You can get a config for about 500 with quad 8 core 2011 chips, then put another 256 gb of ram in and put some ssds in to store the os. You can do this for about a thousand usd.

 

SInce a basic windows 10 system is about 20gb of storage, you probably want 2tb+ of usable ssd storage. You have compression to save some space though.

[DJ_Anuz]

1 minute ago, Spotty said:

You should speak to the developers about what you want to do with it, and inquire as to why it can't run multiple instances. They might be able to tailor the program to better suit your needs.

I have been speaking with them. The solution is relatively simple it just requires writing office specific files to the installation folder rather than to the OS registry. I'm just exploring other options before telling the CEO that dev resources are the only way to accomplish this realistically.

[Wh0_Am_1]

 

8 minutes ago, Electronics Wizardy said:

Your probably ram limited here the most

 

So if the program really doesn't need much from a system, then probably a vm with 2gb of ram is fine. Cpu wise, you probably won't need too much.

 

You want about 256gb+ of ram here

 

Im assuming you don't mind some power consumption and don't mind buying used servers.

 

Id get a dell r820 or similar. Cheap ram, lots of cpu power You can get a config for about 500 with quad 8 core 2011 chips, then put another 256 gb of ram in and put some ssds in to store the os. You can do this for about a thousand usd.

 

SInce a basic windows 10 system is about 20gb of storage, you probably want 2tb+ of usable ssd storage. You have compression to save some space though.

He/She could setup an SSD as RAM to save costs.

[emosun]

23 minutes ago, DJ_Anuz said:

I need to be able to run about 150 instances of a program, and there can only be 1 program installed per OS

how much are your devs being paid again?

[Electronics Wizardy]

Just now, Wh0_Am_1 said:

 

He/She could setup an SSD as RAM to save costs.

you can, but it will kill performance if that ram is needed, and with that many vms, I wouldn't go much under 256gb, maybe 128 would work.

 

But used ddr3r is pretty cheap, so that won't be that bad.

[emosun]

also you don't really build a system to do this as much as you just buy them. this will be about as fast as anything on earth can be within a single non cluster system

http://www.supermicro.com/products/system/7U/7088/SYS-7088B-TR4FT.cfm

[Wh0_Am_1]

Just now, Electronics Wizardy said:

you can, but it will kill performance if that ram is needed, and with that many vms, I wouldn't go much under 256gb, maybe 128 would work.

 

But used ddr3r is pretty cheap, so that won't be that bad.

Indeed it would, but 256GB of DDR3 (used) is still worth ~$400 USD

[Electronics Wizardy]

Just now, emosun said:

also you don't really build a system to do this as much as you just buy them. this will be about as fast as anything on earth can be within a single non cluster system

http://www.supermicro.com/products/system/7U/7088/SYS-7088B-TR4FT.cfm

you can go clusters, and thats probably a good option here, but really I wouldn't get that blade system here. Get somehting like used dual 2011 systems or dual 1366 systems.

 

1 minute ago, Wh0_Am_1 said:

Indeed it would, but 256GB of DDR3 (used) is still worth ~$400 USD

Yea its not cheap, You can try less ram and add more later, but thats the cheapest way to get that much ram.  With vms you are normally ram limited, and its gonna be very slow if you need to swap.

[Wh0_Am_1]

Just now, Electronics Wizardy said:

you can go clusters, and thats probably a good option here, but really I wouldn't get that blade system here. Get somehting like used dual 2011 systems or dual 1366 systems.

 

Yea its not cheap, You can try less ram and add more later, but thats the cheapest way to get that much ram.  With vms you are normally ram limited, and its gonna be very slow if you need to swap.

An SSD would probably be fast enough.

[emosun]

2 minutes ago, Electronics Wizardy said:

Get somehting like used dual 2011 systems or dual 1366 systems.

and.......there going to run 150 instances of windows 10 on that then

[Electronics Wizardy]

Just now, Wh0_Am_1 said:

An SSD would probably be fast enough.

Have you had a system swapping on a ssd? Its very slow, it will make the difference between usable, and everything locking up for a few seconds to minutes. You really don't want to rely on swap on a ssd for data that is used actively.

 

[Electronics Wizardy]

Just now, emosun said:

and.......there going to run 150 instances of windows 10 on that then

clusters, run like 50 on each. 

 

A windows 10 vm with that one app won't need much ram or cpu power.

[Wh0_Am_1]

1 minute ago, Electronics Wizardy said:

Have you had a system swapping on a ssd? Its very slow, it will make the difference between usable, and everything locking up for a few seconds to minutes. You really don't want to rely on swap on a ssd for data that is used actively.

 

How much of that data will be used actively though? 16GB? 32GB?

[emosun]

1 minute ago, Electronics Wizardy said:

clusters, run like 50 on each. 

k well they didn't ask for a cluster

Although now that i re read it.....

37 minutes ago, DJ_Anuz said:

this program can run just fine on old dental servers that are running windows 2000

why not run 150 instances of windows 2000 instead seeing as the program is so simplistic.

windows 2000 only needs 96mb of ram but is capable of running well under that

[Electronics Wizardy]

Just now, Wh0_Am_1 said:

How much of that data will be used actively though? 16GB? 32GB?

Probably about a gig per vm, thats seems to be about the min for windows, its gets bad much under that.

 

If you start swapping that out it gets pretty ugly fast.

Just now, emosun said:

why not run 150 instances of windows 2000 instead seeing as the program is so simplistic.

Don't run 2000 if you care about security.

[emosun]

1 minute ago, Electronics Wizardy said:

Don't run 2000 if you care about security.

they are vm's , running a single program

not humans making web browsing mistakes

[Wh0_Am_1]

4 minutes ago, Electronics Wizardy said:

Probably about a gig per vm, thats seems to be about the min for windows, its gets bad much under that.

I mean how much data per second?

[Electronics Wizardy]

1 minute ago, emosun said:

they are vm's , running a single program

not humans making web browsing mistakes

there are lots of remote access cve's out there. You don't need a user for there to be a security problem

 

Also with old oses in vms driver support is limited and can be a pain to get working right with some hypervisors and just more of a pain to work with. Windows 7 will run with 256gb of ram, and windows 10 should run with about 256-512mb and they work much better in vms.

 

2 minutes ago, Wh0_Am_1 said:

I mean how much data per second?

Hard to measure ram usage bandwidth in a vm. But you don't want the core os swapping.

[Wh0_Am_1]

3 minutes ago, Electronics Wizardy said:

there are lots of remote access cve's out there. You don't need a user for there to be a security problem

 

Also with old oses in vms driver support is limited and can be a pain to get working right with some hypervisors and just more of a pain to work with. Windows 7 will run with 256gb of ram, and windows 10 should run with about 256-512mb and they work much better in vms.

 

Hard to measure ram usage bandwidth in a vm. But you don't want the core os swapping.

True, but then what about throwing in a single 2GB stick and then using m.2 SSDs for the rest.

[Electronics Wizardy]

4 minutes ago, Wh0_Am_1 said:

True, but then what about throwing in a single 2GB stick and then using m.2 SSDs for the rest.

the problem with ssds, esp nand ones are much slower than ram is. The responce time on a normal nvme ssd is about 20000 ns, optane can get than down a bit more, but ram is at about 20ns. Thats why it gets slow slow when you need data from swap. 

 

If OP is on a very tight budget, Id try something lik e64gb, and see how it goes, but Id personally get the extra ram if you have the budget.