Home lab server for VM's recommendation...

[Gerr]

I need to obtain several IT certifications over the next 2 years and most of my studying will be self studying at home.  I need a home lab server that I can use to virtualize the different environments I will encounter.  I will likely have to have a good 4-8 VM's running at the same time and while I'll try to use Linux as much as possible, some Windows 10 and even Windows Server VM's will be needed.

 

My main question is if my current system is powerful enough, or if I should sell it and get something more powerful?  My current system is a Xeon E3-1271v3(4c8t-3.6Ghz) CPU on an C226 mobo with 32GB ECC RAM(max).  If that is not powerful enough, then for around the $550 range I could pick up a used HP Z420 with an 8-core 2.6Ghz Xeon E5 CPU and 64GB RAM or a HP Z620 with two 6-core 2.2Ghz Xeon E5 CPU's and 48GB RAM.  I know those systems are actually older than my current one, but they are more powerful.  I am up for other recommendations, but not a fan of rack mount systems as they are hot and loud.

[leadeater]

You should be fine with what you have, ram is the biggest limitation to the number of VMs you can run but you shouldn't need to be giving each VM 8GB for testing/dev.

[Gerr]

I am more worried about CPU cores than RAM.  With 32GB RAM, I can reserve 8GB for the host giving 24GB for the VM's, which is 6x4GB, 12x2GB, or 24x1GB VM's.  I think that should be plenty.

 

When it comes to cores, I only have 4 of then,  8 threads with HT on.  If I assign 1 core per VM and have 6 VM's running, I have already over provisioned my CPU cores.  If I need a couple VM's that require 2 cores, I could quickly reach a bottleneck if those VM's are CPU intensive.  Granted that my fast 3.6Ghz CPU will help with that, it's still a concern.

[Blake]

6 hours ago, Gerr said:

I am more worried about CPU cores than RAM.  With 32GB RAM, I can reserve 8GB for the host giving 24GB for the VM's, which is 6x4GB, 12x2GB, or 24x1GB VM's.  I think that should be plenty.

 

When it comes to cores, I only have 4 of then,  8 threads with HT on.  If I assign 1 core per VM and have 6 VM's running, I have already over provisioned my CPU cores.  If I need a couple VM's that require 2 cores, I could quickly reach a bottleneck if those VM's are CPU intensive.  Granted that my fast 3.6Ghz CPU will help with that, it's still a concern.

Your learning, you don't need everything to be ultra fast. FYI, virtual CPU cores <> Physical CPU cores. Also assume the old 4:1 vCPU to CPU core count, make sure to if you start 'bogging down' on the host OS, reserve a single core.

[leadeater]

11 hours ago, Gerr said:

When it comes to cores, I only have 4 of then,  8 threads with HT on.  If I assign 1 core per VM and have 6 VM's running, I have already over provisioned my CPU cores.  If I need a couple VM's that require 2 cores, I could quickly reach a bottleneck if those VM's are CPU intensive.  Granted that my fast 3.6Ghz CPU will help with that, it's still a concern.

I've run about 50 VMs on a single E5-2620v1 which is a 6c/12t @ 2.0Ghz, in a lab CPU is barely used at all since there are no users actively using the servers it's only you and you won't be hitting all of them. You will notice slowness compared to only running a few VMs but that shouldn't be a big deal and anything not required can be powered off.

[kladzen]

i would suggest a ryzen based system like 1700(8c/16t) with 64gb ram.... lots of cores and a nice amount of ram unless you have some very specific requirements for hardware..

[NelizMastr]

12 minutes ago, kladzen said:

i would suggest a ryzen based system like 1700(8c/16t) with 64gb ram.... lots of cores and a nice amount of ram unless you have some very specific requirements for hardware..

For virtualization, cores don't matter that much, especially in a lab environment. And 64GB of DDR4 ECC is pretty darn expensive. I'd stick with DDR3.

[Gerr]

I would love a Ryzen 7, but the prices of DDR4 are way too high, so sticking with a DDR3 solution at the moment.

[Ddarlington36]

Is it a baremetal setup or hyper-v. I know some don't see the difference but sounds like if you could pull off a BM you could save resources on the host so that would leave more ram to allocate to guest vm/os or setup an allocation pool of shared ram so it doesn't necessarily need all the vm to allocate all the ram if its working from a pool of ram. 

 

As for the cores/threads don't see you having to many issues aslong as you don't over commit resources.

[leadeater]

1 hour ago, Ddarlington36 said:

Is it a baremetal setup or hyper-v. I know some don't see the difference but sounds like if you could pull off a BM you could save resources on the host so that would leave more ram to allocate to guest vm/os or setup an allocation pool of shared ram so it doesn't necessarily need all the vm to allocate all the ram if its working from a pool of ram. 

 

As for the cores/threads don't see you having to many issues aslong as you don't over commit resources.

Hyper-V works just the same as something like ESXi and doesn't require any more resources to run the hypervisor system, if you use Core or Windows Server 1709 which is core only it'll use very little at all.

 

ESXi, Hyper-V and KVM all allow over-committing of RAM as well, Hyper-V is actually one of the best at it.

 

Bare metal means no virtualization at all btw, that wording is used to distinguish between a server OS running directly on the hardware or in a VM.

[Ddarlington36]

47 minutes ago, leadeater said:

Bare metal means no virtualization at all btw, that wording is used to distinguish between a server OS running directly on the hardware or in a VM.

I always understood it as no actual os apart from it being the server so it doesn't have a host os ontop of it just serving the guest virtualization envoirments and its managed remotely. 

[leadeater]

3 minutes ago, Ddarlington36 said:

I always understood it as no actual os apart from it being the server so it doesn't have a host os ontop of it just serving the guest virtualization envoirments and its managed remotely. 

You'll always need an OS on the server to actually run the VMs, operating systems written specifically for this task are called hypervisors. There are different types of hypervisors but the one you are referring to is Type 1 which Hyper-V is considered as.

https://en.wikipedia.org/wiki/Hypervisor

[Gerr]

I have a spare copy/license for Windows Server 2016 Essentials.  Can I use that for Hyper-V?  I know nothing about the other options.

[leadeater]

1 hour ago, Gerr said:

I have a spare copy/license for Windows Server 2016 Essentials.  Can I use that for Hyper-V?  I know nothing about the other options.

Windows Hyper-V Server is actually free, you can only use it for Hyper-V though. Personally I'd still use ESXi.

[Gerr]

I ended up getting a used system off of eBay that should suit my needs...

 

Xeon E5-2680 v2 (10c/20t-2.8Ghz)

CM 212 EVO cooler

x79 ATX mobo - unknown make/model

16GB DDR3-1866 ECC RAM - unknown brand.

240GB PNY SSD

1TB WD Black HDD

Blu-Ray Combo drive

GTX 1060 3GB by Asus

Corsair TX650W PSU - unknown if Gold or Bronze.

Corsair 100R case

Windows 10 Pro 64

 

$625 shipped.

 

I figure I'll sell the GTX 1060 and use that money to buy more RAM, 32GB min, trying for 48GB or 64GB, pending on mobo capacity.  Outside of that, I thought it was a good deal and should be a good VM & Handbrake system.

[KirbyTech]

If you want to get the most out of it use ESXi, it is the most common level 1 hyper-visor far as I know. I have used it both in a lab setting (getting certs like you and home dicking around) and in production for client servers (I run a small MSP).

[Gerr]

Studying for F5, CheckPoint, PaloAlto, etc, so at the moment, need to focus on those topics and not learning a new VM system.  I am a network admin, so let the server admins learn about ESXi.

[dalekphalm]

On 1/12/2018 at 7:49 PM, Gerr said:

I ended up getting a used system off of eBay that should suit my needs...

 

Xeon E5-2680 v2 (10c/20t-2.8Ghz)

CM 212 EVO cooler

x79 ATX mobo - unknown make/model

16GB DDR3-1866 ECC RAM - unknown brand.

240GB PNY SSD

1TB WD Black HDD

Blu-Ray Combo drive

GTX 1060 3GB by Asus

Corsair TX650W PSU - unknown if Gold or Bronze.

Corsair 100R case

Windows 10 Pro 64

 

$625 shipped.

 

I figure I'll sell the GTX 1060 and use that money to buy more RAM, 32GB min, trying for 48GB or 64GB, pending on mobo capacity.  Outside of that, I thought it was a good deal and should be a good VM & Handbrake system.

Keep in mind that most E series Xeons DO NOT have an iGPU. Meaning, once you get rid of the GTX 1060, you might need to replace it with a cheaper GPU to get actual console display output.

 

With that in mind, literally the lowest end, cheapest GPU would suffice, such as an AMD R5450 or GTX 710 (Both can usually be had for $20-$40 ballpark).

 

What a bizarre PC build. So mixed, and unbalanced. Hopefully you can sell some of the odd components, like the BD-Drive and 1060 GPU, and get some more RAM and an ultra low power GPU (unless you have an old spare GPU lying around).

 

Anyway, I'd personally use ESXi. We use it at work, running a 2-node cluster, with about 2 dozen VM's. At home, I also use ESXi, running 3 VM's (well, 2 most of the time).

[Gerr]

Actually, it's got most of the hardware I want.  It will not only be a VM training system, but will also be a Blu-Ray ripper & transcoder as well as a secondary gaming PC.  Selling the 1060 3GB as I already have a 1060 6GB waiting to install.  Getting a new mobo that accepts registered ECC and getting 64GB of that so I have plenty of memory to go along with the 10c20t CPU.  Blu-Ray drive is needed for ripping, then the high core count will help with the transcoding.  SSD is good for quick OS boots and running the VM's, but the WD Black is an ideal gaming drive.  Replacing the PSU with a new 650W 80+ Bronze and putting the old PSU into a spare system to sell.

 

At some point I might setup ESXi, but for now, just running Windows 10 Pro and all VM's through Oracle's Virtualbox.

[RJCuthbertson]

@Gerr If you're already using Windows 10 Pro, at what point during your decision process in choosing a hypervisor to use did you select a Type 2 (Virtualbox in your instance) instead of the Type 1 Hyper-V support built into Windows? I am legitimately curious. Choosing ESXi makes sense to me, as does choosing a Type 2 on your home desktop environment where a fully featured Type 1 hypervisor is not directly available via your daily OS. Did you just not know about Hyper-V, or not feel comfortable using it due to lack of experience with it?  Did you not know it's a Type 1? Was the Type 1 vs Type 2 comparison a consideration?

[Gerr]

Couple reasons.  First is that I am ONLY familiar with Virtualbox at the moment and learning how to do a full type-1 hypervisor isn't something I want to do at this time.  Second is that this system will spend most of it's time as a general purpose PC, including gaming.  Not sure I want to run that in a VM.

[Jay Deah]

Virtual box is not the best. I would strongly recommend VMware workstation for your use case but it’s not cheap. VMware player is like feature equivalent with virtual box and free.

 

[leadeater]

19 hours ago, Gerr said:

Couple reasons.  First is that I am ONLY familiar with Virtualbox at the moment and learning how to do a full type-1 hypervisor isn't something I want to do at this time.  Second is that this system will spend most of it's time as a general purpose PC, including gaming.  Not sure I want to run that in a VM.

The inbuilt Hyper-V role is actually really simple to use and has much better memory management, it's dynamic allocation is much better than ESXi on a low resource host (a PC). You don't have to install Windows Server or anything, it's included in Windows Professional and up.

[Jay Deah]

You got any links around hyper-v memory management? When I last read up on this it was the 2008R2 days and HV memory management was terrible, infact everything except esxi was terrible.

 

the memory management in esxi is frankly mindblowingly awesome so I’d love to know if other products are close to matching or beating it

[Gerr]

5 hours ago, leadeater said:

The inbuilt Hyper-V role is actually really simple to use and has much better memory management, it's dynamic allocation is much better than ESXi on a low resource host (a PC). You don't have to install Windows Server or anything, it's included in Windows Professional and up.

So Windows 10 Pro has a built in Type-1 hypervisor?  Any links to tutorials, either written or video?