Tips on Virtualization

[KuJoe]

33 minutes ago, leadeater said:

Yea I'm not saying don't use blade, just not in this case where it will only be one chassis.

I read his reply as he was going to have multiple clusters with 2-3 nodes in each one for only a handful of VMs. If he's only going to have a single chassis then yeah, go rackmount servers all the way just to save money.

[TapfererToaster]

7 hours ago, Kadah said:

The i3-4170 supports VT-x.

So?

VT-x - for virtualization performance

VT-d - for passthrough of hardware to the VM

 

So without VT-d, he would NOT be able to directly pass the HDDs to FreeNas.

That means, FreeNas does not see the HDDs SMART values, the HDD health, and has to write over "a layer" ESXi puts over them. The latter is a bad idea - if you read around in the FreeNas forum, they are strongly against any option that doesnt give FreeNas direct access to the drives.

[dalekphalm]

6 hours ago, leadeater said:

Personally I wouldn't go with blade servers, we use 55 (ish) HP DL360 Gen 9 2x E5-2690v3 386GB RAM with onboard dual 10Gb with another dual 10Gb network card and use NFS storage. Each host depending on site either has an average of 20 VMs or 40 VMs, we have around 1000 so it averages out to about 18 per host but half are DR standby so effectively double that. 

 

On the face of it blade severs might seem like a good choice but have limited expandability options and the single chassis in your proposed deployment is a single point of failure (backplane and power).

 

Not sure what storage you are also planning but I would invite a Nutanix sales engineer in to get a brief on what they can offer. If both storage and servers are going to be new this may actually be your cheapest option and is an excellent product. We use Nutanix for our development VM cluster and for one of the DR sites.

 

Also 4-12 VMs is on the low side so 3 nodes would make for a very expensive $/VM figure, but once you start virtualizing VM counts typical jump up very quickly so getting 3 now might be a good choice. 

Sorry guys - I misspoke originally - We're not going Blade Servers specifically, just standard 2U Rackmount servers.

 

We're likely going to end up with 2x Dell R430 servers - the reason why we're using 2x nodes in a cluster is not because we need that much power (In fact, for quite a while, it's likely going to be overkill), but for two reasons:

1. Failover/load balancing - if one server bites the dust (Failed motherboard, CPU fries itself, etc), we just spin up the VM's onto the 2nd machine.

2. Room to grow - we're starting out with ~4 VM's right from the start, but plan to expand that to upwards of 12 within 6-12 months. After that, our needs could grow even farther, so we want that flexibility.

 

For storage, we're looking at a Dell Equilogix SAN, and a couple of QNAP NAS's for offsite mirrored backup (the SAN backs up to the on-site NAS, which replicates a mirror to an offsite location).

 

Some of the vendors suggested offsite DR (Disaster Recovery) nodes, but we don't have the network infrastructure for that to even make sense. If our main site failed in a spectacular enough fashion that we need to use the DR, and we spun up an offsite DR node, it would just sit there with no way to communicate with the rest of the network, since the Internet, Router, Firewall, load balancer, WIFI controller, and master switches are all at the main-site, with no failover (Creating full failover would likely cost us an additional $500,000 or more, in infrastructure and equipment).

5 hours ago, KuJoe said:

I'm going to disagree here. Blades are awesome for ESXi hosts in terms of deployment, management, and capacity per cabinet. I'm not a big fan of HP hardware anymore (we're phasing our the DL380/580s and BL460s for Cisco blades and only using rackmount servers for management hosts), but they do many some solid blades and the chassis aren't too bad although a bit power hungry. The single point of failure is a pain, but mulitple chassis will eliminate that and if you leave some slots open on other chassis you can easily move the blades to a working chassis if something fails on another chassis (and don't put all the hosts for one cluster in a single chassis).

 

As for storage, we've started phasing out EMC for Pure Storage because they out perform EMC's XtremeIO in every test, are extremely easy to manage and setup, the price per TB is much cheaper, and takes up only a quarter cabinet compared to the cabinet sized SANs EMC offers.

 

4 hours ago, KuJoe said:

I read his reply as he was going to have multiple clusters with 2-3 nodes in each one for only a handful of VMs. If he's only going to have a single chassis then yeah, go rackmount servers all the way just to save money.

See above, I did not explain clearly what our proposed new setup is going to be. There will only be one cluster, with (probably) 2 nodes (possibly 3) - each a 2U Dell R430.

[dalekphalm]

23 minutes ago, TapfererToaster said:

So?

VT-x - for virtualization performance

VT-d - for passthrough of hardware to the VM

 

So without VT-d, he would NOT be able to directly pass the HDDs to FreeNas.

That means, FreeNas does not see the HDDs SMART values, the HDD health, and has to write over "a layer" ESXi puts over them. The latter is a bad idea - if you read around in the FreeNas forum, they are strongly against any option that doesnt give FreeNas direct access to the drives.

Yeah, if FreeNAS cannot have direct access to the HDD's, it pretty much defeats the point of FreeNAS - you might as well just go with UnRAID, Windows Server w/ RAID or Storage Pools, or Linux MDADM, etc.

[leadeater]

6 hours ago, dalekphalm said:

-snip-

The Dell Equilogix SANs are actually very nice and have extremely good prices compared to competitor products. Your doing almost exactly what I have done for many clients, 2x IBM x3650 + IBM DS3500/V3700. This type of setup works very well. I also prefer QNAP too.

[dalekphalm]

26 minutes ago, leadeater said:

The Dell Equilogix SANs are actually very nice and have extremely good prices compared to competitor products. Your doing almost exactly what I have done for many clients, 2x IBM x3650 + IBM DS3500/V3700. This type of setup works very well. I also prefer QNAP too.

Yep we are looking forward to it. We're likely going to be using ESXi w/ vSphere 6.0 as the Hypervisor. We'll know for sure once the board of directors gives their official approval on one of the vendors proposals (Next week probably).

 

Most of the vendors all had pretty similar proposals, with just different equipment here and there.