Home lab- First server/workstation for virtualization

[Twigg]

Hi all,

I am currently working at IT helpdesk position and I wanted to expand my knowledge on servers and network.

Currently I am running my vms on alienware 12 laptop which is quite ok for running a domain controller with one workstation.

 However I would want to create a bigger environment with couple workstations in domain, and few dcs to play with dns zones/ exchange server in a future.

Therefore I am thinking about a new device that would serve me to run couple vms (probably around 10). Some of them less cpu heavy like windows 10 hosts but also at leats 4 dcs and a exchange server which i guess would need some resources to go into it.

 I understand I need something with hefty ram (16-32 gb) but not sure about processor

Just started to wrap my head around server hardware so wanted to consult with you my purchase ideas. Please let me know which one of those would be a good option in your opinion or let me know if you have your own suggestions.

Some of my requirements:

 

-run 10 vms for lab.

- I have around 2000-3000 polish złoty I would like to spend (700 -800 us $)

-I will be running it under my desk so it can't be jet level noisy as my girlfriend is going to kill me.

- I would prefer something on lower end with possibility to upgrade.

- I would like to play mostly with windows hyper-v, however having a possibility to play with ESXi would be great.

-It would be added benefit if it had raid 1 /0 for messing around.

 

.

By one hand it would be great to have some hands on experience with server hardware. Only concern is to find something not sounding like a jet.

a) New HPE micro server gen10 with X3421

  On one of the guides I fallowed on udemy they used and recommended HPE micro server gen10.

+It is quiet

+there are some sales on it and I could buy it for around 500 us dollars brand new with  X3421 version

+I was thinking about using it as a nas storage later

+It has raid 0/1

Anyone used this server and what is your experience?

 

 

b) used dell t610

I found used one near me for around 500$. It has  two 2x SixCore E5645 2,40Ghz 12M  and comes with 24gb ram.

This sounds great for me as it has nice specs and by throwing some ram I could easily upgrade it.

My only concern is how loud it can be:

https://www.dell.com/community/PowerEdge-Hardware-General/Noise-levels-for-PowerEdge-T610-T710-compared-with-R610-R710/td-p/3893425

 

 

Workstations

I have no experience with those but some suggested it for virtual lab, and they can be found used quite cheap.

 

c) used workstation lenovo d20

I found those being sold for around 200$ with shipping.

They come with 1x XEON E5620 with possibility to upgrade them with another one dirt cheap.

Also 8gb ddr3 ram. I would possibility have to buy some ram and I would throw another cpu as its so cheap to do.

However I dont know if old zombie like this would run good with esxi. (They are from 2011 god damn!).

Also I probably could get even a better deal if I bought 2 of them as local company seems to be getting rid of them so that would be two esxi/hyper v hosts

 

d) dell precision workstation t7500

Those were suggested to me by one of my co workers. They used one of those to run some company-lab vms.

I could get one with 24 gb ram and ntel Xeon W3530, 4x2.8 GHz for 200$

I found they can run up to 96 gb ram so also a huge benefit.

 

 

[PineyCreek]

You could also get an older rackmount HPE server from ebay electronics recyclers.  Otherwise I do like the refurb Dell and HPE workstations  Microservers not so much...mainly for the jet engine noise...sometimes worse than your average rackmount unit.

 

You'll probably be looking at DDR3 ECC for refurb units, but that's not much of a hindrance since it's not a production unit in the field.

[PineyCreek]

Might also be worthwhile to check the requirements of the latest version of whatever VM software you're thinking of running (example, if you're running a newer version of an ESX server) just to see if they've started stripping out older CPUs.  Not much chance of you running into a conflict...but just a thought.

[Jarsky]

Old R610's/R710's are great, but they can be quite power hungry. I currently have a R610 + 2 x R710's with X5650's (+ NAS with 2 x E5540's). When they're all on and running at a decent load I can watch the power meter spin....

 

I originally got them to do my VMware learning, but since then with the invent of nested virtualisation, i'm just in the process of virtualising my ESXi hosts for my lab onto a single Ryzen 2700X system. Cost me US$700 for CPU + Motherboard + 16GB Ram + Samsung 960 EVO 250GB with the option to upgrade the ram to 64GB. I'm not sure if it would have enough power to run as much as you're looking to but i've found i typically have my clients switched off unless im testing GPO's or package pushes, and i've never had a reason to run more than 2 DC's in my forest (I'm a T3 Wintel engineer). 

[NelizMastr]

19 hours ago, PineyCreek said:

Might also be worthwhile to check the requirements of the latest version of whatever VM software you're thinking of running (example, if you're running a newer version of an ESX server) just to see if they've started stripping out older CPUs.  Not much chance of you running into a conflict...but just a thought.

ESXi is dropping everything pre Sandy Bridge-EP in the next version. 6.5 will still run on Clovertown, Harpertown, Nehalem and Westmere, though. Hyper-V will run on anything with VT-x or AMD-V for the time being.

 

19 hours ago, Twigg said:

 

a) New HPE micro server gen10 with X3421

  On one of the guides I fallowed on udemy they used and recommended HPE micro server gen10.

+It is quiet

+there are some sales on it and I could buy it for around 500 us dollars brand new with  X3421 version

+I was thinking about using it as a nas storage later

+It has raid 0/1

Anyone used this server and what is your experience?

I have the previous version, the G8 and it's pretty darn quiet. The loudest components in mine are the SAS drives I added later  No experience with the G10 yet, but I'd definitely get the quad core Opteron and max the RAM at 32GB. Make sure you get ECC UDIMMs and not registered memory or regular DDR4 as that won't work. Alternatively, you could just buy a 8 or 16GB stick for it now and upgrade later as memory is pretty costly.

 

19 hours ago, PineyCreek said:

You could also get an older rackmount HPE server from ebay electronics recyclers.  Otherwise I do like the refurb Dell and HPE workstations  Microservers not so much...mainly for the jet engine noise...sometimes worse than your average rackmount unit.

DL380 G6 and G7 servers are fairly cheap, but still more noisy than a run of the mill workstation. From my experience the Dell T-series is noisier than the HPE Microserver line and my Microserver is only a tad loud on startup. The main benefit to the older rackservers is cost as the servers are cheap to buy and cheap to upgrade. 

 

@Twigg

If you go with a used HPE rackserver make sure you only use drives with HP firmware or the server will run at deafening noise levels. This is a firmware limitation that can't be worked around. You'd have to look on the other side of the fence (Dell R710/R720).

[AbsoluteFool]

Honestly i wouldn't put any of the eterprise servers in my own room and have them runing.. 

Sure the loudest is porobably supermirco servers. But they are all noisy every now and then.

A 6 core CPU custom build would serve the same features without the jet engine sound or sound at all.

[Twigg]

11 hours ago, Jarsky said:

Old R610's/R710's are great, but they can be quite power hungry. I currently have a R610 + 2 x R710's with X5650's (+ NAS with 2 x E5540's). When they're all on and running at a decent load I can watch the power meter spin....

 

I originally got them to do my VMware learning, but since then with the invent of nested virtualisation, i'm just in the process of virtualising my ESXi hosts for my lab onto a single Ryzen 2700X system. Cost me US$700 for CPU + Motherboard + 16GB Ram + Samsung 960 EVO 250GB with the option to upgrade the ram to 64GB. I'm not sure if it would have enough power to run as much as you're looking to but i've found i typically have my clients switched off unless im testing GPO's or package pushes, and i've never had a reason to run more than 2 DC's in my forest (I'm a T3 Wintel engineer). 

I was actually thinking about the r710 first as those go at decent prices post lease with really nice specs .  Also they seem to be recommended left and right for home lab.

 The main problem is the noise. I wouldn't be running those 24/7, mostly 2-3h after work to practice however not sure if keeping one of those under my desk would be a good idea while my girlfriend is siting next to me trying to watch TV.

Expandability and perspective of throwing a bit of ram into those is great but I would rather have something that won't make me deaf

[Twigg]

10 hours ago, NelizMastr said:

ESXi is dropping everything pre Sandy Bridge-EP in the next version. 6.5 will still run on Clovertown, Harpertown, Nehalem and Westmere, though. Hyper-V will run on anything with VT-x or AMD-V for the time being.

 

I have the previous version, the G8 and it's pretty darn quiet. The loudest components in mine are the SAS drives I added later  No experience with the G10 yet, but I'd definitely get the quad core Opteron and max the RAM at 32GB. Make sure you get ECC UDIMMs and not registered memory or regular DDR4 as that won't work. Alternatively, you could just buy a 8 or 16GB stick for it now and upgrade later as memory is pretty costly.

 

DL380 G6 and G7 servers are fairly cheap, but still more noisy than a run of the mill workstation. From my experience the Dell T-series is noisier than the HPE Microserver line and my Microserver is only a tad loud on startup. The main benefit to the older rackservers is cost as the servers are cheap to buy and cheap to upgrade. 

 

@Twigg

If you go with a used HPE rackserver make sure you only use drives with HP firmware or the server will run at deafening noise levels. This is a firmware limitation that can't be worked around. You'd have to look on the other side of the fence (Dell R710/R720).

I was thinking about microserver as a great compromise. Its said to be quiet, it can be expanded to 32 gb (and those 16gb could server me for some time before I upgrade). It is ok if its going to be louder at start I just don't to have a hover dam under my feet while I try to focus on understanding the udemy course .

[Twigg]

9 hours ago, AbsoluteFool said:

Honestly i wouldn't put any of the eterprise servers in my own room and have them runing.. 

Sure the loudest is porobably supermirco servers. But they are all noisy every now and then.

A 6 core CPU custom build would serve the same features without the jet engine sound or sound at all.

 So it seems. I will have to go either with HP micoroserver or buy a used workstation. Would be great to play with a server/take it apart etc but I would probably pull my hair after having run it next to me for 3h.

 Aren't there any workstation noise level servers out there?

 

[Twigg]

13 hours ago, Jarsky said:

Old R610's/R710's are great, but they can be quite power hungry. I currently have a R610 + 2 x R710's with X5650's (+ NAS with 2 x E5540's). When they're all on and running at a decent load I can watch the power meter spin....

 

I originally got them to do my VMware learning, but since then with the invent of nested virtualisation, i'm just in the process of virtualising my ESXi hosts for my lab onto a single Ryzen 2700X system. Cost me US$700 for CPU + Motherboard + 16GB Ram + Samsung 960 EVO 250GB with the option to upgrade the ram to 64GB. I'm not sure if it would have enough power to run as much as you're looking to but i've found i typically have my clients switched off unless im testing GPO's or package pushes, and i've never had a reason to run more than 2 DC's in my forest (I'm a T3 Wintel engineer). 

@Jarsky So this is custom build PC or pre- build? To be honest was thinking about building something similar but I lack when it comes to hardware knowledge and was afraid I am just going to throw money into not working cluster. Would you mind telling more about this build (ebay links to parts or part names would be great)

[Razor Blade]

The R710 is actually pretty quiet for a 2U rack mounted server as long as you keep it cool. If you're wanting quiet you'll want a work station form factor with larger fans. They're also not that power hungry for what they are... compared to a previous generation server like an IBM X3650 7979 which eats 350 watts at idle... I'll take the 135 watt idle of my R710 all day long.

 

Just keep in mind that the newer the server the more expensive to buy and the more expensive to repair.

[Jarsky]

8 hours ago, Twigg said:

@Jarsky So this is custom build PC or pre- build? To be honest was thinking about building something similar but I lack when it comes to hardware knowledge and was afraid I am just going to throw money into not working cluster. Would you mind telling more about this build (ebay links to parts or part names would be great)

 

I'm just using consumer hardware mostly to rebuild it. 

 

New Hardware:

Gigabyte X470 Aorus Ultra 

Ryzen 2700x

Corsair Vengeance DDR4-3000 LPX 16GB

Samsung 960 EVO 250GB

Seasonic 650w PSU

Norco RPC-3116

 

Existing Hardware:

LSI MegaRAID Card

Intel RES2SV240 SAS Expander

2 x Samsung 850 EVO 500GB

 

If you check my signature, you can see my current NAS server. The idea is to move all the disks and raid setup from this, attach it to my new Ryzen system and rebuild it into the Norco case (still waiting for the case to arrive + some new disks). 

 

I'm considering just running Windows natively on the server with my storage and plex, and my VM's in VMware Workstation as I already have my entire lab setup with ESXi and VM's that can be deployed to VMware Workstation. My VMUG EVALExperience license gives me Workstation Pro for free 

 

Just trying to simplify, as its crazy running so many physical servers, and having it so convoluted. I'm also upgrading and expanding my 8TB disk raid, hence doing a full upgrade at the same time. 

 

If you wanted to do something similar, i'd recommend going for an HBA such as an LSI 9211-8i rather than a RAID card, and do a software solution. You could do something like unRAID - configure your storage in there, and then virtualise your ESXi hosts on that. if you want to do VMware learning. The only thing is though Hyper-V nested virtualisation isn't supported on Ryzen. 

 

[Jarsky]

RE: The R710's - theyre fantastic. Mine arent noisy at all really but you might be surprised about just how big they are if you wanted to put it under your desk. I got mine off a friend with 48GB DDR3 each and 6 x 600GB SAS drives. I sold the drives and cpu's and replaced them with Samsung 850 EVO 500GB's, and 2 x X5650's. If you do end up getting R610/710's, then you might want to consider the L series cheaps which are low power versions of the X series. The L5640 is similar to the X5650 but are 60W TDP vs 95W TDP. 

 

Generally i've found the best thing is to find a fairly barebones system with plenty of RAM. You can swap any crappy 4C CPU's like the E5504 for 5600 series 6C chips for very cheap. I also found the servers are much quieter and cooler after switching the SAS drives to SSD's. So again barebones with lots of RAM and cheap cpu's/little or no drives are the best buy from my experience. 

[RiggzieTech]

so I know this is a tad old now but created an acct to see where the OP went with this.

With ESXI 6.7 out.. any W or X Cpu is no longer going to work. You need an E3/E5 or E7 cpu.

 

now not sure where the OP went with what he got if anything, but I am using a Dell D30 I got off of ebay cheap!!!!! and it is a dual E5-2620 cpus and each are 6 core. I then have 128gb ddr3 memory, 3x2tb wd black drives and 2x 500gb ssd drives.

 

now this thing is QUIET!!! I mean SUPER quiet.. I'm an HP fanboy and at work I have an HP Z800 (which will not run 6.7) and it is way louder but not as loud as an actual server. You would need an HP Z820/840 but they are extremely expensive.

 

so this thing is a tower, just an oversized tower fairly small and can sit under a desk and be nice and quiet and fast with a budget at mind.

 

my problem now is I'm trying to figure out what pro workstation class has 4cpu's !!! HAH.. don't really need it but why not...

 

fyi - check out my videos on youtube as I have how to setup a VMware lab, how to setup some home automation, Altiris (my main job function in life)...

 

https://www.youtube.com/c/JeffreyRiggs

 

 

 

[Jarsky]

ESXi 6.7 does work on 5600 series CPU's, but it just isn't supported. The nehalem cpu's also wont support EVC and other enhanced functions which is why they arent supported for the latest version. Generally you should stick with 6.5 for anything older than Skylake. 

 

Most servers these days have headed towards 2 cpu configurations with the larger core count CPU's.

You'll find modern quad socket servers will be extremely expensive for the likes of the Dell R930 or the HPE 500 series. 

 

[RiggzieTech]

interesting information there... agreed quad socket severs are not cheap. years ago I had an ML570 G2 that was quad cpu... I know 6.7 wont install but someday these servers will be better priced...

 

quad dual core.. hah

 

[NelizMastr]

1 hour ago, RiggzieTech said:

interesting information there... agreed quad socket severs are not cheap. years ago I had an ML570 G2 that was quad cpu... I know 6.7 wont install but someday these servers will be better priced...

 

quad dual core.. hah

 

 

Should work well during winter 

[RiggzieTech]

1 hour ago, NelizMastr said:

Should work well during winter 

it was in the basement so didn't care.. I sold it as a few things.. I quadrupled my return on selling it in pieces and while it was loud, was in my basement.. but yeah.. electric bill was it with it... hah...

 

 

******edited...

some day I'm going to run an hp c7000 chassis with G8 or higher blades... then let the heat begin!