Home Server Build - Lab for learning

[learningnewstuff]

Hi everyone. For a really long time, I have been thinking of building a lab at home for learning and gaining some hands on experience. 

To be better at my day job, I need to gain experience with Red Hat System Administration, Networking, VMWare ESXI, Mongodb, Sybase, LDAP, Oracle SGD, Tomcat, NAS and many other things.

We are a very small team and take care of the whole infrastructure. I also want to study for the below certifications which will help me with my day job and future.

(RHCE, RHCSA, CEH, CISSP, CISM). I don't have much knowledge about Networking. I want to setup routers, firewall, pfsense and make a setup with DMZ and stuff to learn about security.

 

A year ago, I got a job at a finance corporation as Tech Support Engineer where I don't have any access to anything. I cannot learn or experiment anything at work.

I am now put into a Cloudera Apache Hadoop Project. I need to build clusters with at least 8 to 10 machines.

(1 Management node, 2 Master nodes, 6 to 7 Worker nodes - All these nodes which will be VMs require at least 16 to 18 GM RAM)

I am going to to have to build a server to install Cloudera's Apache Hadoop Distribution and do a lot of experimenting at home.

So, I will be needing a server with a lot of RAM so that I can spin up a lot of VMs and allocate a good chunk of RAM to each VM.

 

I need a setup to learn about things used at work and also learn about things I am interested in.

I didn't want to waste time researching on a motherboard and CPU that will support ESXI. So, I looked at the below server and bid $680 on it. Someone just outbid me.

 

I live in a very small apartment and don't want to get rack servers which are going to be very loud. But if there are no other options I will have to get a rack server.

I am willing to spend up to $2000 to $3000 because this will help me with doing well at my day job and also get some certifications.

 

I also thought about buying the below ESXI supported motherboard and CPU to build a home server.

I see that the below motherboards are supported.

Partner Name	Model	CPU Series	Supported Releases
Acer Inc.	Altos R360 F3	Intel Xeon E5-2600-v4 Series	ESXi	6.7	6.5 U2	6.5 U1	6.5
Acer Inc.	Altos R380 F3	Intel Xeon E5-2600-v4 Series	ESXi	6.7	6.5 U2	6.5 U1	6.5
Advantech Corporation	ASMB-815T2	Intel Xeon Platinum 8100 (Skylake-SP) Series	ESXi	6.7	6.5 U2	6.5 U1
Advantech Corporation	ASMB-825T2	Intel Xeon Platinum 8100 (Skylake-SP) Series	ESXi	6.7	6.5 U2	6.5 U1
Advantech Corporation	ASMB-925T2	Intel Xeon Platinum 8100 (Skylake-SP) Series	ESXi	6.7	6.5 U2	6.5 U1
Advantech Corporation	ASMB-935T2	Intel Xeon Platinum 8100 (Skylake-SP) Series	ESXi	6.7	6.5 U2	6.5 U1
Advantech Corporation	ASMB-975T2	Intel Xeon Platinum 8100 (Skylake-SP) Series	ESXi	6.7	6.5 U2	6.5 U1
Advantech Corporation	ECU-4784	Intel Xeon E3-1500-v5 Series	ESXi	6.7	6.5 U2	6.5 U1
AIC Inc.	21D-B312-03	Intel Xeon E5-2600-v4 Series	ESXi	6.7	6.5 U2	6.5 U1	6.5
AIC Inc.	21D-B312-04	Intel Xeon E5-2600-v3 Series	ESXi	6.7	6.5 U2	6.5 U1	6.5
AIC Inc.	LIBRA	Intel Xeon E5-2600-v4 Series	ESXi	6.7	6.5 U2	6.5 U1	6.5
bluechip Computer AG	bluechip SERVERline R52302i	Intel Xeon E5-2600-v4 Series	ESXi	6.7	6.5 U2	6.5 U1	6.5
Intel	Intel S2600CWTS	Intel Xeon E5-2600-v4 Series	ESXi	6.7	6.5 U2	6.5 U1	6.5
Intel	Intel S2600WT2	Intel Xeon E5-2600-v4 Series	ESXi	6.7	6.5 U2	6.5 U1	6.5
Intel	S2600CW2	Intel Xeon E5-2600-v4 Series	ESXi	6.7	6.5 U2	6.5 U1	6.5
Intel	S2600CWTR	Intel Xeon E5-2600-v4 Series	ESXi	6.7	6.5 U2	6.5 U1	6.5
Intel	S2600KP	Intel Xeon E5-2600-v4 Series	ESXi	6.7	6.5 U2	6.5 U1	6.5
Intel	S2600KPF	Intel Xeon E5-2600-v4 Series	ESXi	6.7	6.5 U2	6.5 U1	6.5
Intel	S2600TP	Intel Xeon E5-2600-v4 Series	ESXi	6.7	6.5 U2	6.5 U1	6.5
Intel	S2600TPF	Intel Xeon E5-2600-v4 Series	ESXi	6.7	6.5 U2	6.5 U1	6.5
Intel	S2600WTT	Intel Xeon E5-2600-v4 Series	ESXi	6.7	6.5 U2	6.5 U1	6.5
Maguay	211-x4-2U	Intel Xeon E5-2600-v4 Series	ESXi	6.7	6.5 U2	6.5 U1	6.5
Maguay	212-E5-2U	Intel Xeon E5-2600-v4 Series	ESXi	6.7	6.5 U2	6.5 U1	6.5
TAROX Systems & Services GmbH	ParX G5	Intel Xeon E5-2600-v4 Series	ESXi	6.7	6.5 U2	6.5 U1	6.5
TAROX Systems & Services GmbH	ParX R1082i G5	Intel Xeon E5-2600-v4 Series	ESXi	6.7	6.5 U2	6.5 U1	6.5
TAROX Systems & Services GmbH	ParX R2082i G5	Intel Xeon E5-2600-v4 Series	ESXi	6.7	6.5 U2	6.5 U1	6.5
Tyan Computer	S5630	Intel Xeon Gold 6100/5100, Silver 4100, Bronze 3100 (Skylake-SP) Series	ESXi	6.7	6.5 U2	6.5 U1	6.5
Tyan Computer	S7100	Intel Xeon Gold 6100/5100, Silver 4100, Bronze 3100 (Skylake-SP) Series	ESXi	6.7	6.5 U2	6.5 U1	6.5
Tyan Computer	S7106	Intel Xeon Gold 6100/5100, Silver 4100, Bronze 3100 (Skylake-SP) Series	ESXi	6.7	6.5 U2	6.5 U1	6.5
Tyan Computer	S8026	AMD EPYC 7XX1 Series	ESXi	6.7	6.5 U2	6.5 U1
Tyan Computer	S8026GM2NRE	AMD EPYC 7XX1 Series	ESXi	6.7	6.5 U2	6.5 U1
Wortmann AG	TERRA-SERVER-7230-G2-(2x10GbE)	Intel Xeon E5-2600-v4 Series	ESXi	6.7	6.5 U2	6.5 U1	6.5
Wortmann AG	TERRA-SERVER-7420-G2-(2x10GbE)	Intel Xeon E5-2600-v4 Series	ESXi	6.7	6.5 U2	6.5 U1	6.5
Wortmann AG	TERRA_SERVER_7120_G2_(2x10GbE)	Intel Xeon E5-2600-v4 Series	ESXi	6.7	6.5 U2	6.5 U1	6.5
Wortmann AG	TERRA_SERVER_7120_G2_(2x1GbE)	Intel Xeon E5-2600-v4 Series	ESXi	6.7	6.5 U2	6.5 U1	6.5
Wortmann AG	TERRA_SERVER_7130_G2_(2x10GbE)	Intel Xeon E5-2600-v4 Series	ESXi	6.7	6.5 U2	6.5 U1	6.5
Wortmann AG	TERRA_SERVER_7130_G2_(2x1GbE)	Intel Xeon E5-2600-v4 Series	ESXi	6.7	6.5 U2	6.5 U1	6.5
Wortmann AG	TERRA_SERVER_7220_G2_(2x10GbE)	Intel Xeon E5-2600-v4 Series	ESXi	6.7	6.5 U2	6.5 U1	6.5
Wortmann AG	TERRA_SERVER_7220_G2_(2x1GbE)	Intel Xeon E5-2600-v4 Series	ESXi	6.7	6.5 U2	6.5 U1	6.5
Wortmann AG	TERRA_SERVER_7420_G2_(2x1GbE)	Intel Xeon E5-2600-v4 Series	ESXi	6.7	6.5 U2	6.5 U1	6.5

I was looking at these boards that are compatible. https://www.intel.com/content/www/us/en/products/compare-products.html/servers?productIds=88276,88275,88278,88273

Also looking at these processors.

https://ark.intel.com/products/series/91287/Intel-Xeon-Processor-E5-v4-Family

 

It would be helpful if someone can guide me choose the Case, PSU, RAM. 

Is it worth it? Should I build a Home server or should I be buying an old rack server from ebay?

 

[Electronics Wizardy]

look at a dell r720. Can have cheap ram, and they normally go for about 800usd with a good spec. Then put like 128 or 256gb of ram in and you have a nice system. Get a few ssds for vms and put esxi on it.

 

The 430 will be fine, but you can save a lot going back a gen.

 

Edit also look at the dell t620 as its a tower server and will be quieter.

[learningnewstuff]

2 hours ago, Electronics Wizardy said:

look at a dell r720. Can have cheap ram, and they normally go for about 800usd with a good spec. Then put like 128 or 256gb of ram in and you have a nice system. Get a few ssds for vms and put esxi on it.

 

The 430 will be fine, but you can save a lot going back a gen.

 

Edit also look at the dell t620 as its a tower server and will be quieter.

I think you saved me hundreds of hours and a lot of money. Thank you so much!

Looking at this https://www.newegg.com/Product/ProductList.aspx?Submit=Property&N=100283198 600415412 600339035&IsNodeId=1&bop=And&Order=PRICE&PageSize=96

They look like a steal. 

 

Thinking about getting this.

https://www.newegg.com/Product/Product.aspx?Item=9SIACSJ80S6559

 

 

[Electronics Wizardy]

1 hour ago, learningnewstuff said:

I think you saved me hundreds of hours and a lot of money. Thank you so much!

Looking at this https://www.newegg.com/Product/ProductList.aspx?Submit=Property&N=100283198 600415412 600339035&IsNodeId=1&bop=And&Order=PRICE&PageSize=96

They look like a steal. 

 

Thinking about getting this.

https://www.newegg.com/Product/Product.aspx?Item=9SIACSJ80S6559

 

 

that will work fine, Id make sure to get a ssd or two as there much faster.

 

You also want a raid card as esxi doesn't have software raid. You want a h700 here.

[learningnewstuff]

3 hours ago, Electronics Wizardy said:

that will work fine, Id make sure to get a ssd or two as there much faster.

 

You also want a raid card as esxi doesn't have software raid. You want a h700 here.

Thank you! Yes, I intend to get few 2.5" PCIe SSDs. I see that it already has a PERC H310 RAID controller. Do I need a H700?

 

Not sure what kind of SSD to buy.

[Electronics Wizardy]

31 minutes ago, learningnewstuff said:

I intend to get few 2.5" PCIe SSDs.

Id stay away from those here, and you need a certian backplane for them, the one you picked doesn't have it.

 

Get used enterprise ssds, like a intel s3500, pretty cheap, fairly fast.

 

33 minutes ago, learningnewstuff said:

Do I need a H700?

Seing that you will be running esxi and probably want raid, id get the raid card. Hunt on ebay, there pretty cheap, check for formfactor in the server though, you may need A H710

 

what drive setup do you want? How much storage?

[learningnewstuff]

39 minutes ago, Electronics Wizardy said:

Id stay away from those here, and you need a certian backplane for them, the one you picked doesn't have it.

 

Get used enterprise ssds, like a intel s3500, pretty cheap, fairly fast.

 

Seing that you will be running esxi and probably want raid, id get the raid card. Hunt on ebay, there pretty cheap, check for formfactor in the server though, you may need A H710

 

what drive setup do you want? How much storage?

Wow, I didn't even know that there was a category for Enterprise SSDs. Thank you so much!

Gonna buy 4 of these used SSDs for $60.

I am not sure about the drive setup I want. Don't know about how much storage I would need. 

[Electronics Wizardy]

1 minute ago, learningnewstuff said:

Wow, I didn't even know that there was a category for Enterprise SSDs. Thank you so much!

Gonna buy 4 of these used SSDs for $60.

I am not sure about the drive setup I want. Don't know about how much storage I would need. 

Those are some bad prices, ebay is much better.

 

Id just do a raid 10 of 4 480gb class drives. That gives you about 1tb usabe and pretty good speed for a reasonable price(about 400usd)

[learningnewstuff]

17 hours ago, Electronics Wizardy said:

Those are some bad prices, ebay is much better.

 

Id just do a raid 10 of 4 480gb class drives. That gives you about 1tb usabe and pretty good speed for a reasonable price(about 400usd)

Thank you so much! I am planning to buy SSDs from ebay. I see that they are cheaper there.

 

I have decided to buy this Tower that doesn't have any hard drives. https://www.newegg.com/Product/Product.aspx?Item=9SIACSJ80S6709

I want to use only SSDs and no hard drives. I have asked the seller if they can replace the PERC H310 with a refurbished PERC H710.

 

Newegg page says it comes with 4x Trays. I don't exactly know what that means.

I don't know which SSDs to buy.

i) From the technical guide below, I see that there are four 2.5 inch internal storage configurations. Am I getting the "16 x 2.5-inch hot-plug drive bays" or "16 x 2.5-inch hot-plug drive bays plus four Express Flash PCIe SSDs" configuration from the seller?

ii) I see that the 2.5 inch configuration supports SAS, SATA or PCIe SSD. I don't know which ones this Tower will support and which one should I bet getting - SAS, SATA or PCIe SSD?

iii) And should they be exactly the capacity mentioned in the technical guide? Will the Tower recognize different capacities?

Below snapshot is from owner's manual.

I have been reading the manuals to make sure I don't get the wrong SSDs and waste my money and time. I don't know much about servers and I am just more confused now after reading these. They have so many different configurations. It's crazy.

 

[Electronics Wizardy]

11 minutes ago, learningnewstuff said:

Newegg page says it comes with 4x Trays. I don't exactly know what that means.

There are 16 drive bays in the server. this server comes with 4 trays that hold the drives in the bays. If you want more than 4 drives you need to buy trays. There cheap on ebay though(normally like 20 bucks)

 

11 minutes ago, learningnewstuff said:

ii) I see that the 2.5 inch configuration supports SAS, SATA or PCIe SSD. I don't know which ones this Tower will support and which one should I bet getting - SAS, SATA or PCIe SSD?

This appears to have 16 sata/sas 2.5 drive bays. No pcie or 3.5 drive support.

 

11 minutes ago, learningnewstuff said:

i) And should they be exactly the capacity mentioned in the technical guide? Will the Tower recognize different capacities?

you can mix sizes, but you have to plan how it will work in the raid array.

 

 

Also look in ebay, it can often get this cheaper on ebay, esp if you will to change cpus and ram.

 

 

 

How much storage do you need? For vm lab 1tb should be fine, if you want this to be a nas you probably want more hdds.

 

also you want the battery for the raid card, it really helps performance.

[learningnewstuff]

43 minutes ago, Electronics Wizardy said:

There are 16 drive bays in the server. this server comes with 4 trays that hold the drives in the bays. If you want more than 4 drives you need to buy trays. There cheap on ebay though(normally like 20 bucks)

 

This appears to have 16 sata/sas 2.5 drive bays. No pcie or 3.5 drive support.

 

you can mix sizes, but you have to plan how it will work in the raid array.

 

 

Also look in ebay, it can often get this cheaper on ebay, esp if you will to change cpus and ram.

 

 

 

How much storage do you need? For vm lab 1tb should be fine, if you want this to be a nas you probably want more hdds.

Thank you so much!

I will be adding only 4 drives for now. So that's perfect. If I need more, $20 sounds great.

 

Thinking of getting 4 of these SAS SSDs for RAID 10.

800 GB should be fine for now I guess.

Don't have much knowledge about NAS yet but will be setting that up in the future just to get some hands on experience.

[Electronics Wizardy]

25 minutes ago, learningnewstuff said:

Thank you so much!

I will be adding only 4 drives for now. So that's perfect. If I need more, $20 sounds great.

 

Thinking of getting 4 of these SAS SSDs for RAID 10.

800 GB should be fine for now I guess.

Don't have much knowledge about NAS yet but will be setting that up in the future just to get some hands on experience.

those drives should work fine here. 

 

You may want some hdds if you want to bulk store data that doesn't need to be fast.

[Jarsky]

You only need SAS SSD's for Enterprise Production - especially for database work because of the queue depths. 

In my R710's at home I just got Samsung 860 EVO 500GB SSD's , brand new theyre the same price as those used SAS drives and you have the warranty. 

 

Personally I switched to using a consumer desktop for my virtualization lab. It's far quieter under load. Its new so all the parts have warranty. You can build an entire vCenter environment on a single host with nested virtualization.

 

You can also get a cheap US$150 licence for the entire vCenter Suite from a Premium VMUG subscription. 

[learningnewstuff]

On 8/27/2018 at 8:38 AM, Jarsky said:

You only need SAS SSD's for Enterprise Production - especially for database work because of the queue depths. 

In my R710's at home I just got Samsung 860 EVO 500GB SSD's , brand new theyre the same price as those used SAS drives and you have the warranty. 

Thank you, I didn't know that. I bought the enterprise SSDs though.

On 8/27/2018 at 8:38 AM, Jarsky said:

Personally I switched to using a consumer desktop for my virtualization lab. It's far quieter under load. Its new so all the parts have warranty. You can build an entire vCenter environment on a single host with nested virtualization.

I will have to experiment with nested virtualization.

On 8/27/2018 at 8:38 AM, Jarsky said:

You can also get a cheap US$150 licence for the entire vCenter Suite from a Premium VMUG subscription

I would love that. Thank you! I just signed up as a basic member. I will explore more and become Premium member.

[learningnewstuff]

On 8/26/2018 at 8:56 PM, Electronics Wizardy said:

those drives should work fine here.

So my SSDs and Server have arrived 

 

Didn't know how to insert the SSDs. lol

Took a while to figure out how to insert the SSDs correctly. Lol

 

A cable was hanging when I opened the panel

Any idea what that cable is for and where it needs to be plugged?

[Razor Blade]

That's an internal mini SAS SFF-8087 cable typically used to connect back planes to a RAID card.

[Electronics Wizardy]

18 minutes ago, learningnewstuff said:

Any idea what that cable is for and where it needs to be plugged?

what raid card do oyu have?

 

where does that cable go to?

[Razor Blade]

12 minutes ago, Electronics Wizardy said:

what raid card do oyu have?

 

where does that cable go to?

I'll bet dollars to donuts it is for the tape drive seen in photo 6. It looks like a Dell RN757 which is a SAS tape drive.

[Electronics Wizardy]

1 minute ago, Razor Blade said:

I'll bet dollars to donuts it is for the tape drive seen in photo 6. It looks like a Dell RN757 which is a SAS tape drive.

yep thats probably it.

[learningnewstuff]

1 hour ago, Razor Blade said:

what raid card do you have?

I paid a few bucks more to upgrade the RAID controller to H710.

1 hour ago, Razor Blade said:

I'll bet dollars to donuts it is for the tape drive seen in photo 6. It looks like a Dell RN757 which is a SAS tape drive.

Thank you!

Sadly, I didn't get a power cord  to turn on the server. Ugh

What kind of power cord should I be getting?

Is there any kind of specification for that? Looking at these.

[Razor Blade]

It should take a standard computer power cord such as the Tripp Lite in the centre. If you have a desktop PC you could just borrow one from that.

 

That unit likely has 2 power supplies (redundant) but should be able to run on one just fine.

[learningnewstuff]

Was reading the owner's manual to get started. I don't see the embedded SD card inside the server like shown here.

https://www.dell.com/support/contents/us/en/19/videos/videoplayer/PowerEdge-T620-Embedded-SD-Card/g1eXlhMzqveTEZBXuPWk4GyGXEEaiKQK

 

So, I believe I need a SD vFlash card to install some type 1 hypervisor (vmware esxi/Xenserver)

What kind of SD card should I buy? Based on below snapshot, it can be only 16 GB or less.

I am buying these as well. More expense. 

 

[learningnewstuff]

@Electronics Wizardy What kind of SSD would you recommend to install type 1 hypervisor on this machine?

[Electronics Wizardy]

10 minutes ago, learningnewstuff said:

@Electronics Wizardy What kind of SSD would you recommend to install type 1 hypervisor on this machine?

what hypervisor are you running?

 

If you have a sd card in the system, just put a 8 or 16gb sd card(model doesn't really matter, get a midrange one)

 

If you don't have the sd card reader, get a usb stick.

 

 

[learningnewstuff]

2 minutes ago, Electronics Wizardy said:

what hypervisor are you running?

I plan to try VMware ESXI for free first and later pay for it.

If it becomes very expensive, I will have to switch to Xenserver.