Server Upgrade Log

[dalekphalm]

Hi everyone! It's finally time for me to upgrade my server. It's actually done quite well for itself so far, but the OS isn't ideal, and it's time for a bit of change.

 

An overview of my current setup:

ProximaServer

 

Hardware
CASE: LianLi something.. it's an older model
PSU: 550 Watt
MB: ASUS P5G41T-M LX PLUS
CPU: Intel E8600 Core 2 Duo @ 3.33 GHz
HS: Generic Tower style
RAM: 8GB (2x 4GB) DDR3 1333 Patriot
RAID CARD: LSI 8888-ELP w/ BBU and 256MB Cache - The BBU has been giving me errors for ages now (Doesn't keep a charge for long), but finding a replacement BBU of the right model is quite difficult now
SSD: ADATA SX900 64GB
HDD 1: 2x 3TB Seagete ST3000DM001 - One of these actually died last week - replaced with same model.
HDD 2: 1x 3TB Western Digital WD30EZRX
HDD 3: 1x 2TB Seagate ST2000DM001 - I have also since added a second 2TB Seagate HDD of the same model.
HDD 4: 2x 500GB Hitachi something - These hard drives dropped out and my SAS expander would refuse to read them anymore - they were removed.

Expander Case: SE3016 16-bay SAS Expander case (The 2TB and 500GB drives are in here).

OS: Windows Home Server 2011

 

Detailed config here:

New Build/Upgrade

 

Hardware

 

Case: Cooler Master HAF 922 - Picked this up on Kijiji for $30 (Missing one of the feet)

PSU: Antec 550W from original build

MB: EVGA X58 SLI LE - recycled from my old gaming PC

CPU: Intel Xeon L5520 4C8T @ 2.4GHz

HS: Cooler Master Air Cooler w/ Cougar 120mm Fan (The CM Fan was making bad noises)

RAM: 12GB (3x4GB) DDR3

GPU: MSI R5450 1GB - I just needed video out, as my old gaming board doesn't have onboard video

RAID Card: LSI 8888-ELP w/ BBU and 256MB Cache

SSD: ADATA SX900 64GB

HDD 1: 2x 3TB Seagate ST3000DM001

HDD 2: 1x 3TB Western Digital WD30EZRX

HDD 3: 2x 2TB Seagate ST2000DM001

HDD 4: 2x 300GB Dell 15K SAS drives

HDD 5: 2x 146GB Dell 15K SAS drives

Expander Case: SE3016 16-bay SAS Expander - I now have the 2TB drives and the SAS drives in here

OS: Windows Server 2012 R2 Standard

 

I will post some pictures and more details when I get home.

 

As of right now, I finished installing the new OS last night - 2012 R2 picked up all my drivers right away. I didn't have the RAID Card installed during OS installation though - so when I get home tonight I'll have to install the drivers for the Raid Card. I did the OS install outside of the case, in case I needed to swap parts.

 

A teaser shot:

[scottyseng]

@dalekphalm

 

Looks pretty good. I hope the OS install goes well (64GB SSD is kind of tiny to me...but yours is just a file server..I use my NAS as a workstation when my 2500K can't handle the workload).

 

Curious, how loud are the 15K drives? I have four WD Re SAS drives, and they run really loud...I can hear them seeking from the other end of my room.

[dalekphalm]

1 minute ago, scottyseng said:

@dalekphalm

 

Looks pretty good. I hope the OS install goes well (64GB SSD is kind of tiny to me...but yours is just a file server..I use my NAS as a workstation when my 2500K can't handle the workload).

 

Curious, how loud are the 15K drives? I have four WD Re SAS drives, and they run really loud...I can hear them seeking from the other end of my room.

The OS Install went just fine. The OS itself uses like 20GB or less, since I have no extra Roles installed. I don't need the SSD to store anything more than a few apps/addons/programs, like Plex. It will definitely never be used as a workstation lol.

 

The 15K drives are loud as fuck. WHICH kinda sucks, since my Server lives right below my HTPC, which lives right below my main 50" TV

 

At idle they sound like a car in neutral, a constant low whirring. With that in mind, I'm moving next month, and at the new place, the server is gonna be in a storage room in the basement

[scottyseng]

3 minutes ago, dalekphalm said:

-snip-

Yeah, I kind of went severe overkill in my NAS using a E5-2695V3 (14 core, 28 thread)...I should've put the Xeon into my main PC now that I think about it (Currently CPU bottlenecked for my work)

 

Oh, so the 15k drives are as loud as I've read online. Yeah, last night I turned the server off randomly for the night and wow, my room was so quiet (no fans / drive noise).

[dalekphalm]

11 minutes ago, scottyseng said:

Yeah, I kind of went severe overkill in my NAS using a E5-2695V3 (14 core, 28 thread)...I should've put the Xeon into my main PC now that I think about it (Currently CPU bottlenecked for my work)

 

Oh, so the 15k drives are as loud as I've read online. Yeah, last night I turned the server off randomly for the night and wow, my room was so quiet (no fans / drive noise).

I wouldn't keep it in my room - way too loud, even before the 15K drives. What in gods name would you need an E5-2695v3 in a NAS for? If you're running a bunch of VM's, though, that could be super useful.

 

I don't plan on running any VM's yet, but we'll see. I don't want to rebuild using ESXi, so I would need to use HyperV or a software level VM like VMWare Player.

[scottyseng]

4 minutes ago, dalekphalm said:

I wouldn't keep it in my room - way too loud, even before the 15K drives. What in gods name would you need an E5-2695v3 in a NAS for? If you're running a bunch of VM's, though, that could be super useful.

 

I don't plan on running any VM's yet, but we'll see. I don't want to rebuild using ESXi, so I would need to use HyperV or a software level VM like VMWare Player.

It was 100% an impulse buy of a engineering sample Xeon from ebay. I don't regret it though, it eats my overclocked 2500K alive at rendering. But usually it spends most of its life idling at 1% at everything as a file server. I'm trying to resist buying more engineering sample Xeons to replace my 2500K...

[dalekphalm]

2 minutes ago, scottyseng said:

It was 100% an impulse buy of a engineering sample Xeon from ebay. I don't regret it though, it eats my overclocked 2500K alive at rendering. But usually it spends most of its life idling at 1% at everything as a file server. I'm trying to resist buying more engineering sample Xeons to replace my 2500K...

You gotta be careful with Engineering Samples - they're the silicon lottery times 10. You might get one that's glitchy or unstable. Not all of them will be like the one you already have

[Mikensan]

Very nice, a large improvement with only a small amount of $$, I love it. I have that exact same motherboard, still chugging along. That CPU is a workhorse.

[dalekphalm]

13 minutes ago, Mikensan said:

Very nice, a large improvement with only a small amount of $$, I love it. I have that exact same motherboard, still chugging along. That CPU is a workhorse.

Yeah it's been pretty good. I was running a Xeon W3520 in that motherboard when it was my gaming computer, but the L5520 has like half the TDP and the clock rate is only a few hundred MHz slower - both are 4C8T, so I figured for a server I'm running 24/7, I'll take the lower power consumption. The video card also only draws around 25-30w, so when it's idle, it shouldn't be drawing much at all.

 

I completed the build inside my case:

 

The cable management is much better then before, even around back. The side panels do bulge out a bit to help with cable bunching, but not nearly as much as my Carbide 300R, so getting the rear side panel on was a bit tight, as all the SATA connectors for those HDDs are straight, not right angle. Ideally, I would replace them all with right angle SATA cables, but meh.

 

Now the problem:

My Rackables SE3016 SAS Expander bay seems to have died, or isn't cooperating. When I press the power button on the expander bay, the power light comes on and the fans spin up for a few seconds, then the fans die. My RAID Card doesn't see any of the drives in the SAS Expander...

 

This is not good.

 

Fortunately, 4 of the 6 drives in the bay were empty (newly added, so no data made it on to them yet). The motherboard does have 6x SATA ports, but there are only 5x 3.5" HDD bays. There are a ton of 5.25" Drive bays, but I'd need to buy some HDD Cage adapters. I can still use the RAID Card, since it was a 8i/8e model, so it has both internal and external SAS connectors, but I'd have to buy some breakout cables for the 4x SAS drives.

 

Hmm. I'm going to do some more work soon to see if I can resurrect the SAS Expander bay, but I might have to migrate all the drives into the HAF 922, and hope my Antec 450w PSU can handle 9x HDD's and an SSD...

[dalekphalm]

Can anyone recommend a reasonably priced 5.25" HDD bay cage? Preferably something like 5-to-3 (5x 3.5" HDD's inside 3x 5.25" bays), or a 3-to-2?

 

Keep in mind I'm Canadian, so Newegg.ca, NCIX.com, CanadaComputers.com, or Amazon.ca are my preferred retailers.

[Jarsky]

30 minutes ago, dalekphalm said:

Can anyone recommend a reasonably priced 5.25" HDD bay cage? Preferably something like 5-to-3 (5x 3.5" HDD's inside 3x 5.25" bays), or a 3-to-2?

 

Keep in mind I'm Canadian, so Newegg.ca, NCIX.com, CanadaComputers.com, or Amazon.ca are my preferred retailers.

Not sure what your idea of 'reasonably priced' is, but I REALLY love the IcyDock MB455SPF I have - its been fantastic.

[dalekphalm]

14 minutes ago, Jarsky said:

Not sure what your idea of 'reasonably priced' is, but I REALLY love the IcyDock MB455SPF I have - its been fantastic.

http://www.newegg.ca/Product/ProductList.aspx?Submit=ENE&DEPA=0&Order=BESTMATCH&Description=MB455SPF&N=-1&isNodeId=1

 

$200 CAD - is not what I would call "reasonable" I was hoping more around the $50 range, hopefully. I'll look on ebay too for used ones and see the difference.

 

Edit: Just checked eBay - no Canadian or US sellers have them - only Overseas sellers, who are mostly charging more than NewEgg.ca does.

[Jarsky]

3 minutes ago, dalekphalm said:

http://www.newegg.ca/Product/ProductList.aspx?Submit=ENE&DEPA=0&Order=BESTMATCH&Description=MB455SPF&N=-1&isNodeId=1

 

$200 CAD - is not what I would call "reasonable" I was hoping more around the $50 range, hopefully. I'll look on ebay too for used ones and see the difference.

 

Oh I think I listed the wrong one - this one here http://www.amazon.com/FatCage-MB155SP-B-EZ-Tray-Hot-Swap-Backplane/dp/B00DWHLFMA

MB155SP

 

 

[dalekphalm]

1 minute ago, Jarsky said:

 

Oh I think I listed the wrong one - this one here http://www.amazon.com/FatCage-MB155SP-B-EZ-Tray-Hot-Swap-Backplane/dp/B00DWHLFMA

MB155SP

 

 

http://www.newegg.ca/Product/Product.aspx?Item=N82E16817994155&cm_re=MB155SP-_-17-994-155-_-Product

$115 CAD - much more reasonable. I'll definitely think about that one. I'll also shop around - ICY DOCK makes some damn good products.

[scottyseng]

7 minutes ago, dalekphalm said:

http://www.newegg.ca/Product/Product.aspx?Item=N82E16817994155&cm_re=MB155SP-_-17-994-155-_-Product

$115 CAD - much more reasonable. I'll definitely think about that one. I'll also shop around - ICY DOCK makes some damn good products.

Have to say, I'm amazed that you can cram 5 3.5" drives into 3 5.25" bays. I got a three 5.25 to 4 3.5" drive convertor from Silverstone (not hot swap though) back in 2013, this one here: http://www.newegg.com/Product/Product.aspx?Item=N82E16811999164 It's not sold anymore though...I'm not even using mine after I got my SuperMicro 24 bay chassis...it just sits in the corner of my room. Sad to hear your SAS backplane died though.

[Jarsky]

27 minutes ago, scottyseng said:

Have to say, I'm amazed that you can cram 5 3.5" drives into 3 5.25" bays.

You should check out this one then, which I have installed above the MB455SP for my SSD data stores:

http://www.amazon.com/DOCK-ToughArmor-MB998SP-B-Metal-Backplane/dp/B00TL4US8K

 

Yes that is eight x 2.5" disks in one x 5.25" slot.

[dalekphalm]

Now I did find this on eBay:

http://www.ebay.ca/itm/5-25-to-5x-3-5-SATA-SAS-HDD-Cage-Rack-Hard-Driver-Tray-Caddy-with-Fan-Space-/182089476908?hash=item2a6560eb2c:g:oIwAAOSwq19XC8Cr

Just under $40 shipped. It would simplify the process, certainly.

 

I'll tinker with the SAS Expander bay some more before making a decision.

 

I still need to reinstall FlexRAID and Plex and recreate the network shares, etc. But I'm going to wait until I get my drive situation figured out first.

[leadeater]

9 hours ago, Jarsky said:

You should check out this one then, which I have installed above the MB455SP for my SSD data stores:

http://www.amazon.com/DOCK-ToughArmor-MB998SP-B-Metal-Backplane/dp/B00TL4US8K

 

Yes that is eight x 2.5" disks in one x 5.25" slot.

Damn I'd love one of these with a mini SAS/mini SAS HD connector.

[alex75871]

This is what I would change in your setup:

• Replace the 15k drives with WD RE or SE (enterprise grade SATA drives 7200 RPM)
• Buy WD Red drives for your storage. Reds are rated for 24/7 operation and RAID
• Get a server-grade motherboard. They are not that expensive and support ECC RAM
• ECC RAM. Seriously. Just do it <3
• Run FreeNAS OS
• Use a enterprise grade SSD for L2Arch, just because

This is just my personal opinion.

 

Also that cooler is seriously overkill haha, nothing wrong with that

[dalekphalm]

1 minute ago, alex75871 said:

This is what I would change in your setup:

• Replace the 15k drives with WD RE or SE (enterprise grade SATA drives 7200 RPM)
• Buy WD Red drives for your storage. Reds are rated for 24/7 operation and RAID
• Get a server-grade motherboard. They are not that expensive and support ECC RAM
• ECC RAM. Seriously. Just do it <3
• Run FreeNAS OS
• Use a enterprise grade SSD for L2Arch, just because

This is just my personal opinion.

 

Also that cooler is seriously overkill haha, nothing wrong with that

So, you suggest I build an entirely new storage server?

 

Mhmm... Or, I could continue to use all the parts I have, at no extra cost

 

I do plan on eventually doing a FreeNAS build, but that's probably gonna cost $1500+ after buying the components and HDD's.

 

The 15K drives I got for free, and threw in the server for fun, because why not? I'm not going to pay money to replace them with RE or SE drives that cost several hundred per drive, compared to the zero dollars worth of my own drives.

 

I don't mean to sound harsh, but I already know all of the suggestions you're offering (FYI, ECC RAM is mainly useful in FreeNAS - if using other OS's for file servers, the difference it makes is significantly less - FreeNAS loves ECC RAM because of the nature of the bitrot protection it has).

 

If I had $2000 to spend on a hotswap storage server case, ECC RAM, an HBA Card (Because, as you know, my SAS Card would be useless in FreeNAS, doubly so since you cannot flash it to IT mode), and a ton of enterprise or NAS grade HDD's, then I sure as hell would But that would be a waste of money, when I can spend that money on better things right now instead. My server is hella McGyver'd, but it works.

[alex75871]

5 minutes ago, dalekphalm said:

So, you suggest I build an entirely new storage server?

 

Mhmm... Or, I could continue to use all the parts I have, at no extra cost

 

I do plan on eventually doing a FreeNAS build, but that's probably gonna cost $1500+ after buying the components and HDD's.

 

The 15K drives I got for free, and threw in the server for fun, because why not? I'm not going to pay money to replace them with RE or SE drives that cost several hundred per drive, compared to the zero dollars worth of my own drives.

 

I don't mean to sound harsh, but I already know all of the suggestions you're offering (FYI, ECC RAM is mainly useful in FreeNAS - if using other OS's for file servers, the difference it makes is significantly less - FreeNAS loves ECC RAM because of the nature of the bitrot protection it has).

 

If I had $2000 to spend on a hotswap storage server case, ECC RAM, an HBA Card (Because, as you know, my SAS Card would be useless in FreeNAS, doubly so since you cannot flash it to IT mode), and a ton of enterprise or NAS grade HDD's, then I sure as hell would But that would be a waste of money, when I can spend that money on better things right now instead. My server is hella McGyver'd, but it works.

ECC provides overall better system stability. For example you won't get a BSOD if your RAM decides to bit flip. But I agree, ECC is unnecessary for a home file server. It all depends on your use case, and how much you love your data.

 

You could start with a server mobo and RAM and then slowly replace your drives as they expire

[dalekphalm]

Just now, alex75871 said:

ECC provides overall better system stability. For example you won't get a BSOD if your RAM decides to bit flip. But I agree, ECC is unnecessary for a home file server. It all depends on your use case, and how much you love your data.

 

You could start with a server mobo and RAM and then slowly replace your drives as they expire

ECC can indeed provide additional stability, but I'm not having any stability issues, so it's not exactly something I care about right now. And a bit flip won't necessarily cause a BSOD - it entirely depends on the situation. In most cases, you probably won't even notice when a bit flip happens, except a file becomes corrupted, or there's a slight amount of artifacting in an image file, etc.

 

No need to start with a server mobo and RAM. Pointless and unnecessary expense. When I'm ready to do a full replacement, I'll be getting all the proper parts. Also switching drives as they die, can sometimes work. But it has to be planned very carefully. Again, when the time comes to do a replacement, I'll be looking most likely at Red Pro's or SE drives, depending on how many total drives I settle on.