LTT Storage Rankings

[unijab]

Did you run wdidle3 on those WD Green drives?

[NWArkGuy]

Did you run wdidle3 on those WD Green drives?

 

no, not yet.  I had a drive go bad while still under warranty a couple of months ago and I just reinstalled the drive in earlier today. 

 

I have not had time yet to reload some of the software on this box yet to get it back where it was at.

 

Fortunately, I did not lose any data that I could not live without.

[unijab]

no, not yet.  I had a drive go bad while still under warranty a couple of months ago and I just reinstalled the drive in earlier today. 

 

I have not had time yet to reload some of the software on this box yet to get it back where it was at.

 

Fortunately, I did not lose any data that I could not live without.

 

are you keeping an eye on their Load Cycle Count?

Depending on their age, the count may be unusually high if you didnt wdidle3 them.

[aznmonk]

This is my first server and the raid 6 is still initializing in the background.

 

Hardware
CASE: Fractal Design Define R4
PSU: EVGA SuperNOVA 750 G2
MB: ASRock E3C224-4L 
CPU: Intel i3-4130
HS: Stock Intel heatsink
RAM: Kingston KVR16E11/8 8GB
RAID CARD 1: LSI MegaRAID SAS 9271-8i

SSD: Samsung 840 EVO 120GB
HDD 1: 8x 6TB WD RED WD60EFRX

HDD 2: 1x 3TB Seagate ST3000DM001

Software and Configuration:
The server is running a HW raid 6 with all the WD RED Drives on Debian Wheezy 7.7. The single Seagate drive is connected to the motherboard sata port.

Usage:
This will primarily used as media storage center with MySQL to store XBMC/Kodi database. It will also be a torrent box for myself and SFTP enable for a few friend and family members.

Backup:
Currently I only have 2 external seagate 3TB drive as backup.

Additional info:
The motherboard has 5 on-board lan, where one is dedicated for ipmi. The 4 other NICs will be bonded later on when I get a new network switch. I also has a APC USP which isn't connected yet.

Photo's:

[MrBucket101]

CLICK HERE FOR ORIGINAL POST

CLICK HERE FOR UPDATE 2

CLICK HERE FOR UPDATE 3

 

 

It's upgrade time!!!
 
Hardware

CPU: Intel Xeon E3-1220 (Sandy Bridge)
Mobo: Asus P8B WS C206
Case: Norco RPC-4224
RAM: Crucial 16GB (2x8GB) DDR3 1600 ECC Unbuffered
GPU: Sapphire Radeon HD5450
PSU: Corsair HX850
OS Drive: OCZ Vertex 4 128GB SSD
Storage HDD: 9x 3TB Seagate Barracuda ST3000DM01, 2TB WD, 320GB WD Black, 120GB Hitachi
RAID: LSI MegaRaid 9260-8i w/ BBU
 
Software and Configuration:
 
OS: Ubuntu 14.04 LTS

 
I finally got around to upgrading my server and I wish I could say it's a doozie (installing it all sure felt tough)...but it's not in the grand scheme of things lol
 
I purchased a new case, the Norco RPC-4224, a used HP SAS Expander, 8 SFF8087->SFF8087 (SAS) cables, and an old ancient decrepit HP P410. I got everything off ebay, and I was pleasantly surprised with the cost savings. The case was nearly $100 off, and was still in the MFG box unopened with the plastic seals still on everything.
 
Some of you may be wondering why on earth I would buy an old HP P410 raid card, especially when I have a much nicer LSI-9260. Well HP in their infinite wisdom, only allows you to upgrade the firmware of the expander from specific genuine HP hardware. So I had to buy the card just to upgrade the firmware on the expander. The seller was unsure of the cards current firmware, so I took it upon myself to make sure I could upgrade the card to the latest firmware. The reason I needed to upgrade was FW >= 1.52 enabled 3gbps/6gbps versus the previous 1.5gbps/3gbps link speeds. So upgrading the firmware was a definite priority...and now I've got an old P410 lying around. I may throw it in my gaming rig and run some sort of SSD RAID0. if that will even work, who knows lol.
 
If anyone is interested in the HP SAS expander, and the terrible firmware upgrade experience, read the next paragraph. Otherwise skip it.
 
Things got a bit hairy when I was trying to actually upgrade the firmware on the card. In order to upgrade the firmware you HAVE to use an HP provided firmware upgrade DVD. The only problem is that HP only allows users with a valid warranty to access the URL to download the .iso. At this point I'm pulling my hair out. I managed to figure out the exact file name of the .iso, and thanks to a few generous folks on reddit. I was able to get a copy of the official firmware update .iso. But wait!!! it gets better, HP prevented booting of the firmware update DVD unless you were using a genuine HP Proliant server...which I don't own and didn't plan on. After a bit of googling, I found out there is a plain text file containing blacklisted hardware ids. I blanked out the blacklist file, and after a couple errors I was able to boot into the DVD. At this point, the firmware upgrade went smoothly and took a few minutes.
 
I now had a cheap 9 port 6gbps SAS expander, yay!!
 
I took apart my old server case, pre-wired the backplane and SAS cables, and then began to install everything in the new enclosure. Everything went off without a hitch, and I was up and running again a few hours later.
 
Now that I had a case capable of holding more than 8 hard drives, I had to test it out!
 
I put the 8 drives from my server into the top 8 drive bays, and booted up with no issues. I had an extra 3TB drive laying around that I was using as a spare. I put the drive in each bay on the other 4 backplanes and just verified that everything worked.
 
Last month I experienced 3 (yeah thats right THREE) drive failures. They were all from my original batch I purchased, and each drive failed staggered a week from the previous failure. If It wasn't for me having a spare, I would have been screwed!!! But I got all the drives replaced under warranty, and I still have my spare drive. Due to the fact that I still have 2TB of free space left on my array, I put the spare drive in my new enclosure, and configured it as a global hot spare. I'll add it to the array when I get down to a couple hundred gigs of space. Till then I want my piece of mind, I'm positive another drive will fail on me soon. Learned my lesson about buying drives at the same time all at once.
 
In summary, I now have a total space of 29TB...and my next upgrade post should only be a quick paragraph saying...put in new drive, expanded array

And of course, here are some quick pictures I took. I was in a hurry to get it assembled, and forgot to take pics along the way. If you need to see the inside of the case just let me know. but I assure you its all there!
 

 

Spoiler

SERVER---
Server IP: 10.254.6.250
Server Name: SERVER
OS name: Linux
OS Version: 3.13.0-43-generic
OS Architecture: x86_64
Driver Name: megaraid_sas
Driver Version: 06.700.06.00-rc1
Application Version: MegaRAID Storage Manager - 14.08.01.02

HARDWARE---
Controller: LSI MegaRAID SAS 9260-8i(Bus 3,Dev 0)
Status: Optimal
Firmware Package Version:12.14.0-0167
Firmware Version: 2.130.393-2551
BBU: YES
Enclosure(s): 1
Drive(s): 9
Virtual Drive(s): 1

BBU---
BBU Type: IBBU
BBU Status: Optimal

Enclosures---
PRODUCT NAME TYPE STATUS
HPSASEXPCard Ses OK

Drives---
CONNECTOR PRODUCT ID VENDOR ID STATE DISK TYPE CAPACITY POWER STATE
Port 0 - 3 ST3000DM0011CH1 ATA Online SATA 2.729 TB On
Port 0 - 3 ST3000DM0011ER1 ATA Online SATA 2.729 TB On
Port 0 - 3 ST3000DM0011CH1 ATA Online SATA 2.729 TB On
Port 0 - 3 ST3000DM0011CH1 ATA Online SATA 2.729 TB On
Port 0 - 3 ST3000DM0011CH1 ATA Online SATA 2.729 TB On
Port 0 - 3 ST3000DM0011ER1 ATA Online SATA 2.729 TB On
Port 0 - 3 ST3000DM0011CH1 ATA Global Hot Spare SATA 2.729 TB Powersave
Port 0 - 3 ST3000DM0019YN1 ATA Online SATA 2.729 TB On
Port 0 - 3 ST3000DM0019YN1 ATA Online SATA 2.729 TB On

Virtual Drive(s):---
TARGET ID NAME CAPACITY STATE RAID LEVEL MegaRAID RECOVERY
0 - 16.371 TB Optimal RAID 6 NO

 

 
 

 

also, if anyone has a spare server rack they don't want, I'll take if off your hands

[jmannik]

Hardware
CASE: Silverstone DS380
PSU: Silverstone ST45SF-G
MB: Supermicro A1SRi-2758F
CPU: Intel Atom C2758
HS: Stock Supermicro heatsink
RAM: 16GB DDR3 SODIMM 1333
RAID CARD 1: HP Smart Array P410 512MB With BBU
SSD: Kingmax SMP35 240GB
HDD 1: 5x 2TB Western Digital Green


Software and Configuration:
The server is running ESXi 5.5 and currently hosts 3 virtual machines with more to be added once the RAM is upgraded.

Usage:
I use the server for storage of audio recordings from gigs (24 individual channels and a stereo channel, all in raw WAV audio) and also to store TV Shows and Movies for watching on the media center pcs, the storage server is running on Windows Server 2012 R2. Two other virtual machines run linux, One runs Turnkey MineOS, the other Runs Ubuntu server to run the Mumble server.

Backup:

The server is backed up via an LTO 2 Tape drive on a workstation set up for backup running VEEAM

Additional info:
This server was built specifically for low power usage while still being powerful enough

Photo's:

To be Added once I return home and can take pictures (currently away for work).

[alpenwasser]

- snip -

 

- snip -

- snip -

 

Very nice, thanks for your submissions, and welcome to the list (and the forums ).

 

- snip -

 

Oh, how nice of HP to try to lock down their hardware...

And of course, they still fail, but they do manage to make our lives miserable.

Nice upgrade though

[jmannik]

Very nice, thanks for your submissions, and welcome to the list (and the forums ). Oh, how nice of HP to try to lock down their hardware...And of course, they still fail, but they do manage to make our lives miserable.Nice upgrade though

My hp raid card is stuck a firmware version 3.66 because newer firmware doesn't play nice with the supermicro motherboard... I'm currently putting money aside for an Areca 8 port sas card with bbu so I can replace the hp. It's not the route I wanted and will be a pain to back up the array data and switch cards but hp are a pain so I've got no other choice (I haven't tested the latest firmware for the hp yet, no sign of new firmware for the supermicro yet either)

Edit: also thanks for the welcome

[looney]

parts for my new server should arrive tomorrow excluding 4 2.5" drivebays, 2 4TB HDD, 1 USB stick. 

[MrBucket101]

My hp raid card is stuck a firmware version 3.66 because newer firmware doesn't play nice with the supermicro motherboard... I'm currently putting money aside for an Areca 8 port sas card with bbu so I can replace the hp. It's not the route I wanted and will be a pain to back up the array data and switch cards but hp are a pain so I've got no other choice (I haven't tested the latest firmware for the hp yet, no sign of new firmware for the supermicro yet either)

Edit: also thanks for the welcome

You can download the firmware for just about any HP device from their website. If you can find the firmware you need, I'll happily share the upgrade .iso if you need it

[jmannik]

You can download the firmware for just about any HP device from their website. If you can find the firmware you need, I'll happily share the upgrade .iso if you need it

I have all the firmware and can run the higher versions of the firmware in my main PC, just not on the supermicro motherboard, it gets stuck in a BIOS loop

[MrBucket101]

I have all the firmware and can run the higher versions of the firmware in my main PC, just not on the supermicro motherboard, it gets stuck in a BIOS loop

bios upgrade on the supermicro board perhaps?

Does the new firmware provide any additional functionality? 

 

Also, instead of purchasing an 8-port RAID/HBA you might consider a lower-end RAID card and an SAS expander instead. That's what I went with, and I like to think I saved a bit of cash doing it that way.

[jmannik]

bios upgrade on the supermicro board perhaps?

Does the new firmware provide any additional functionality?

Also, instead of purchasing an 8-port RAID/HBA you might consider a lower-end RAID card and an SAS expander instead. That's what I went with, and I like to think I saved a bit of cash doing it that way.

Support for larger drives and stability/functionality updates and fixes. There are no updates for the system board BIOS as of yet but it seems to be a common issue between supermicro boards and HP smart array cards. I don't want to downgrade in raid card, also an expander is not an option in this case, it only has a mitx board and can't take full height cards unless you lose one drive bay. I'm looking at getting an Areca 1883 with battery backup and getting the remaining 3 drives I need to fill the case.

Also cash saving is not the aim here, reliability and performance are.

[alpenwasser]

My hp raid card is stuck a firmware version 3.66 because newer firmware doesn't play nice with the supermicro motherboard... I'm currently putting money aside for an Areca 8 port sas card with bbu so I can replace the hp. It's not the route I wanted and will be a pain to back up the array data and switch cards but hp are a pain so I've got no other choice (I haven't tested the latest firmware for the hp yet, no sign of new firmware for the supermicro yet either)

Edit: also thanks for the welcome

 

Damn, that sounds annoying.

 

parts for my new server should arrive tomorrow excluding 4 2.5" drivebays, 2 4TB HDD, 1 USB stick.

Yay!

[looney]

To whom this my be of interest, >here< is the build log for my new server

[Tim Drake]

 

 

How the HELL have you used 70TB?! I'm guessing you record a lot of max quality lossless videos?

[MrBucket101]

In summary, I now have a total space of 29TB...and my next upgrade post should only be a quick paragraph saying...put in new drive, expanded array

 

also, if anyone has a spare server rack they don't want, I'll take if off your hands

Mini update2

 

I purchased 2 128-gb 840 EVO's off amazon.

 

I set them up in RAID 1, and cloned the OS from my previous SSD onto the array.

 

I then took my old SSD, and I will be using it as a SSD Cache for my RAID 6. The hardware license key is in the mail currently. (Not sure what the rules are for SSD Cache drives, do they count?)

 

Funny story if you'd like some lolz. I guess I hadn't inserted one of my SSDs all the way, shortly after I had closed the lid to the case. The alarm went off and from the lights I could tell an SSD had dropped out. In my last update, I had configured an extra 3TB drive as a GLOBAL hot-spare. So as soon as I closed the lid, the 3TB hot spare went into action rebuilding a 111GB RAID1 that didn't even have a partition table on it. Needless to say, I booted into the webbios, blew up the RAID1, and retagged the hotspare as a DEDICATED hot-spare to prevent that from happening again, and reconfigured everything again lol

 

After that I just cloned the OS partition from my solo SSD, and then reinstalled the bootloader. All in all, about 30 minutes of upgrades including the hot-spare debacle lol.

 

SERVER---

Server IP: 10.254.6.250

Server Name: SERVER

OS name: Linux

OS Version: 3.13.0-44-generic

OS Architecture: x86_64

Driver Name: megaraid_sas

Driver Version: 06.700.06.00-rc1

Application Version: MegaRAID Storage Manager - 14.08.01.02

HARDWARE---

Controller: LSI MegaRAID SAS 9260-8i(Bus 3,Dev 0)

Status: Optimal

Firmware Package Version:12.14.0-0167

Firmware Version: 2.130.393-2551

BBU: YES

Enclosure(s): 1

Drive(s): 12

Virtual Drive(s): 2

BBU---

BBU Type: IBBU

BBU Status: Optimal

Enclosures---

PRODUCT NAME TYPE STATUS

HPSASEXPCard Ses OK

Drives---

CONNECTOR PRODUCT ID VENDOR ID STATE DISK TYPE CAPACITY POWER STATE

Port 0 - 3 ST3000DM0011CH1 ATA Online SATA 2.729 TB On

Port 0 - 3 ST3000DM0011ER1 ATA Online SATA 2.729 TB On

Port 0 - 3 ST3000DM0011CH1 ATA Online SATA 2.729 TB On

Port 0 - 3 ST3000DM0011CH1 ATA Online SATA 2.729 TB On

Port 0 - 3 ST3000DM0011CH1 ATA Online SATA 2.729 TB On

Port 0 - 3 ST3000DM0011ER1 ATA Online SATA 2.729 TB On

Port 0 - 3 ST3000DM0011CH1 ATA Dedicated Hot Spare SATA 2.729 TB On

Port 0 - 3 ST3000DM0019YN1 ATA Online SATA 2.729 TB On

Port 0 - 3 ST3000DM0019YN1 ATA Online SATA 2.729 TB On

Port 0 - 3 SamsungSSD840 ATA Online SATA 111.281 GB On

Port 0 - 3 SamsungSSD840 ATA Online SATA 111.281 GB On

Port 0 - 3 OCZVERTEX4 ATA Unconfigured Good SATA 118.719 GB On

Virtual Drive(s):---

TARGET ID NAME CAPACITY STATE RAID LEVEL MegaRAID RECOVERY

0 Share 16.371 TB Optimal RAID 6 NO

1 Ubuntu_OS 111.281 GB Optimal RAID 1 NO

 

 

 

All the blinky lights 3rd bay from the bottom left has the SSD Cache drive in it

[looney]

 

[Shaqalac]

 

 

So you just went from 48TB of raw HDD storage to, I presue, 101TB of what I presume is formatted storage?

[looney]

So you just went from 48TB of raw HDD storage to, I presue, 101TB of what I presume is formatted storage?

yep.

[Atomika]

This might sound stupid but what the hell do you guys even store here? 

 

Like all the movies in the world or what?!?!?

[handruin]

This might sound stupid but what the hell do you guys even store here? 

 

Like all the movies in the world or what?!?!?

 

For me, I built my NAS (see signature) as a learning device to get better with ZFS management (personal and job reasons).  In addition to that I use mine to backup all my home PCs to it using the local (free) features of CrashPlan.  My fiancee's PC is a photo-editing workstation for her personal business that has Terabytes of Canon RAW files from the work she does.  Those get backed up to my (NAS 5+TB and growing) and also to CrashPlan cloud as an off-site protection.  The cloud backup takes considerably longer so having a local option helps to stage the backups as we wait for the cloud to sync.

 

My personal workstation also has a couple TB of Canon RAW files for my own personal photo hobby along with other content.  I also have a large collection of FLAC music that I've ripped from my personal CD collection over the years.  I'm also working on ripping all my DVD's and Bluray into mp4 files to serve up to my TV via Plex.  My workstation probably qualifies for this thread in storage space but since it's not a dedicated storage server I can't list it.  :-)  I also run multiple Linux VMs and I plan to move those over to my NAS to reduce volatility from my workstation.  I host some basic gaming servers for friends and mumble chat server on my Linux VM via a domain name I registered.  Those will eventually get hosted on the same machine I use as a NAS since it has decent cpu power.

 

I'm working on revamping my old NAS (8 x 1.5TB | Dell Perc 6I | Server 2012 | AMD 4600+) to ship off to my dad's house so I can replicate my data there and also give him a place to backup some data locally.  I haven't started that project yet and not sure how I want to manage that.  One of the 1.5TB drives failed recently and I can't find a replacement to match.  I plan to convert the system into Linux with ZFS and make the remaining 7 into its own pool for slower storage.  I'll possibly buy a few 4-6TB drives to add on to that system in a different ZFS pool.

[MrBucket101]

This might sound stupid but what the hell do you guys even store here? 

 

Like all the movies in the world or what?!?!?

I have a very large media collection that I've built up over the past 10 years. I have around 1300 Movies, 150 of which are DVD quality, with the rest primarily being 1080p. The same goes for TV shows, except I rip those in 720p for size constraints. I tend to wait till shows are over, and I'll buy the bluray off craigslist or ebay for cheap, and rip it to my server. I love binge watching.

 

like @handruin, I also run quite a few VM's along with a database server.

 

I also have a VPN network established between me and the rest of my family. If anyone wants access to my library, I simply make them purchase a chromecast and a mikrotik router. I'll setup the router for direct VPN access, and then I show them how to use Plex Sync to access my library. I don't have a big enough Upload Connection to stream 1080p, so its really the only option unless they want to deal with crap tons of buffering lol

 

I don't mind paying for movies and tv shows, I just love being able to access it whenever/wherever I want.

 

My server has also provided me with valuable experience that I'm able to put on my resume. Over the past 8 years I have forced myself to learn linux and now I'm VERY proficient with it. A skill that is very applicable in the tech industry. This combined with my familiarity with hardware RAID and high capacity storage arrays; I believe it gives me a leg up over other interview candidates. Many times when I am interviewing for jobs I will be asked about my server. It may not be directly applicable to the job at hand, but I feel that it shows off some ingenuity and desire to learn. Another skill which is very applicable.

 

It all started as a pet project, but I'm very proud of what it's grown into.

[Atomika]

@MrBucket101 & @handruin

 

Thanks for explaining to me what you use your servers for

 

Now that I've read this, I might make my own NAS server now, as I have wanted to for a while. Might be able to clear my DVD / CD collection up a little as there's alot of space needed for all of this content.

Seeing that it looks good on your resume, I might learn how to use Linux but I own a VPS right now, that has Linux on it, accessed with PuTTY etc.

 

Anyways, I'll see what happens in the next year or so

[jmannik]

Hardware

CASE: Silverstone DS380

PSU: Silverstone ST45SF-G

MB: Supermicro A1SRi-2758F

CPU: Intel Atom C2758

HS: Stock Supermicro heatsink

RAM: 16GB DDR3 SODIMM 1333

RAID CARD 1: HP Smart Array P410 512MB With BBU

SSD: Kingmax SMP35 240GB

HDD 1: 5x 2TB Western Digital Green

Software and Configuration:

The server is running ESXi 5.5 and currently hosts 3 virtual machines with more to be added once the RAM is upgraded.

Usage:

I use the server for storage of audio recordings from gigs (24 individual channels and a stereo channel, all in raw WAV audio) and also to store TV Shows and Movies for watching on the media center pcs, the storage server is running on Windows Server 2012 R2. Two other virtual machines run linux, One runs Turnkey MineOS, the other Runs Ubuntu server to run the Mumble server.

Backup:

The server is backed up via an LTO 2 Tape drive on a workstation set up for backup running VEEAM

Additional info:

This server was built specifically for low power usage while still being powerful enough

Photo's:

To be Added once I return home and can take pictures (currently away for work).

Im finally back home now so I took a few pics of my server to add to the album above.