Jump to content
looney

LTT Storage Rankings

Recommended Posts

Hay Everyone i Though I show You my Server ;)

 

And Sorry for my really bad English but is not my Native language

 

UPDATE 1

I Updated some of the specs due to failure and some thinks what i have planned what i changed i marked with *xxxxx*

 

Hardware
CASE: MS-Tech CA-0210 Midi Tower
PSU: *Corsair CS Series 850 Watt*
MB: Asus M5A78L-M/USB3
CPU: AMD Phenom II X6 T1100 Black Edition
HS: *Thermalright HR-02 Macho Rev. B Tower Kühler*
*RAM: 16GB DDR3 1600*
RAID CARD 1: Dell PERC 5/i
RAID CARD 2: Cheap No Name 4 Port Sata + 1 Pata
HDD 1: 5x 2TB Seagete ST2000DM001
HDD 2: 1x 2TB Hitachi HDS5C3020ALA632
HDD 3: 2x 500GB Hitachi (I don't remember the Model)
*SSD 1: 4x 250GB SanDisk SSD*

Total RAW Capacity :14,5TB
Total Usable Capacity : 12,1TB

Software and Configuration:

*I Use Synology DiskStation as Main System with Virtalbox and a Web Frontent to virtualise the Servers and the Test Clients etc I need.
For the Storage i have all 2 TB Drives in RAID 0 .
Then i have a Win 2012R2 VM for my AD and RIS and also Acronis Backup and Restore Server to back up all my PC´s to my 2x 500GB RAID 0.
All the VM run of 4x 250 GB SSD in RAID 0.
The main Use fort he Server is Data Storage and Backup Storage and VM Testing.
But in the Future I would Like to Change some Things oft he Hardware.


Usage:
The main Use for the Server is Data and Media Storage and Backup Storage also VM Testing.

Backup:
All my Drives are in RAID 0 and I have no Internal Backup but
I have 5x 5TB External HDD´s by my Fathers House for Backup the complet Server every 2 Month.

Additional info:
I'm using the AMD processor because it was a leftover from a previous PC, but i have many plans to this Machine.
Some Ideas i have are:

A Proper Fan Splitter
Another MoBo (Gigabyte GA-990FXA-UD5 AMD 990FX) and a PCIEx1 Graphics Card (1024MB Club 3D Radeon R5 230 Noiseless )
Matching 32GB DDR 1600MHz Ram
2 more Dell Perc 5/i + FanMod
1x ICY BOX IB-543SSK and 1x 545SSK
8 – 10x 2TB Seagete ST2000DM001
Change the Data raid from RAID 0 to RAID 5 or RAID 6

I Would Like to Hear you Thoughts to My System.
I know this Config is Very Special but is´t what I need!

Thanks for Reading ;)

Photo's:

 

Hear are some Pics i have taken While i redone the Cable Management

 

(Girlfrend for Scale)

 

Link to post
Share on other sites

snip

 

Think it will go something like this:

 

lehman_bros_1.gif

 

Will be fine for a good while but will end up crashing and burning.

 


Gaming Build:  i5 4690K, RX 480 8GB, 16GB G-Skill 2166MHz RAM, Samsung 840 Evo 250GB, MSI Z97 Gaming 3, 700W Cooler Master G-Series PSU, Windows 10 Pro (Build Thread).

Network Gear: Dell PowerEdge T110, Netgear ReadyNAS 104 (4x3TB), TP Link Gigabit 24 Port Switch, Sky Router, Asus Wireless Access Point.

Mobile Machine: Lenovo Thinkpad T420, i5-2520M,  4GB DDR3 1333MHz RAM, 500GB HDD, Windows 10 Pro

Other Tech: Surface Pro 4, Galaxy S8, Gear Fit 2, Moto G4, Xbox One, Xbox 360, PS3

Link to post
Share on other sites

@Sousuke and @MrBucket101

Don't forget i have an Offside Backup I do regularly

From 10 Drives i only lost 2 in 4 years and round about 100 GB of non Important Data.

My Important Data are Mirrored to a 4 TB Cloud on a Root Server

 

Was only kidding anyway, I thought that you would have offsite backups when you said you had no internal backups.

 

RAID 0 is known as Striping however other people in the tech industry call it Scary RAID as all you need is a small error or a single drive failure to bring the array down. However I am sure you knew that anyway :lol: I'd only risk RAID 0 on an SSD Array which is backed up nightly. 


Gaming Build:  i5 4690K, RX 480 8GB, 16GB G-Skill 2166MHz RAM, Samsung 840 Evo 250GB, MSI Z97 Gaming 3, 700W Cooler Master G-Series PSU, Windows 10 Pro (Build Thread).

Network Gear: Dell PowerEdge T110, Netgear ReadyNAS 104 (4x3TB), TP Link Gigabit 24 Port Switch, Sky Router, Asus Wireless Access Point.

Mobile Machine: Lenovo Thinkpad T420, i5-2520M,  4GB DDR3 1333MHz RAM, 500GB HDD, Windows 10 Pro

Other Tech: Surface Pro 4, Galaxy S8, Gear Fit 2, Moto G4, Xbox One, Xbox 360, PS3

Link to post
Share on other sites
Jep i know ist but the term Scary RAID is new to me ;)

When i get more 2 TB Drives i would Like to Change to RAID 5 or RAID 6 to be a little bit Saver.

But i´m happy its a little and nice Project wenn i have monny to throw away B)

And i´m allways out for new ideas for my Server

Link to post
Share on other sites

Are you sure you are using the consumer 3TB versions? Those are the drives we're referring to, ST3000DM001

 

You're quite right actually... I assumed that $employer went with their original plan to cheap out and populate our NetApp platform with the barracuda drives, looks like we're using ST3000NM0033 Constellation series instead... which is quite a relief :)


I'm on a horse...


Gaming Rig | Storage Server | Virtual Server | HTPC

Link to post
Share on other sites

 

 

Keep in mind Backblaze's methodology is extremely flawed, and I would not use it as a reliable reference. 

 

 

In the end, you're just as likely to get dead drives from any company, personally, I've only ever had 1 HDD fail on me and it was a WD Caviar Blue 1TB after 1 year. Does that mean I'll never buy WD again? Nope. I also have 4 Seagate NAS drives that have been running (perfectly) for a total of over 20,000 drive hours (about 9 months of 24/7 use for 2 and 7 months of 24/7 for the other 2) and none have died.

EDIT: No need to paraphrase! Alp's quote is now pasted here.

 

Keep in mind I've had something like 6ish Seagates fail the past year. 2 3TBs out of 5, and 3 2TBs out of 7, and 1-3 (1 for sure, i cant remember when exactly the other 2+ 1TB drives failed, could have been a year prior) 1TB drives.

 

Yes, I just had a second 3TB drive fail. Out of an array that has barely been used the past year, and is purely a backup array which is rsynced to from the main nas array once a day, which its only been doing for the past month or two. Before that, it literally wasn't doing anything.

 

If I could remember the total count of failed drives the past 15 years, it'd be pretty shocking. Its over 1 disk a year on average at this point. A few years ago, it was at about 10 disks, and around 1 disk a year on average. I just assumed it was bad luck, but at this point? I will not be buying another seagate for quite some time.

 

And before anyone asks, my computers are all on UPSs so they get decent power, the controllers for the two big arrays are LSI SAS2008 cards that likely aren't killing drives (can a controller card cause a drive to report uncorrectable sectors and reallocate hundreds or thousands of sectors?), everything is cabled decently with SAS breakout cables, temperatures are reasonable at around 35c all the time. The drives were also not all in the same system, or on the same UPS during the various failures.

 

Some of the drives suffered from seagates power management firmware bugs, causing heads to park and unpark many times a day, even when not in use apparently.

 

I just have no trust left to give them. I used to be a fan for many years. I bought nothing but seagates. Maybe in a few years when they figure out their problems I'll buy more, but so far it seems WD and HGST drives have better customer reviews. Sure, call me a statistical anomaly, but from my point of view, they are currently not worth buying. I think there's a reason Seagate's desktop drives are cheaper than almost everyone else's. You get what you pay for.

 

append: fun story, one of the failed 1TB disks is a previous RMA return. thats at least the second time I had a replacement fail on me. one brand new (from memex), the other an actual rma replacement.

Edited by Tomasu
Link to post
Share on other sites

-snip-

You're running the drives 24/7. They aren't meant for it.

I made the same mistake, but regardless, they should last more than 1yr 24/7. Pull up the smart info on your bad drives if you can and see how many hours they were powered on.

The consumer sea gate drives are only meant for light workloads. 8 hours a day, 5 days a week.

Linus is using the enterprise grade 6tb seagates in the storinator. If he trusts the enterprise drives then so can we. It's just a shame that seagates really cheaps out on their consumer drives. It seems as though they make profits off of consumers who dont want to pony up for WD Drives of the same size. Lots of corners are cut to meet that price point.

Link to post
Share on other sites

I am in the process of a heavy upgrade to my storage system. Instead of running everything virtualized on a single machine I'm going to have a single purpose storage server providing iSCSI over infiniband to a pair of small VMWare hosts. I've already built the 2 hosts which work well over teamed 4x port gigabit NICs but seriously cant wait for the infiniband cards to arrive.

 

The storage server itself is going to be substantially rejigged to provide a high speed SSD datastore and large HDD datastore for the vmware boxes and a NAS datastore for the rest of the storage. Estimating 4x 512GB SSDs, 4x 6TB enterprise drives (with 128GB cache SSD) and 6x 6TB NAS drives should do a nice job.

 

Will post build pics as it all comes together :) for now here's one of the VMWare esxi mini Hosts:

 

17638243562_4c5e372ed7_b.jpg

New VMWare ESXi Host by TJ Stamp, on Flickr


I'm on a horse...


Gaming Rig | Storage Server | Virtual Server | HTPC

Link to post
Share on other sites

The storage server itself is going to be substantially rejigged to provide a high speed SSD datastore and large HDD datastore for the vmware boxes and a NAS datastore for the rest of the storage. Estimating 4x 512GB SSDs, 4x 6TB enterprise drives (with 128GB cache SSD) and 6x 6TB NAS drives should do a nice job.

 

 

My god this is relevant to my interest. Sounds like a much more extreme version of what one of my friends was doing with NUC's and a storage server. I look forward to seeing more on this :) pictures, setup, configuration, etc...sounds really well planned out.

 

I'm just awaiting a few more drives to build my new array then will be posting an update to mine soon too - config has changed substantially from what's in the LTT 10TB+ list :P


Spoiler

Intel i7 3770K @ 4.6ghz | EVGA Z77 FTW | 2 x EVGA GTX1070 FTW | 32GB (4x8GB) Corsair Vengeance DDR3-1600 | Corsair H105 AIO, NZXT Sentry 3, Corsair SP120's | 2 x 256GB Samsung 850EVO, 4TB WD Black | Corsair 600T (White) | OCZ ZX 1250w | Samsung 28" 4K Display | Ducky Shine 3 Keyboard, Corsair M50 Mouse, MicroLab Solo 7C Speakers, Razer Goliathus Extended, X360 Controller | Windows 10 Pro | SteelSeries Siberia 350 Headphones

 

Spoiler

Corsair 400R, IcyDock MB998SP & MB455SPF, Seasonic X-Series 650w PSU, 2 x Xeon E5540's, 24GB DDR3-ECC, Asus Z8NA-D6C Motherboard, AOC-SAS2LP-MV8, LSI MegaRAID 9271-8i, RES2SV240 SAS Expander, Samsung 840Evo 120GB, 2 x 8TB Seagate Archives, 12 x 3TB WD Red

 

Link to post
Share on other sites

Looking awesome guys!

My Netgear ReadyNAS Is still going strong with 86% of the space still free ... won't be for long.


Gaming Build:  i5 4690K, RX 480 8GB, 16GB G-Skill 2166MHz RAM, Samsung 840 Evo 250GB, MSI Z97 Gaming 3, 700W Cooler Master G-Series PSU, Windows 10 Pro (Build Thread).

Network Gear: Dell PowerEdge T110, Netgear ReadyNAS 104 (4x3TB), TP Link Gigabit 24 Port Switch, Sky Router, Asus Wireless Access Point.

Mobile Machine: Lenovo Thinkpad T420, i5-2520M,  4GB DDR3 1333MHz RAM, 500GB HDD, Windows 10 Pro

Other Tech: Surface Pro 4, Galaxy S8, Gear Fit 2, Moto G4, Xbox One, Xbox 360, PS3

Link to post
Share on other sites

Update #1 Click Here

 

This is my first dedicated server/nas. Previously I had been serving the media and file sharing for my home network with my main desktop machine, it has a 17TB Flexraid setup that is still going but now only for redundancy. The new machine is running the latest FreeNAS with various plugins.

 

Specs below.

 

 

24TB Advertised, I am using a Raidz2 8 disk array, for 16.4TB Usable and can lose 2 drives before game over.

 

FreeNas is running off a 16GB USB 2.0 drive

 

CASE: Fractal Design R5 Black No window
PSU: Corsair CS450M GOLD Cert
MB: Supermicro X10SLL-F
CPU: Intel Xeon E3-1231V3 Haswell 3.4GHz 
HS: Noctua NH-U9S
RAM: 32GB Crucial DDR3 ECC 4x8GB
RAID CARD 1: LSI 9211-8i flashed with P16IT Firmware
HDD: 8x TOSHIBA PH3300U-1I72 3TB 7200 RPM

Fans: 2xFractal 140mm / 2x Noctua 120mm / 3x Noctua 140mm

 

P1010658_zps5om7dzjj.jpg

 

P1010687_zpsgsqd9gcj.jpg

 

P1010668_zpsadapckis.jpg

 

P1010671_zpsgnmiqcpe.jpg

Link to post
Share on other sites

@Cheatdeath : Nice idea, those extra fans on the back of the hard drive tray. Do you notice a difference in hard drive temperature?

Mine are 38 to 40°C (100-104°F) under load with 2 front fans (I ditched the default Fractal fans for 140mm BeQuiet SilentWings2) at 5V.

Based on Google's evaluation of the 100k+ hard drives in their datacenters those temps are perfect, but I'm worried what happens if I turn on 8 drives simultaneously instead of 2. (currently doing the burn-in 2 disks at a time while I wait for the NAS' motherboard to arrive)

Link to post
Share on other sites

@Cheatdeath : Nice idea, those extra fans on the back of the hard drive tray. Do you notice a difference in hard drive temperature?

Mine are 38 to 40°C (100-104°F) under load with 2 front fans (I ditched the default Fractal fans for 140mm BeQuiet SilentWings2) at 5V.

Based on Google's evaluation of the 100k+ hard drives in their datacenters those temps are perfect, but I'm worried what happens if I turn on 8 drives simultaneously instead of 2. (currently doing the burn-in 2 disks at a time while I wait for the NAS' motherboard to arrive)

 

The problem with the R5 is that the front intake fans don't work so good with the filter on and door closed. Adding the extra 2 120mm lowered my drive temps 5C. Drives are now 35C under load. Also there is not a factory mounting setup to add these fans, I had to use zip ties, which sounds fine till you try to do it lol, larger hands and small zip ties....sigh. Now that I think about it more I bet you could drill some holes and use the rubber Noctua things to mount the fans.

Link to post
Share on other sites

The problem with the R5 is that the front intake fans don't work so good with the filter on and door closed. Adding the extra 2 120mm lowered my drive temps 5C. Drives are now 35C under load. Also there is not a factory mounting setup to add these fans, I had to use zip ties, which sounds fine till you try to do it lol, larger hands and small zip ties....sigh. Now that I think about it more I bet you could drill some holes and use the rubber Noctua things to mount the fans.

I know there are no holes, the first thing I did after seeing your pics was to open up my own R5 and check for mounting holes.

Sounds like I'll be adding a couple of 120mm fans to mine too then. Seeing as I have a cat, I like to keep the filter on the front.

I'll probably drill some holes and use the plastic pins that BeQuiet includes. The only tricky part will be the alignment, I don't want the fan mounting points to interfere with the drive caddies.

Do you use low-profile screws to hold the drive caddies in place or didn't you bother with screwing those in? I have the original thumb screws in there right now and that feels a lot more secure than having the caddies move about.

P.S. : did you do anything about the optical bays? If you remove those covers you can see that there's a serious unfiltered gap around those, where dust can come in through the slits in the front-side and get in the case through the bays themselves.

I plan on using duct tape to close that gap even though I'll have positive pressure anyway.

Link to post
Share on other sites

I know there are no holes, the first thing I did after seeing your pics was to open up my own R5 and check for mounting holes.

Sounds like I'll be adding a couple of 120mm fans to mine too then. Seeing as I have a cat, I like to keep the filter on the front.

I'll probably drill some holes and use the plastic pins that BeQuiet includes. The only tricky part will be the alignment, I don't want the fan mounting points to interfere with the drive caddies.

Do you use low-profile screws to hold the drive caddies in place or didn't you bother with screwing those in? I have the original thumb screws in there right now and that feels a lot more secure than having the caddies move about.

P.S. : did you do anything about the optical bays? If you remove those covers you can see that there's a serious unfiltered gap around those, where dust can come in through the slits in the front-side and get in the case through the bays themselves.

I plan on using duct tape to close that gap even though I'll have positive pressure anyway.

 

The drive cages and thumb screw holes on my case do not line up and are not usable for me. Do you think this is a major problem for my drives? They feel secure and well mounted so I don't know. I have good positive pressure, probably going to pick up another hot swap bay off monoprice to match the top one though. 

 

And I feel your pain in the cat department, for some reason this is now the place to be for our cats....

Han%20Server_zpsvjbyac5p.jpg

Link to post
Share on other sites
And I feel your pain in the cat department, for some reason this is now the place to be for our cats....

Computers give off heat -> cats like heat -> cats like computers -> cats are on the Internet.

 

It all makes sense!

Link to post
Share on other sites

I'm terribly sorry for the delay, things have been rather busy in college in recent months.

 

-snip-

 

Updated. :)

 

-snip-

Nice, an actual server chassis! :D

 

 

-snip-

Updated your config accordingly.

 

 

-snip-

Nice and compact, welcome to the list!

 

I know this Config is Very Special but is´t what I need!

Hey, special systems is what this topic is about! :D

 

-snip-

Nice and clean, I like it.

 

 

Computers give off heat -> cats like heat -> cats like computers -> cats are on the Internet.

 

It all makes sense!

It is the law of the interwebz, indeed.


BUILD LOGS: HELIOS - Latest Update: 2015-SEP-06 ::: ZEUS - BOTW 2013-JUN-28 ::: APOLLO - Complete: 2014-MAY-10
OTHER STUFF: Cable Lacing Tutorial ::: What Is ZFS? ::: mincss Primer ::: LSI RAID Card Flashing Tutorial
FORUM INFO: Community Standards ::: The Moderating Team ::: 10TB+ Storage Showoff Topic

Link to post
Share on other sites

Time to join, methinks.  32.5TB
 
 
Tentagon
Hardware :
 
Case : Fractal Design Define R5
PSU : BeQuiet Dark Power Pro P9 650W
MoBo : Intel Server S1200V3RPS
CPU : Intel Core i3-4170 (ECC compatible)
Heatsink : BeQuiet Shadow Rock 2
Ram : 16GB (2x 8GB) Kingston ValueRAM ECC DDR3-1333
Network : on-board 1Gb
HBA : LSI 9211-8i flashed to IT mode
SSD : 2x Crucial MX200 250GB
HDD : 8x WD Red 4TB

Case fans : 4x BeQuiet SilentWings2 140mm, 2x BeQuiet SilentWings2 120mm
Power usage : 50-55W idle, up to 100W during heavy writing
 

Software and configuration :

OS : Freenas running from a 16 GB PNY stick
RAID Config : SSDs mirrored, HDDs in RAIDZ2
RAID hot spare : 2x 4 TB
File system : zfs
File system capacity :
- Total Advertised : 32.5 TB
- SSDs actually available : 222 GB
- HDDs actually available : 19.9 TB
 
 
Usage :
 
NAS for 1 PC and 2 laptops.
The SSDs are for personal and work-related data, where access/seek time matters most to me.
The HDDs are for backup of the SSDs, storage of software installers + ISO files and mostly video storage (full-size Blu-Ray rips)
 
 
Backup :
- SSDs are backed up to the HDDs (500GB share just for that purpose), as well as to 2 external 2.5" HDDs, one of which for off-site storage
- HDDs are backed up once to external HDDs. If that's not enough, I guess I'll have to rip my Blu-Rays yet again.
 
 
Photos :
 
 
post-139790-0-30072600-1434409890.jpg
 
 
 
post-139790-0-93993200-1434411052.jpg
 
 
 
Seeing as Fractal warned me to hook up no more than 3 fans to the integrated fan controller, I slapped my old Laptron FC5v2 in there. 
the 4 temperature sensors are on HDD 2, 4, 6 and 8 when counting from the top.
The white stuff around the fan controller is the edge of the duct tape.  The R5 has a huge unfiltered gap around the 5.25" bays, so I closed that off to prevent unfiltered air from being sucked in.
 
post-139790-0-55888800-1434409912.jpg
 
 
The disks in the Freenas menu
 
post-139790-0-56086900-1434409920_thumb.
 
 
 
The disks in Windows. 
The NAS is brand new and I still need to copy almost everything over, so for now I have both old NASes connected as well as the new one. 
Due to corruptions (one bad HDD probably) I already threw close to 2TB of Blu-Ray rips away.  I'm not even going to bother copying the other rips, I'll just rip everything again.
Those corrupt files are also why I built this NAS to begin with.  If it weren't for those, I'd have been using the 2Big and the EX4 for a while longer. 
 
post-139790-0-18411700-1434409929.jpg

Link to post
Share on other sites

Time to join, methinks.

 

Cool build man, I think we are on the same page when it comes to our Nas boxes lol. I see you went with the extra fans and looks like zip ties which is what I did also.

 

Have been extremely happy with my build as I am sure you will be, it has really been fun and I feel so much better about my data.

Link to post
Share on other sites

Cool build man, I think we are on the same page when it comes to our Nas boxes lol. I see you went with the extra fans and looks like zip ties which is what I did also.

 

Have been extremely happy with my build as I am sure you will be, it has really been fun and I feel so much better about my data.

Yeah, I went out and bought 2 more fans because of your build.  You didn't patent that idea, did you?

I didn't have trouble attaching them, probably because of the different mounts BeQuiet uses.  Those holes were easy to put a zip-tie through.

 

I might add a fan in the side panel over the weekend to get extra cooling for the card. 

I'm also considering putting a 1TB HDD in the top 5.25" bay to replace the backup share, but then it'd have 11 drives and would need a new name.

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now


  • Recently Browsing   0 members

    No registered users viewing this page.


×