LTT Storage Rankings

[Spannerhands]

Very nice

Thanks...

No longer raid 5 using the new storage pool and virtual disk stuff very cool, you can make virtual disks using different parity settings which is just amazing.... things have moved on since windows NT lol...

[stevv]

Here's my entry .

 

Hardware

 

CPU:    Intel Xeon E3 1230v2 (ivy bridge)

MB:     Supermicro X9SCM-F-O

HS:      Stock Intel

RAM:   32GB DDR3 1600 ECC (Kingston)

PSU:    Corsair AX750

 

Case:          LIAN LI PC-V700X 

hotSwap:  2 x iStarUSA BPN-DE340SS
 

RAID:    Areca 1880i + BBU

SSD:     Intel 330 160GB SSD

HDD :   5x 2TB WD 20EARS 6x 4TB WD REDS (RAID 6)

 

Software and Configuration:
Server is running windows 7 on intel SSD.

Large Storage: RAID5 through Areca card.

All data is also indexed (faster search) and snapshotted (previous versions)

 
Usage:

Storage for data and multimedia content.  

Centralized Back up for data and os's of other systems.

Run VM's for variety of purposes.

 
Backup:
Data is backed up to online backup service (currently Backblaze, trying out Crashplan).

2014-07-30 - backup to Crashplan since Jan 2014.  

 
Additional info:
I run the server headless.  All KVM is through the IPMI and through network.
Why win 7?  Less costly online backup plan for non-server OS, and not using windows server specific functions at the moment.  (if I do, I'll probably run it as a VM.  I'm interested in serer 2012's new "self healing" file system, but that's for another thread.

 

Photo's:

FRONT:  I stripped that case of everything except fans and filters.  Everything meaning power buttons, headers lights, useless stuff. Server board layouts are the best.  +1 for those who notice the differences =).

 

BACK:  Did a quick job... noticed the red sata cable from motherboard to hotswap =).  The power cable just barely made it to the top.  I like the power cable location as opposed the the typical ATX being in two different places!  

 

 

 

INSTALLED:  In a Ikea bookshelf.  All drives in hotswap bay and their own activity lights (very useful).  Only power cable and three network cables (IPMI control, RAID card control, network for OS) to the server, no keyboard/mouse/video.  Three empty bays, so I threw SSD in there.

 

 

 

Case Closed.  APC  backup batteries on right work as one.  That battery was a lifesaver during hurricane sandy last year!  Used it for light for a week! 

 

 

Screenshot: don't mind the 1TB black drive, it might go back to my main system for game storage (darn humble bundles!)

 

Future Plans:

 

UPGRADING! from 10TB to 20TB.  B)

From these

 

To These

 

 

 

Maybe Build log of this?

 

edit: Oh yeah.. dont' forget the lighting.. plenty of UV lighting... =)

[AidanJMessenger]

Can someone please come and wipe the dribble from my floor? hahaha.

That's a damn fine system @stevv

[alpenwasser]

Oh la la, that is noice!

Build log of this?

I would not be against that.

[Raboo]

Here's my entry .

Very nice

[madcow]

2014.08 Added a second server. Motherboard and CPU upgrade on main box.

2013.11 Upgraded. Bigger case, more drives and more RAM.

 

Hardware:
 

Main

 

CASE: Supermicro SC846BE26-R920B
PSU: Built-in redundant 920W
MB: Supermicro X9SRL-F
CPU: Xeon E5-2630 v2
RAM: 64GB ECC
RAID CARD: LSI 9211-8i flashed to IT mode. Chassis backplane has built in SAS expanders.
SSD: Intel 710 100GB (OS), Intel 730 480GB (VM)
HDD: Seagate 4TB x16 (64TB raw)

 

Backup

 

CASE: Supermicro SC846BE26-R920B
PSU: Built-in redundant 920W
MB: Supermicro X9SCM
CPU: Xeon E3-1220
RAM: 16GB ECC
RAID CARD: LSI 9211-8i flashed to IT mode. Chassis backplane has built in SAS expanders.
SSD: Intel 710 100GB (OS)
HDD: Seagate 4TB x8, Hitachi 3TB x10 (62TB raw)

Software and Configuration:

Gentoo Linux

ZFS with 2 raidz2 vdevs (about 48TB usable on each box)

 

Usage:

ZFS, NFS, KVM hypervisor

 

See for details:

link

 

Photos:

Main above, backup pulled out below:

 

Main box:

 

Backup:

vm2 ~ # zpool listNAME   SIZE  ALLOC   FREE    CAP  DEDUP  HEALTH  ALTROOTz1      58T  44.7T  13.3T    77%  1.00x  ONLINE  -

vm1 ~ # zpool listNAME   SIZE  ALLOC   FREE    CAP  DEDUP  HEALTH  ALTROOTz2    56.2T  41.8T  14.5T    74%  1.00x  ONLINE  -

[RandomNOOB]

 

Upgraded. Bigger case, more drives and more RAM.

 

Original post: http://linustechtips.com/main/topic/21948-ltt-10tb-storage-show-off-topic/?p=662131

 

Hardware

CASE: Supermicro SC846BE26-R920B

PSU: Built-in redundant 920W

MB: Supermicro X9SCM

CPU: Xeon E3-1220

RAM: 32GB ECC

RAID CARD: IBM M1015 flashed to IT mode. Chassis backplane has built in SAS expanders.

SSD: Samsung 840 pro 256GB

HDD: 10x Seagate 4TB, 6x Hitachi 3TB, 4x Seagate 3TB

Software and Configuration:

Gentoo Linux

10x 4TB mdadm raid6

10x 3TB raidz2

Trying ZFS on Linux after seeing a number of them in this thread. Thanks for sharing.

 

I think we have a new nr.1

 

Gz m8!

[atv127]

Hardware
CASE: CM Storm Trooper
PSU: Cooler Master Silent Pro 850w
MB: Asus Sabertooth x58
CPU: Intel I7 930
HS: Noctua NH-L12

RAM: 24GB DDR3
RAID CARD 1: Adaptec AAR-1220SA
RAID CARD 2: Adaptec AAR-1430SA
SSD: Corsair Force 3 SSD 240GB
HDD 1: 1x 3TB WD 30EZRX
HDD 2: 2x 2TB WD 20EADS
HDD 3: 2x 2TB WD 20EURS
HDD 4: 1x 2TB WD 2003FYYS
HDD 5: 1x 2TB WD 20 02FAEX
HDD 6: 1x 2TB Hitachi HDS722020

HDD 7: 2x 1TB WD 10EAVS
HDD 8: 2x 750GB Seagate ST375064

 
Software and Configuration:
My computer is running Windows 7 and basic windows raid (striped). Mainly using raid cards for more sata ports.

My main array consist of 4 WD green and 2 WD black 2TB disks. 
 
Usage:
I use the storage for movies and series, i have media players around the house that access it.
I also use it for storing pictures and documents.
 
Backup:
I'M BADASS and dont need no Backup! My drives will never fail

 
Photo's:

 

 

 

[alpenwasser]

<...noice...>

1. Welcome to the forum!

2. Nice server you got there.

[Alexdaman]

Could you give use some benchmarks on that storage pool?

Because I'm doubting if I should use it or get a raid card.

 

I always saturate my Microserver on Gigabit .

[stevv]

Thanx guys!

 

Wish me safe journey on changing over disk! hope none of the disk fails during that time 

 

 

Can someone please come and wipe the dribble from my floor? hahaha.

That's a damn fine system @stevv

 

 

Oh la la, that is noice!


I would not be against that.

will probably write it up after I migrate my drives.

 

Very nice

[looney]

Hardware

CPU:    Intel Xeon E3 1230v2 (ivy bridge)

MB:     Supermicro X9SCM-F-O

HS:      Stock Intel

RAM:   32GB DDR3 1600 ECC (Kingston)

PSU:    Corsair AX750

Case:          LIAN LI PC-V700X 

hotSwap:  2 x iStarUSA BPN-DE340SS 

RAID:    Areca 1880i + BBU

SSD:     Intel 330 160GB SSD

HDD :   5x 2TB WD 20EARS

 

 

Hardware

CASE: Supermicro SC846BE26-R920B

PSU: Built-in redundant 920W

MB: Supermicro X9SCM

CPU: Xeon E3-1220

RAM: 32GB ECC

RAID CARD: IBM M1015 flashed to IT mode. Chassis backplane has built in SAS expanders.

SSD: Samsung 840 pro 256GB

HDD: 10x Seagate 4TB, 6x Hitachi 3TB, 4x Seagate 3TB

 

 

Hardware

CASE: CM Storm Trooper

PSU: Cooler Master Silent Pro 850w

MB: Asus Sabertooth x58

CPU: Intel I7 930

HS: Noctua NH-L12

RAM: 24GB DDR3

RAID CARD 1: Adaptec AAR-1220SA

RAID CARD 2: Adaptec AAR-1430SA

SSD: Corsair Force 3 SSD 240GB

HDD 1: 1x 3TB WD 30EZRX

HDD 2: 2x 2TB WD 20EADS

HDD 3: 2x 2TB WD 20EURS

HDD 4: 1x 2TB WD 2003FYYS

HDD 5: 1x 2TB WD 20 02FAEX

HDD 6: 1x 2TB Hitachi HDS722020

HDD 7: 2x 1TB WD 10EAVS

HDD 8: 2x 750GB Seagate ST375064

Have all been added (sorry it took so long )

[Alexdaman]

Just decided that im gonna destroy my raid 5 : those Green drives are crap... getting a lot of aborted commands and really slow speed, around 30 mb sec , its gonna take a while to move all my data (+6tb) to the other disks ,

 

tuesday im gonna buy 4 4Tb WD Red´s those should be raid- able right ? sigh...

[L.E.D]

Just decided that im gonna destroy my raid 5 : those Green drives are crap... getting a lot of aborted commands and really slow speed, around 30 mb sec , its gonna take a while to move all my data (+6tb) to the other disks ,

 

tuesday im gonna buy 4 4Tb WD Red´s those should be raid- able right ? sigh...

 

There should be no problem with those drives,

[MG2R]

Just decided that im gonna destroy my raid 5 : those Green drives are crap... getting a lot of aborted commands and really slow speed, around 30 mb sec , its gonna take a while to move all my data (+6tb) to the other disks ,

 

tuesday im gonna buy 4 4Tb WD Red´s those should be raid- able right ? sigh...

I'm running RAID5 myself with green drives and saturating my gigabit line with ease. I think your problem lies somewhere else...

[Appleby]

Sooo I kind of added my specs a while ago but everything was in such a mess I didn't have any pictures but here we go with a proper post.

Hardware
CASE: Logic Case -SC-4320 (Norcotek RPC4220)

PSU: Corsair TX750w
MB: Supermicro X8DTL-IF
CPU: Intel Xeon L5639
HS: Intel stock 1366
RAM: 12GB DDR3 1333 Corsair XMS
RAID CARD 1: Intel RS2BL080
RAID CARD 2: Intel RS2BL080
SSD: About to put a 160GB Intel x25m but not yet
HDD 1: 5x 2TB Seagete ST2000DM001
HDD 2: 4x 1TB WD Black
HDD 3: 1x 320GB WD Blue (OS drive to be replaced with SSD)
HDD 5: 1x 1TB WD Green 10EARS
 
Software and Configuration:
Server is running 2012 Essentials, 2TB are configured in RAID5 (want to add at least 3-7 drives more) which are set up for media storage and backups. WD Blacks are in RAID0 set up for ISCSI for various machines in my network. 1TB for messing around and WDS images.
 
Usage:
Storage is used for media streamed to HTPC, tablet etc. ISCSI is used to store photos to be processed meaning I have no spinning drives in any of my machines.
 
Backup:
All critical data is backed up to 5x 1TB drives in a second server.  
 
Additional info:
Its still a work in process. I need to add more storage and a RAID expander. I will also need to add in some form of faster networking probably 10Gbe or Infiniband but it all comes down to time and money! 
 
Photo's:

 

Empty Case
 

 

Motherboard
 

 

Some drives

 

 

Starting the build

 

 

More drives (still warm from being pulled from old server)

 

 

In its temporary home (gonna build a rack in the near future, I want to add a new switch, router, UPS)

 

[Raboo]

Sooo I kind of added my specs a while ago but everything was in such a mess I didn't have any pictures but here we go with a proper post.

Very nice server, need more storage

[alpenwasser]

@Appleby: I almost bought that M/B for my new server, went with an X8DT3-LN4F instead

though (found it for better price).

+1 for LGA1366, I just love that platform.

[Alexdaman]

I'm running RAID5 myself with green drives and saturating my gigabit line with ease. I think your problem lies somewhere else...

 

same thing was happening with my old server and the PERC 5/i. On a Lot of forums they are telling me to switch from those greens.

Im just going to invest 750euro's   and lets hope its fixed .

[MG2R]

same thing was happening with my old server and the PERC 5/i. On a Lot of forums they are telling me to switch from those greens.

Im just going to invest 750euro's   and lets hope its fixed .

There have been issues with the Green drives dropping out of RAID because of the way they are designed to react to bad sectors and read errors, but I've never heard of issues with general bad performance... ANyway, let us know if the investment solved the issue

[stevv]

Just decided that im gonna destroy my raid 5 : those Green drives are crap... getting a lot of aborted commands and really slow speed, around 30 mb sec , its gonna take a while to move all my data (+6tb) to the other disks ,

 

tuesday im gonna buy 4 4Tb WD Red´s those should be raid- able right ? sigh...

 

I also have 5 x Green drives (2TB) on an areca 1880i.  They have worked good for me (except when using it as VM's OS storage... get hiccups).  Is your card set with only one array dedicated to your WD Green drives?

 

benchmark with %70 full.. 

 

same thing was happening with my old server and the PERC 5/i. On a Lot of forums they are telling me to switch from those greens.

Im just going to invest 750euro's   and lets hope its fixed .

 

The issue might stem from the TLER settings in conjunction with the parking feature of the green drives.  You might be getting the 30MB/s because of the dropout and the raid is rebuilding again.

 

Some raid cards can handle it better.  The areca has the option to ignore it.  (even though I left it default)

 

The REDs with the following difference:

-Less aggressive head parking (no IntelliPark feature)

-Configurable Time Limited Error Recovery (TLER), with a default of 7 seconds

 

Should aleviate the issue your having with your RAID card.

[RandomNOOB]

Just decided that im gonna destroy my raid 5 : those Green drives are crap... getting a lot of aborted commands and really slow speed, around 30 mb sec , its gonna take a while to move all my data (+6tb) to the other disks ,

 

tuesday im gonna buy 4 4Tb WD Red´s those should be raid- able right ? sigh...

 

I run all green's and I'm seeing 700-800MB/s Read and Write. Don't think it's the drives

[Alexdaman]

just bit the bullet , bought 4 extra 4Tb RED´s , my raid5 array is almost empty , so in 3-4 hours I will be installing them , the raid initialization will take 6 days 5x4Tb... 

I am actually thinking to skip raid and go further with JBOD , I only use raid for the ease of having 1 huge drive. 

 

so Im replacing 8x 2tb green , with 5x 4Tb RED´s , curious how much my power usage will drop.

at least 4x 2Tb are going into my HP microserver , so its almost an 10tb machine

[MG2R]

just bit the bullet , bought 4 extra 4Tb RED´s , my raid5 array is almost empty , so in 3-4 hours I will be installing them , the raid initialization will take 6 days 5x4Tb... 

I am actually thinking to skip raid and go further with JBOD , I only use raid for the ease of having 1 huge drive. 

 

so Im replacing 8x 2tb green , with 5x 4Tb RED´s , curious how much my power usage will drop.

at least 4x 2Tb are going into my HP microserver , so its almost an 10tb machine

What, no RAID? You have offsite backup?

 

Damn it man, I really wanted to know what the performance was like.

[Alexdaman]

What, no RAID? You have offsite backup?

 

Damn it man, I really wanted to know what the performance was like.

 

I have a Microserver , the most important files are synced . 

 

Well I just did a 4 drive Raid 5 with quick initialization 383MB's Read and 260MB's write . looks good , need to empty a  4tb RED that I already had . Doing this via network , constant 98% Gigabit utilization . Nice!  

 

update: I build the new array 5x4tb RAID 5 = 16Tb usable . 

 

Also Found 2 Drive with bad clusters (green drives) so I will need to send them back to WD...

the array is building itself now. takes ages.... 

also the PERC 5i Lost a configuration and couldn't import it as an foreign config. weird ! (luckily its only one 2Tb drive)  R-studio FTW!

 

I once had a better week