LTT Storage Rankings

[RandomNOOB]

Word of advice - watch out raiding WD greens because you can run into your array dropping out.

 

Not happend to me in my 3 years of running greens in raid, Im running sw raid thow.

[.Jay]

Your running a big raid 5 with the greens? If you are and haven't had issues that's good then. I have had issues with multiple non-enterprise drives in the past but have been in the same field for many years. I even had an issue with velociraptors running in raid 10 on a dell 1950. Every 6 months I would have to remove a hot swap drive and let it rebuild.

One thing to remember is that its the rebuild also where you can run into issues. I'm doing some performance testing and its taking me over 15hours to rebuild. If something goes wrong during the rebuild you can looses your data also.

Just some fun facts that might be useful for some of ya.

[looney]

Not all greens have the problem, the older ones work fine in RAID, and a lot of software RAID solutions also have no problems with new green drives. (like FlexRAID)

[wpirobotbuilder]

im still going to school :P

Undergrad, or graduate?

 

Also, what do you use your iSCSI targets for?

[RandomNOOB]

Your running a big raid 5 with the greens? If you are and haven't had issues that's good then.

 

20 drive RAID6.

[looney]

Undergrad, or graduate?

 

Also, what do you use your iSCSI targets for?

I'm studying Industrial Automation (PLC programming)

 

The iSCSI targets are for my workstation and my laptop, they store VM's and general storage as I only have SSD's in my workstation.

[looney]

Well then, looks like I found this topic just in time for my finished server. :)

Note: This rig also does some BOINC computing, hence the overpowered CPU and

cooling solution.

For those interested, build log is here.

Hardware

CASE: Fractal Design R4 (modded)

PSU: Be Quiet 550 W

MB:  MSI Z77A-GD65

CPU: Intel i7 2600k (@ stock for the moment)

HS:  Custom water loop

RAM: 4 GB 2133 MHz Kingston HyperX

SSD: Intel 330 60 GB

HDD 1: 4x 2 TB Wester Digial RE4

HDD 2: 3x 3 TB Western Digital Red

Total Theoretical Capacity: 17 TB

Actually usable Capacity: 12 TB

 

Congrats on the Build of the Week !!

[alpenwasser]

Congrats on the Build of the Week !!

Thanks! :)

I just got back from putting some bread into the oven for breakfast (3 AM here) when

I was featured, perfect timing and very happy. :)

[looney]

still hoping that linus will post his server here :(

[L.E.D]

That'd be cool, In time, he will, maybe lol

[alpenwasser]

still hoping that linus will post his server here :(

Oh right, that is a pretty neat machine from what I recall. :)

[looney]

Oh right, that is a pretty neat machine from what I recall. :)

yeah, should get a nice place in the list.

[.Jay]

Not all greens have the problem, the older ones work fine in RAID, and a lot of software RAID solutions also have no problems with new green drives. (like FlexRAID)

 

 

Good to know - thanks

[.Jay]

looney what is your preferred drive count for around 5-9 drives in raid6?

[Rudde]

"Update" on my server. Switching out the DVB-C with a Quad DVB-C (on the way) to watch different channels on up to 4 devices at the same time. TVheadend and Softcam is at lest up an running, and I love the result of it, TV anywhere the recoding feature and everything about Tvheadend! :)

[raphaex]

I remember that to buid my array, took me almost 3 days. 8 x 2tb drives in RAID5

[.Jay]

I remember that to buid my array, took me almost 3 days. 8 x 2tb drives in RAID5

 

WOW - I never saw anything near that.  Just this alone should make people think about raid6 incase of rebuild failures.  Are you sure you didn't add drives and it took that long?  A new volume shouldn't take more then a day at the max.

[raphaex]

Yeah, i'm sure! The controller didnt help much!

 

INTEL RS2WC080

 

But yeah, was about 3 days.

[Ramaddil]

Total Storage 

111.0TB

 

 

Server Name

Titan

 

Hardware

CASE: Lian Li 343B
PSU: Corsair HX1000i
MB: Asus X370 Prime

CPU: Ryzen 1700X (Stock)
HS: Noctua NH-D14
RAM: 64GB DDR4 Corsair Vengance
RAID CARD 1: Adaptec 52445 28 Port Raid Card

SSD: Samsung 840 EVO 120GB
Enclosures: Startech 2/5.25 for 3 HDD (8) 

HD's: 

21x 2TB Western Digital (Red) WD20EFRS
8x 2TB Western Digital Greens (Various Models) 

1x 3TB Western Digital WD30EZRS

8x 6TB Western Digital WD60EFRX

 

Update 1. Added 7TB to Server

Update 2. Added 10TB More To Server7/12/14

Update 3. Replaced a Failed 2TB//Added Additional 2TB//Changed Operating System//CPU, Motherboard Upgraded

Update 4. Changed Raid card//Added Additional 20TB of Storage//Added 16Gb more of Ram (total 32gb)

Update 5. Added an additional 2TB

Update 6. Added an additional 6TB/Replaced MB 

Update 7. Added an additional 2TB///Changed to Platinum PSU HX1000i

Update 8. Added Additional 12TB//reconfigure backup solution to a Raid 5 solution with a New 9260-8I

Update 9. Upgraded Sever hardware, New Motherboard, Processor and Ram.

Update 10. Added 16TB to the Server

 

Software and Configuration:
My server is running Windows 10 I have one Raid 6 Array with 27 drives (27 X 2TB =54TB plus 2TB hotspare).  This array holds all my movies/tv shows and data.  I have one 3TB drive that I use for backup for my software that is synced to nightly with Syncback Pro on top of my cloud storage which is mentioned in the backup section.  Second Array is 8 x 6TB (48TB) for Backup Purposes). 
 
Usage:
I use my server as my media hub for my entire house and beyond Using Plex, it also serves as my security system monitoring cameras placed around the house, and FTP Server.
 
Backup:
My Server is connected to Crashplan which stores my data off site, And Synced to 29TB of USB Drives,
 

Photo's:

 

[alpenwasser]

Ha, I used to run a 343B! :)

Still have it, though it's currently in pieces (took it apart for modding and repairs

and haven't reassembled it yet).

[looney]

 

Total Storage 

28.0TB

 

 

Server Name

Titan

 

Hardware

CASE: Lian Li 343B

PSU: Corsair TX850

MB: Gigabyte GA-MA790GPT

CPU: AMD Phenom II X6 @3.2Ghz

HS: Noctua NH-D14

RAM: 16GB DDR3 1600 GSkill Ram

RAID CARD 1: LSI Megaraid 8708EM2

RAID CARD 2: LSI Megaraid 8708EM2

SSD: Crucial M4 64GB 

Enclosures: Startech 2 5.25 for 3 HDD (4) 

HD's: 

7x 2TB Western Digital (Red) WD20EFRS

2x 2TB Western Digital WD20EARS

2x 2TB Western Digital WD20EARX

1x 2TB Western Digital WD20EADS

1x 1TB Western Digital WD10FALS

1x 3TB Western Digital WD30EZRS

nice build, I like the 343B, it was also on my list but I decided on a norco instead.

Added to the list with 28TB

[alpenwasser]

nice build, I like the 343B, it was also on my list but I decided on a norco instead.

To be honest it's a bit too flimsy for my taste. When I filled mine up I could hardly

lift it anymore (lots of HDDs, two Aquadrives and a W/C loop). Not because it was too

heavy to lift, but because it would bend too much. The alu is quite thin (~1 mm IIRC),

which doesn't give it much stability.

There are Lian Li cases which are quite sturdy (i.e. use thicker alu, at least in some

parts of their construction), but for many of their cases they have been using these

thin sheets for the past few years. Of course this is somewhat understandable from an

economic point of view (Caselabs cases are as expensive as they are sturdy), but it's

a bit of a shame imho. They could at least reinforce them a bit with some steel here

and there. I'd rather have a non-pure alu case which is made sturdy by some steel

reinforcements than a flimsy pure-alu case. But that's just a personal preference of

course.

I still quite like the 343B though, it was my first high-end-ish case and will always

have a special place in my heart for that. :wub:

[Eric1024]

UPDATED 1/4/2013

Hardware
CASE: Supermicro CSE-822T-400LPB
PSU: Supermicro 400W (came with case)
MB: MSI Z77MA-G45
CPU: i7 3770k
HS: Noctua NH-L9i
RAM: 32GB Corsair Vengance 1600MHz
RAID CARD 1: LSI 9260-4i (using as an HBA)
SSD: Crucial m4 120GB

SSD2: Corsair Force GT 120GB

SSD3: Intel 530 series 240GB (ZIL, L2ARC)

HDD 1: 6x3TB WD Red

NIC: Mellanox connect-X2 something something (8Gb/s Infiniband HCA, got 2 of em on eBay)

 
Software and Configuration:

CentOS 6.5 running ZFS on Linux. Also I've grown out of the GUI, SSH for life  :D  

 

Array is pretty much the same.

[root@######### ~]# zpool iostat -v                                                   capacity     operations    bandwidthpool                                            alloc   free   read  write   read  write----------------------------------------------  -----  -----  -----  -----  -----  -----tank                                            3.19M  16.2T      0      0     30      4  raidz2                                        3.19M  16.2T      0      0      3      3    sdc                                             -      -      0      0     24      5    sdb                                             -      -      0      0     24      6    sdf                                             -      -      0      0     34      5    sdg                                             -      -      0      0     24      6    sdh                                             -      -      0      0     24      5    sdi                                             -      -      0      0     26      6logs                                                -      -      -      -      -      -  scsi-3600605b0052bc3b01a5a6b2c099f2d74-part1      0  1016M      0      0     26      0cache                                               -      -      -      -      -      -  scsi-3600605b0052bc3b01a5a6b2c099f2d74-part2    41K   194G      0      0      2      1----------------------------------------------  -----  -----  -----  -----  -----  -----

[root@hypno ~]# zfs listNAME   USED  AVAIL  REFER  MOUNTPOINTtank  1.53M  10.7T   272K  /tank

^ Note the pool is only empty because I just put this thing back together and I have yet to restore everything from my external drives.

 

 

Old:

 

Finally I can be on this list  :D

Hardware
CASE: Fractal Design Define Mini
PSU: Corsair TX650
MB: MSI Z77MA-G45
CPU: i7 3770k
HS: Stock (soon to be replaced with an NH-L9i from an itx server that I don't use anymore
RAM: 32GB Corsair Vengance 1600MHz
RAID CARD 1: LSI 9260-4i (using as an HBA)
SSD: Crucial m4 120GB (boot drive)

SSD2: Crucial m4 64GB (for ZIL and L2ARC)

HDD 1: 6x3TB WD Red

NIC: Mellanox connect-X2 something something (8Gb/s Infiniband HCA, got 2 of em on eBay)

 
Software and Configuration:
Server is running Linux Mint (weird OS choice I know, but my reasons were simple: I needed linux for ZFS, I had almost zero linux experience, and Mint is a very beginner-friendly distro

ZFS kernel module obviously. The 6 WD reds are in raid-z2.

 

 

 
Usage:
I use this to host a few minecraft servers, as well as a for storing media and backups. The IB interface is fast enough to record with fraps at 1080p & 60fps with no hickups.

 

At some point I also plan to use this computer for cluster rendering (just for fun). 
 
Backup:
None ): I don't have a fast enough network to do anything cloud based... (about 2Mb/s up with 4TB to backup... not really possible   :huh: )

BUT. I have hopes of making another server in the future that would mirror this one
 
Additional info:
Network benchmark over IB.

 

 

IB is amazing. I paid <100$ for the two HCAs and the cable.
 
Photo's:

 

 

 

IB card:

 

Edit: holy crap those pictures are big. Also I suppose I should wrap it up by saying that the whole server is 18TB unformatted and a little under 12TB formatted. 

[alpenwasser]

CASE: Fractal Design Define Mini

RAM: 32GB Corsair Vengance 1600MHz

Server is running Linux Mint (weird OS choice I know, but my reasons were simple: I needed linux for ZFS, I had almost zero linux experience, and Mint is a very beginner-friendly distro

ZFS kernel module obviously. The 6 WD reds are in raid-z2.

'

Yay, another Linux/ZFS build, and in a Fractal case as well! :)

I wouldn't exactly say Linux Mint is a weird OS choice. From what I've seen and read

it seems to be an excellent distro, well suited for beginners and advanced users as

well if they so choose (I'm running Arch, but that's a different story ;) ).

If I may ask: Why 32 GB of RAM? Are you running deduplication on the ZFS pool? That

uses a crapload of RAM, so if you're not running it yet it might actually be worth

looking into if you have some data which can be deduplicated (I don't, plus I only

have 4 GB of ram, about a third of which is usually used by ZFS).

Anyway, awesome little machine you have there. :)

[looney]

@Eric1024 nice build, added to the list with 18TB :)