LTT Storage Rankings

[Jonny]

On 21/03/2016 at 2:30 PM, Jonny said:

Original Post

LINK

 

Updated My Original Post.

[Jarsky]

On 4/1/2018 at 6:45 PM, Cynagen said:

If somebody could clarify if I qualify, I'll do up a full post with pics, but if I don't qualify, I'll show myself out until I can get on FreeNAS at some point in the future.

I don't think it really qualifies mainly because its an external eSata enclosure that rather than a SAS one which is a native extension of SAS. 

But this ranking hasn't been maintained in over a year since no one has taken it over, so it doesnt realy matter unless we kill this thread and start a new one

[kiddsupreme]

Hardware:
I had been wanting to add my name to list for a while now. I have the following:

 

• HDD
  • 2 x 8TB Seagate ST8000AS0002
  • 2 x 4TB Seagate ST4000DM000
  • 1 x 8TB Seagate ST8000VN0002
    • (All drives utilizing ext4, NON-RAID, JBODs)
• OS
  • Fedora 28 (64 bit of course)
    • ext4
    • Located on a Plextor 128GB SSD
• Motherboard
  • ASRock Z77 Extreme4
• Memory
  • 2 x 8GB G.Skill Memory (DDR3, 1600Mhz)
• CPU
  • Intel Core i5-2320
• Case
  • Corsair 900D

Usage

Multiple uses (File server, Plex Server, home to some VM's, etc)

[unijab]

On 9/30/2016 at 6:24 PM, unijab said:

Hardware

CASE:    Fractal Define R5

PSU:      EVGA SuperNOVA 650

CPU:      Intel core i5 8500

RAM:     32GB - G.SKILL Ripjaws V Series 

MB:       Asrock Z370 pro4

 

HBA CARD:     LSI 9207-8i

HDD 1:   1x 500G WD Blue 2.5" (OS)

HDD 2:   1x 480GB Intel 905p SSD

HDD 3:   7x 4TB Seagate ironwolf

 

 

Software and Configuration:

My server is running Fedora 27 .

ZFS on linux.

I'm using fileio with targetd to export storage over 4Gb Fibre channel to my proxmox VM server. There likely a bug in zvols... still testing.

 

 

 

 

So many changes... again..

[SakuraChan]

Server1

Hardware:

4U 24 bay AIC (old) tripple 500w PSU's

Mobo: Supermicro model: H8SGL

CPU: 12-core AMD Opteron 6238 @ 2.6GHz

RAM: 16GB DDR3 Registered ECC (4x4GB)

Network: Mellanox MT26448 [ConnectX EN 10GigE PCIe 2.0 5GT/s]

RAID Controller: 3ware 9650SE-24M8

HDDs: 24x4TB (96TB raw storage) RAID6 formatted 80.04TB usable Hardware RAID

 

Server 2 will get details later

 

Server2

Hardware:

4U 24 bay Supermico dual 900watt PSU's

Mobo: Supermicro model: H8SGL

CPU: 12-core AMD Opteron 6238 @ 2.6GHz

RAM: 32GB DDR3 Registered ECC (8x4GB)

Network: Mellanox MT26448 [ConnectX EN 10GigE PCIe 2.0 5GT/s]

RAID Controller: LSI MegaRAID SAS 9280-24i4e

HDDs: 24x4TB 80TB (96TB raw storage) RAID6 usable 80TB Hardware RAID

 

Edited August 2, 2018 by SakuraChan

[Thorn57]

Hardware:

CASE: Nanoxia Deep Silence 1 Rev. B

PSU: 350W FSP

MB: Fujitsu MB from old office PC

CPU: i5-2400

RAM: 4 x 8GB 

NIC1: Integrated NIC on MB

NIC2: 2 Port Intel 82576NS Gigabit Network Card (Previously used for SMB-Multichannel under    FreeNAS which I couldn't get to work in Proxmox)

HBA: 8 Port LSI SAS 3Gb/s HBA which was 18€ from Ebay

TV-Tuner: DVBSky S952

SSD: 128GB Samsung 840 Evo for booting and some VMs

HDD1: 3 x 1 TB (2 x Seagate + 1 x WD from old Computers)

HDD2: 3 x 2 TB (2 x HGST Ultrastar A7K2000 + 1 x Seagate)

HHD3: 4 x 3 TB (1 x HGST Ultrastar 7K4000 + 3 x Toshiba DT01ACA)

Raw capacity: 21 TB

Power: 110-160W

 

Software, Configuration and Usage:

OS: Proxmox 5.1

ZFS Config: 

• RAIDZ: 3 x 1 TB for Media
• RAIDZ: 3 x 2 TB for Backups of Computers and Phones
• Striped Mirror: 2 x 2 x 3TB: Random files like Games and VMs 

VMs: 

• UniFi Controller
• OpenVPN
• TS3 server
• Zoneminder
• Plex
• tvheadend with a Tunercard passed through (hardly used though)
• multiple testing VMs (Win10, pfsense, ...)

Backup:

Don't have the budget for a cloud backup and neither money nor a location for a offsite Backup. Important stuff is stored on my PC and backed up to the server though.

 

Photos:

[Ziggidy]

On 2/23/2018 at 4:04 AM, GreasyGaming said:

UPDATED - 360TB

 

Added 9 more 8TB drives

Added Some more SSD

 

RAM: 32GB ECC Memory (4x 8GB PC3-12800)

I am so jealous! If I may inquire, what is the total power draw on that 45 drive setup, and also, I saw from the thread you posted (2017) that you WERE running Raidz2. Is that still the case, and if so, how the heck are you only using 32 Gigs of memory? Maybe I am doing something wrong, but my 14 TB setup with Rz2 is using a bunch of memory, which I have 72 gigs of. (actually I just checked, and now it says there is only 60 Gigs, wtf, and 46 gigs in use.). I think I need to actually monitor my server...... I suck.

[gcs8]

.

[gcs8]

My old 2011 build still has me in 23ed place, let's see where my current build gets me. Let me know if I missed anything else you may want me to cover, this is just kinda everyday life for me, so if you wanna know more, just ask.

Hardware

768GB DDR4-2133(24x32GB) LRDIMM

Supermicro Rackmount 4U w/ Red. 1280W Platinum P/S

24x: Samsung DDR4 2133MHzCL15 32GB (PC4 2133) Internal Memory M386A4G40DM0-CPB

2x: Intel 2.20GHz Xeon E5-2630 v4 Deca-Core (10-Core), 25MB Intel Smart Cache, Socket-2011-v3 (FC-LGA14A)

24x: HGST Hard Drive [HUH728080AL5200] 8TB SAS 12Gb/s 7200RPM 3.5in, 128MB Buffer, Internal

Supermicro Motherboard S-2011 R3 for 2x E5-2600 v3 MFG Part Number: X10DRi-T4+

LSI Logic Controller Card H5-25573-00 9300-8i SGL SAS 8Port 12Gb/s PCIE3.0 HBA Brown Box

QLE2562 dual 8Gbps Fibre Channel

2x: Supermicro 4U Active CPU Heatsink f/ X9 Socket 2011 MFG Part Number: SNK-P0050AP4

2x: ASUS HYPER M.2 x16 PCIe Expansion Card

Chelsio T62100-SO-CR

Software and Configuration:

FreeNAS

Usage:

NAS and SAN usage, My desktop boots off of FiberChannel from the FreeNAS box so I can have snapshots every 5 min on my OS drive and I keep my main OS drive small enough to cache it in RAM on FreeNAS. It hosts a lot of VMs and Jails that do stuff and things, I try and keep my digital hoarding inside the rack.

 

Photo's:

 

[GDRRiley]

this thread never quite seems to die. I don't know how it still lives on

[brwainer]

4 hours ago, gcs8 said:

My old 2011 build still has me in 23ed place, let's see where my current build gets me. Let me know if I missed anything else you may want me to cover, this is just kinda everyday life for me, so if you wanna know more, just ask.

 

The listing hasn't been updated for at least a year, maybe two. In the time since the last update, I added one new eligible system and changed most of the specs of my other two eligible systems - and yet I only have one in there to my credit. I don't know how far your 2011 system will drop when the listing is finally (if ever) updated, but it certainly isn't in the top 30 anymore. That system is still rad and beats out my three individually.

[MariusJS]

On 06/07/2018 at 11:11 AM, brwainer said:

The listing hasn't been updated for at least a year, maybe two. In the time since the last update, I added one new eligible system and changed most of the specs of my other two eligible systems - and yet I only have one in there to my credit. I don't know how far your 2011 system will drop when the listing is finally (if ever) updated, but it certainly isn't in the top 30 anymore. That system is still rad and beats out my three individually.

It was last updated the 6th of november 2016, sad to see the thread die...

[timdine]

Hardware

CASE: Antec Fusion 430 (Still with the LCD working)

PSU: EVGA SuperNOVA 850 G2

MB: ASUS P5QL Pro (Flashed to support Xeon)

CPU: Intel Xeon E5450 (Pin Swapped for 771 to Socket 775 Compatibility)

HS: Coolermaster HyperZ 600 Passive Cooler

RAM: 8GB OCZ Reaper (https://youtu.be/-P8_ZoOBmF0?t=4m28s)

RAID CARD 1: IOCrest SI-PEX40064 PCIE 2.0 SATA III

SSD: Samsung SSD 850 EVO 250GB

HDD 1: 1x 2TB Hitachi HDS722020ALA330

HDD 2: 2x 8TB Seagate ST8000AS0002-1NA17Z

HDD 3: 2x 8TB - Seagate ST8000DM004-2CX188

HDD 4: 2x 4TB - Seagate ST4000VN008-2DR166

HDD 5: 2x 3TB 2x 4TB - Seagate ST4000DM000, 2x WDC WD30EZRX, WDC WD40EZRX

Disk Enclosure: 2x Mediasonic ProBox HF2-SU2S2 4 Bay

 

Points: 134    56TB x ln(11 drives)

 

Software and Configuration:

My server is running Windows 8.1 and im using a mix of techniques for the disk sets.  Each was used as a bit of an experiment.  I'm using Software RAID (Windows Disk Manager Mirroring) and Windows Storage Spaces (Extend, Mirror, and Parity for different drive sets)

My main array consists of 2x4TB Seagate Iron Wolf drives mirrored for the most important files.  There is also 2x8tb Seagate Archive Drives in Storage Spaces Extended, 2x8TB in Storage Space Mirrored, 2x3TB + 2x4TB drives in Storage Spaces Parity.  Four of the 8TB drives and the Parity setup are housed in Mediasonic ProBox and connected via eSata through a pcie card.  2TB misc drive which includes a Steam Library.

 

Usage:

The machine is used as a home file server, PLEX media server, Calibre and Ubooquity for eBooks, and a living room media / Couch Gaming setup with some bluetooth Wiimotes, SNES USB controls, or Wireless 360 Controllers.  It is also accessed via VPN for all the same uses from remote locations.

 

Backup:

Only critical files are backed up to a set of 1Tb USB drives which are accessed through a USB 3.0 dock.  Backup drives are cycled on a random interval.  I'm relying on the RAID to protect me from random drive failures.

 

Additional info:

The pcie esata card is connected via two pcie extension cables to make the port accessible under the EVGA 650TI Boost card with the plastic shroud removed for clearance.  There aren't nearly enough pcie slots on the board, or conveniently placed in the case.  Another extension is necessary to connect the USB 3.0 card I've added. 

The 8TB drives are all in the ProBoxes because the on board SATA controller doesn`t support drives that large.  The case has been recycled through multiple builds and has had the case internals cut and modified to fit different hardware over time, the lid obviously doesn`t fit because of the large passive cooler.  I have issues with the drive mirror breaking down and needing to be recreated if the server is powered off unexpectedly.  The ProBoxes prevent any sort of hardware raid with the drives in the box.  During boot they don't give unique IDs to all the drives confusing the SATA card, which I suspect is the same issue I'm having with my software mirror breaking.  I haven't quite gotten the right order of steps to "import foreign drives" to have the mirror be healthy.  Once in windows everything looks fine which is why I've used Storage Spaces for the other arrays.  

Most drives are shucked external USB drives picked up overtime, they were mostly running JBOD over usb 2.0 until I started reading through this thread sometime last year and I got tired of too many cords and the mess of enclosures.  It is only at that time that I added any redundancy to the system.  Power consumption is around 240W.

A third party fan controller has been added to help get rid of some of the excess summer heat in the media stand.  I'm also installing a tiny arduino accessed on internal USB connections to make using the volume knob on the case easier again as the original drivers have gotten too old to get working easily on Windows 8.

 

Build Thread: (Will continue to update)

 

 

Photo's: (Will add shortly, July 19)

[timdine]

On 7/6/2018 at 1:26 AM, gcs8 said:

My old 2011 build still has me in 23ed place, let's see where my current build gets me.

Without adding anyone else that has updated or added new builds since the last update...

 

First!!

 

 

[timdine]

On 4/1/2018 at 3:45 AM, Cynagen said:

Before I go posting full specs here, I just wanted to throw out a couple of my concerns before I attempt to opt in to be counted.

I'm running a QNAP TS-431 (4-bay) + 4-bay eSATA port replicator for a total of 8 disks, the 4 disks in the eSATA enclosure are not in a RAID configuration (I popped them in one by one and had to manually transfer stuff from disk to disk so I never attempted to RAID them).
My concerns are that my base enclosure (the TS-431) won't qualify me in the first place, and that the 4 disks in the external enclosure won't count (for whatever reason) thus completely disqualifying me.

They're all for storage as per the requirement; I'm not including the USB2.0 RAID0 enclosure attached to the NAS; One of the disks in the external enclosure is the destination of incoming sources including my own Plex DVR captures (HDHomeRun Prime + CableCard ftw) others acting as sorted semi-deep storage (before it gets mirrored to DBox), and the internal bays are configured together as a RAID5 right now to store my movie collection for Plex. Overall, there's a 4x3TB RAID5 in the internal, and 3x2TB + 1x2.5TB individuals in the external enclosure for a total of 8 drives and 20.5TB total.

If somebody could clarify if I qualify, I'll do up a full post with pics, but if I don't qualify, I'll show myself out until I can get on FreeNAS at some point in the future.

I'd be curious to check it out.  eSATA still relies on the same single motherboard for control.  It isn't just slamming external USB drives onto the system.  I assume that's really what the original rule was attempting to avoid.

[timdine]

@gcs8 a moment of glory until @GreasyGaming came along...

 

[timdine]

I may miss a few people that were updates but some new systems have been added:

@snortingfrogs

@brwainer

@Poohy14

@Lurick

@TopWargamer

@Nemockulous

@handruin

@JK Vengeance

@porina

@Darkside447

@GreasyGaming

@deafboy

@Mikensan

@H3llscr3am

@dkuhn

@grimreeper132

@LAwLz

@kiddsupreme

@SakuraChan

@Thorn57

@gcs8

 

updates:

@b348xyx

@AnonymousGuy

@Mkvarner

@djdwosk97

@Helly

@Green_man_5

@unijab

@Night_Rider0

@Dr_Frankenstein

@dalekphalm

@Jarsky

 

I had to add a bunch of new OS and Drive types.  I also am sure I missed at least one system of note that didn't have five drives.  I'll go back and find that to add next time.  I was able to get colonel_mortis, one of the mods, to put in the html for me into the front page.  Big thanks for the help on that.  I'm going to update the github repository with the updates I've made to the configuration pages.  I'll keep an eye out for some updates and maybe try making another update then.  Or maybe just when I add some drives to try to get a better rank myself

[TopWargamer]

Oh oops, I kind of forgot about this thread. Don't really post anymore. Mine's been updated. It's still in a cardboard box, and still 16TB, but it has 4 drives now and is running Windows 10 (a cardinal sin, I know).

[brwainer]

4 hours ago, timdine said:

I may miss a few people that were updates but some new systems have been added:

Thanks! Unfortunately, none of the graphs are showing in the posting, and if I click the link for the .svg files I get an error that the attachment isn't available to me (like a permissions issue).

[Mattias Edeslatt]

8 hours ago, timdine said:

I may miss a few people that were updates but some new systems have been added: 

 

......

 

I'm going to update the github repository with the updates I've made to the configuration pages.  I'll keep an eye out for some updates and maybe try making another update then.  Or maybe just when I add some drives to try to get a better rank myself

Sweet. Glad to see that this thread lives on and you have taken over the torch 

 

It's this thread that lured me in to this forum, many, many moons ago, dreaming and drooling over humongous storage and dual Xeon systems   

[alpenwasser]

Just a general heads-up:

 

I was abroad doing a crapload of overtime during spring, which meant basically no private live in the past few months. The good news is that I get to take lots of time off during the second half of the year, so I was planning to get around to this in the coming weeks. Nice to discover somebody has decided to help out! Many thanks to @timdine (and, uhm, apologies for the messy code ).

[timdine]

On 7/29/2018 at 12:54 AM, brwainer said:

Thanks! Unfortunately, none of the graphs are showing in the posting, and if I click the link for the .svg files I get an error that the attachment isn't available to me (like a permissions issue).

I see what is happening with the graphs, they're hosted out of a message conversation which I can see and the mod who helped post the code can see so it looked good when we tested it.  We'll find a way to change that shortly.  

 

Drive Heatmaps

Spoiler

2018-07-26--22-57-13--drive-heatmap.svg

2018-07-26--22-57-13--drive-heatmap-contribs.svg

2018-07-26--22-57-13--drive-heatmap-systems.svg

OS Heatmaps

Spoiler

2018-07-26--22-57-13--os-heatmap.svg

2018-07-26--22-57-13--os-heatmap-caps.svg

2018-07-26--22-57-13--os-heatmap-drvc.svg

Rankings

Spoiler

2018-07-26--22-57-13--rankings.svg

2018-07-26--22-57-13--rankings-caps.svg

2018-07-26--22-57-13--rankings-drvc.svg

 

 

Storage Systems

Spoiler

2018-07-26--22-57-13--storage_sys.svg

 

Timelines

Spoiler

2018-07-26--22-57-13--timeline_caps.svg

2018-07-26--22-57-13--timeline_drvc.svg

2018-07-26--22-57-13--timeline_sys.svg

 

[timdine]

4 hours ago, alpenwasser said:

Just a general heads-up:

 

I was abroad doing a crapload of overtime during spring, which meant basically no private live in the past few months. The good news is that I get to take lots of time off during the second half of the year, so I was planning to get around to this in the coming weeks. Nice to discover somebody has decided to help out! Many thanks to @timdine (and, uhm, apologies for the messy code ).

No problem.  It wasn't that bad to figure out.  You had it all hosted so that I could just fork it and go.  The documentation was close enough even if some of it still mentioned PERL that I added a couple modules to my installation and it worked.  The config json files were easy to figure out too.  Any critique of code for something of this purpose would be pretty subjective, it solves a problem, that's reasonable code.

 

This is absolutely one of the threads that I got reading, and still come back to, the forum for.  May as well keep it going.  It's a great idea building thread for something that isn't gaming.

[Pyranicus]

Goro

 

Hardware:
CASE: NorcoTek RPC-4224

PSU: Corsair AX1200i

MB: Asus X99-E-10G WS

CPU: Intel Xeon E5-2620 v4

HS: Noctua NH U9DX i4

RAM: 128GB Crucial DDR4 2400 ECC

Network: Myricom 10G SFP+ 2 Ports

RAID CARD: 3 x LSI 9361 8i

Boot Disk: 2 x Samsung Fit 32GB USB (Mirror)

SSD: 2 x 250GB Samsung 850 EVO

HDD 1: 12 x 6TB WD RED Pro

HDD 2: 12 x 6TB WD RED Pro

Rack: Tripp-Lite 25U SmartRack 4-Post Open Frame Rack

 

 

Software and Configuration:
OS: FreeNas

RAID config: Raidz2 + Raidz2

File system: ZFS
File system capacity: 144 TB Total (105.5 TB Usable)

 

Usage:
General Storage / Backup / VMs

Backup:
Snapshots

Important Documents syncs to Drobo 5N with 5 x WD 4 TB Red Pro in Raid6

 

Photo's:

 

[sh4z]

@Pyranicus How are the temps with the Norco case and the 6TB Drives? I found the WD Red Pro 8TB / WD Red 8TB are too fat/tall for adequate airflow to go over them in this chassis ... I imagine the 6TB drive might be a tad less tall, the 4TB drives are fine ... but I wasn't forcing the air over them with Delta fans or anything crazy (With enough static pressure it'd probably work) I was trying to keep things fairly quiet ...