LTT Storage Rankings

[ChalkChalkson]

@Jarsky sort of convinces me to enter, I got a pretty simple NAS sitting below my desk (and sometimes rest my feet) as well as a really ghetto server that is the backup for said NAS, I will only enter with my main NAS though, since I can't make pictures of my (offsite) backup atm.

 

Hardware:

Spoiler

 

Kabylake G4560

ASRock H110M ITX

2x4GB GSkill value DDR4

240GB Crucial BX200

3x2TB Seagate Ironwolf 5900 rpm

 

Meaning I got 4TB of accessible storage and 240GB of cache.

 

Software:

I am running unraid, with one parity drive, to access my offsite backup I use OpenVPN and this server also hosts my Plex.

 

Storage setup:

My servers both are organized in in 2 public shares each, one of which is read only, and holds a copy of the other server as well as timed versions of important folders of that server, the other is holding my movies, images and important data, which is then (in a weekly fashion) copied onto the other server in a similar way, giving me weekly rollbackpoints for everything but pictures and movies.

[deafboy]

FreeNAS Box (Black 4224): 12x4TB (2 6 drive vdevs each in RaidZ2)

Cold Storage Box (Green Google 4u):  Currently 1x8TB, 2x2TB, 1x1TB

ESXi Box (Google Yellow 2u): 2x512GB

 

Definitely wanting to get more drives to fill up the FreeNAS box

[dalekphalm]

I have totally reconfigured my Server Setup, which is as follows now:

 

Running on a Dell T410 Tower Server

Base OS: ESXi

OS drive: Samsung 850 Evo 120 GB SSD

 

ESXi Datastore drives:

Dell PERC H200 RAID Card in IR Mode

2x 146GB 15K SAS - RAID1

2x 300GB 15K SAS - RAID1

 

Main Storage Drives:

LSI 9207-8e RAID Card in IT Mode

4x Toshiba 3TB SATA; 2x Seagate 3TB SATA - RAIDZ1

 

Guest Operating Systems:

1. FreeNAS - using PCI Passthrough to have direct access to the above LSI Controller

- Runs the main pool RAIDZ1

- Shares the pool in variouns SMB Shares across the network

- Runs a Transmission Jail

2. Windows Server 2016 Standard

- Hosts a Plex Server

[Dr_Frankenstein]

It has been a while since I last updated my post, my 412+ has been on backup duties for a while, not sure if I should update the old post with its current setup?

 

Hardware

Synology DS1815+

PSU:  Power Brick 250w

MOBO: Custom Synology Board

CPU:  Intel Atom C2538 2.4 GHz

RAM: 6GB Kingston DDR3
HDD 1: Seagate Barracuda 4TB (ST4000DM000)
HDD 2: Seagate Barracuda 4TB (ST4000DM000)
HDD 3: Seagate Barracuda XT 4TB (ST4000DX00)
HDD 4: Seagate Barracuda XT 4TB (ST4000DX00)
HDD 5: Toshiba 3TB (DT01ABA300)

HDD 6: WD Blue 4TB (WD40EZRZ)

HDD 7: WD Red 4TB (WD40EFRX)

HDD 8: Drevo 256GB SSD (X1 Pro)

 

Software, Configuration & Usage

 

My thirst for data outgrew my now older 412+ Synology, I love their products so stuck with them and upgraded to a 1815+ around 18 months ago, I currently have a hodge podge of drives, this is due to looking out for drives on eBay as you can pick up some real bargains. I quite often shuck externals, the two 4TB Seagate's are from Expansions the WD red I picked up by shucking a WD Studio external. So all in I have just over 27TB of raw storage in this box.

 

I currently have the 256gb SSD dedicated to download duties and stores all the Docker containers to keep things snappy, the remaining drives are setup in SHR2 aka RAID 6.

 

I store mainly linux ISO's, family photos etc on the box and it happily runs a Minecraft server for my son and his friends, I have everything setup in Docker containers for usenet automation and alike, I write guides over on my website. I never seem to be able to stratch my storage itch so across my other drives in the 412 and a Microserver I have another 15TB!

 

Almost forgot I have a Cyberpower UPS for backup power in case anyone accidently unplugs the box or trips off the mains power as the last thing I wont is drives lost to power issues.

 

 

[Oxydoreduction]

Here's my storage setup for Home:

 

1 Wd Green 3Tb in my main rig which is a backup of the files stored on the NAS.

 

NAS server:

4U Inter Tech 4098

350W Corsair PSU

Asrock Rack e3c224-4l

Pentium G3420 with stock aircooler

2x4Gb DDR3 1600 ECC

Intel 10Gb AF-DA sfp+ NIC

Icy Box SATA Backplane 4x3.5" in 3x 5.25"

3x3Tb WD Red and 1 3Tb Seagate Ironwolf in RAID 10

 

Offsite Backup (at work ) for the most important files:

1U chassis

220w FSP PSU

Asrock Rack e3c224d2i mITX

Intel Core i3 4370 (Which I plan to install later in the Home NAS)

2x4Gb DDR3 1600 ECC

3x1Tb Hitachi 2.5" drives in RAID 5 (Gonna see if I can update to 4x1Tb RAID 5 this week)

 

In the picture below, the Home NAS is still in the 2U chassis but I swapped not long ago to the 4U chassis (and added the Seagate 3Tb drive) which dramatically improved the cooling of the drives.

 

And the setup of my rack as it was in april:

 

It's a lot cleaner now, added 10Gb ethernet, rails for the 4U chassis (retailer had not the other 2 pair I need for the 2U chassis) and a proper KVM.

 

Some pictures of the renewed setup a bit later

[GreasyGaming]

On 19/05/2017 at 2:13 PM, GreasyGaming said:

--snip--

UPDATED

 

Added 9 more 8TB drives

Changed CPU

Added 10Gbit NIC

Added another HBA

 

CASE: 45Drives 

MB: Supermicro X9SCM 

CPU: Intel(R) Xeon(R) CPU E3-1275 @ 3.40GHz

HS: Standard

RAM: 32GB ECC Memory (4x 8GB PC3-12800)

HBA: 2 x HighPoint Rocket 750

SSD: Toshiba 120GB

HDD 1: 36x 8TB WD RED 5400 RPM

NIC: Onboard 2x 1Gigabit

NIC: HP Mellanox 10GbE Connext-2

[Oxydoreduction]

Hi guys,

 

As promised, here are some more recent pictures of my setup

 

I finally have the last pair of rails for the 2U chassis.

 

Rack as it is now, the top still requires some work and shelves to serve for miscellaneous storage place

 

And here's a little close-up for the hardware bit:

 

 

From top to bottom:

Netgear access point

TP-Link TL-ER6020 VPN router

Cable manager (That kind of manager is a really a must have for a rack, wonder how I did without it for so long)

D-Link DGS 1510-20 L3 switch with 2 10Gb ports

RDP Server (not a Xeon anymore but Ryzen 7 1700X with 32Gb DDR4 2133 ECC)

NAS in it's 4U chassis with it's 4x3Tb backplane (Not a Ryzen CPU, it's the wrong sticker)

2 UPS's to protect everything.

I placed pieces of paper over the Power on LED's cause they are quite aggressive

[snortingfrogs]

Workstation/Gaming computer
Intel i7-2600K running at 4.5GHz
Corsair 16GB DDR3 1600MHz
MSI GTX 1070
Samsung 850 EVO 500GB SSD for Windows and games.
17TB HDD (1x2TB, 2x3TB, 1x4TB, 1x5TB)

 

NAS
HP ProLiant Microserver Gen8
G1610T
4GB
2x8TB (Seagate IronWolf)

FreeNAS

 

Fileserver
Intel Xeon 1230v2
8GB
15TB
Debian

[GDRRiley]

I will make it on the odd list when I finally get my stupid hp dl380 g6 to run linux 

[leadeater]

On 6/1/2017 at 7:25 AM, GreasyGaming said:

Ok, someone else is running enterprise high speed spinners and the drive temps were astronomical!

 

Would be interested to know you get on.

Been running 10k and 15k RPM disks since SCSI Ultra Wide & Ultra 2, back then the full metal sleds HP used really helped. Now they are just plastic and require a bit more cooling, especially now that most are in 2.5" size. I've had disks too hot to even hold before, lack of airflow is a bitch lol.

[Dark]

QNAP TVS-EC880 w/ 8x 3TB @RAID6 (10Gb SFP+)

Supermicro server (esxi) w/ 2x 1TB SSD + 1x 5TB (10Gb SFP+)

 

Network backend is on a Ubiquiti US-16-XG 10Gb switch.  Most clients are served from AC WAPs and a Ubiquiti US-24. 

 

 

[MrBucket101]

@alpenwasser @looney

 

CLICK HERE FOR ORIGINAL POST

CLICK HERE FOR UPDATE 1

CLICK HERE FOR UPDATE 2

CLICK HERE FOR UPDATE 3

 

 

UPDATE #4:

 

Time for another storage upgrade

 

I added an additional 4-4TB HGST NAS drives

 

 

[Mikensan]

Hardware

CASE: Rosewill RSV-L4411

PSU: Rosewill CAPSTONE-450

MB: SUPERMICRO MBD-X9SCL

CPU: Intel Xeon E3-1220v3

HS: Stock Intel heatsink

RAM: Mushkin Enhanced PROLINE 32GB DDR3 1600 ECC

RAID CARD 1: IBM M1015, Flashed with "IT" firmware

NIC: Mellanox MNPH29D-XTR

SSD: Intel 128gb (not sure of model, had it laying around)

HDD 1: 5x 4TB Seagate ST4000DM000 (Bigboy)

HDD 2: 4x 3TB WD red WD30EFRX (Tinyboy)

HDD 3: 3x 512GB Samsung EVO SSD (SSD1)

 

Software and Configuration:

My server is running FreeNAS 11u3.

My main array consists of 5 4TB disks in RaidZ1 using the 128GB SSD for SLOG with NFS and SMB shares.. The second array is RaidZ2 with a SMB share. It is joined to active directory to manage permissions to the smb shares. I'm not using any plugins/jails.

Ranking: 12 disks totaling 33.5TiB, score of 83.244

 

Usage:

It is effectively silo storage for my hypervisors. These are connected via 10gb Mellanox DAC using iSCSI. I also store movies / torrents / data on bigboy. I host plex on a VM within my vmware lab and mount a SMB share, running it within a jail is a hassle*. 

 

Backup:

I use veeam for my backups which mounts to a smb share on tinyboy. I also drop my manual backups of network devices to tinyboy. At this time I do not use any cloud storage but hoping to find a really good friend or a nearby co-location site to stick a server. I use replication to backup some of the datasets to the backup volume (tinyboy).

 

Additional info:

*When FreeNAS updates the FreeBSD kernel, the current jails do not get updated. As a result when you go to update your ports/plugins it wants the new kernel. To update the jail's kernel you have to delete it, update the template, and create a new jail. Too much of a headache.

Rack is a Startech 4POSTRACK25U (not sure if I'm allowed to link to where I bought it or not?)

The Rosewill case is great except that it's pointless as a "rackable" case... They made it too wide for rails of any kind. So it's sitting on a shelf -_-.

 

Photo's:

Just the rack in the dark

Spoiler

Rack with the lights on. Top left: Cable modem (150/10) Top Right: Custom box for pfSense

Stray cable shooting out the left above the switch - runs out to another power outlet where I'm using powerline ethernet. Drooping cable with PoE injector is used for a Ubiquiti AP-Pro mounted higher up on the wall.

Spoiler

16 port netgear - has a nice web GUI, bought it for VLANs (ignore the poor unplugged blue cable) 

Dell C1100 (32GB, L5520) is off, was a ESXi node. May switch from the custom pfSense box to the C1100.

Dell R610 (96GB, X5670) running ESXi

IBM x3650m2 (32GB, X5670) running ESXi

FreeNAS-1 (what this post is about)

Spoiler

TP-Link switch is used for IPMI, piss poor cable management by yours truly (always changing things, hard to keep up). Little 8 port gigabit switch ontop of the TP-Link was something I was testing.

Spoiler

Cyber Power 1500 UPS, just bought new batteries for it few months ago.

Spoiler

 

Volume overview

Spoiler

Bigboy volume status

Spoiler

 

[MG2R]

On 9/29/2017 at 1:14 AM, Dark said:

-snip-

You mounted a rack server to the wall...

 

That's what I call... vertical thinking! <yeeeeaaaaaaahhhhhhhh!!!!!.wav>

 

Seriously though, super neat setup on that wall!

 

EDIT: actually, how did you mount them exactly?

[brwainer]

33 minutes ago, MG2R said:

You mounted a rack server to the wall...

 

That's what I call... vertical thinking! <yeeeeaaaaaaahhhhhhhh!!!!!.wav>

 

Seriously though, super neat setup on that wall!

 

EDIT: actually, how did you mount them exactly?

Something like this I'm assuming: https://www.amazon.com/Startech-19-Inch-Vertical-Mountable-Server/dp/B001YHYVEY

Generally these are intended for switches and patch panels, but they can hold servers if you secure them to the wall properly.

[H3llscr3am]

I wanna play

 

Hardware

CASE: U-NAS NSC-810A

DRIVE CADDY: iStarUSA BPU-350SATA 5 x 3.5" Drive Cage

PSU: SeaSonic 350M1U

MB: Asrock Rack E3C224D4I-I4S

CPU: Intel Xeon E3-1275L v3

HS: Noctua NH-l9i

RAM: 32GB Crucial (4 x 8GB) 240-Pin DDR3 SDRAM ECC Unbuffered DDR3L 1600 (PC3L 12800)

RAID CARD 1: LSI 2308, Flashed with "IT" firmware

SSD: Patriot PSK128GS25SSDR Spark Series 128GB SSD (L2ARC)

HDDs: 13x 3TB HGST Ultrastar 7K4000 3TB 64MB Cache 7200RPM SATA 3 (one was a spare drive, but swapped it for the SSD, so it's lying around in bubble armor)

Software and Configuration:

OS: FreeNAS

In Jails, I run Transmission, Plex, Headphones, and some other applications for media acquisition, management, and local streaming.

Usage:

I use the storage for movies, shows, and music mainly. But it also stores some educational/certification materials, and other junk.

Photo's:

Drives in Freenas UI:

Server:

[Dark]

The office sent me home with some new bits the other day-- older TS-EC1679U-RP (E3 Xeon) and 8x 850 Pro 2TB SSDs.  Got it setup on the 10g network with an NFS share for ESXI, still need to figure out what I want to store on the rest.

 

[koolerone]

6 hours ago, Dark said:

The office sent me home with some new bits the other day-- older TS-EC1679U-RP (E3 Xeon) and 8x 850 Pro 2TB SSDs.  Got it setup on the 10g network with an NFS share for ESXI, still need to figure out what I want to store on the rest.

 

holy cow your company let you have that for free?!  If so nice freakin score on those ssds and the server

[snortingfrogs]

Installed another 8TB drive in my NAS today so it now has 24TB, another 8TB drive will be added later on.

 

 

Computer

HPE Microserver Gen8 (G1610T, 16GB RAM)

3x8TB Seagate IronWolf HDD's

FreeNAS 9.10.2-U6 running from a 16GB MicroSD card on the internal port.

[dkuhn]

I'm gonna post mine here, but I was curious what the rules were for mounted storage (via NFS)?  I distribute my storage across two systems then present it logically to the docker apps as a single storage pool.

[brwainer]

9 minutes ago, dkuhn said:

I'm gonna post mine here, but I was curious what the rules were for mounted storage (via NFS)?  I distribute my storage across two systems then present it logically to the docker apps as a single storage pool.

for the ranking, it only matters where the physical disks are. So if you have two servers that somehow pool storage together, each one has to meet the minimum requirements for total size and number of disks.

[dkuhn]

Just now, brwainer said:

for the ranking, it only matters where the physical disks are. So if you have two servers that somehow pool storage together, each one has to meet the minimum requirements for total size and number of disks.

good deal, thanks

[dkuhn]

so it looks like my Synology won't qualify here, but that's fine...it's only 32TB

 

instead, here's my unRAID system that I use to house my modest Plex system

I'm not providing my case/psu/mobo as I honestly don't remember what they are and I'm too lazy to look

 

Hardware:

CPU: AMD Ryzen 5 1600

HS: Stock cooler

RAM: 16GB of <insert brand>

SSD: 1 x 525GB Crucial MX300

HDD 1: 5 x 8TB WD80EFZX

UPS: 2x APC SmartUPS 3000XL SU3000RMXL3U (not pictured)

Rack: Startech 12U

Network Stack: Cisco Meraki MS225-24 switch, MX84 firewall

 

Software and Configuration:
OS: unRAID

RAID config: unRAID High Water

File system: btrfs
File system capacity: 32TB presented, 40TB raw (add'l 22TB presented via NFS from my Synology DS1515 w/4x 8TB WD80EFZX)

 

Usage:
Plex. I have a lot of stuff. 724 movies, 233 TV shows with 18,224 episodes. I share this with close friends and family so uptime and performance is very important. I have at least 6 concurrent transcodes going at any point in time, most 1080P with some 4K. It's essentially full at this point so I'm looking at replacing it with a Norco RPC-4224 and consolidating all of my storage into a single chassis. Going to get rid of the R710/MD1200 and the Xserve/Xserve RAID - they're power hogs and are not expandable any further

 

Backup:
I back up my most important media to my grandfathered Google Apps account.

 

Photos:

The system in question is the top one.  Couldn't help but show off the rest, though.  Might do other posts for them later.  Yes, I know it's dusty as shit.  I'm in the process of moving and all this will be torn apart, cleaned piece by piece and reassembled with dust filtration in the new rack.

[Mikensan]

Updated my post, moved my SSDs over and added 2x port 10gb mellanox card.

 

NIC: Mellanox MNPH29D-XTR

HDD 3: 3x 512GB Samsung EVO SSD

[alpenwasser]

So... the reason I've not really been around much in the past year was because I was doing the final year of my degree and didn't really have any spare time. But the good news is: I'm finally an officially certified electrical engineer (albeit an unemployed one at the moment)! I will be updating the list in the near future (no, seriously! :D), but it turns out that there have been some updates to the Python libraries I'm using and the plot output is kinda broken (very nice of them, isn't it?), so I need to debug that first before I can update the rankings. Yay!