LTT Storage Rankings

[MyNameIsNicola]

On 2/9/2016 at 9:54 PM, Mikensan said:

Ah ok very cool and well done. I find it useful as well, I like having my network/servers documented and this is right up my ally. The GPTID explains why I had an empty row with nothing but a GPTID - I partitioned out 2GB on my SSD for ZIL/SLOG - (seeing if it helps my iSCSI connections) and I guess that's the ID for this 2gb partition. Neat :-)

I think I've fixed the script. I've not got a setup that's the same to test against though, but I think it'll parse the output for those scenarios now.

 

https://nicolaw.uk/freenas_disk_info.pl

 

It also now outputs the adaptor, bus, target and lun that the disk is connected to. Especially useful for me given how many different HBAs I have.

 

I've tidied up the code a little more to run with full taint checking etc to be a little safer (not that there should be any risk anyway).

 

Let me know if you have any warnings from the output.

[brwainer]

so... my build hasn't made it into the rankings yet (which I totally understand, I'm not asking for it to be immediate) but I already have a change to the config - added 2x 512GB SSDs. Should I update my post or make a new one?

[dalekphalm]

16 minutes ago, brwainer said:

so... my build hasn't made it into the rankings yet (which I totally understand, I'm not asking for it to be immediate) but I already have a change to the config - added 2x 512GB SSDs. Should I update my post or make a new one?

i would edit your original post, then tag a mod in a new post. If you're over 10TB, you should be added to the list.

[weeandykidd]

I'm back for a quick update, I'm going to have stop reposing every time I add one more drive but this is a bit of a bigger upgrade!

Hardware
CASE: Dell 12-Bay 2U 
MB: 3CESL1
CPU: Dual Opteron 2373 Quads
HS: OEM w/ added fans
RAM: 64GB ECC DDR2
RAID CARD: PERC H310
SSD: SanDisk 120gb
HDD: 6x4TB RED
24TB Total


Software and Configuration:
Win Server 2012R2

Drivepool

 

Usage:
Plex, IIS, VMWare, IRC, OwnCloud, Backups, Game Servers

Backup:
Crashplan

Additional info:
LACKRACK!

 

Photo's:

 

[SnorryHobo]

there are some really nice setups in here !

[paps511]

Hey guys,

Just got my server stuff today. Still figuring it out, but here is what I have.

 

 

I have it set up for 1 drive failover, so the capacity I have is 24tb (21.81 tb). 30 tb of storage with 5 drives puts me at about 50th. Not bad for having no idea what the fuck i'm doing!

 

Hardware
CASE: Synology 1815+
CPU: 2.4ghz quad core
HS: OEM
RAM: 6gb of 1600 mhz
HDD: 5x6TB WD RED
30TB Total

Software and Configuration:
Synology software and custom raid. 1 drive failover

 

Usage:
Plex, Backups, remote storage

Additional info:
I have more dollars than sense

 

Photo's:

 

 

edited formatting

[Ramaddil]

On 7/4/2013 at 8:20 PM, alpenwasser said:

On 7/4/2013 at 7:25 PM, Ramaddil said:

Total Storage 

89.0TB

 

 

Server Name

Titan

 

Hardware

CASE: Lian Li 343B
PSU: Corsair HX1000i
MB: Asus TUF Sabertooth 990FX

CPU: AMD FX - 8120 @ Stock
HS: Noctua NH-D14
RAM: 32GB DDR3 1600 GSkill Ram
RAID CARD 1: Adaptec 52445 28 Port Raid Card (Raid 6)

Raid Card 2: LSI 9260-8I 8 Port Card (Raid 5)

SSD: Samsung 840 EVO 120GB
Enclosures: Startech 2/5.25 for 3 HDD (8) 

HD's: 

16x 2TB Western Digital (Red) WD20EFRS
9x 2TB Western Digital Greens (Various Models) 

1x 3TB Western Digital WD30EZRS

6x 6TB Western Digital WD60EFRX

 

((( Update Added 7TB to Server )))

((((( Update # 2 Added 10TB More To Server ))))) 7/12/14

((((((( Update #3 Replaced a Failed 2TB//Added Additional 2TB//Changed Operating System//CPU, Motherboard Upgraded)))))))

((((((((((Update #4 Changed Raid card//Added Additional 20TB of Storage//Added 16Gb more of Ram (total 32gb) ))))))))

((((((((((((Update #5 Added an additional 2TB/Reconfiguring Fans for external Fan controllers (Pics to Follow)  ))))))))))))))

(((((((((((((((Update #6 Added an additional 6TB/Replaced MB/Scrapped outside Fan controller/Added 2U for future Expansion)))))))))))))) 

(((((((((((((((((Update #7 Added an additional 2TB///Changed to Platinum PSU HX1000i))))))))))))))))))))

(((((((((((((((((Update #8 Added Additional 12TB//Reconfigure backup solution to a Raid 5 solution with a New 9260-8I))))))))))))))))

 

Software and Configuration:
My server is running Windows 8.1, I have one Raid 6 Array with 23 drives (24 X 2TB =48TB).  This array holds all my movies/tv shows and data.  I have one 3TB drive that I use for backup for my software that is synced to nightly with Syncback Pro on top of my cloud storage which is mentioned in the backup section.  Second Array is 6 x 6TB (36TB) for Backup Purposes). 
 
Usage:
I use my server as my media hub for my entire house and beyond.  I have 2 WD TV Media players in different rooms in the house which can remotely access my server.  I also utilize my server to access Air Video remotely on my Idevices, and it also serves as my security system monitoring cameras placed around the house.
 
Backup:
My Server is connected to Crashplan which stores my data off site.
 
Additional info:
Server Is Running 8.1 til Further.
 

(((((((((((((((((Update #8 Added Additional 12TB//Reconfigure backup solution to a Raid 5 solution with a New 9260-8I))))))))))))))))

 

 

 

 

 

 

 

 

 

 

 

[Bhav]

So how does my storage setup compare to the rest?

 

2 x 4 Tb Seagate HDDs (main drive + raid / documents backup)

2 x 1 Tb Samsung 840 Evos in Raid 0 (gaming drives)

512 Gb Crucial M4 (old gaming drive, now movie maker + office, and other apps drive)

128 Gb Crucial M4 OS + Drivers (OS drive so doesnt count)

 

I got the 1 Tb EVOs wen the 840s went EOL for a good price, and they are super fast:

 

 

I would have liked to have sold either one of the M4s, but they are worthless second hand now.

 

Also I just got a big improvement to random read and writes by enabling Samsung Rapid mode on one of my Raid 0 drives:

 

 

[Eric The Tech Guru]

So I've posted before but I've kinda made a bunch of revisions to my server.

 

Hardware

CASE: Fractal Design Define R5 (windowless)

PSU: Corsair RM650

MB: Asus M5A78L-M

CPU: AMD FX-8350 Black Edition

HS: Coolermaster Hyper 212 Evo

RAM: 24GB DDR3 1600MHz

OS SSD: Samsung 850 Evo 250GB

HDD: 6 x 4TB Western Digital Caviar Red

 

Software and Configuration:

OS is Ubunbu Linux 15.10:

Using Zsh as my shell with 'norm' theme.

Drives are configured in RAID 10 using MDADM, 12TB:

 

Usage:

Backup and media storage for household. Running Plex Media Server on the server as well, using the 8 cores for transcoding media for everyone in the household.

Data usage:

Backup:

No offsite or onsite backups, running only redundancy.

Primary desktop documents do have Backblaze backup running continuously though.

 

I can add photos of the actual system in a bit once I open it up again.

 

Thanks!

- Eric

 

EDIT 1 - Performance Photos:

No Zsh here because root user and not my normal user.

[Jonny]

WinterFell (Ubuntu 16.04)

 

CPU: Intel® Core™ i7-2600 Processor (8M Cache, up to 3.80 GHz) 

Motherboard: Asus P8Z77-V LX

Boot Drive: OCZ Agility 3 60GB

RAM: 32GB DDR3 (4x8GB HyperX FURY)

HDD Config 1: (3x 3TB 3.5in WD RED in ZFS RAIDZ1) (3x 10TB 3.5in Seagate IronWolf in ZFS RAIDZ1 again) (22.9GB usable)

HDD Config 2: 2x 1TB 2.5in WD RED in ZFS Mirror for Virtual Machine Storage. Mainly for Linux Containers, which run EMBY, Torrent Server, Sonarr, Radarr, VPN Server, Apache, all in their own containers, which stops package version conflicts. Also backups up important data from HDD config 1

SSD Cache Drive: Samsung SSD 850 EVO 250GB for Main Storage Pool (L2ARC)

SSD Cache Drive: Transcend 128GB for LXC Container Pool (L2ARC)

UPS: APC Back-UPS Pro 900

HBA: Fujitsu D2607 (LSI 9211-8i IT MODE) Not in pictures as taken before hand.

 

Updated (02/04/2018)

 

See Below for images....

 

Future Plans....... Maybe

• Moving to different case and add another Icydock for the 3x 3.5in inside (FatCage MB155SP-B or FatCage MB153SP-B). Maybe case like the Storm Trooper where it has 9 5.25in bays, or something similar. 
• Going to Ubuntu 18.04 little while after release.

Inside shots (Updated 09/04/2015)

 

 

 

[scottyseng]

@alpenwasser:

I've added two more 4TB Re SAS drives to my NAS. Now have four 4TB Re SAS drives in RAID10 instead of the existing two 4TB RAID0.

[Kiades]

Hardware

CASE: HP Proliant DL180 G6

PSU: HP 460W, part 511777-001

MB: (Part number unknown at this time, but the one that comes standard with the HP Proliant DL180 G6)

CPU: 2 x Intel Xeon X5570

HS: Stock HP Heatsink

RAM: 24 GB DDR3 1333 ECC Registered

RAID CARD 1: HP Smart Array P410

HDD 1: 9 x 4TB Seagete ST4000DM000

Software and Configuration:

My server is running windows server 2012. Booted off two HP 146 GB SAS drives in RAID 1, this machine runs my deduplicated file storage, a 2 TB iSCSI target, and VMware vCenter 6.

 

My main array consists of 9 x 4 TB in RAID 6, totaling in a 25.5 TB usable disk space within Windows (28 TB raw).

Usage:

The iSCSI target is consumed by my four ESXi hosts, which are controlled by my vCenter instance. Several VM's that are hosted on the iSCSI share consume multimedia files stored on the storage server, such as Universal Media Server and Plex.

Backup:

Essential and non-replaceable files are backed up remotely to colocated Dell PowerEdge C6100 in a different state.

Additional info:

An upgrade is scheduled for next month to upgrade from the HP Smart Array P410 to a HP Smart Array P822 and add a HP StorageWorks D2700 with four Samsung EVO 850 250 GB SSD's and 21 x 320 GB Western Digital 

Photo's:

 

During a recent server rack upgrade, I took quite a few pictures, to include me installing my server. You can check out more pictures during this upgrade, to include some animated photos at the end!

The local arrays within the server itself.

During a recent upgrade, I installed two more 4 TB drives into my HP Proliant DL180 G6, giving me the now total 9 drives.

Before I installed the drives with the server still running. More pictures and even more animated photos are included in this link!

And this picture was taken this this past weekend (26 March 2016). This server and it's hard drives have been on nearly 24/7 since early 2013 and is scheduled for upgrades very soon.

[mGrytbak]

On 23.6.2015 at 9:43 PM, mGrytbak said:

Hardware:

 

CPUs: 2x Xeon E5-2603 v3

RAM: Kingston DDR4 2133MHz ECC 32GB (4x8gb)

Motherboard: Supermicro MB X10DRL-i Dual Xeon E5 v3

PSU: Redundant 960w 80Plus Platinum PSU (Comes with the case)

Case: Supermicro SuperChassis 846BA-R920B

Boot-drive: 128Gb OCZ Agility 3

Cache/encoding drive: 600Gb WD VelociRaptor

Controllers: 2x LSI MegaRAID SAS 9211-8i 

 

Software:

 

The server is running Microsofts StorageSpaces on Windows Server 2012. It fits my drive config, and the speed is perfectly fine for gigabit connectivity. 

 

Server is running Plex and some other media-stuff.

 

The storage is configured as one drive-pool with two virtual disks. One big disk for media configured as parity, and one smaller disk for documents and more important stuff which is configured as a mirror over three drives, and is backed up using Jottacloud aswell.

 

Update! *yay*

 

I have some new drives and whatnot

 

This is now the full list of drives in this system:

8x 4TB WD Red

6x 2TB Seagate NAS HDD

2x 1,5TB Seagate Barracuda

1x 3TB Seagate Barracuda

 

The server is still running Windows Storage Spaces, but has been upgraded to Server 2012 R2.

 

[Pancake]

How do I update my ranking? I am currently sitting at 29TB. My setup has changed a lot lately

Hardware

CASE: Thinkserver SA120 JBOD

PSU: 550w Hotswap

MB: N/A

 

The VM its connected to has 4 x vCPU cores of a Xeon E3-1265L V2 and 4GB of RAM. It also has the IBM M1015 RAID Card assigned to it (Flashed to IT mode)

 

Specs of my main ESXi Server:

E3-1265L V2, 32GB ECC, Supermicro Server board with IMPI and Dual lan etc, Samsung 850 Pro 512GB, iStarUSA 213 2u Case, ICY DOCK 6 x 2.5" Hotswap bay, 430w Antec Earthwatts PSU

Specs of my second ESXi Server:

i3-3220, 32GB ECC, Supermicro Server board with IMPI and Dual lan etc, Crucial MX100 256GB, iStarUSA 213 2u Case, ICY DOCK 6 x 2.5" Hotswap bay, 430w Antec Earthwatts PSU, And a Dual Port Intel Pro1000 NIC for pfSense (Used as my main router)

 

 

Software and Configuration:

I am now using a Lenovo Thinkserver SA120 JBOD, connected to a IBM M1015 which is using hardware passthrough to my file server VM in my ESXi server. I am running Windows Server 2012 R2 with Stablebit Drivepool

 

Usage:

Storage for PLEX movies and TV Shows, and all general files 

 

 

Backup:

 

Local backup of important file to another disk not in the standard pool, and also I am using Syncovery to backup all 15TB of my data to Amazon Cloud Drive unlimited 

 

 

 

[thekeemo]

I know this is off topic I am sorry

but how do you make tables and graphs in this? also headers.

[jscho]

19 hours ago, Pancake said:

How do I update my ranking? I am currently sitting at 29TB. My setup has changed a lot lately.

Heyy, saw this over on /r/homelab. Such a clean setup, I love it.

Those SA120 machines look fantastic, too. You've got me on the hunt for one now.

[Mikensan]

@Pancake How do you connect the Thinkserver to the IBM 1015? Is the SFF cable just running through an open expansion slot? 

[Pancake]

2 minutes ago, Mikensan said:

@Pancake How do you connect the Thinkserver to the IBM 1015? Is the SFF cable just running through an open expansion slot? 

I use a low profile sff-8087 to sff-8088 adapter, I do wish I had just bought an HBA with external ports in the beginning, but it works

[dalekphalm]

Minor update to my Server:

1x Seagate ST3000DM001 3TB SATA HDD died - no warranty unfortunately - but I did pull it out of a Seagate 3TB USB3.0 external Drive that I got for a really good deal (It was definitely worth losing the warranty).

Bought a replacement, same model (This one I bought new from Canada Computers, so it has the full warranty - though Seagate only offers 2 year warranty on their Desktop drives now).

 

I also picked up the following drives for free (Work was tossing them):

2x 146GB 15K SAS

2x 300GB 15K SAS

 

Just threw them into my FlexRAID pool as a new DRU with ~900GB. As long as each DRU doesn't exceed the capacity of my PPU (Parity Protection Unit - a HDD or series of HDD's dedicated as a virtual parity disk) of 3TB, then the size of each DRU actually doesn't matter that much in FlexRAID (Using Snapshot RAID - cannot say if their newer Real Time RAID has different requirements).

[AidanJMessenger]

UPDATE 2

I've managed to somehow use 4TB in the last year since upgrading, so I've had to upgrade yet again. Unfortunately, I was wrong in thinking that you could easy add 2 more drives to a raid z and have it automatically expand. Instead, I had to add another 3x3tb drives, in another raid z to expand the volume. Have to learn these things somehow! I went with 3TB WD Blue drives for $164 NZD each. A total of $492 NZD, or ~$340USD. So the upgrade costed more than expected, but ah well. I plan on eventually building some form of backup solution, at which point I'll backup all the data, and reconfigure the disks in either a raidz or raidz2 and then copy all the data back to it. For now however, this solution is sufficient and provided me with an additional 6TB of storage. I also upgraded the boot drive from an old 8gb flash drive I bought in 2008 for $80 (I have no idea how this thing is still working), and replaced it with 2x8GB USB 3.0 drives which are mirrored. The web UI environment for freenas is certainly much snappier now which is good. It was slowly getting slower and slower and I wanted to replace the boot drive before it died. Even though the process of a fresh reinstall was intimidating, restoring from the small config backup file actually put everything back exactly the way it was beforehand, so the upgrade was actually a breeze. 

Also recently upgraded to 16GB of ram, taking advantage of my friend visiting America and bringing back some stuff for me. Server is much happier with the 16gb of ram in there!

My cache drive has been removed, as i only have 6 sata ports and now have 6 hard drives. I don't mind, given that it didn't seem to have any affect for me at all anyway and was basically going to waste. 

 

My server hardware is now as follows:

 

Hardware

CASE: Zalman Z1
PSU: Some cheap thing I salvaged. (I really need to upgrade this. Unfortunately power supplies are a huge rip off in NZ)
MB: Gigabyte B85M-D3H
CPU: Intel Core i3 4130 3.4Ghz
HS: Stock Intel heatsink
RAM: 16GB DDR3 1333
HDD 1: 3x 3TB WD Green WD30EZRX (Main RAIDZ Array)
HDD 2: 3x3TB WD Blue WD30EZRZ (also in a raidz array)


Software, Configuration & Usage:
My server is running Freenas, with the 3x3TB drives in one RAIDZ array, and another 3x3TB drives in another RAIDZ array, with 11.5TB of usable storage after redundancy and formatting, and 18TB of raw storage. 500GB of this volume is dedicated to time machine backups for the 2 MacBooks and my main Hackintsoh.  The server is running plex to manage all my media and lets me stream it throughout the house, and I also share it with a few of my friends. I've recently set up Sonarr to automatically download my TV shows, and couch potato to download movies automatically for me. These download using SabNZBD mainly to take advantage of my 100mbps (hopefully soon to be 200/200) connection, and also transmission for torrents. I also have Syncthing running to sync files locally to my laptop and hackintosh so I don't need to rely on always being connected to my network to access some of my important files. 

Backup:
No current backup. Currently looking into either building a backup solution (potentially a raspberry pi with a couple large external HDDs, or something else with faster networking capabilities), or potentially crashplan. Though I've been unsuccessful at setting up crashplan thus far. 

Additional info:
When I first built this server in 2013, I had it running windows server, with 2x1TB drives in Raid 0. I was using some cheap ASrock board, and a pentium G2020. The server ran great, and the G2020 handled transcoding really well - I never noticed any hiccups. However when I upgraded, I wanted to replace the boot drive (an old 250GB drive I'd salvaged), with a 64GB SSD. When trying to clone the old drive to the new SSD, I accidentally wiped the boot drive, and had to figure something out. I decided to try Freenas. I installed windows 8, moved my data to an external, sourced an old 1TB green drive from an old external drive my dad no longer used, and set up freenas. I now had a 64GB SSD to do something with, so I decided to use it as a cache. The server has come a long way since then. With now 18TB of raw storage, 16GB of ram, an i3 and mirrored boot drives. 

 

I have all of this set up in the "server wardrobe", aka the wardrobe in my "office". I have no new interesting pictures of the server itself (you can see old ones in my previous post here. However here is an image of the server "wardrobe" as a spoiler:

 

Spoiler

 

I do however, have a picture of the closet in case anybody is interested. I also aquired a very ancient UPS which I'm using for fun. However unfortunately it's from the 90s and only has a serial port, and cannot get it to work with freenas and a serial to USB adapter. But that's fine. Not really necessary anyway. My router is a Netgear Nighthawk R7000 which I love to bits. Used to have 3 different access points around the house. Since then moved into a smaller house and just bought a single fantastic router. I regret nothing, I'm getting the best speeds ever. I get 130MBPS down, 25MBPS up. I pay for 100/20 (and with the stock ISP router, that's all I used to get). 

 

You may also notice the raspberry pi on top of the server. If you're interested this is currently being used to automate the lighting in my house, and is used to control my lighting with amazon dash buttons. I may also end moving it elsewhere and using it as a backup server for the main server, and attach a couple external HDDs. This solution would be quite cheap, but would be quite slow with only USB 2.0 and 100mbps ethernet. 

 

Total Raw Capacity: 18TB

Total Formatted Capacity: 11.5TB

 

 

[raid0yolo]

Here is my new all-in-one whitebox server. It acts as my router, server, NAS and gaming computer.
 

Raw storage: 63TB
Usable: 42TB
 

OS: ESXi6.0
NAS OS: debian 7 openmediavault + LSI SAS3008 HBA (IT) passthrough + mergerfs + snapraid 
 

Boot: SanDisk 32GB Extreme CZ80 USB 3.0
Datastore#1: SAMSUNG 850 EVO 500GB
Datastore#2: 5TB WD Black
Storage: 5 x 8TB Segate SMR archive
Storage: 6 x 3TB Segate PMR
 

Case Fan: 3 x COUGAR CF-V12HPB 120mm
Case Fan: 2 x SilenX EFX-08-15 80mm fans
CPU Cooler: Noctua NH-U9DXi4 90mm SSO2
PSU: CORSAIR AX series AX760 760W ATX12V
Motherboard: SUPERMICRO MBD-X10SRH-CLN4F-O
RAM: 2 x Kingston 16GB DDR4 KVR21R15D4/16
GPU: MSI Radeon HD 6950 DirectX 11 R6950 Twin Frozr 
CPU: Intel Xeon E5-2670 v3 (ebay)
 

Case: SUPERMICRO 846E16-R1200B (ebay)
Case Backwindow: Norco 4224_backwindow
Case Fanwall: Norco 120mm fan wall: ASIN:B0075790SO




After the fan/case mods the server is dead silent. The only time I can hear it is when the GPU fan kicks in. 


Server only uses 150 Watts at idle. When my desktop VM is running it uses around 200 watts due to the GPU power draw. 


I had to mount the WD 5TB Black drive inside the case because everything connected to the SAS controller gets passed through to the Openmediavault VM. I use the Black drive as backup for the VMs on the SSD. I also use it for jobs too large for the SSD.




I could only plug in one of the SAS cables because the GPU is in the way. In the future I will get a smaller GPU or just build a new gaming computer.




It's not a perfect fit, but it gets the job done. I have 4 large screws securing the backwindow to the case.


GPU passthrough worked on first try without any issues. I gave this VM 24 cpus just so i could take a picture of Task Manager.


Here is my snapraid config. I'm using two 8TB archive drives for parity.


It turns out it's tricky to turn on a desktop VM. My solution was to standup a small webserver that runs a PHP script to SSH into the esxi host to turn on/off my desktop. It's not the most secure solution but it works for now.

Kitty likes the new server.

[RageInvader]

Quote

It turns out it's tricky to turn on a desktop VM. My solution was to standup a small webserver that runs a PHP script to SSH into the esxi host to turn on/off my desktop. It's not the most secure solution but it works for now.

 

I'd love to know more about how you accomplished this, a look at the php code you used would be great, after removing your personal details obviously

 

Cheers

[raid0yolo]

1 hour ago, RageInvader said:

 

I'd love to know more about how you accomplished this, a look at the php code you used would be great, after removing your personal details obviously

 

Cheers

Here are all the files I used for this. Hope it helps.
 

turnOnDesktopVM.zip

[RageInvader]

18 minutes ago, raid0yolo said:

Here are all the files I used for this. Hope it helps.
 

turnOnDesktopVM.zip

Thanks thats fantastic, adapted it and got it working on my unRAID server, now my girlfriend can start her gaming VM without logging into the webUI and me fearing she's going to break something.

 

[Chaosratt]

Wait, I've got that same case. I ended up removing the fan wall and laying some 140MMs across the spot where it was, do you mean that I can just buy the Norco 120mm fanwall and it'll mount up where the supermicro fanwall went, or will I have to drill?

Same for the rear panel, that can be replaced as well?

On 5/4/2016 at 1:41 PM, raid0yolo said: