LTT Storage Rankings

[.:MARK:.]

2 hours ago, Jonny said:

What do you run on these servers?

Thats alot of $$$£££

He runs things...

 

But seriously, the servers are used for a variety of things.

The new storage server will run a hypervisor which will run the storage host and also some VMs for rtorrent, sonarr, couchpotato.

The server will also be used to store CCTV footage a few times a year.

 

I should also point out that I know this because me and @looney share a RFC1918 block and have an IPsec tunnel between us, and our servers are joined and managed together.

Edited January 2, 2017 by .:MARK:.

[looney]

9 hours ago, Jonny said:

Thats alot of $$$£££

Cost me 0$  and 0£

 

€ however... 

 

[Jonny]

10 hours ago, .:MARK:. said:

He runs things...

 

But seriously, the servers are used for a variety of things.

The new storage server will run a hypervisor which will run the storage host and also some VMs for rtorrent, sonarr, couchpotato.

The server will also be used to store CCTV footage a few times a year.

 

I should also point out that I know this because me and @looney share a RFC1918 block and have an IPsec tunnel between us, and our servers are joined and managed together.

I run couchpotato, sonarr, transmissions, emby, Jackett, gitlab, nagios and more on mine.

You must have a good connection speeds between you and looney

1 hour ago, looney said:

Cost me 0$  and 0£

 

€ however... 

 

[.:MARK:.]

41 minutes ago, Jonny said:

I run couchpotato, sonarr, transmissions, emby, Jackett, gitlab, nagios and more on mine.

You must have a good connection speeds between you and looney

The storage server is designed to be functional standalone and also somewhat portable (as portable as several Us of rackmount gear can be).

The virtualisation servers run a whole lot more VMs running many services.

The connection between me and looney isn't great, but the most important services are HA and failover.

I'll work on a network diagram for me and @looney when were done with most of the work.

[aakopa]

Even though this is not quite 10TB (Hopefully it will be someday), but I wanted to post it anyway 

 

Hardware

CASE: Dell Precision T7400

PSU: Dell PSU

MB: Dell MB

CPU: 2x Intel Xeon E5440

HS: Stock Dell heatsink

RAM: 16GB DDR2 ECC FB-DIMM

GPU: Nvidia Quadro FX 4600 Nvidia Geforce GT 220

HBA: Dell SAS 1068E, Flashed with LSI IT firmware

HDD 1: 1TB Hitachi HDT721010SLA360 (VMWare ESXi)

HDD 2: 500GB Western Digital WD5000AAKS (ZFS mirror in FreeNAS)

HDD 3: 500GB Seagate ST3500418AS (ZFS mirror in FreeNAS)

Software and Configuration:

I run VMWare ESXi 6.0 as the base OS on this server. I have the 1TB Hitachi running on the onboard Sata controller so it's independent from the 500GB drives which are running off the HBA. Here are the few virtual machines I have running:

• FreeNAS: General data storage and backups of my computers (with Rsync). I am using hardware pass-through to get the HBA to this VM so I can run ZFS mirror on the 2 500GB hard drives.
• Windows Server 2016 Datacenter: Currently only runs my VPN so I can access my network remotely.
• OpenSUSE: Plex server that I use to access my movie collection. I have mounted the freenas volume in this vm so I can store my movies there.
• Debian: Torrent server (Deluge).

Usage:

Primary role of this server is to do daily backups of my computers and act as media/general storage server. I have Plex Media Server running in OpenSUSE virtual machine with fstab entry so I can store movies on FreeNAS. I have been slowly ripping my kinda large movie collection to this over the last few months. I also have Deluge running in Debian virtual machine for bigger overnight downloads, so I don't have to leave my computer running.

Backup:

As this is the backup for my computers I haven't done anything about backing this up yet.

Additional info:

Due to the location where I had to put the server (heat and noise), I have to use powerline to run networking. So I only get around 50Mb link to it from my other computers which are on gigabit network otherwise. If anyone has any ideas how to make this faster I am open for suggestions.

Photos:

(I have the keyboard on the top of the server because I am too lazy to search the bios for option to disable the warning )

 

 

 

 

 

[TopWargamer]

I did a thing and put it in a cardboard box. 

 

Specs

Xeon X5472 (2x)

32GB of RAM

16TB of storage

(4x 2TB Seagate ST2000DM001) 

(4x 2TB Hitachi A7K2000) 

Running FreeNAS

 

Huge thanks to @Vitalius for helping me set it up and providing some of the parts!

And now here are some crappy photos.

 

 

 



  
 

 

So glorious.

[brwainer]

1 hour ago, TopWargamer said:

I did a thing and put it in a cardboard box. 

 

Specs

Xeon X 5472 (2x)

32GB of RAM

16TB of storage

Running FreeNAS

 

Huge thanks to @Vitalius for helping me set it up and providing some of the parts!

And now here are some crappy photos.

  Hide contents

 

 



  
 

 

So glorious.

Please follow the template on the first page (the very first build post in the thread) so that we have details like the size, count, and manufacturer of you hard drives. Those are used to place you in the ranking and make the graphs about how many drives we have combined.

[MG2R]

6 hours ago, TopWargamer said:

I did a thing and put it in a cardboard box.

  Hide contents

 

 

Love it!

[TopWargamer]

11 hours ago, brwainer said:

Please follow the template on the first page (the very first build post in the thread) so that we have details like the size, count, and manufacturer of you hard drives. Those are used to place you in the ranking and make the graphs about how many drives we have combined.

Thanks for the heads up. Fixed it. The specific Seagate drive may be wrong, because I edited the post while on the go. Will double check it later. 

[handruin]

On 1/6/2017 at 2:23 AM, TopWargamer said:

I did a thing and put it in a cardboard box. 

 

Specs

Xeon X5472 (2x)

32GB of RAM

16TB of storage

(4x 2TB Seagate ST2000DM001) 

(4x 2TB Hitachi A7K2000) 

Running FreeNAS

 

Huge thanks to @Vitalius for helping me set it up and providing some of the parts!

And now here are some crappy photos.

  Hide contents

 

 



  
 

 

So glorious.

Wow, first I've seen of this style case. 

 

What are the HDD temperatures like in that setup?  I'd be fairly concerned with them being sandwiched together like that with no direct air flow.

 

[looney]

Small update on the lack of updates:

 

This is taking a while... 

[Nemockulous]

Hardware

SERVER DESIGNATION: "Vanaheim" (Norse: World of Vanir, Group of Gods/Goddesses)

CASE: Fractal Design Node 804

PSU: Corsair RM 550W 80+ Gold - Fully Modular

MB: Supermicro X10SL7-F Micro ATX LGA1150

CPU: Intel Xeon E3-1220 V3 3.1Ghz Quad Core

HS: Cooler Master Hyper 212 EVO

RAM: Crucial 16GB DDR3-1600 ECC

RAID CARD: N/A

SSD: N/A

HDD 1: 8x 4TB WD Red 5400 RPM (WD40EFRX) (32TB)

HDD 2: 4x 3TB WD Green 5400 RPM (WD30EURS) (12TB)

 

Software and Configuration:

Running FreeNAS 9.10.1 (d989edd) - Designated "Æsir" (The primary race of Norse Gods)

I have setup two Volumes:

ZFS RAIDZ2 - 8x 4TB Drives - Usable storage of 20.0TB (RAW 32TB) - Designated "Vanir" (A Group of Norse Gods/Goddesses)

ZFS RAIDZ2 - 4x 3TB Drives - Usable storage of 5.1TB (RAW 12TB) - Designated "Asynjur" (Another Group of Norse Gods/Goddesses, Belonging to the Æsir)

 

Usage:

The main Volume "Vanir" is used primarily as a secure redundant storage for my streaming media and content creation storage (Podcast, Gaming footage). Full system drive backups will eventually go here once setup.

The secondary Volume "Asynjur" is used as storage for backups of programs, DVR storage for my Antenna DVR system, and other miscellaneous non-critical data. These are just spare drive I had, and as you'll see below, are hacked to fit in the system.

Backup:

My entire household is in the process of being backed up offline to SOS Online Backup Crashplan. I have CenturyLink Gigabit internet, but the backup is still taking a few weeks.

Photo's:

[Mikensan]

@Nemockulous - do you have crashplan running inside a jail, or on a separate box (windows/linux)?

[Nemockulous]

1 minute ago, Mikensan said:

@Nemockulous - do you have crashplan running inside a jail, or on a separate box (windows/linux)?

Separate box, My windows Desktop. I know FreeNAS has a plugin for Crashplan, but I haven't explored it.

[Mikensan]

1 minute ago, Nemockulous said:

Separate box, My windows Desktop. I know FreeNAS has a plugin for Crashplan, but I haven't explored it.

Ah ok. Their plugin last I checked was on an older version and won't update. If you try to manually install the port / package inside a jail it wants the linux kernel module which can't be installed inside a jail. Issue with it being an older version is the newer client uses a different method to authenticate/connect to the crashplan server, and thus not backwards compatible. Nice setup btw.

[leadeater]

On 12/24/2016 at 8:48 PM, .:MARK:. said:

Someone is incredibly close to looney on those graphs...

It may take time, but there may be a small surprise on looney's setup, I think all he needs is a little motivation.  

 

I think there should be a different list for all SSD servers, then I'd bother listing mine .

[Nemockulous]

16 minutes ago, leadeater said:

I think there should be a different list for all SSD servers, then I'd bother listing mine .

Here I am just happy to be sitting at 27th (whenever this gets updated iunno)

[.:MARK:.]

15 hours ago, leadeater said:

I think there should be a different list for all SSD servers, then I'd bother listing mine .

Personally I want a homelab thread, where people can actually explain and show off their systems/clusters and networks. That's when I'll bother posting my stuff.

[Lurick]

6 minutes ago, .:MARK:. said:

Personally I want a homelab thread, where people can actually explain and show off their systems/clusters and networks. That's when I'll bother posting my stuff.

That's what I was using this thread for:

 

[handruin]

Hardware

 

CASE: Supermicro SC846E16-R1200B chassis with BPN-SAS2-846EL SAS 2 backplane

PSU: Supermicro dual PWS-1K21P-1R 1200 watt PSUs

MB: Supermicro X9DRi-LN4F+ dual socket LGA2011 motherboard

CPU: 2x Intel Xeon E5-2670 (SR0H8)

HS: 2x Supermicro SNK-P0048P CPU Coolers

RAM: 192GB ECC Memory (24x 8GB PC3-10600R DIMM)

RAID CARD 1: Supermicro SAS2308 SAS2 HBA (flashed to IT mode, fw 20.00.04.00)

SSD: Samsung SV843 enterprise 960GB

HDD 1: 20x 6TB HGST NAS 7200 RPM

HDD 2: 4x 1.5TB Samsung Ecogreen 5400 RPM

NIC: Mellanox ConnectX 2 - MNPA19-XTR HP 10Gb ethernet (SFP+ DAC cable)

 

Software and Configuration:

The OS is Ubuntu Server 16.04 with ZFS on Linux configured.  I also have samba installed to us for SMB/CIFS/NFS network file sharing.  I will also be using rsync to keep files synchronized between my two NAS systems.  Other than those components there are a few other basics to lock down the system to keep it protected.  Given the CPU and RAM on this system this may end up becoming my primary Plex server to take over for my other NAS.

 

Right now I have no SLOG or L2ARC configured until I do more performance testing.  I've partitioned a little less than half of the Samsung SV843 SSD to be used for either of these tasks.  I realize that a dedicated SSD should be used for this purpose but I don't have many synchronous writes to take advantage of a small SLOG and my system has tons of memory to use for ARC that I think configuring an L2ARC will only detract from performance.

 

My primary zpool is configured as 2 vdevs in raidz2 with 10 disks in each.

The secondary pool will be 1 vdev in raidz with 4 disks.

Both my zpools will be configured with compression=lz4 and only the primary pool will use ashift=12 since it's using 4k drives.

 

Usage:

Backups and media warehouse.  Potentially a new and/or replacement Plex server.  This will be used for other processing of data in the background.

 

Backup:

This is the backup to my other NAS as well as a backup to data on my desktop systems.

 

Additional info:

This is my second large NAS.  The first one is listed earlier in this thread.

The Mellanox 10Gb NICs will be direct connected to each of the NAS systems to allow for higher bandwidth syncs.

 

Photo's:

 

 

 

 

[Jarsky]

On 1/14/2017 at 6:50 PM, handruin said:

 

 

Beautiful 

I wish that wasn't about $10,000 worth of drives in my currency....

[mymouthandi]

I have enough drives (8x2TB, 3x4TB) and hardware to get on this list, I just need to build something. 

[Dark]

Had 32x 850 Pro 2TB drives arrive today.

 

Would be fun to play with them in a massive RAID0 config if I wasn't on a time crunch.

 

[79wjd]

5 minutes ago, Dark said:

Had 32x 850 Pro 2TB drives arrive today.

 

Would be fun to play with them in a massive RAID0 config if I wasn't on a time crunch.

 

Wanna send 10 of those my way? Thanks. 

[Gronnie]

1 hour ago, Dark said:

Had 32x 850 Pro 2TB drives arrive today.

 

Would be fun to play with them in a massive RAID0 config if I wasn't on a time crunch.

 

What are you building, and is it for business or pleasure?