Jump to content
looney

LTT Storage Rankings

Recommended Posts

Orion Server:

Storage: 24x5TB WD Red drives

RAM: 256GB of DDR3

CPU: 2x E5-2620 v2 @ 2.10GHz

Mobo: ASRock EP2C602-4L/D16 SSI EEB

OS: Ubuntu Linux

Network: 2x 10Gb Ports

Case: SuperMicro 4RU (can't find exact model right now)

RAID: 10

Controller: 2x  LSI LSI00244 (9201-16i)

 

(Sorry about the mess, picture was taken while I was adjusting some networking gear)

zXHucJT.jpg

 

BtTWt69.jpg

carb3Dd.jpg

 

YolRfpV.jpg

Link to post
Share on other sites
Posted · Original PosterOP
37 minutes ago, scottyseng said:

Hmm, curious, how loud is that 1U supermicro?

 

But wow, amazed at how many drives are being bought by both you and @handruin

not that loud really, I mean, its in the basement for a reason but not bad.


Respect the Code of Conduct!

>> Feel free to join the unofficial LTT teamspeak 3 server TS3.schnitzel.team <<

>>LTT 10TB+ Topic<< | >>FlexRAID Tutorial<<>>LTT Speed wave<< | >>LTT Communies and Servers<<

Link to post
Share on other sites
23 hours ago, looney said:

Hate to break it to ya but I'm also planning a little update.

ESX will stay the same.

Current NAS node will also become ESX, I have just bought 2x 2660 v2 for that and 128GB of ram

 

As for the NAS, I bought a 1U supermicro, I'm not sure which CPU's I'll run on them, probably 2x E5-2630 with 192GB of ram.

Also buying 8 more 4TB drives.

 

On 12/29/2016 at 0:55 PM, handruin said:

Even if I don't surpass him, it's fun trying.  In true LTT fashion I made a sad HDD christmas tree from the first delivery of my drives.  :-)

20161229_102733.jpg

 

Buy more, more, more! Bonuses should be good this quarter!


SSD Firmware Engineer

 

| Dual Boot Linux Mint and W8.1 Pro x64 with rEFInd Boot Manager | Intel Core i7-4770k | Corsair H100i | ASRock Z87 Extreme4 | 32 GB (4x8gb) 1600MHz CL8 | EVGA GTX970 FTW+ | EVGA SuperNOVA 1000 P2 | 500GB Samsung 850 Evo |250GB Samsung 840 Evo | 3x1Tb HDD | 4 LG UH12NS30 BD Drives | LSI HBA | Corsair Carbide 500R Case | Das Keyboard 4 Ultimate | Logitech M510 Mouse | Corsair Vengeance 2100 Wireless Headset | 4 Monoprice Displays - 3x27"4k bottom, 27" 1440p top | Logitech Z-2300 Speakers |

Link to post
Share on other sites
Posted · Original PosterOP

Just ordered all the materials for the upgrade, so new system will be running this hardware:

 

Server: Supermicro Server 6017R-NRF4+

CPU: 2x Intel XEON E5-2650v1

RAM: 24x 8GB 1600 DDR3 ECC RDIMM  (192GB)

SSD: 2x 512GB Samsung 850 Pro

HBA: SAS 9200-8E HBA

NIC: Dual port 10G SFP+ NIC

JBOD: SC847 E16-RJBOD1

HDD1: 37x 4TB ST4000DM000 Seagate Desktop Drive

HDD2: 8x 4TB ST4000NM0033 Seagate Constellation ES Drive

 

The new server will run EXSi of USB.

The 2 SSD's will be set up in RAID 1 to server as the ESXi datastore.

I will be running a FreeNAS VM with 8 cores and 160GB RAM allocated to it with the HBA in passthrough mode.

There will be 2 pools, one with the desktop drives and one with the enterprise drives

The desktop drive pool will be 3 vdev's of 12 drives each in RAID-Z2.

The enterprise drive pool will be 1 vdev of 8 drives in RAID-Z2

The remaining 4TB desktop will be a hot-spare.

 

I may still buy a extra small SSD for the ZIL.

 

This will bring the ranking TB's to 176TB excluding hot-spare with 44 drives giving me 666 storage ranking points!

Server of the Beast!

 


Respect the Code of Conduct!

>> Feel free to join the unofficial LTT teamspeak 3 server TS3.schnitzel.team <<

>>LTT 10TB+ Topic<< | >>FlexRAID Tutorial<<>>LTT Speed wave<< | >>LTT Communies and Servers<<

Link to post
Share on other sites

I liked your post...but I really didn't want to.  ;-)

 

I got all my drives in.  I had to ask friends and family to help ordering since Newegg has limits of 5 per customer in a 48-hour window.  I'm still working on getting the rest of the parts but it is looking like it will be the following:

  • Supermicro SC846E16-R1200B chassis, dual PWS-1K21P-1R PSU, 24x hot swap 3.5" drive carriers, 3x 80mm mid plane fans, 2x 80mm rear fans.
  • Supermicro X9DRi-LN4F+ dual socket LGA2011 motherboard
  • 2x E5-2670 (SR0H8)
  • 192GB Memory ( 24x 8GB PC3-10600R DIMM)
  • 1x Supermicro SAS2308 (pcie 3.0) SAS2 HBA (flashed to IT mode, fw 20.00.04.00)
  • 18 x 6TB HGST NAS HDDs

I'm still working out the SSDs I'll use for L2ARC and OS boot.

 

This will put me at 156TB via 12 x 4TB and 18 x 6TB using 30 drives total.

 

While I wait on the chassis, I'm testing my drives inside another system to make sure they're all healthy.

 

 

20170101_135057.jpg


Workstation 1: Intel i7 4790K | Thermalright MUX-120 | Asus Maximus VII Hero | 32GB RAM Crucial Ballistix Elite 1866 9-9-9-27 ( 4 x 8GB) | 2 x EVGA GTX 980 SC | Samsung 850 Pro 512GB | Samsung 840 EVO 500GB | HGST 4TB NAS 7.2KRPM | 2 x HGST 6TB NAS 7.2KRPM | 1 x Samsung 1TB 7.2KRPM | Seasonic 1050W 80+ Gold | Fractal Design Define R4 | Win 8.1 64-bit
NAS 1: Intel Intel Xeon E3-1270V3 | SUPERMICRO MBD-X10SL7-F-O | 32GB RAM DDR3L ECC (8GBx4) | 12 x HGST 4TB Deskstar NAS | SAMSUNG 850 Pro 256GB (boot/OS) | SAMSUNG 850 Pro 128GB (ZIL + L2ARC) | Seasonic 650W 80+ Gold | Rosewill RSV-L4411 | Xubuntu 14.10

Notebook: Lenovo T500 | Intel T9600 | 8GB RAM | Crucial M4 256GB

Link to post
Share on other sites
1 hour ago, looney said:

Just ordered all the materials for the upgrade, so new system will be running this hardware:

 

Server: Supermicro Server 6017R-NRF4+

CPU: 2x Intel XEON E5-2650v1

RAM: 24x 8GB 1600 DDR3 ECC RDIMM  (192GB)

SSD: 2x 512GB Samsung 850 Pro

HBA: SAS 9200-8E HBA

NIC: Dual port 10G SFP+ NIC

JBOD: SC847 E16-RJBOD1

HDD1: 37x 4TB ST4000DM000 Seagate Desktop Drive

HDD2: 8x 4TB ST4000NM0033 Seagate Constellation ES Drive

 

The new server will run EXSi of USB.

The 2 SSD's will be set up in RAID 1 to server as the ESXi datastore.

I will be running a FreeNAS VM with 8 cores and 160GB RAM allocated to it with the HBA in passthrough mode.

There will be 2 pools, one with the desktop drives and one with the enterprise drives

The desktop drive pool will be 3 vdev's of 12 drives each in RAID-Z2.

The enterprise drive pool will be 1 vdev of 8 drives in RAID-Z2

The remaining 4TB desktop will be a hot-spare.

 

I may still buy a extra small SSD for the ZIL.

 

This will bring the ranking TB's to 176TB excluding hot-spare with 44 drives giving me 666 storage ranking points!

Server of the Beast!

 

What do you run on these servers?

Thats alot of $$$£££


PC: Intel i7 3770k | Asus P8Z77-V | Cool Master Storm Trooper | ASUS GTX 1070 Strix | 12 GB DDR3 Ram | 840EVO 500GB | 850EVO 1TB | 2x DELL U2414H | Windows 10 Pro

NAS: AMD Phenom II X6 1090T | Asus M5P97 PRO | 32GB DDR3 RAM | 60GB SSD (BOOT) | Ubuntu 16.04.1 | 6x 3TB WD Red in ZFS Raidz1 (240GB Cache Drive) | 2x 1TB WD Red ZFS Mirror for Containers, Backup and VMs. (120GB Cache Drive)

Laptop: MSI PX 60 2QD | 8GB RAM| 500GB SSD and 256GB SSD | Windows 10 Pro

Ultralight: Toshiba Z930 | 6GB RAM | 120GB SSD | Linux: KDE Neon

Link to post
Share on other sites
Posted (edited)
2 hours ago, Jonny said:

What do you run on these servers?

Thats alot of $$$£££

He runs things... xD

 

But seriously, the servers are used for a variety of things.

The new storage server will run a hypervisor which will run the storage host and also some VMs for rtorrent, sonarr, couchpotato.

The server will also be used to store CCTV footage a few times a year.

 

I should also point out that I know this because me and @looney share a RFC1918 block and have an IPsec tunnel between us, and our servers are joined and managed together.

Edited by .:MARK:.

Comb it with a brick

Link to post
Share on other sites
Posted · Original PosterOP
9 hours ago, Jonny said:

Thats alot of $$$£££

Cost me 0$  and 0£

 

€ however... 

 

:P


Respect the Code of Conduct!

>> Feel free to join the unofficial LTT teamspeak 3 server TS3.schnitzel.team <<

>>LTT 10TB+ Topic<< | >>FlexRAID Tutorial<<>>LTT Speed wave<< | >>LTT Communies and Servers<<

Link to post
Share on other sites
10 hours ago, .:MARK:. said:

He runs things... xD

 

But seriously, the servers are used for a variety of things.

The new storage server will run a hypervisor which will run the storage host and also some VMs for rtorrent, sonarr, couchpotato.

The server will also be used to store CCTV footage a few times a year.

 

I should also point out that I know this because me and @looney share a RFC1918 block and have an IPsec tunnel between us, and our servers are joined and managed together.

I run couchpotato, sonarr, transmissions, emby, Jackett, gitlab, nagios and more on mine.

You must have a good connection speeds between you and looney

1 hour ago, looney said:

Cost me 0$  and 0£

 

€ however... 

 

:P

:D


PC: Intel i7 3770k | Asus P8Z77-V | Cool Master Storm Trooper | ASUS GTX 1070 Strix | 12 GB DDR3 Ram | 840EVO 500GB | 850EVO 1TB | 2x DELL U2414H | Windows 10 Pro

NAS: AMD Phenom II X6 1090T | Asus M5P97 PRO | 32GB DDR3 RAM | 60GB SSD (BOOT) | Ubuntu 16.04.1 | 6x 3TB WD Red in ZFS Raidz1 (240GB Cache Drive) | 2x 1TB WD Red ZFS Mirror for Containers, Backup and VMs. (120GB Cache Drive)

Laptop: MSI PX 60 2QD | 8GB RAM| 500GB SSD and 256GB SSD | Windows 10 Pro

Ultralight: Toshiba Z930 | 6GB RAM | 120GB SSD | Linux: KDE Neon

Link to post
Share on other sites
41 minutes ago, Jonny said:

I run couchpotato, sonarr, transmissions, emby, Jackett, gitlab, nagios and more on mine.

You must have a good connection speeds between you and looney

:D

The storage server is designed to be functional standalone and also somewhat portable (as portable as several Us of rackmount gear can be).

The virtualisation servers run a whole lot more VMs running many services.

The connection between me and looney isn't great, but the most important services are HA and failover.

I'll work on a network diagram for me and @looney when were done with most of the work.


Comb it with a brick

Link to post
Share on other sites

Even though this is not quite 10TB (Hopefully it will be someday), but I wanted to post it anyway :D

 

Hardware

CASE: Dell Precision T7400

PSU: Dell PSU

MB: Dell MB

CPU: 2x Intel Xeon E5440

HS: Stock Dell heatsink

RAM: 16GB DDR2 ECC FB-DIMM

GPU: Nvidia Quadro FX 4600 Nvidia Geforce GT 220

HBA: Dell SAS 1068E, Flashed with LSI IT firmware

HDD 1: 1TB Hitachi HDT721010SLA360 (VMWare ESXi)

HDD 2: 500GB Western Digital WD5000AAKS (ZFS mirror in FreeNAS)

HDD 3: 500GB Seagate ST3500418AS (ZFS mirror in FreeNAS)

Software and Configuration:

I run VMWare ESXi 6.0 as the base OS on this server. I have the 1TB Hitachi running on the onboard Sata controller so it's independent from the 500GB drives which are running off the HBA. Here are the few virtual machines I have running:

  • FreeNAS: General data storage and backups of my computers (with Rsync). I am using hardware pass-through to get the HBA to this VM so I can run ZFS mirror on the 2 500GB hard drives.
  • Windows Server 2016 Datacenter: Currently only runs my VPN so I can access my network remotely.
  • OpenSUSE: Plex server that I use to access my movie collection. I have mounted the freenas volume in this vm so I can store my movies there.
  • Debian: Torrent server (Deluge).

Usage:

Primary role of this server is to do daily backups of my computers and act as media/general storage server. I have Plex Media Server running in OpenSUSE virtual machine with fstab entry so I can store movies on FreeNAS. I have been slowly ripping my kinda large movie collection to this over the last few months. I also have Deluge running in Debian virtual machine for bigger overnight downloads, so I don't have to leave my computer running.

Backup:

As this is the backup for my computers I haven't done anything about backing this up yet.

Additional info:

Due to the location where I had to put the server (heat and noise), I have to use powerline to run networking. So I only get around 50Mb link to it from my other computers which are on gigabit network otherwise. If anyone has any ideas how to make this faster I am open for suggestions.

Photos:

4xfJsXu.jpg

(I have the keyboard on the top of the server because I am too lazy to search the bios for option to disable the warning :D)

 

FquuwUf.jpg

 

DZq0D0A.png?1

 

gRZfGUs.png

 

JN8ADIk.jpg

 

vLaHf6J.png


Devices:

Desktop(s): Main Rig | CPU: R7 1700x, Ram: 16GB, GPU: Radeon HD 7970

Server(s): My Server 

Laptop(s): Macbook Pro 13" (2015) | Running Windows 10 with Bootcamp 

Phone(s): iPhone SE 64GB, Nokia Lumia 925 

Link to post
Share on other sites

I did a thing and put it in a cardboard box. 

 

Specs

Xeon X5472 (2x)

32GB of RAM

16TB of storage

(4x 2TB Seagate ST2000DM001) 

(4x 2TB Hitachi A7K2000) 

Running FreeNAS

 

Huge thanks to @Vitalius for helping me set it up and providing some of the parts!

And now here are some crappy photos.

 

 


 

127d596aaf.jpg

c82fde2d72.jpg   27c36452d2.jpg872416037e.jpg
 

 

So glorious.


Still just happy that flaming pink comic sans wasn't nuked in the forum update. Now. We. Shall. ENLARGE THE FLAMING PINK COMIC SANS!

Link to post
Share on other sites
1 hour ago, TopWargamer said:

I did a thing and put it in a cardboard box. 

 

Specs

Xeon X 5472 (2x)

32GB of RAM

16TB of storage

Running FreeNAS

 

Huge thanks to @Vitalius for helping me set it up and providing some of the parts!

And now here are some crappy photos.

  Hide contents

 


 

127d596aaf.jpg

c82fde2d72.jpg   27c36452d2.jpg872416037e.jpg
 

 

So glorious.

Please follow the template on the first page (the very first build post in the thread) so that we have details like the size, count, and manufacturer of you hard drives. Those are used to place you in the ranking and make the graphs about how many drives we have combined.

Link to post
Share on other sites
11 hours ago, brwainer said:

Please follow the template on the first page (the very first build post in the thread) so that we have details like the size, count, and manufacturer of you hard drives. Those are used to place you in the ranking and make the graphs about how many drives we have combined.

Thanks for the heads up. Fixed it. The specific Seagate drive may be wrong, because I edited the post while on the go. Will double check it later. 


Still just happy that flaming pink comic sans wasn't nuked in the forum update. Now. We. Shall. ENLARGE THE FLAMING PINK COMIC SANS!

Link to post
Share on other sites
On 1/6/2017 at 2:23 AM, TopWargamer said:

I did a thing and put it in a cardboard box. 

 

Specs

Xeon X5472 (2x)

32GB of RAM

16TB of storage

(4x 2TB Seagate ST2000DM001) 

(4x 2TB Hitachi A7K2000) 

Running FreeNAS

 

Huge thanks to @Vitalius for helping me set it up and providing some of the parts!

And now here are some crappy photos.

  Hide contents

 


 

127d596aaf.jpg

c82fde2d72.jpg   27c36452d2.jpg872416037e.jpg
 

 

So glorious.

Wow, first I've seen of this style case. 

 

What are the HDD temperatures like in that setup?  I'd be fairly concerned with them being sandwiched together like that with no direct air flow.

 


Workstation 1: Intel i7 4790K | Thermalright MUX-120 | Asus Maximus VII Hero | 32GB RAM Crucial Ballistix Elite 1866 9-9-9-27 ( 4 x 8GB) | 2 x EVGA GTX 980 SC | Samsung 850 Pro 512GB | Samsung 840 EVO 500GB | HGST 4TB NAS 7.2KRPM | 2 x HGST 6TB NAS 7.2KRPM | 1 x Samsung 1TB 7.2KRPM | Seasonic 1050W 80+ Gold | Fractal Design Define R4 | Win 8.1 64-bit
NAS 1: Intel Intel Xeon E3-1270V3 | SUPERMICRO MBD-X10SL7-F-O | 32GB RAM DDR3L ECC (8GBx4) | 12 x HGST 4TB Deskstar NAS | SAMSUNG 850 Pro 256GB (boot/OS) | SAMSUNG 850 Pro 128GB (ZIL + L2ARC) | Seasonic 650W 80+ Gold | Rosewill RSV-L4411 | Xubuntu 14.10

Notebook: Lenovo T500 | Intel T9600 | 8GB RAM | Crucial M4 256GB

Link to post
Share on other sites
Posted · Original PosterOP

Small update on the lack of updates:IMG_20170109_164524.jpg

 

This is taking a while... 


Respect the Code of Conduct!

>> Feel free to join the unofficial LTT teamspeak 3 server TS3.schnitzel.team <<

>>LTT 10TB+ Topic<< | >>FlexRAID Tutorial<<>>LTT Speed wave<< | >>LTT Communies and Servers<<

Link to post
Share on other sites

Hardware

SERVER DESIGNATION: "Vanaheim" (Norse: World of Vanir, Group of Gods/Goddesses)

CASE: Fractal Design Node 804

PSU: Corsair RM 550W 80+ Gold - Fully Modular

MB: Supermicro X10SL7-F Micro ATX LGA1150

CPU: Intel Xeon E3-1220 V3 3.1Ghz Quad Core

HS: Cooler Master Hyper 212 EVO

RAM: Crucial 16GB DDR3-1600 ECC

RAID CARD: N/A

SSD: N/A

HDD 1: 8x 4TB WD Red 5400 RPM (WD40EFRX) (32TB)

HDD 2: 4x 3TB WD Green 5400 RPM (WD30EURS) (12TB)

 

Software and Configuration:

Running FreeNAS 9.10.1 (d989edd) - Designated "Æsir" (The primary race of Norse Gods)

I have setup two Volumes:

ZFS RAIDZ2 - 8x 4TB Drives - Usable storage of 20.0TB (RAW 32TB) - Designated "Vanir" (A Group of Norse Gods/Goddesses)

ZFS RAIDZ2 - 4x 3TB Drives - Usable storage of 5.1TB (RAW 12TB) - Designated "Asynjur" (Another Group of Norse Gods/Goddesses, Belonging to the Æsir)

 

Usage:

The main Volume "Vanir" is used primarily as a secure redundant storage for my streaming media and content creation storage (Podcast, Gaming footage). Full system drive backups will eventually go here once setup.

The secondary Volume "Asynjur" is used as storage for backups of programs, DVR storage for my Antenna DVR system, and other miscellaneous non-critical data. These are just spare drive I had, and as you'll see below, are hacked to fit in the system.

Backup:

My entire household is in the process of being backed up offline to SOS Online Backup Crashplan. I have CenturyLink Gigabit internet, but the backup is still taking a few weeks.


Photo's:

NAS-Vanir-Drives.jpg

NAS-Vanir-Labels.jpg

NAS-Asynjur.jpg

NAS-Front.jpg

NAS-FreeNAS-Drives.jpg

Link to post
Share on other sites
1 minute ago, Nemockulous said:

Separate box, My windows Desktop. I know FreeNAS has a plugin for Crashplan, but I haven't explored it.

Ah ok. Their plugin last I checked was on an older version and won't update. If you try to manually install the port / package inside a jail it wants the linux kernel module which can't be installed inside a jail. Issue with it being an older version is the newer client uses a different method to authenticate/connect to the crashplan server, and thus not backwards compatible. Nice setup btw.

Link to post
Share on other sites
On 12/24/2016 at 8:48 PM, .:MARK:. said:

Someone is incredibly close to looney on those graphs...

It may take time, but there may be a small surprise on looney's setup, I think all he needs is a little motivation. xDxDxD 

 

I think there should be a different list for all SSD servers, then I'd bother listing mine :P.

Link to post
Share on other sites
15 hours ago, leadeater said:

I think there should be a different list for all SSD servers, then I'd bother listing mine :P.

Personally I want a homelab thread, where people can actually explain and show off their systems/clusters and networks. That's when I'll bother posting my stuff.


Comb it with a brick

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now


  • Recently Browsing   0 members

    No registered users viewing this page.

×