LTT Storage Rankings

The_Ellimist · July 11, 2020

I really like the way you managed to do that. Very cool 3d printed case mod.

thedarkdad3 · August 1, 2020

I'll play. I should add. I bought the unit on ebay (new) but as a shell more or less. Alot of these parts I got used or new off ebay and slowly built it out. It came with like 8GB ram and like a BRONZE CPU and no RAID and no cages etc etc

Actual Server unit ML110 GEN 10
https://buy.hpe.com/us/en/servers/tower-servers/proliant-ml100-servers/proliant-ml110-server/hpe-proliant-ml110-gen10-server/p/1010192782

SPECS

Power HPE Redundant Power Supply Enablement Kit system redundancy kit

x2 865409-001 FS HPE 800W Flex Slot Platinum Hot Plug Low Halogen Power Supply Kit

OS Windows Datacenter 2019

CPU 4112 Intel Xeon Silver 4112 Processor

Memory 128GB Crucial CT32G4LFD4266 32Gb Ddr4-2666 Lrdimm 32Gb Ddr4-2666 Lrdimm 1.2V Cl19

RAID CARD 408i-p HPE Smart Array P408i-p SR Gen10 Controller

GPU P200 NVIDIA VCQP2000-PB Quadro P2000

Storage WD 6TB (x8) WD Red 6TB NAS 5400 RPM Class, SATA 6 Gb/s, SMR, 256MB Cache, 3.5" - WD60EFAX

CAGES x2 HPE ML350 Gen10 4LFF Non Hot Plug Drive Cage Kit

10GBE
Intel X520-SR2 2-Port 10G SFP+ Ethernet, PCI-e E10G42BFSR 82599ES Converged Network Adapter, PCI-E

jagdtigger · September 5, 2020

Are virtual storage servers allowed for entry?

(XCP-NG host running truenas in VM, HW raid card passed over to VM)

Edited September 5, 2020 by jagdtigger
added some clarification

Jarsky · September 6, 2020

17 hours ago, jagdtigger said:

Are virtual storage servers allowed for entry?

(XCP-NG host running truenas in VM, HW raid card passed over to VM)

yeah that configuration is fine, just put the details in your post

jagdtigger · September 6, 2020

Hardware

Motherboard: MSI MS-7A33 (x370 chipset)
CPU: AMD R7 1700X
Cooling: be quiet 120mm AIO
Case: Aerocool Strike-X
Network: integrated, Intel I350T4
GPU: GeForce GT 710
RAM: 2x CORSAIR 16GB Vengeance LPX DDR4 3000MHz CL16 KIT CMK16GX4M2C3000C16
PSU: be quiet modular (cant see the sticker)
HBA: HP H220 IT mode
Boot drive: XCP-NG virtual drive
Drives: 3xST10000VN0004-1Z, 2xST10000VN0008-2J
Hot Spares: None

Total capacity: 50TB (Raw) 26,6 TB(Usable)

50xln(5) = 80,47

Software and Configuration:

XCP-NG on bare metal, truenas core inside VM, HBA passed over to VM directly. VM has 16GB RAM and 4 cores.

Usage:

stores music, movies, backups, photos, videos, and downloads.

Backup:

DS416 via syncthing, then from DS416 to backblaze via Hyper Backup(encrypted)

Photos:

(yes i know my room is a mess and dusty)

looney · October 19, 2020

Hey guys, long time no see, I can't be bothered to fill in the template, way to much info, whoever made that template is truly as massive a idiot.

But here is a picture of what my home rack(s) look like now:

brwainer · October 19, 2020

45 minutes ago, looney said:

Hey guys, long time no see, I can't be bothered to fill in the template, way to much info, whoever made that template is an idiot.

But here is a picture of what my home rack(s) look like now:

Very nice. What switches are you using? They almost look like white-label ones that you can put open source switch OS onto.

looney · October 19, 2020

Just now, brwainer said:

Very nice. What switches are you using? They almost look like white-label ones that you can put open source switch OS onto.

It's an arista DCS-7124SX-SSD-F, never really looked into ONIE / SONiC compatibility.
I'm also looking to replace it in the near future, the availability of firmware updates is bugging me even though its one hell of a switch.

Top switch is just a Cisco WS-C3750X-48P-S, needed something for ipmi and access points / patch points throughout the house.

Jakoski360 · October 22, 2020

TRITIUM: 3 years ago it was a prebuild Packard Bell 2GB Athlon X2 4000+ box with two 500GB SATAII HDDs...

Hardware

CASE: Corsair 750D modified to support a SilverStone SST-FS304B 4 Bay Caddy

PSU: Corsair HX1200

MB: Intel Server Board S2600CP4

CPU: 1x Intel Xeon E5-2680 v2 (2nd one on the way soon)

HS: 1x Cooler Master Hyper 212 Black (see above)

RAM: 4x16GB ECC DDR3-1600 (for now)

HBA: 2x LSI 9207-8i IT mode, HDDs running from HBA 1, HBA 2 unused but in place for 8 more ST4000s.

NETWORK: 4x1Gb via integrated Intel I350-T4, 1x10Gb via SFP+ Mellanox ConnectX2

SSD: 3x 180GB Intel SSD Pro 1500

HDD: 8x 4TB Seagate IronWolf ST4000VN008

Software and Configuration:

OS/Software: Ubuntu Server 20.04, standard LAMP stack for website hosting, QEMU+KVM for VMs, ZoL for storage pools shared via Samba and iSCSI, UniFi controller,

tank: 8x4TB zfs raidz2 ~32TB raw ~22TB usable over SMB

dozer: 3x180GB zfs raidz1 ~0.54TB raw ~0.34TB usable over iSCSI

Usage:

Music, TV & Movies via Plex on dozer, Backups destination (zfs-auto-snapshot every 15 minutes server-side for peace of mind), VMs on dozer replicated to tank nightly, general file storage via Samba shares, Games (picky ones that don't like network shares live on an iSCSI zvol).

Backup:

~8TB raw ~5.4TB usable, my usually powered off box (ARGON) is a backup for anything "mission critical" for lack of a better term. Every couple weeks I manually boot it up triggering a couple hours of rsync.

Additional info:

I swear 22TB is enough... nervously sweats at 81% allocated

Photo's:

TBC

jagdtigger · November 8, 2020

On 10/19/2020 at 11:08 PM, looney said:

Hey guys, long time no see, I can't be bothered to fill in the template, way to much info, whoever made that template is truly as massive a idiot.

But here is a picture of what my home rack(s) look like now:

Spoiler

Nice.

A bit of an update to mine, one more seagate hdd is on its way out. Replacement will be probably a wd hc510. (Ill edit this post after i got the replacement.)

/EDIT

One Seagate out (ST10000VN0004), one HGST in(HUH721010ALE600).

bonox · November 9, 2020

I found this thread looking for other stuff while building my recent storage box, so thought i'd join in.

Hardware

CASE: Rack (primary server) and Custom (second server)

PSU: All sorts of OEM - EMC and HP in the server and disk shelves, Corsair AX850 in the second server

MB: HP DL360, Z68XP-UD3R

CPU: two E5645, one i5-2500K

HS: HP DL360, Thermalright IFX14

RAM: 120GB DDR3 ECC, 16GB DDR3 vanilla

RAID CARD 1: LSI 9200-16e

RAID CARD 2: LSI 9240-8i , Flashed with "IT" firmware

SSD: Samsung EVO 860 500GB

HDD 1: 45x 2TB Seagate and Hitachi - Constellation ES.1 ST2000NM001 and HUS723020ALS640

HDD 2: 8x 4TB Seagate mixed shucked SMR crap

HDD 3: 4x 8TB Seagate Ironwolf ST8000VN004

HDD 4: Assorted stuff like the four 10Krpm HP 146GB units in the DL360. Nothing more than boot drives.

NIC: Two Mellanox ConnectX-3 10Gbe

Software and Configuration:

freeNAS 11.3U4 on one server, TrueNAS 12.0 on the HP

Server one: HP DL360, with three attached drive shelves and four internal drives

Neo: 28 Drive array: 4 stripe of 7 Disk RAIDZ2 of 2TB, 4 hot spares, 5 cold spares

Mouse: One 8 drive array; RAIDZ2 of 2TB

Morpheus: 4 disk array RAID1+0 of 8TB

Server two: i5/Z68, internal drives only

Trinity: One 8 drive array; RAIDZ2 of 4TB

Usage:

Two datasets on the main array - 1MB record size for vlogs and my travel video, 128kB for photos, music, backups etc (Neo)

Second array as backup of first (Mouse)

Others - misc scratch space and games (Morpheus)

Backup:

External USB drives as a last ditch effort offsite, and Server number two (Trinity) onsite

Additional info:

Started the freeNAS journey with the old PC when a friend bought an $800 4 bay Synology NAS and I thought I could do way better for the money. Got way out of hand afterwards once I started buying cheap used enterprise gear. If you can handle the noise, I can't recommend proper servers highly enough, especially for the ipmi/remote management capability.

Photo's:

maxtch · November 25, 2020

I am consolidating and migrating my NAS to a 24-drive unit filled with old drives - either my own old drives or recertified refurbishments, so expect an update.

Spoilers:

Spoiler

Hardware

CPU: Intel Core 2 Quad Q9550S
Cooler: Lenovo OEM LGA775 cooler
Motherboard: Asus P5BV-C
RAM: 4x Micron 2GB DDR2-800 Unregistered ECC = 8GB
PSU: TBD 650W+
SAS RAID controller: Broadcom MegaRAID 9271-8iCC
SAS expander: HP-branded LSI card with x8 uplink, x24 internal downlonk and x4 external downlink
Boot drive: Ramsta 120GB SATA SSD (TBD)
Data drive: 10x Hitachi HUS72303CLAR3000 3TB SAS HDD
Data drive: 5x WD WD30EFRX 3TB SATA HDD
Data drive: 1x Hitachi HUA5C3030ALA640 3TB SATA HDD
Data drive: 4x WD WD20EFRX 2TB SATA HDD
Data drive: 3x WD WD20EZRX 2TB SATA HDD
Data drive: 1x TBD 2TB SAS HDD
Case: TBD 4U 24-tray rack-mount case

Software

OS: Ubuntu Server 20.04 LTS, configured to be headless and operated over serial port.
RAID setup: two VD's: one 8-drive RAID6 from the 2TB units forming a 12TB array, one 16-drive RAID60 from the 3TB drives forming a 36TB array. The 36TB array holds my downloads and media library, while the 12TB array holds backups of all my other computers.

Usage: This is my main NAS.

Backups: It is too large to backup using any of my current solutions. However the 4 external-facing SAS ports from the expander is giving me hope for a possible tape based solution.

maxtch · December 15, 2020

Here is the final update, with me updating the base platform as well. Now my storage server features 24 drives and 64TB total capacity.

Hardware

CPU: Intel Xeon L3426 (Lynnfield 4C/8T, 1.86/3.2GHz)
CPU cooler: Deep Cool's imitation of an Intel stock cooler with a copper slug.
Motherboard: Asus P7F-M with ASMB4-iKVM
RAM: 2x Samsung DDR3-1600 Unbuffered ECC SDRAM (underclocked to DDR3-1333)
GPU: Gigabyte GeForce GTX 1060 3GB G1 Gaming
PSU: Antec NeoEco 650W JD version semi-modular PSU, NE650M is the closest approximation
SAS RAID controller: Broadcom MegaRAID 9271-8i with CacheCade Pro 2.0 hardware key and LSICVM01 CacheVault module
SAS expander: HPE SAS Expander card, with 2.10c firmware for SATA 3Gbps and SAS 6Gbs support
Boot drive: Gigabyte 480GB SATA SSD
Data drive: 10x Hitachi HUS72303CLAR3000 3TB SAS HDD
Data drive: 5x WD WD30EFRX 3TB SATA HDD
Data drive: 1x Hitachi HUA5C3030ALA640 3TB SATA HDD
Data drive: 4x WD WD20EFRX 2TB SATA HDD
Data drive: 3x WD WD20EZRX 2TB SATA HDD
Data drive: 1x Hitachi HUS72302CLAR2000 2TB SAS HDD
Case: Local-brand 4U 24-bay hot swappable case.

Software & Usage

OS: Proxmox VE 6.3 installed on the SSD, with all hardware HDD arrays passed through directly into a VM. The SSD also serves as boot drives for all VMs.
RAID setup: two arrays. One 8-unit RAID-6 with the 2TB drives forming a 12TB volume, one 16-unit RAID-60 with the 3TB drives forming a 36TB volume. Total useable capacity is 48TB.
VMs: two Ubuntu Server VM's. One have the two RAID arrays passed through serving as my Nextcloud NAS. The other have the GPU passed through serving as my NAS-backed Jellyfin server.

Backup

This server is currently too large for me to backup using existing means.
The SAS expander has a x4 external port which I plan to add a tape drive to down the road.

Photos & terminal dumps

Terminal dump listing all my drives:

root@kagayaki-homare:~# storcli /c0/e16/sall show
Controller = 0
Status = Success
Description = Show Drive Information Succeeded.


Drive Information :
=================

------------------------------------------------------------------------------
EID:Slt DID State DG     Size Intf Med SED PI SeSz Model                   Sp 
------------------------------------------------------------------------------
16:0     17 Onln   0 1.818 TB SAS  HDD N   N  512B HUS72302CLAR2000        U  
16:0     18 Onln   0 1.818 TB SATA HDD N   N  512B WDC WD20EZRX-00D8PB0    U  
16:0     19 Onln   0 1.818 TB SATA HDD N   N  512B WDC WD20EZRX-00D8PB0    U  
16:0     20 Onln   0 1.818 TB SATA HDD N   N  512B WDC WD20EZRX-00D8PB0    U  
16:0     21 Onln   0 1.818 TB SATA HDD N   N  512B WDC WD20EFRX-68EUZN0    U  
16:0     22 Onln   0 1.818 TB SATA HDD N   N  512B WDC WD20EFRX-68EUZN0    U  
16:0     51 Onln   0 1.818 TB SATA HDD N   N  512B WDC WD20EFRX-68EUZN0    U  
16:0     52 Onln   0 1.818 TB SATA HDD N   N  512B WDC WD20EFRX-68EUZN0    U  
16:0     40 Onln   1 2.728 TB SATA HDD N   N  512B WDC WD30EFRX-68EUZN0    U  
16:0     41 Onln   1 2.728 TB SATA HDD N   N  512B WDC WD30EFRX-68EUZN0    U  
16:0     42 Onln   1 2.728 TB SATA HDD N   N  512B WDC WD30EFRX-68EUZN0    U  
16:0     43 Onln   1 2.728 TB SAS  HDD N   N  512B HUS72303CLAR3000        U  
16:0     44 Onln   1 2.728 TB SAS  HDD N   N  512B HUS72303CLAR3000        U  
16:0     45 Onln   1 2.728 TB SAS  HDD N   N  512B HUS72303CLAR3000        U  
16:0     46 Onln   1 2.728 TB SAS  HDD N   N  512B HUS72303CLAR3000        U  
16:0     47 Onln   1 2.728 TB SAS  HDD N   N  512B HUS72303CLAR3000        U  
16:0     23 Onln   1 2.728 TB SAS  HDD N   N  512B HUS72303CLAR3000        U  
16:0     24 Onln   1 2.728 TB SAS  HDD N   N  512B HUS72303CLAR3000        U  
16:0     37 Onln   1 2.728 TB SATA HDD N   N  512B Hitachi HUA5C3030ALA640 U  
16:0     38 Onln   1 2.728 TB SATA HDD N   N  512B WDC WD30EFRX-68EUZN0    U  
16:0     39 Onln   1 2.728 TB SATA HDD N   N  512B WDC WD30EFRX-68EUZN0    U  
16:0     48 Onln   1 2.728 TB SAS  HDD N   N  512B HUS72303CLAR3000        U  
16:0     49 Onln   1 2.728 TB SAS  HDD N   N  512B HUS72303CLAR3000        U  
16:0     50 Onln   1 2.728 TB SAS  HDD N   N  512B HUS72303CLAR3000        U  
------------------------------------------------------------------------------

EID-Enclosure Device ID|Slt-Slot No.|DID-Device ID|DG-Drive Group|
Intf-Interface|Med-Media Type|SED-Self Encryptive Drive|PI-Protection Info|
SeSz-Sector Size|Sp-Spun|U-Up|D-Down|T-Transition|UGood-Unconfigured Good|UBad-Unconfigured Bad|Offln-Offline|Onln-Online|Rbld-Rebuild|Cpybck-Copyback|GHS-Global Hot Spare|DHS-Dedicated Hot Spare

Photo of the guts of the server, pre-insertion in the rack, taken when I was still planning to use a traditional BBU and a Crucial BX200 boot SSD on the server:

The server after it is inserted into the rack and loaded with the drives:

Green_man_5 · March 22, 2021

My Storage server V3,

V2 (December 2016)

V1 (January 2015)

Hardware

CASE: Norcotek RPC4224

PSU: Corsair RM-850

MB: Asus X99-A

CPU: Intel Xeon E5 2620 v4

HS: Noctua NH-U9DX i4

RAM: 96GB DDR4 2400

GPU: GTX 970

HBA CARD: 9305-24i

SSD: 2x Intel 520 series 240GB

SSD: Samsung 970 EVO 500GB

HDD: 24x 8TB WD Red

NIC: Intel X520-DA2 10Gb

Software and Configuration:

TrueNAS core 12 on mirrored intel 240GB ssd

3 vDevs, each vDev is 8 drives in z2

L2ARC Samsung 970 EVO 500GB

192TB Raw, 120TB Usable

Usage:

Plex Media storage (server is run on a different box)

Edit Videos off it

will be used as VM storage well I get NUC cluster sorted

Backup:

The data is backed up to a few 4TB and 5TB external drives that are only connected during the backup.

I plan on using some of my 10x 4TB and Raid card from V2 server and old desktop parts to make a backup server

Photo's:

will edit with photos later

Kapoue · April 17, 2021

Hardware

CASE : inter-tech 4U 4416

PSU : Corsair HX 850

MB : HUANANZHI X79

CPU : Intel® Xeon® CPU E5-2690

HS : Noctua NH-D9DX i4 3U

RAM : 32GB ECC Memory (2x 16GB PC3-10600R DIMM)

GPU : Asus GeForce GT 710 BRK

CONTROLLER CARD: LSI 9201

NVME : Samsung 970 evo 500 Go (for cache)

HDD 1 : 12x 4TB Iron Wolf NAS (Array)

HDD 2 : 2x 4TB Iron Wolf NAS (Parity)

NIC : Asus Xg-c100c

Software and Configuration:

The OS is UnRAID 6.9.1. I use a large array of 12 drive plus 2 parity drive. No VM (for now)

Usage:

Backups and Storage for large video and timelapse project.

I use a Soft raid 5 volume on my main station ( a 2009 MacPro).

Problem : very slow transfer speed with folder with lot of JPG (every timelapse project).

Photo's:

besterino · September 25, 2021

Been an opportunistic / sporadic reader-only from time to time. This time looking for a lancache thread and while browsing the subforum stumbled into this one. Me likey. =)

Thought I'd give something back and show off my current main homeserver.

Hardware

Case: Intertech 4U-4416 (same as directly above this post) =)

CPU: Threadripper 3960X
Mainboard: ASRockRack TRX40D8-2N2T
RAM: 8x16GB Kingston ECC UDIMM (mixed 4x2666 and 4x3200)
HBA: Avago SAS3224 24Port 12GSAS
NIC: Mellanox ConnectX-4 Dual Port (1x 100Gbit, 1x10Gbit with adapter) = 8 SRIOV-Devices (4+4)
PSU: Seasonic Prime 1300W
Storage Total 19,5 SSD-only:

4x1TB Samsung 980 Pro (PCIE passthrough)
4x2TB ADATA (PCIE passthrough)
2x1TB Samsung PM9A1 (PCIE passthrough)
4x1TB Samsung 860 EVO (RDM passthrough)
4x256GB ADATA SATA SSD (RDM passthrough)
2x240GB Samsung SM883 (VM Storage)
1x120GB Samsung SM863 (VM Storage)
1x120GB SanDisk SSD (crap/playground/backup)
1x 256GB Samsung 830 (crap/playground/backup)

Cooling

(bit unusual for a server maybe)

Watercool CPU-Block Heatkiller IV
Watercool Heatkiller Tube 100 (multitop / DDC pump)
Pump DDC 1Plus T PWM
Aquacomputer HighFlow flow sensor
Aquacomputer QUADRO
HWLabs 360 radiator
Various (Barrow fittings / snake adapter, Bitspower quick disconnects, Watercool EPDM tube)
6x Noctua NF-A12x25 PWM (push/pull)
2x Noctua NF-A8 PWM

Software & Configuration:

ESXi "all-in-one": ESXi 6.7U3 + Storage-VM(s) - in particular serving back NFS-storage for VM-OS-VMDKs

Usage

Main-Server with:

1. Solaris-Storage-VM (main storage)

2. HyperV-Storage-VM (for smb direct only, high speed storage)

3. APCUPSD-VM (for shutting down various stuff in a controlled way)

4. SophosUTM-VM (Firewall)

5. Office-VM (hate reinstalling stuff just because I screwed up something bare metal)

6. Game-Server-VMs: Minecraft (various), Conan Exiles (2x), Ark Evolved (2x), Valheim

7. Teamspeak-Server-VM

Formerly: Win10-GPU/USB-Passthrough-VM (discontinued, was able to play crysis)

Currently Testing: TrueNAS-VM (seeking to replace Solaris 11.x / HyperV)

Work-in-progress: lancache-VM - speed still sucks atm though)

Won't be winning any capacity competition, but maybe worth an "honorable mentioning" when it comes to speed. This is a CDM result from a windows client accessing a high speed storage pool on the server via 100gbit RDMA-powered smb direct:

WTF916 · October 26, 2021

Hardware

Case: Fractal Design Define 7 Light Tempered Glass Black
PSU: Seasonic PRIME-Fanless-PX-450
MBO: ASRock Rack X470D4U
CPU: AMD Ryzen 7 2700x
CPU Cooler: be quiet! Pure Rock Slim
RAM: 4x Kingston KSM26ED8/16ME 16GB ECC DDR4
NIC: 1x IPMI (onboard) + 2x 1Gbit (onboard) + 1x 10Gbit Mellanox ConnectX-3
HBA: HP Smart Array H240 (IT Mode)
SSD: 2x Kingston A400 SSD SA400M8 / 120GB + 1x Crucial P1 SSD 1TB, M.2
HDD: 4x Western Digital WD Red Pro 2TB
RGB Controller: Shelly RGBW2

Software

Hypervisor: VMware ESXI vSphere 7 Hypervisor on 120GB SSD with 1TB SSD for storing VMs
- VM: TrueNAS Core on second120GB SSD with directly passed HBA controller managing 4x 2TB HDD drives in RAIDZ2 + 10Gbit NIC
- Other VMs:
  - Unifi Controller
  - Home Assistant
  - Plex Media Server
  - Nextcloud
  - Minecraft Server

Update: added some photos, had to disconnect front panel buttons so my son would not restart the system lol

King_PIN · January 7, 2022

I'm not as fancy as you guys mostly because I have no idea about this stuff.

Just a Qnap tvs-882t with expansion bay.

96tb's of raw storage. 6x10tb and 6 x 6tb drives installed.

Another 66tb's of external drives attached to that.

The NAS is only used for Emby and stores all my movies and shows.

External drives store them movies as backup.

TopHatProductions115 · January 17, 2022

Not sure if this counts, but here I go:

Had a cloning operation go wrong recently, that would have boosted my overall storage capacity even more. But, I'll hopefully recover from that pretty soon

EDIT: I'm back in business!

loadnabox · February 1, 2022

ZFS pool on Ubuntu, 41T total capacity, 27 drives

# zpool status
pool: tank
state: ONLINE
scan: scrub in progress since Tue Feb 1 10:01:35 2022
8.62T scanned at 260G/s, 39.6G issued at 1.16G/s, 8.62T total
0B repaired, 0.45% done, 0 days 02:05:55 to go
config:

NAME STATE READ WRITE CKSUM
tank ONLINE 0 0 0
raidz2-0 ONLINE 0 0 0
ata-HGST_HUS724040ALA640_PN1331PAJJSWGS ONLINE 0 0 0
ata-HGST_HUS724040ALA640_PN1334PBJE61GS ONLINE 0 0 0
ata-HGST_HUS724040ALA640_PN2334PBG2AKBT ONLINE 0 0 0
ata-HGST_HUS724040ALA640_PN1338P4HBXWSB ONLINE 0 0 0
ata-HGST_HUS724040ALA640_PN2331PAKNA8WT ONLINE 0 0 0
ata-HGST_HUS724040ALA640_PN2331PAHUYBJT ONLINE 0 0 0
ata-HGST_HUS724040ALA640_PN2331PBKH9LLT ONLINE 0 0 0
ata-ST4000NE001-2MA101_WJG1ME6E ONLINE 0 0 0
ata-HGST_HUS724040ALA640_PN2334PBGE6M6T ONLINE 0 0 0
ata-HGST_HUS724040ALA640_PN2331PCGJLZPB ONLINE 0 0 0
ata-HGST_HUS724040ALA640_PN2334PEJE6T3T ONLINE 0 0 0
ata-HGST_HUS724040ALA640_PN1338P4GWVYGB ONLINE 0 0 0
special
mirror-1 ONLINE 0 0 0
ata-PNY_CS900_1TB_SSD_PNY40191910041011E5A ONLINE 0 0 0
ata-PNY_CS900_1TB_SSD_PNY40191910041011E53 ONLINE 0 0 0
cache
ata-TEAM_T253X2001T_AA000000000000000624 ONLINE 0 0 0

errors: No known data errors

pool: tank2
state: ONLINE
scan: scrub in progress since Tue Feb 1 10:01:39 2022
1.03T scanned at 35.0G/s, 1.12M issued at 38.4K/s, 7.43T total
0B repaired, 0.00% done, no estimated completion time
config:

NAME STATE READ WRITE CKSUM
tank2 ONLINE 0 0 0
raidz2-0 ONLINE 0 0 0
ata-WDC_WD10SPSX-00A6WT0_WD-WX61A89KZKL0 ONLINE 0 0 0
ata-WDC_WD10SPSX-00A6WT0_WD-WX91AB9273RY ONLINE 0 0 0
ata-WDC_WD10SPSX-00A6WT0_WD-WX91AB97N1N2 ONLINE 0 0 0
ata-WDC_WD10SPSX-22A6WT0_WD-WX61AB97C5VY ONLINE 0 0 0
ata-WDC_WD10SPSX-22A6WT0_WD-WX91AB927E67 ONLINE 0 0 0
ata-WDC_WD10SPSX-22A6WT0_WD-WX91AB927EEV ONLINE 0 0 0
ata-WDC_WD10SPSX-22A6WT0_WD-WX91AB927FDS ONLINE 0 0 0
ata-WDC_WD10SPSX-22A6WT0_WD-WX91AB927HJ4 ONLINE 0 0 0
ata-WDC_WD10SPSX-22A6WT0_WD-WX91AB97N8U6 ONLINE 0 0 0
ata-WDC_WD10SPSX-22A6WT0_WD-WXT1E2943NMP ONLINE 0 0 0
ata-WDC_WD10SPSX-22A6WT0_WD-WXT1E2972UYX ONLINE 0 0 0
ata-WDC_WD10SPSX-22A6WT0_WD-WXT1E297EDFJ ONLINE 0 0 0

errors: No known data errors

# df -h /tank
Filesystem Size Used Avail Use% Mounted on
tank 33T 6.6T 26T 21% /tank
# df -h /tank2
Filesystem Size Used Avail Use% Mounted on
tank2 8.1T 5.7T 2.4T 71% /tank2
#

noodle with crab · February 3, 2022

Looks fun on this.

Hardware:

Case: Supermicro SC-846

Backplane: Supermicro BPN-SAS3-846EL1

PSU: 2x Supermicro PWS-920P-SQ

MB: Supermicro H11SSL-i v2.0

HS: Noctua NH-U12S

CPU: EPYC 7501

RAM: Samsung 32G DDR4-2400 RECC

Network 1: 2x Intel I210 LACP

Network 2: Mellanox MCX354A-FCBT (40/56G VPI)

RAID card: Dell PERC H730P

GPU: Nvidia T1000

SSD (OS): Samsung 970 EVO 500G

SSD (Cache1/2): 2x Samsung 860 EVO 500G

HDD1: 7x 10T ST10000NM0086

HDD2: 1x 16T ST16000NM001G Bulk and one-time data storage

HDD3: 1x 10T WD102KRYZ Backup

HDD4: 1x 6T WDD6003FFBX Backup

HDD5: 1x 10T HGST HDN721010ALE604 Backup

HDD6: 1x 16T ST16000NM001G backup of HDD2

HDD7: 1x 4T HGST HDN726040ALE614 Backup

HDD8: 1x 3T WD30EZRZ OS Backup

UPS: 2x CyberPower CP1500AVRLCD

Rack: Sysracks 32U 30'' network rack

Power: 214W - 310W Peak 353W

Software:

OS: Windows Server 2019

RAID configuration: 6x10T RAID6 + 1 Hot Spare

Filesystem: NTFS

Capacity: 40TB/37254.0GiB (Only counting the RAID disk)

Disk monitoring: HD Sentinal 5.7 Pro

Administration tool: MegaRAID Storage Manager

Virtualization: VMWare Workstation Pro

Cache provided by PrimoCache

Using Windows iSCSI target & VHDX to store actual data.

Usage:

Storage, some recording/transcoding of VTuber lives, internal debian-like apt source (VM), compiler, VPN server.

Backup:

Manual written biweekly auto backup task by copying VHDX files.

Pictures/Snapshots:

SubbyDew · March 2, 2022

Hardware

CASE: Silverstone DS380

PSU: Cooler Master V 550W Gold SFX

MB: ASRock B150M-ITX

CPU: Intel Core i7-6700K

HS: Noctua NH-L9i

RAM: 16 GB GSkill DDR4

HBA: LSI 9300-8I

SSD: 1TB Samsung 860 EVO

HDD 1: 6 x 12TB WD Ultrastar DC HC520

HDD 2: 2 x 14TB WD Ultrastar DC HC530

Total Theoretical Capacity: 100 TB

Actually usable Capacity: 86 TB

UPS: CyberPower PFC Sinewave 1500VA/900Watt UPS

Software and Configuration:

Unraid with a bunch of Docker containers.

Using 1 of the 14TB drives for the parity and the 1TB SSD for the Cache

Usage:

Primarily gets used for media storage and streaming, with some data backup.

Additional info:

Replaced the case fans with Noctuas and got these air ducts https://www.tindie.com/products/patshead/cooling-fan-duct-for-silverstone-ds380b-case/ to help with the airflow to better cool the HDDs

Eventually plan to get a proper rack mounted data storage server with a lot more drive bays and add a bunch more drives, but that is still a long while down the track.

Photo's:

OddOod · March 21, 2022

Hardware

Core system: Levnovo TS 440
CPU: Xeon E3-1245 v3 @ 3.40GHz
RAM: 2x 8GB noname DDR3 1600MT/s
PSU: 290W non-redundant
HBA: 2x LSI raid
Boot drive: 240GB SanDisk SDSSDA24
Drives: 8x10TB WDC WD100EMAZ-00, 8x14TB WDC WD140EDGZ-11
Hot Spares: None
Cold spares: 1 each
UPS: CyberPower CP1500AVRLCD

Total capacity:

Raw: 192TB
Pool1: 54.3TB (RAID6)
Pool1: 76.0TB (RAID6)

Total points: 192*ln(16) = 532.3

Software and Configuration:

Ubuntu with MDADM soft raid. 8x10TB drives are in the 2x4 built in hotswap bays, 8x14TB drives are in a diskshelf connected with some sort of SAS-like cable

Usage:

stores TV shows, movies, backups, photos, etc.

Backup:

TV shows and movies sync'd weekly to an array of external HDDs connected to a rPi. Everything else is sync'd nightly to gDrive

Photos:

The Rig:

The Backup:

Helly · June 30, 2022

Felt like posting an update. Because a whole lot has changed in my setup.

Hardware

CASE: Cooler Master HAF Stacker 935.
PSU: Corsair HX1200
MB: Asus ROG Zenith Extreme Alpha
CPU: AMD Ryzen Threadripper 2950x
HS: Noctua NH-U14S TR4-SP3
RAM: 128GB Kingston Fury Beast 3200Mhz (4x 32Gb)

RAID CARD 1: Areca 1883ix16

Boot Drive: USB 32Gb Drive

Raid Array 1: 8x 6TB WD Red
Raid Array 2: 6x 8TB WD Red

Raid Array 3: 12x 6TB WD Red (mostly SMR unfortunately)

UnRAID array drives: 2x Kingston KC2500 1TB

UnRAID pool drives: 1x Samsung 950pro 512GB

PCPartpicker link

Software and Configuration:
Running UnRAID with several dockers and VM's. A webserver (windows server 2022), ubuntu for a pi-hole (yes can do this is docker, don't feel like it) and a windows server 2019 for work. Dockers include NZBget, Redis and Vaultwarden as well as a few game servers.

Array's 1 and 2 are connected to the Areca card, both running in RAID-6. Array 3 is a Promise 1830i SAN connected through iSCSI with 3 1Gbit connections. Also in RAID-6. It has trouble reaching high speeds though and took almost 2 weeks to initialize the array . Tiny configuration error though, it should be able to do it faster.

Usage:

Storing my media files (movies and tv shows). currently almost 24TB in size. Files are usually copied over from my main PC over a 10Gbit connection to the server, which is a separate network between a few of my PC's. Connected through a 8 port 10Gbit switch

Backup:
LOL? Maybe on a few old drives but in no way ever up-to-date.

dkuhn · August 26, 2022

On 11/3/2017 at 10:46 PM, dkuhn said:

so it looks like my Synology won't qualify here, but that's fine...it's only 32TB

instead, here's my unRAID system that I use to house my modest Plex system

I'm not providing my case/psu/mobo as I honestly don't remember what they are and I'm too lazy to look

Hardware:

CPU: AMD Ryzen 5 1600

HS: Stock cooler

RAM: 16GB of <insert brand>

SSD: 1 x 525GB Crucial MX300

HDD 1: 5 x 8TB WD80EFZX

UPS: 2x APC SmartUPS 3000XL SU3000RMXL3U (not pictured)

Rack: Startech 12U

Network Stack: Cisco Meraki MS225-24 switch, MX84 firewall

Software and Configuration:
OS: unRAID

RAID config: unRAID High Water

File system: btrfs
File system capacity: 32TB presented, 40TB raw (add'l 22TB presented via NFS from my Synology DS1515 w/4x 8TB WD80EFZX)

Usage:
Plex. I have a lot of stuff. 724 movies, 233 TV shows with 18,224 episodes. I share this with close friends and family so uptime and performance is very important. I have at least 6 concurrent transcodes going at any point in time, most 1080P with some 4K. It's essentially full at this point so I'm looking at replacing it with a Norco RPC-4224 and consolidating all of my storage into a single chassis. Going to get rid of the R710/MD1200 and the Xserve/Xserve RAID - they're power hogs and are not expandable any further

Backup:
I back up my most important media to my grandfathered Google Apps account.

Photos:

The system in question is the top one. Couldn't help but show off the rest, though. Might do other posts for them later. Yes, I know it's dusty as shit. I'm in the process of moving and all this will be torn apart, cleaned piece by piece and reassembled with dust filtration in the new rack.

Oh my...how far we've come.

Here we go again?

Some updates...everything posted above is gone except for those 4 8TB HDDs (more on those later) and the Google Search Appliance. Rack is gone, Xserve and Xserve RAID ended up in a dumpster, R710 and the MD1200 was donated to my high school.

Those 4 8TB drives started something that hasn't slowed down. I have a problem.

I'm (currently) rocking a massive (for me) ZFS cluster with 552TB of storage directly attached to a single host, and another 96TB of "scratch" storage on a Synology, and another 19.2TB all flash Synology for a whopping total of 667.2TB of raw storage. I'm definitely missing a few dozen here or there. Easy to lose track of at this point.

Hardware:

Server: Dell R720xd (reflashed Google Search Appliance G100 T4)

CPU: 2x Intel Xeon E5-2697 v2

RAM: 768GB (24x 32GB 4Rx4 PC3-14900L DDR3-1866 ECC REG LRDIMM)

SSD: 12x 1.6TB HP G8-G10 1.6-TB 2.5 SAS 12G WI SSD

HDD 1: 45x Mixed 8TB 7.2K SAS and SATA HDDs

HDD 2: 12x Seagate Exos X18 16TB SATA HDDs

HDD 3: 8x Seagate 1TB 6G 7.2K 2.5 SAS HDDs

HDD 4: 12x Seagate Exos 7E8 8TB 7.2K SAS HDDs

UPS: Worthless TrippLite POS that shall not be named

Rack: APC NetShelter 42U

Network Stack: Ubiquiti Unifi - UDM-Pro, US-48-G1, USW-Pro-Aggregation, plus a bunch of U6-LR Unifi APs...also a bunch of Meraki 8-port PoE switches.

Software and Configuration:
OS: CentOS 7

File system: ZFS -

RAIDZ Configuration: RAIDZ-1 (7x 6-8TB drive vdevs, 2x 6-16TB drive vdevs; for a total of 9 vdevs)
File system capacity: 479T

Usage:
Plex. I have a lot of stuff. 3000+ movies, 1100+ TV shows with 73,000+ episodes. I (still) share this with close friends and family so uptime and performance is very important. The S/O approval factor means downtime/unexpected maintenance is unacceptable. The 4K stuff is starting to dwarf the 1080p content. I am actually in the process of converting all of my 1080p content into h.265 HEVC. I know the process isn't lossless, but that's what I have the remux 4K stuff for. Everything else can get converted to save space. And bandwidth. I had 1080p stuff that would buffer because the bitrate was unnecessarily high. HEVC fixes that. I've saved over 100TB by converting so far. Kinda insane.

Backup:
YOLO (I do snapshots but lmao lets be serious I am not backing this shit up)

Story time:

So, a lot has changed in the last 5 years. Still amazing that it's been 5 years since I made that post. I bought my first house shortly after I made that post. Moved in December of 2017. That's when I got my big boy APC cabinet. I also got 600+lbs worth of APC 30A 120V UPS. APC Smart-UPS XL 3000VA RM 3U 120V if memory serves. With two expansion units. I had the whole house wired for ethernet, APs on each floor...it was great. PoE fed from the server cabinet meant that even in the event of a power outage, I'd still have blazing fast Wi-Fi inside, outside, in the yard, everywhere. That UPS could keep it all powered for like 11 hours. Nucking futs. I moved Plex into a Chenbro 48-bay chassis, and we were off to the races. The system outgrew that after about 3 years. That's when I got an HGST 4U60 JBOD shelf. That thing is a fucking tank. I'm keeping it forever. But, it required 240V power. So, a new breaker in the electrical panel literally 1.5 feet from the cabinet and a new 240V UPS later, it was all up and running! Moved from Unraid to ZFS, and kept expanding. Got up to 52 drives in the HGST JBOD, then it was time to move again! Also, don't ask me what my power bill was. I never looked. Pretty sure I'd have a heart attack if I did.

That brings us to 3 months ago. New place identified, ready to move, but shit, there's literally no way to get 240V power economically to where I need it. SOLUTION! I scored some free NetApp DS4246 shelves from work! They're SAS 6Gb/s instead of the 12Gb/s of the HGST JBOD, but it's not like they're full of SSDs, so who cares!! Moved all the drives in to the NetApp shelves, and we were off! They gobbled up the 120V power with ease! Also significantly quieter. And somehow puts out less heat. Not sure how that's possible, but it is. So, a lot has changed. Lots of hardware has been disposed of, lots upgraded, lots added, etc. This is my hobby and passion so I enjoy the hell out of it.

Latest upgrade was going to 2x Intel Xeon E5-2697 v2 CPUs, 768GB of RAM, and a new LSI 9206-16e HBA. The HBA was a bad move. I need to swap it out. I wanted a card that had four SFF8644 connectors so I could single path each DS directly to the HBA (I know they stack, but if I've got the ports, why not?). That works great. The throughput numbers are insane. And since I've mixed SAS and SATA drives multipathing won't get me anywhere. The problem is the temperature. Holy shit. I don't know how LSI/Avago/Broadcom or whomever got this thing by the regulators. It's a fire hazard. The first time I turned the server on it kernel panic'd and shut down because of how hot it got. Last reported temperature I saw was 131C. I cranked the R720xd fans up and moved it to a different slot with more airflow which has it down to a more reasonable 75C but still, that's too hot and the fans are too loud now. So, back to the drawing board...

Photos:

Getting the rack installed at the new place in December/January 2017 -

Spoiler