LTT Storage Rankings

[dfsdfgfkjsefoiqzemnd]

Do you rip your blu-rays on another PC and just transfer them over to your NAS?

Indeed, I use an external Blu-Ray reader that's connected to the main PC.

 

 

30TB Advertised, I am using a RaidZ2 10 disk volume for 21.8TB actual usable space.

RAIDZ2 is a funny thing. You're getting almost 22TB out of a 10-drive 30TB pool, I'm barely getting 21TB out of an 8-drive 32TB one.

That's 1TB less usable space with 2TB more advertised space. Go figure.

[MG2R]

RAIDZ2 is a funny thing.

[alpenwasser]

RAIDZ2 is a funny thing. You're getting almost 22TB out of a 10-drive 30TB pool, I'm barely getting 21TB out of an 8-drive 32TB one.

That's 1TB less usable space with 2TB more advertised space. Go figure.

 

Interesting:

24e12/1024^4 = 21.83

So, 24 TB are pretty much 21.8 TiB. Give or take a bit for metadata, it seems rather

weird that you'd end up with 19.9 TiB (I'm assuming it's TiB, not TB?) of usable space.

I mean, obviously metadata will take up some space, but that seems like a lot of metadata.

On my own RAIDZ2 pool (6 x 3 TB), I seem to lose about 250 GiB (12 TB = 10.9 TiB, I have

about 10.65 TiB available).

@Cheatdeath on the other hand doesn't seem to lose any space to metadata (again, assuming

it's reported in TiB, not TB), which seems weird as well (good for him?).

So, yeah:

[Cheatdeath]

 

Interesting:

24e12/1024^4 = 21.83

So, 24 TB are pretty much 21.8 TiB. Give or take a bit for metadata, it seems rather

weird that you'd end up with 19.9 TiB (I'm assuming it's TiB, not TB?) of usable space.

I mean, obviously metadata will take up some space, but that seems like a lot of metadata.

On my own RAIDZ2 pool (6 x 3 TB), I seem to lose about 250 GiB (12 TB = 10.9 TiB, I have

about 10.65 TiB available).

@Cheatdeath on the other hand doesn't seem to lose any space to metadata (again, assuming

it's reported in TiB, not TB), which seems weird as well (good for him?).

So, yeah:

 

T

 

 

Interesting:

24e12/1024^4 = 21.83

So, 24 TB are pretty much 21.8 TiB. Give or take a bit for metadata, it seems rather

weird that you'd end up with 19.9 TiB (I'm assuming it's TiB, not TB?) of usable space.

I mean, obviously metadata will take up some space, but that seems like a lot of metadata.

On my own RAIDZ2 pool (6 x 3 TB), I seem to lose about 250 GiB (12 TB = 10.9 TiB, I have

about 10.65 TiB available).

@Cheatdeath on the other hand doesn't seem to lose any space to metadata (again, assuming

it's reported in TiB, not TB), which seems weird as well (good for him?).

So, yeah:

 

The TiB thing is kind of new to me, I had never heard of TiB till I started using FreeNAS.  FreeNAS reports 20.1 TiB usable and that should be about 21.8 TB correct? What's strange is I just noticed my CIFS share viewed on my desktop says 20.1TB, does it not realize that is TiB or what?

[alpenwasser]

The TiB thing kinda is new to me, I had never heard of TiB till I started using FreeNAS.  FreeNAS reports 20.1 TiB usable and that should be about 21.8 TB correct? What's strange is I just noticed my CIFS share viewed on my desktop says 20.1TB, does it not realize that is TiB or what?

Ah, I see. 20.1 TiB is about 22.1 TB if I haven't miscalculated. As for CIFS, are

you using a Windows client? If I remember right, Windows says 'TB', but actually

means 'TiB'. That's why a 1000 GB drive shows up as 931 GB. It should actually

say 931 GiB, but it seems Windows doesn't want to confuse users by saying 'GiB'

(which, ironically, actually leads to confusion ).

So it seems you lose quite a bit of space to metadata as well.

[dfsdfgfkjsefoiqzemnd]

 

Interesting:

24e12/1024^4 = 21.83

So, 24 TB are pretty much 21.8 TiB. Give or take a bit for metadata, it seems rather

weird that you'd end up with 19.9 TiB (I'm assuming it's TiB, not TB?) of usable space.

I mean, obviously metadata will take up some space, but that seems like a lot of metadata.

On my own RAIDZ2 pool (6 x 3 TB), I seem to lose about 250 GiB (12 TB = 10.9 TiB, I have

about 10.65 TiB available).

@Cheatdeath on the other hand doesn't seem to lose any space to metadata (again, assuming

it's reported in TiB, not TB), which seems weird as well (good for him?).

So, yeah:

 

I even missed the TB vs TiB thing and completely forgot how much space I had.  I thought I had 19.9 + 0.5 + 0.5, but it was 18.9 + 0.5 + 0.5.  So it's a full TB worse than my previously mentioned number.

 

 

Ignore The top 2 shares, those NASes are disconnected and shut down by now.

 

"Data" are the mirrored SSDs, so that's a different pool than the one that the HDDs are in.

 

"Video", "Software" and "Data_Backup" are the shares from the HDDs' pool.  I assigned 500GB to the latter 2 and let "Video" take the rest of the pool.  That's 19.9TiB total (equates to 21.88TB) out of a 24TB (+8 TB for parity) setup.  So the actual numbers are even worse.

I double-checked the list of drives, and all 8 HDDs are indeed online and part of the pool.

 

Should I backup my data and completely reconfigure the NAS to see what's going on there?  This is a LOT of lost space. 

[alpenwasser]

- snip -

 

Updated.

 

Should I backup my data and completely reconfigure the NAS to see what's going on there?  This is a LOT of lost space.

Hm, good question. It really could just be ZFS doing some sort of weird magic. But

honestly, I think for this question to be answered competently you probably need

somebody with knowledge of ZFS' internals, and I don't know too much about that I'm

afraid.

As long as pool integrity checks out I would wait with wiping until you have a good

answer or are confident that you won't get one. You could try serverfault.com

or if that doesn't help and you really want an answer, maybe even open an issue about

it on the github page for OpenZFS and politely ask if they might know anything about

this.

[Cheatdeath]

Ah, I see. 20.1 TiB is about 22.1 TB if I haven't miscalculated. As for CIFS, are

you using a Windows client? If I remember right, Windows says 'TB', but actually

means 'TiB'. That's why a 1000 GB drive shows up as 931 GB. It should actually

say 931 GiB, but it seems Windows doesn't want to confuse users by saying 'GiB'

(which, ironically, actually leads to confusion ).

So it seems you lose quite a bit of space to metadata as well.

 

Yes a Windows 7 Pro machine. 

[Ramaddil]

UPDATE #4 - ((((((((((Update #4 Changed Raid card//Added Additional 20TB of Storage//Added 16Gb more of Ram (total 32gb) ))))))))

 

 

 

Total Storage 

67.0TB

 

 

Server Name

Titan

 

Hardware

CASE: Lian Li 343B
PSU: Corsair TX850
MB: Gigabyte 990FXA

CPU: AMD FX - 8120 @ Stock
HS: Noctua NH-D14
RAM: 32GB DDR3 1600 GSkill Ram
RAID CARD 1: Adaptec 52445 28 Port Raid Card

SSD: Samsung 840 EVO 120GB
Enclosures: Startech 2 5.25 for 3 HDD (6) 

HD's: 

13x 2TB Western Digital (Red) WD20EFRS
9x 2TB Western Digital Greens (Various Models) 

1x 3TB Western Digital WD30EZRS

3x 6TB Western Digital WD60EFRX

 

((( Update Added 7TB to Server )))

((((( Update # 2 Added 10TB More To Server ))))) 7/12/14

((((((( Update #3 Replaced a Failed 2TB//Added Additional 2TB//Changed Operating System//CPU, Motherboard Upgraded)))))))

((((((((((Update #4 Changed Raid card//Added Additional 20TB of Storage//Added 16Gb more of Ram (total 32gb) ))))))))

 

Software and Configuration:
My server is running Windows 8.1, I have one Raid 6 Array with 23 drives (22 X 2TB =44TB).  This array holds all my movies/tv shows and data.  I have a 5TB (3TB + 2TB) drive that I use for backup for my software that is synced to nightly with Syncback Pro on top of my cloud storage which is mentioned in the backup section.  Second Array is 3 x 6TB (18TB) for Backup Purposes). 
 
Usage:
I use my server as my media hub for my entire house and beyond.  I have 2 WD TV Media players in different rooms in the house which can remotely access my server.  I also utilize my server to access Air Video remotely on my Idevices, and it also serves as my security system monitoring cameras placed around the house.
 
Backup:
My Server is connected to Crashplan which stores my data off site.
 
Additional info:
Server now running 8.1 after some tweaks to add start menu and better functionality, looking forward to Windows 10!
 

 

 

Photo's:

 

 

 

 

 

 

 

 

 

 

 

 

[Commander_Z]

UPDATE #4 - ((((((((((Update #4 Changed Raid card//Added Additional 20TB of Storage//Added 16Gb more of Ram (total 32gb) ))))))))

 

 

 

Total Storage 

67.0TB

 

 

Server Name

Titan

 

Hardware

CASE: Lian Li 343B

PSU: Corsair TX850

MB: Gigabyte 990FXA

CPU: AMD FX - 8120 @ Stock

HS: Noctua NH-D14

RAM: 32GB DDR3 1600 GSkill Ram

RAID CARD 1: Adaptec 52445 28 Port Raid Card

SSD: Samsung 840 EVO 120GB

Enclosures: Startech 2 5.25 for 3 HDD (6) 

HD's: 

13x 2TB Western Digital (Red) WD20EFRS

9x 2TB Western Digital Greens (Various Models) 

1x 3TB Western Digital WD30EZRS

3x 6TB Western Digital WD60EFRX

 

((( Update Added 7TB to Server )))

((((( Update # 2 Added 10TB More To Server ))))) 7/12/14

((((((( Update #3 Replaced a Failed 2TB//Added Additional 2TB//Changed Operating System//CPU, Motherboard Upgraded)))))))

((((((((((Update #4 Changed Raid card//Added Additional 20TB of Storage//Added 16Gb more of Ram (total 32gb) ))))))))

 

Software and Configuration:

My server is running Windows 8.1, I have one Raid 6 Array with 23 drives (22 X 2TB =44TB).  This array holds all my movies/tv shows and data.  I have a 5TB (3TB + 2TB) drive that I use for backup for my software that is synced to nightly with Syncback Pro on top of my cloud storage which is mentioned in the backup section.  Second Array is 3 x 6TB (18TB) for Backup Purposes). 

 

Usage:

I use my server as my media hub for my entire house and beyond.  I have 2 WD TV Media players in different rooms in the house which can remotely access my server.  I also utilize my server to access Air Video remotely on my Idevices, and it also serves as my security system monitoring cameras placed around the house.

 

Backup:

My Server is connected to Crashplan which stores my data off site.

 

Additional info:

Server now running 8.1 after some tweaks to add start menu and better functionality, looking forward to Windows 10!

 

 

 

Photo's:

 

 

 

 

 

 

 

 

 

 

 

 

the front of this case................mmmmmmmmmmmmm

Edited July 7, 2015 by alpenwasser
spoiler for long quote

[Eric The Tech Guru]

Total Storage: 16 TB

 

Hardware
CASE: Fractal Design Define R4 (No Window)
PSU: Corsair RM 650
MB: Asus M5A78L-M
CPU: AMD FX 8350 Black Edition (8 Cores, 4.0 GHz)
HS: Hyper 212 Evo
RAM: 32GB DDR3 1600
HDDs:  4x 4TB WD Red
Software and Configuration:
Running Ubuntu 14.04 Server Edition with Plex Media Server, Owncloud, and Deluge Server with Web Interface

Usage:
My household uses it for media transcoding and streaming (we have around 4 people in home all the time)

The high RAM amount is because I had a ton of extra RAM from an old custom-pc building business I had that I never got rid of when I got a new job.

Plex runs like a pro, we're running Deluge on there for seeding and leeching torrents.

Owncloud backs up our critical documents and photos.

Backup:
None, this is our backup

Photos:

[AnonymousGuy]

Specs

 

4790K @ 4.4Ghz

Asus Z87 Gryphon mATX

LSI MegaRAID 9280i-24

24 x Seagate 3TB in raid 60 (~55TB)

1TB Crucial MX200 SSD (Boot + VM storage)

EVGA 740 GT SC

Corsair AX1200i

Windows 7 x64 Ultimate

 

Fractal Node 804 mATX case.

 

Usage:

 

Runs VMs, stores backups.  Average RAM utilization is 16-20GB.  It sits in my laundry room running 24/7 with 95F ambient.  8 of the drives are internal to the case (+1 SSD), 16 are outside the case in a 4x4 configuration.  I could fit 5 drives in each cage, but airflow isn't great doing that way.

 

 

10GbE is planned for later this year.  The cable is already run to point-to-point it to my desktop, but need the NICs.

 

Fun Fact: the startup current of this many drives is something like 600W before settling to ~ 120W static.

[Eric The Tech Guru]

 

Any reason for that particular configuration hardware-wise?

 

Why not go for a larger case that can hold more of your drives, or go for a small 8-12U rack box and slide a rack-mounted server in there?

 

I feel like it'd be worth the price to get a much more compact and aesthetically pleasing or just plain sensible build. (No offense)

 

EDIT: removed unnecessary quoted parts to keep the post shorter.

[AnonymousGuy]

Any reason for that particular configuration hardware-wise?

 

Why not go for a larger case that can hold more of your drives, or go for a small 8-12U rack box and slide a rack-mounted server in there?

 

I feel like it'd be worth the price to get a much more compact and aesthetically pleasing or just plain sensible build. (No offense)

 

EDIT: removed unnecessary quoted parts to keep the post shorter.

cost.  enclosures run several hundred bucks and doing it this way was a bit over $100.

 

plus space considerations.  the shelf I built over my laundry machines cant handle a super deep case.

 

oh and I can swap failed drives really easy when theyre exposed like this.  just need a screwdriver and 5 minutes.

[Eric The Tech Guru]

cost.  enclosures run several hundred bucks and doing it this way was a bit over $100.

 

plus space considerations.  the shelf I built over my laundry machines cant handle a super deep case.

 

oh and I can swap failed drives really easy when theyre exposed like this.  just need a screwdriver and 5 minutes.

Ah, okay.

 

I'm hoping the setup is over the washing machine on a shelf, not sitting directly on the washing machine?

 

I wouldn't like to see a drive spinning at 7,200 RPM while a washing machine thrashes it around a bit during a spin cycle

[AnonymousGuy]

Ah, okay.

 

I'm hoping the setup is over the washing machine on a shelf, not sitting directly on the washing machine?

 

I wouldn't like to see a drive spinning at 7,200 RPM while a washing machine thrashes it around a bit during a spin cycle

 

Yeah I considered that.  It's above but half of the support is coming from the machines beneath them (the other half coming from the wall).  The machines are modern LG's though so they don't really vibrate a significant amount.  If I start seeing unusually high drive fail rates then I'll shock mount everything better than it is.

[Eric The Tech Guru]

Yeah I considered that.  It's above but half of the support is coming from the machines beneath them (the other half coming from the wall).  The machines are modern LG's though so they don't really vibrate a significant amount.  If I start seeing unusually high drive fail rates then I'll shock mount everything better than it is.

Ahahaha, yeah I guess trial by fire works in this case

[Shaqalac]

I think it's about time I move up the list.

I've expanded my Raid 6 array with an additional 4TB Seagate NAS HDD, which gives me a total of 20TB of raw data.

Here's a picture of the result if anyone's interested.

[weeandykidd]

Ginnungagap

16TB of storage teetering on edge

 

I'm a bit of a jack of all trades, master of none. So my 'server' is a bit amateur, but sharing funds between that and my other pursuits is hard

 

Hardware
CASE: Silverstone Milo ML04B
PSU: 300W SFX
MB: GA-J1900n-d3v
CPU: Celeron J1900 
HS: Passive, onboard
RAM: 4GB DDR3L
RAID CARD: Semlos Sil3114
SSD: Kingston v300 120gb
HDD: 4x4TB RED
16TB Total


Software and Configuration:
Running windows server 2008R2, no arrays as of yet.
I occasionally scrape for all filenames just incase, most is replacable. what's left is backed up externally.

 

Usage:
Plex Server, OwnCloud, Steam IHS (rarely), IIS.

Backup:
Investing in CrashPlan

Additional info:
Next step is to get a new case and more drives, move away from Windows and make use of ZFS.

 

Photo's:

This is a (relatively) old photo, the drives are now mounted using SEDNA rubber stands, they stack quite well and fit the case perfectly. 

I also improved my cable management

 

A picture of the case, since I don't have one:

[MG2R]

<gorgeous>

 

That is one looker of a system you've got there!

 

 

 

Ginnungagap

 

Mighty fine system! Welcome to the club!

 

@alpenwasser will add you ASAP

 

just poke him a bit

@alpenwasser

 

 

dude, @alpenwasser, faster!

 

[alpenwasser]

OUCH, what's with all the poking!? :wacko:

Yeah, I'll update the thread next weekend, sorry for the delays.

[MG2R]

OUCH, what's with all the poking!? :wacko:

Yeah, I'll update the thread next weekend, sorry for the delays.

YEAH YOU BETTER BE SORRY

 

let me tag you some more, @alpenwasser

 

never doing your job in time, you BAD @alpenwasser!!!!

 

:wub:  I wanted a devil emoticon but this was the only thing in red. so here, have some more. :wub: :wub: :wub:  I totally don't love you at all. think of devils. so many devils! :wub: :wub: :wub:

[Shaqalac]

That is one looker of a system you've got there!

Cheers!

 

I'm still not pleased with the cable management and the sleeving of the cables that came with the SilverStone SFX PSU, that will have to be re-made at some point 

 

I had to make some customisation to the SATA-power cable, as it didn't had enough connectors. It's not completely straight but as mentioned, it'll be customised at some point. 

[RandomNOOB]

After reading @weeandykidd post, I had a look at my own drives power on hours, and most of my drive has about 18-22k hours on and then there is this one o_O

37824 hours

1576 days

225 weeks

4.3 years.

 

All my drives are old

[Str_Mape]

Just throwing mine out there even though it sadly doesn't qualify due to being my main / gaming PC.

16TB Raw Storage excluding the SSD's.