New File Systems vs Parity (ReFS/Storage Spaces, Btrfs/RAID)

[Jarsky]

Hey guys

 

 

Waiting on my pretty new drives to arrive to update my build in LTT 10TB+ but just wanting to see what others opinions are.

My system has been upgraded with a few SAS expander cards with the plan of continuing software solutions

 

Right now i'm using NTFS with a Win2k12 Software RAID5. I've also used EXT4 with mdadm, and had a play with FreeNAS/Fedora using ZFS in a raidz.

 

I've been waiting on ReFS and Btrfs to mature for a next-gen file system so I don't have to worry as much about bitrot, but i've heard a few horror stories with them.

Im wanting to find out I guess what the features and compatibilities are like for ReFS Storage Spaces and Btrfs Parity?

 

Does anyone have any knowledge or experience in this area?

 

Compatibility I need:

Linux 2.6.32 kernel and newer (Read/Write/Create Directory)

Windows 7 / 2008R2 Server and newer (Read/Write/Create Directory)

 

Features I need:

Folder/File Permissions

Multidisk spanning with Parity

Rebuild array from a failed disk

Ability to add additional disks and grow the filesystem across a larger pool.

 

 

Now I know that Storage Spaces can handled 1 & 2 drive Parity, but Im not sure on what features ReFS supports and if its supported on Linux?

On the Btrfs side, the same questions, but im also concerned is Btrfs Raid still highly experimental?

 

Would I better off forgoing these new Filesystems and going the traditional NTFS/EXT4 route?

 

[Blake]

I would recommend staying clear of btrfs for a raid 5, give it a few more releases and it should be good.

 

ReFS horror stories are mainly from when it was first released, and wasn't stable. nowdays you should be fine, however there is alot less recovery software that supports it so you could have issues should your raid fail.

 

Storage spaces are great for raid 0/1. Raid 5 I was getting ~800MB Reads 4MB writes. but that is just raid 5, upgraded to a dedicated HW raid 5 and get 1000MB reads and 20MB Writes.

 

Ext4 and NTFS meet your requirements and are actually stable.

[Jarsky]

Yeah thats what I was afraid of. The archwiki for Btrfs says it supports Parity but the only other articles I could find, it still didnt officially support it. 

 

 

Given this is for my personal storage and not a test setup, guess its probably best to stick with the tried and true file systems. 

 

I'm going to be virtualising my new server with Hyper-V Server

The next hard decision is wether to go Storage Spaces, mdadm or Z-raid. 

 

Leaning towards ZFS Z-Raid because of the copy-on-write, scrubbing, compatibility and performance. Might have to up my memory though with the size this array is going to be. 

[alpenwasser]

As far as I'm aware, Btrfs is still quite a long way off from being production-ready

when it comes to parity RAID, so as mentioned I'd stay clear from that.

I've been using ZFS on Arch Linux for about two years now, and it has been working very

well for me. As for memory, I wouldn't worry about it as much as other people seem to

be doing. I ran a 17 TB (raw space) array on a 4 GB machine for a year without trouble

(read/write did get slow when I started doing other stuff which ate up RAM, but no crashes

or data loss), and these days I'm running a 18 TB raw space array on a VM with 6 GB of RAM.

Yes, ZFS loves RAM, and it will gobble up as much as it can, and you will get better

performance for it (as long as you don't bump up against another bottleneck), but as best

as I can tell that's about it. When my pool was not yet quite as full as it is now I could

still easily saturate a gigabit network connection with it, and I don't need anything faster

than that (of course, if you have 10Gb Ethernet... different story).

I would however recommend ECC RAM if you're going with ZFS. As for possible alternatives,

mdadm + ext4 (or another stable Linux filesystem) is a well-tested and stable solution

too.

[Jarsky]

Already got ECC memory- its the only way I go for a storage server (serve rin my sig), power isnt a problem either. My current setup is already over 24TB of useable space, and this new array is going to be considerably larger which is why I thought I may need more memory, especially when some of it is already dedicated to other tasks (Rust Servers, Development VM, Test Domain & Clients, etc...)

[alpenwasser]

Already got ECC memory- its the only way I go for a storage server (serve rin my sig), power isnt a problem either. My current setup is already over 24TB of useable space, and this new array is going to be considerably larger which is why I thought I may need more memory, especially when some of it is already dedicated to other tasks (Rust Servers, Development VM, Test Domain & Clients, etc...)

Ah excellent. If you can afford more memory I certainly wouldn't advise against it,

it's just that I've seen some people almost go into panic mode when you don't throw

half your budget into RAM in a ZFS machine.

Of course I can't say how much RAM you'll need for other stuff (VMs etc.), but going

by your signature specs (which I overlooked before...), 24 GB of RAM will get you

quite far with ZFS, even with lots of terabytes. If it was my personal setup, I'd

probably first put everything together, then see if performance is acceptable, and

only if I find it to be unsatisfactory would I then buy more RAM. As said though,

this isn't taking into account VMs and other stuff, so adjust your needs accordingly.

[Eniqmatic]

I too would go with ZFS, BTRFS is pretty stable for RAID 1 and 0 (I think) however quite a bit away from stable for higher levels. ZFS works great IMO and is great if you want data to still be correct for many years to come.

[alpenwasser]

I too would go with ZFS, BTRFS is pretty stable for RAID 1 and 0 (I think) however quite a bit away from stable for higher levels.

Yes, last I checked that was what I read as well. I've seen people who've been using BTRFS for quite

a while, but only in RAID1 or RAID0 (or just JBOD), nobody I've come across has actually used one of

the higher RAID levels in a production machine.