Can I build a raid with a drive that already has files in it?

[ShadoWalkeR]

Hi,
I currently have a x18 18tb exos drive that has like 2tb free space. I was thinking of getting 3 more a going Raid 6 as I really need to keep my files safe.

Do I have to put my files somewhere else first to do the raid and copy them over or is there a way?
Also I was thinking of using HexOS, has this is mainly my archive of BD, VHS, and other things I had lying around. So mostly for Plex.

[Kilrah]

No, you'll have to wipe everything. Also RAID is not a backup so any files you really need safe need at least one, ideally 2 separate copies not counting any array redundancy.

[porina]

I'm not familiar with all RAID systems but if they offer the option to "grow" an array with data on it already that might offer a path.

 

I am familiar with Unraid and that can do something similar to RAID 6. Unraid has a separate dedicated parity and data disks. If you got 3 new drives, you could set up one as parity, and two for data. Then move your data onto that array. The disk that is freed up could then be added to the array as a second parity disk. You can in future expand the array by adding more data disks.

 

For arrays with a low number of drives, having 2 disks for redundancy feels a bit excessive and space inefficient.

 

Not familiar with Plex but I'm sure people have used it with Unraid so you can look that up if this path is of interest.

[ShadoWalkeR]

6 minutes ago, Kilrah said:

No, you'll have to wipe everything. Also RAID is not a backup so any files you really need safe need at least one, ideally 2 separate copies not counting any array redundancy.

It's safe though Raid 6 allows any 2 drives to fail without me losing data. I'd replace them immediately.
One thing tho, image I have the 4 drives in Raid 6, will I need to re-format each time I want to add more drives?

[Kilrah]

4 minutes ago, ShadoWalkeR said:

One thing tho, image I have the 4 drives in Raid 6, will I need to re-format each time I want to add more drives?

With "true" RAID yes. With ZFS as is in HexOS / TrueNAS you can do some expansion but it's very constrained, e.g. you if you have a 4-drive RAIDz2 you have to add a whole 4 drives if you want to expand.

 

If you want to gradually expand use Unraid instead, there you can add any kind of mismatched drives at any time as long as your parity drive(s) are the biggest.

 

6 minutes ago, ShadoWalkeR said:

It's safe though Raid 6 allows any 2 drives to fail without me losing data.

Drive failures are almost the least common way of losing data. With an array if you delete a file by mistake it's gone, the array won't help you. If the array's filesystem somehow corrupts the entire array is gone regardless of redundancy, etc.

[ShadoWalkeR]

8 minutes ago, porina said:

I'm not familiar with all RAID systems but if they offer the option to "grow" an array with data on it already that might offer a path.

 

I am familiar with Unraid and that can do something similar to RAID 6. Unraid has a separate dedicated parity and data disks. If you got 3 new drives, you could set up one as parity, and two for data. Then move your data onto that array. The disk that is freed up could then be added to the array as a second parity disk. You can in future expand the array by adding more data disks.

 

For arrays with a low number of drives, having 2 disks for redundancy feels a bit excessive and space inefficient.

 

Not familiar with Plex but I'm sure people have used it with Unraid so you can look that up if this path is of interest.

That would mean Raid 5/10 right? I was looking into Raid 6 cause any 2 drives can fail, I know with Raid 10 it's faster but if the 2 failing drives are the data and parity then I lost my stuff
Although let's be honest 2 drives failing at the exact same time is just plain bad luck.

[ShadoWalkeR]

1 minute ago, Kilrah said:

With "true" RAID yes. With ZFS as is in HexOS / TrueNAS you can do some expansion but it's very constrained, e.g. you if you have a 4-drive RAIDz2 you have to add a whole 4 drives if you want to expand.

 

If you want to gradually expand use Unraid instead, there you can add any kind of mismatched drives at any time as long as your parity drive(s) are the biggest.

 

Drive failures are almost the least common way of losing data. With an array if you delete a file by mistake it's gone, the array won't help you. If the array's filesystem somehow corrupts the entire array is gone regardless of redundancy, etc.

Yes, I'm planning 4 drives now and add 4 more drives in like a year or two. So I think I'm good in this. RAIDZ2 is Raid 6 right?

So do I get a better chance of not corrupting data with a Raid 10 system which is less complex?

[porina]

5 minutes ago, ShadoWalkeR said:

That would mean Raid 5/10 right? I was looking into Raid 6 cause any 2 drives can fail, I know with Raid 10 it's faster but if the 2 failing drives are the data and parity then I lost my stuff

Unraid isn't exactly like the numbered RAID tiers. I guess it is quite similar to raid 4 but with the possibility to add extra parity disks as well as data disks.

 

Data is written to one data disk, and the parity disk(s) are updated accordingly. You can have as many data disks fail as you have parity disks and still have all data recoverable. You can add data or parity disks as you like. Even if more disks fail than you can fully recover from, the data on the working data disks is still fully accessible.

 

IMO with 4 disks I'd run 1 parity and 3 data. Only if you get to 8+ disks then a 2nd parity may be considered as possibly worthwhile. In all the years I've been running Unraid I never had a disk fail so never had to test it out.

 

Unraid wouldn't be my choice for max performance but it has decent flexibility. License pricing depends on how many disks you have and you can upgrade as needed.

[Somerandomtechyboi]

if you need them available at all times go ahead, otherwise better off buying more drives as backup data as raid is mostly for making sure you have access to your data at all times even when a drive or two fails afaik

[brwainer]

2 hours ago, Kilrah said:

With "true" RAID yes. With ZFS as is in HexOS / TrueNAS you can do some expansion but it's very constrained, e.g. you if you have a 4-drive RAIDz2 you have to add a whole 4 drives if you want to expand.

This isn’t true anymore, raidz expansion was released as a stable feature of OpenZFS and TrueNAS Scale (24.10) and HexOS support it. You can add individual drives to any raidz now.

[brwainer]

2 hours ago, ShadoWalkeR said:

Although let's be honest 2 drives failing at the exact same time is just plain bad luck.

But to repeat what was said before - RAID doesn’t help you with file deletion. Or to expand on that - it doesn’t help with ransomware. Backups should be independent copies that are not instantly updated (meaning tools like syncthing aren’t meant for backups), and ideally one of them is stored off-site.

 

The extent to which this matters depends on the data. I only properly back up about 2TB of my 12TB of total data because of what the different data is.

 

EDIT: Also if drives are purchased at the same time, from the same batch, and used in the say way, they are actually very likely to fail at about the same time - plenty of RAID5 rebuilds have failed because the stress of rebuilding the first failed drive killed a second, which is why RAID6 exists. Some people intentionally get drives from different manufacturers, batches, or mix the ages between pools.

[ShadoWalkeR]

1 hour ago, brwainer said:

But to repeat what was said before - RAID doesn’t help you with file deletion. Or to expand on that - it doesn’t help with ransomware. Backups should be independent copies that are not instantly updated (meaning tools like syncthing aren’t meant for backups), and ideally one of them is stored off-site.

 

The extent to which this matters depends on the data. I only properly back up about 2TB of my 12TB of total data because of what the different data is.

 

EDIT: Also if drives are purchased at the same time, from the same batch, and used in the say way, they are actually very likely to fail at about the same time - plenty of RAID5 rebuilds have failed because the stress of rebuilding the first failed drive killed a second, which is why RAID6 exists. Some people intentionally get drives from different manufacturers, batches, or mix the ages between pools.

This is for my home nas that'll only operate locally I'm not really concerned about ransomwares. I want raid cause of the faster speed an everything is always synched. I could have it manually backed up but that's a lot more work
 

[Jarsky]

7 hours ago, porina said:

I'm not familiar with all RAID systems but if they offer the option to "grow" an array with data on it already that might offer a path.

I'm gonna assume that since he already has this drive in use for some time its probably using the NTFS file system.

Theres no way he can add this to a traditional array (FSoR) without wiping it.

 

He could though a RoFS "Raid over File System" solution.

FlexRAID was the king of this but is long gone. SnapRAID is probably the best one still available. 

 

However, it doesnt provide any speed benefit. Infact it has a negative impact on performance. 

It's for infrequently changed files; for pooling into single shares and providing redundancy with hashing across disks. 

[porina]

7 minutes ago, Jarsky said:

I'm gonna assume that since he already has this drive in use for some time its probably using the NTFS file system.

Theres no way he can add this to a traditional array (FSoR) without wiping it.

I was thinking of something different. They were going to buy 3 more disks. I was wondering if the 3 could be set up as raid. The data from the existing disk is moved onto that array, then that vacated disk would be wiped and added to the array also. I know Unraid can do it. I don't know about other raid systems.

[Lurick]

5 hours ago, brwainer said:

This isn’t true anymore, raidz expansion was released as a stable feature of OpenZFS and TrueNAS Scale (24.10) and HexOS support it. You can add individual drives to any raidz now.

While you can do this in TrueNAS Scale and ZFS there are some major caveats

1) Drives can ONLY be added one at a time and apparently it's a long and slow process. Someone said it took them a week to add 4 drives fully.

2) You need to rebalance everything and hope and pray things mostly balance without killing drives in the process

2a) A rebalance will not guarantee proper distribution of files

3) Some space will not be shown until you expand the array and even afterwards it's possible some space could still be missing, I've not delved too much into this but just pointing it out from what I saw.

 

tl;dr: Rebuild the entire array instead of expanding with vdev expansion if at all possible. OR just add drives to a second vdev and add that to the pool.

[ShadoWalkeR]

Ok, so my best bet is to buy a 5th drive do the raid with the drives and then copy my files hover.