Ubuntu + Software Raid 10?

[Torey]

Is there a package I can install on Ubuntu that will allow me to use software raid 10?

 

Basically I have a motherboard that doesn't support raid and all of the cheap enough raid controllers I've seen have very poor reviews. I'd like to be able to use Raid 10 and keep Ubuntu 16.04. I'm not against switching OS's entirely. But I also use this computer as a streaming box sometimes for Twitch and getting the setup to work on other OS's has proven difficult. 

 

Current Setup - 

Xeon E3-1231V3

8GB DDR3 - Single Stick

Biostar B85MG Ver.6x 1150

SSD Boot Drive

4 Port Sata Controller

2 6TB Drives one for use, the second for backup

2 3TB Drives one for use, the second for backup.

 

I'd like to buy two more 6 TB drives and raid 10 them. Not sure what's the best way to go about this.

[cynexit]

mdadm should be able to do what you want. Google for "mdadm raid10" or something, there are plenty of tutorials.

[Torey]

5 minutes ago, cynexit said:

mdadm should be able to do what you want. Google for "mdadm raid10" or something, there are plenty of tutorials.

Thanks, I had just found this like 5 minutes after posting >.<

[Torey]

3 hours ago, cynexit said:

mdadm should be able to do what you want. Google for "mdadm raid10" or something, there are plenty of tutorials.

Quick question. I'm trying to find a better solution to moving my files before I build the raid 10. If I get two brand new drives and raid 0 them. Then put all the info from one of my old drives onto the raid 0. Then erase the other two drives and raid 0 them. Will I be able to raid 1 the two raid 0's after and keep the data? Or will I have to have all the drives clean first?

 

I have the storage to backup everything on some older 3TB's. I'd just like to be able to keep an extra copy of everything, instead of just one.

[Electronics Wizardy]

10 hours ago, Torey said:

Is there a package I can install on Ubuntu that will allow me to use software raid 10?

 

Basically I have a motherboard that doesn't support raid and all of the cheap enough raid controllers I've seen have very poor reviews. I'd like to be able to use Raid 10 and keep Ubuntu 16.04. I'm not against switching OS's entirely. But I also use this computer as a streaming box sometimes for Twitch and getting the setup to work on other OS's has proven difficult. 

 

Current Setup - 

Xeon E3-1231V3

8GB DDR3 - Single Stick

Biostar B85MG Ver.6x 1150

SSD Boot Drive

4 Port Sata Controller

2 6TB Drives one for use, the second for backup

2 3TB Drives one for use, the second for backup.

 

I'd like to buy two more 6 TB drives and raid 10 them. Not sure what's the best way to go about this.

Id personally use btrfs raid 10. Its has checksums so it can keep your data from being corrupted. It comes preinstalled in almost every distro. It also supports drives with differents sizes in raid 5(which id personally perfer.

 

In order to make an array you can type mkfs.btrfs -m raid1 -d raid10 /dev/sdb /dev/sdc /dev/sdd /dev/sde(put your own drive letters in here.

[Torey]

12 hours ago, Electronics Wizardy said:

Id personally use btrfs raid 10. Its has checksums so it can keep your data from being corrupted. It comes preinstalled in almost every distro. It also supports drives with differents sizes in raid 5(which id personally perfer.

 

In order to make an array you can type mkfs.btrfs -m raid1 -d raid10 /dev/sdb /dev/sdc /dev/sdd /dev/sde(put your own drive letters in here.

Thanks for the tips. But from what I've read everywhere isn't raid 5 not that great of a solution for such few discs. Specifically in my case since they are 6TB drives wouldn't I have a very high chance of a URE when recovering? Everywhere I read told me to just stay away from 5/6 and do 10. It sucks that I lose more space, but at least their is a full exact copy of everything. 

[Electronics Wizardy]

50 minutes ago, Torey said:

Thanks for the tips. But from what I've read everywhere isn't raid 5 not that great of a solution for such few discs. Specifically in my case since they are 6TB drives wouldn't I have a very high chance of a URE when recovering? Everywhere I read told me to just stay away from 5/6 and do 10. It sucks that I lose more space, but at least their is a full exact copy of everything. 

Its defiantly a problem, but with btrfs and other modern fs's they will skip the bad part and only corrupt one file, which you can replace with a backup.

[Torey]

3 hours ago, Electronics Wizardy said:

Its defiantly a problem, but with btrfs and other modern fs's they will skip the bad part and only corrupt one file, which you can replace with a backup.

Ah but that's the problem, one half of my RAID10 will be for backup. Right now I just manually backup with rsync. Which I'm sure is a better system then relying on RAID10. But..., well I guess I could just raid 0 two of the drives and just cronjob the rsync like I am now. But basically I'm trying to use the 4x 6TB drives as my main storage and backup. Just just the main storage. So I wouldn't have a backup to replace the file. Or are you suggesting that it's still safe since most likely only one or two files will fail and I can just replace them. 

[Electronics Wizardy]

7 hours ago, Torey said:

Ah but that's the problem, one half of my RAID10 will be for backup. Right now I just manually backup with rsync. Which I'm sure is a better system then relying on RAID10. But..., well I guess I could just raid 0 two of the drives and just cronjob the rsync like I am now. But basically I'm trying to use the 4x 6TB drives as my main storage and backup. Just just the main storage. So I wouldn't have a backup to replace the file. Or are you suggesting that it's still safe since most likely only one or two files will fail and I can just replace them. 

Unless the data on your drives is something that you don't really care about back it up to something else, like a clould service or tapes. Even though your system will keep you safe from things like drive failure, you still have problems if you somehow got ransomware(could be fixed with snapshotting) or a fire or diaster that destroys your system.

[Torey]

11 hours ago, Electronics Wizardy said:

Unless the data on your drives is something that you don't really care about back it up to something else, like a clould service or tapes. Even though your system will keep you safe from things like drive failure, you still have problems if you somehow got ransomware(could be fixed with snapshotting) or a fire or diaster that destroys your system.

I can't afford a cloud service to store that much data at the moment. But I do have two systems and the backup could be stored on the other. In the event of a disaster or theft I'd be screwed I guess. It's not that it's very important data. It's just years of buying movies, ripping, encoding, then re ripping and re encoding because I found better encoding settings. Lots of time basically.

[Electronics Wizardy]

20 minutes ago, Torey said:

I can't afford a cloud service to store that much data at the moment. But I do have two systems and the backup could be stored on the other. In the event of a disaster or theft I'd be screwed I guess. It's not that it's very important data. It's just years of buying movies, ripping, encoding, then re ripping and re encoding because I found better encoding settings. Lots of time basically.

Also raid 6 should be safer than raid 10 as your can lose any 2 drives and btrfs/zfs do checksums so you know the data is good.

[Torey]

2 hours ago, Electronics Wizardy said:

Also raid 6 should be safer than raid 10 as your can lose any 2 drives and btrfs/zfs do checksums so you know the data is good.

How much of a performance increase on reads and writes would I see from raid 6?

[leadeater]

22 minutes ago, Torey said:

How much of a performance increase on reads and writes would I see from raid 6?

None, technically it should be a little slower since it is doing double parity not single parity so for every write request more disk writes are required to complete it. I also agree with @Electronics Wizardy use btrfs or zfs when creating a new storage array, mdadm while extremely easy and quick to use has no smarts behind it which is not necessarily a bad thing but just a by product of being designed when CPU power was an issue for software storage solutions and disks comparatively to now were much smaller.

[Torey]

47 minutes ago, leadeater said:

None, technically it should be a little slower since it is doing double parity not single parity so for every write request more disk writes are required to complete it. I also agree with @Electronics Wizardy use btrfs or zfs when creating a new storage array, mdadm while extremely easy and quick to use has no smarts behind it which is not necessarily a bad thing but just a by product of being designed when CPU power was an issue for software storage solutions and disks comparatively to now were much smaller.

Hmm. I'd really like some increase in performance. So I think I'm going to do either RAID10 with all four discs, or RAID0 a set of each and use rsync for nightly backups. Would you recommend one over the other? For now it would be on the same system in either situation.

[leadeater]

If possible go through the full set of testing each setup to find the one you like the most, takes ages but is something I always do. Better to get it right now than have to backup the data and reconfigure later due to unacceptable performance.

[Torey]

1 minute ago, leadeater said:

If possible go through the full set of testing each setup to find the one you like the most, takes ages but is something I always do. Better to get it right now than have to backup the data and reconfigure later due to unacceptable performance.

The choice between two raid 0 or one raid 10 isn't for a performance reason, just figured having them separate would be safer in a way. But I have no experience with that, was just an assumption.

 

Anyways, thanks for your help. Between here and reddit I think I have all the information I need.