RAID 10 Ubuntu Server

[juraj]

Hello Guys,

 

I am very happy I have found this forum because I did not know where should I ask for help. I have watched all your videos on youtube about servers and RAIDs, they are really perfect!

 

My interest is to improve my current home server with RAID Array. 

 

Currently, I have a home server with one old 200Gb and new 1Tb HDD. 1TB HDD I am using for owncloud and Plex. The 200Gb is for Ubuntu Server OS and webpage hosting. 

Now I would like to step a bit up to improve my setup with some additional safety. I do not need really large space capacity thus I am considering to buy 4x 1TB WD red HDDs. 

I want to install there again Ubuntu Server, my idea was to use the HDDs as RAID 10 and use my current 1TB HDD as spare HDD (the option ubuntu server offers). I am interested in trade-off between speed and reliability. 

 

The problem is, based on the documentation the /boot partition can be just on RAID 1 max. I can not figure out what is the most convenient and safe way to partition my future 5x1TB HDDs with a condition of obtaining 1x RAID1 for /boot, 1xRAID10 for file system + storage space and SWAP space. 

 

My current idea is based on tons of tutorial which can be seen in the attached image (hopefully you will understand it). I thought about cutting a small part from each HDD for /boot and create RAID1 for /boot from 4 smaller partitions, then cut again small part for SWAP from each HDD and the rest of space use as RAID10 for File System + Storage Space (because each partition of RAID10 has to have the same space I thought about cutting a small part of each drive). This should mean that if any of these two drives fails, all the system should be recoverable. Expected storage space is around 1.5TB.

 

Do you have some better idea how can I do, or a hint? Just to let you know, I do not have many experiences about RAIDs just what I learned from the  internet thus I am very helpful for any kind of help!

 

My second question is regarding one of your videos. You said that RAIDs are not actually backup and I need additional backup. How did you mean it? Like if I imagine I will have a server with 1TB stored files, should I save all this data on some additional place, or did you mean just the os?

 

Thank you all very much for any kind of help!

 

Juraj!

[Electronics Wizardy]

Id personally keep your 200gb for boot then make a raid 5 from the 4x 1tb hdds using btrfs, not mdadm.

 

mdadm is old stuff and doesnt handle errors as well as btrfs does.

 

But all the boot files on the 200gb drive. If you really need only the 1tb drives, id make the /boot and swap on one of the drives, with btrfs using the rest of the space, then you can add more drives to a btrfs raid later.

[juraj]

Thank you very much for your reply!

 

If I will keep all boot stuff as well as OS on the 200gb, if this drive will fail, the system will not boot anymore. Or how did you think it?

 

The single problem with raid5 is that is not as durable as raid10 and requires a lot of calculations and I do not plan to buy external Raid controller. I am afraid it would be too slow, wouldn't it?

 

My decision on RAID10 was based on the fact that I want to keep it lower cost, I do not need a lot of space and I have already 1x1tb. I will be happy if you will suggest me what else to buy in my current situation.

 

The default ubuntu server installer is using mdadm right? So first I need to install the os and then using btrfs the RAID?

 

2 hours ago, Electronics Wizardy said:

If you really need only the 1tb drives

 

No, that was just my idea because I already have 1x1TB. 

[Electronics Wizardy]

15 minutes ago, juraj said:

Thank you very much for your reply!

 

If I will keep all boot stuff as well as OS on the 200gb, if this drive will fail, the system will not boot anymore. Or how did you think it?

 

The single problem with raid5 is that is not as durable as raid10 and requires a lot of calculations and I do not plan to buy external Raid controller. I am afraid it would be too slow, wouldn't it?

 

My decision on RAID10 was based on the fact that I want to keep it lower cost, I do not need a lot of space and I have already 1x1tb. I will be happy if you will suggest me what else to buy in my current situation.

 

The default ubuntu server installer is using mdadm right? So first I need to install the os and then using btrfs the RAID?

 

No, that was just my idea because I already have 1x1TB. 

If you keep the os on the 200 gb you can make a backup on your array using dd, so that if your boot drive fails you can copy all the data to anouther drive and then boot from that drive. You can setup a cron job that will make a new image every week or two.

 

Raid 5 is about the same speed as your using software, not hardware raid. You won't need a raid controller. If you need it to be more reliable you can use raid 6. I normally stay away from raid 10 unless you need to have the most iops(for a vm or database workload.) for a media server raid 5/6 will be more than fast enough.

 

I would personally use raid 5 and then backup to a external 4tb drive every month. Raid isn't going to help you if you delete a file or randsonware starts working.

 

Since you already have a ubuntu server install on the 200gb drive, i wouldn't reinstall and would just add the drives and make the array. Id probably do it like this.

 

1. Add the new drives

 

2. Make a btrfs raid 5 with the 3 new drives using sudo mkfs.btrfs -m raid 1 -d raid5 /dev/sdc /dev/sdd /dev/sde (use your own drive letters) then mount the drive in fstab

 

3. Copy the files from your old drive over using cp or rsync.

 

4. Add the old device using sudo btrfs device add /dev/sdb /mnt

 

5. Then balance the data with sudo btrfs filesystem balance /mnt

[juraj]

15 minutes ago, Electronics Wizardy said:

If you keep the os on the 200 gb you can make a backup on your array using dd, so that if your boot drive fails you can copy all the data to anouther drive and then boot from that drive. You can setup a cron job that will make a new image every week or two.

 

Raid 5 is about the same speed as your using software, not hardware raid. You won't need a raid controller. If you need it to be more reliable you can use raid 6. I normally stay away from raid 10 unless you need to have the most iops(for a vm or database workload.) for a media server raid 5/6 will be more than fast enough.

 

I would personally use raid 5 and then backup to a external 4tb drive every month. Raid isn't going to help you if you delete a file or randsonware starts working.

 

Since you already have a ubuntu server install on the 200gb drive, i wouldn't reinstall and would just add the drives and make the array. Id probably do it like this.

 

1. Add the new drives

 

2. Make a btrfs raid 5 with the 3 new drives using sudo mkfs.btrfs -m raid 1 -d raid5 /dev/sdc /dev/sdd /dev/sde (use your own drive letters) then mount the drive in fstab

 

3. Copy the files from your old drive over using cp or rsync.

 

4. Add the old device using sudo btrfs device add /dev/sdb /mnt

 

5. Then balance the data with sudo btrfs filesystem balance /mnt

 

You are awesome, thanks!

 

well as I am using the server mostly for owncloud every day, there is the necessity for many iops for many database updates when I upload/download many small files (coding). Do you think it still is not so dramatic (I do not know in what scale you are talking, large company or small database)?

 

I can not imagine one thing. If 200gb fails but I would have a copy of the OS on RAID 5. How could I potentially restore the os when I can't boot to that pc anymore? Wouldn't I have to plug all HDDs from raid to another pc where I set them up (what I can not imagine how), copy the OS to another drive and then insert everything back to old pc?

 

 

[Electronics Wizardy]

11 minutes ago, juraj said:

You are awesome, thanks!

 

well as I am using the server mostly for owncloud every day, there is the necessity for many iops for many database updates when I upload/download many small files (coding). Do you think it still is not so dramatic (I do not know in what scale you are talking, large company or small database)?

 

I can not imagine one thing. If 200gb fails but I would have a copy of the OS on RAID 5. How could I potentially restore the os when I can't boot to that pc anymore? Wouldn't I have to plug all HDDs from raid to another pc where I set them up (what I can not imagine how), copy the OS to another drive and then insert everything back to old pc?

 

 

Raid 5 is more than fast enough for what your doing. You won't get near the limits of the drives speeds.

 

If you want to restore the os, you put a new drive in(200gb or bigger to make is easyier. Put a live linux disk in. Boot from the usb and then copy the image back onto the other drive.

[juraj]

18 minutes ago, Electronics Wizardy said:

Raid 5 is more than fast enough for what your doing. You won't get near the limits of the drives speeds.

 

If you want to restore the os, you put a new drive in(200gb or bigger to make is easyier. Put a live linux disk in. Boot from the usb and then copy the image back onto the other drive.

 

Thank you for the help! That solves all my questions for now! 

[juraj]

I still thought about one more question. 

If I would want to change also the 200gb drive. How would you suggest to do it with the raids and the condition on RAID0/1 for /boot (because the 200gb is already very old)?

 

Thank you again for your time!

[juraj]

Based on multiple videos on youtube, they show that is possible to mount whole / partition on raid5, e.g. 

 

I do not understand how that is possible when in documentation is written that /boot partition has to be either on RAID0 or 1 (6:54 in video). Can someone enlighten me? 

 

In case I would install OS on raid5 or raid1, if one drive fails would the computer still boot? 

 

 

[Ramaddil]

Honestly I would not run raid 5 with out a good backup solution. There is to much chance of a failure during a rebuild on a raid 5.  You should have a good backup solution no matter what you choose (I.E. cloud storage, or other solutions) , me personally I prefer raid 6 for extra redundancy.  But with only a few drives it is a little over kill, you would be better off in my opinion to get two 4 TB or two 2TB and run in raid 1 (per TB cost is less on higher capacity).   Raid 5 and 6 don't really become worth it until you have more than 4 drives. As far as the 200GB drive i wouldn't run your OS on your raid array as you previously mentioned.  I would buy a small SSD and run the OS on that.  A 120 GB SSD can be had for 40 dollars, but at that point you could get a 240 for just a little more.  

[juraj]

16 minutes ago, Ramaddil said:

Honestly I would not run raid 5 with out a good backup solution. There is to much chance of a failure during a rebuild on a raid 5.  You should have a good backup solution no matter what you choose (I.E. cloud storage, or other solutions) , me personally I prefer raid 6 for extra redundancy.  But with only a few drives it is a little over kill, you would be better off in my opinion to get two 4 TB or two 2TB and run in raid 1 (per TB cost is less on higher capacity).   Raid 5 and 6 don't really become worth it until you have more than 4 drives. As far as the 200GB drive i wouldn't run your OS on your raid array as you previously mentioned.  I would buy a small SSD and run the OS on that.  A 120 GB SSD can be had for 40 dollars, but at that point you could get a 240 for just a little more.  

 

 

Thanks for the response!

 

I have one spare old pc with Intel xeon so I plan to rebuild all the server as right now it has intel core 2 duo (takes really a lot of electricity) so I it does not bother me if I have to install everything new, plus I know now how to configure everything.

 

The thing is that I have already one 1TB hdd which I would like to use, thus I thought to go for 2 additional 1TB hdds in RIAD5 (as @Electronics Wizardy said)  what gives me 2TB plus one 3TB drive for the whole server back up, what as I counted is cheaper than getting 2x2TB + 1x4TB in raid1. However, I will definitely consider your idea of small ssd for os. Regarding your opinion about raid6, based on multiple videos they state  that Raid6 should be used JUST with raid controller as it requires many computations. Do you have some experiences for smaller (home) scale servers?

 

I am interested, why you do not like RAID10? Based on the internet information I have read, I understand that it as best thread-off between speed and redundancy.

 

It would be very helpful, definitely not just for me but also for others, if you more experienced guys could also state some reason why would not you do that (as we beginners do not have experiences) e.g. run OS on raid (most tutorials are doing this).

 

 

[Ramaddil]

Well I guess given your current situation raid 6 is a little ambitious.  I have a home server with a raid card that i bought used from ebay for < $100.  It supports 24 drives or more with a sas expander.  I run raid 6 with roughly 23 (2TB drives).

 

Running your OS on your array in my humble opinion is a bad idea.  The reason why is given the inherent nature of possibilities of viruses or data corruption,  I would rather my OS be separate from my array with my media.  This is how most people set up a regular desktop pc with ssd+hdd for data.  So my though process holds the same.   Given the possibility of problems with your array 0,1,5,6,10 etc if you have a failure except for raid 1 you will not be able to boot your OS if it is on the array.  This is important because sometimes there is a possibility that the array can be saved or data can be recovered, but if your OS is on the place you store your data you will have to insert a new HD and install your os to attempt to read the contents of your storage pool/raid. 

 

As far as raid 10 it is a viable option for those without serious raid cards given no parity is calculated.  Most decent motherboards have the capacity to do raid 0,1, and even 10.  I dont run it because with massive amount of hard drives or even large hard drives you cut your storage in half. With Raid 10 you have two raid 0's in raid 1.  So with 4 hard drives of equal size say you have 4x1TB you would make a raid 10 that would only have 2 TB.  When you have 20 drives of equal size you only get 10 drives worth of space.  Raid 10 has good speed, but honestly most home networks can only achieve 100mb/s.  

 

Given all this I would rather take a speed hit from raid 5 or 6 and conserve HD space, then go with raid 10.  Especially for media streaming.  Now if you are doing video editing or other tasks on this server and had a 10gbe network then yes a faster raid level would make sense for editing.  Most media apps wont come close to saturate a gigabit network.  

[juraj]

Perfect thanks!

 

How are you handling backup on such a large storage?

[Ramaddil]

I have 3 Backup sources.. First I have a replica server with a different drive setup it hosts (5) 6 TB drives in a raid 5, everyday at a specific time this backup server powers on for 1 hour and checks the primary for differences.  Then the backup server downloads the changes to the appropriate folders and shuts down.  Second I have Crashplan which is 6 dollars a month for unlimited cloud storage.  The Program runs on the server and backup's up all my files and any changes to those files automatically, and allows me to restore those files in the event of a loss.  Lastly I have multiple External hard drives that I use when I travel.  I use these to maintain copies of all my data, and have a program that will check each drive when plugged in and makes appropriate changes based off the folders it is backing up.

[juraj]

Thank you for the description. I have been googling how could I do what you described - "everyday at a specific time this backup server powers on for 1 hour and checks the primary for differences ".

 

I did not find some usable solution. What software and OS did you use?

 

[Ramaddil]

6 hours ago, juraj said:

Thank you for the description. I have been googling how could I do what you described - "everyday at a specific time this backup server powers on for 1 hour and checks the primary for differences ".

 

I did not find some usable solution. What software and OS did you use?

 

So most motherboards have a function to power on the computer at a certain time in the bios.  I use Windows 8.1/Windows 10.  In Control panel/administrative tools/task scheduler you can set up a task to at a certain time of the day run a specific command.  I set it for 1 hour after turning on from bios (I.E. Bios powers computer on at 6Am, and task scheduler runs Shutdown command at 7Am.  

 

I use a Program called Syncback Pro.  With this program you can set up profiles to mirror right/left//sync or other functions between two folders or drives and even over networks.  There are other programs that do this as well. I find that Syncback pro works for me.