Search the Community
Showing results for tags 'flexraid'.
-
Reducing Single Points of Failure in Redundant Storage In lots of the storage builds that exist here on the forum, the primary method of data protection is RAID, sometimes coupled with a backup solution (or the storage build is the backup solution). In the storage industry, there are load of systems that utilize RAID to provide redundancy for customers' data. One key aspect of (good) storage solutions is being resistant to not only drive failures (which happen a LOT), but also failure of other components as well. The goal is to have no single point of failure. First, let's ask: What is a Single Point of Failure? A single point of failure is exactly what it sounds like. Pick a component inside of the storage system, and imagine that it was broken, removed, etc. Do you lose any data as a result of this? If so, then that component is a single point of failure. By the way, from this point forward a single point of failure will be abbreviated as: SPoF Let's pick on looney again, using his system given here. Looney's build contains a FlexRAID software RAID array, which is comprised of drives on two separate hardware RAID cards running as Host Bus Adapters, with a handful of iSCSI targeted drives. We'll just focus on the RAID arrays for now, since those seem like where he would store data he wants to protect. His two arrays are on two separate hardware RAID cards, which provide redundancy in case of possible drive failures. As long as he replaces drives as they fail, his array is unlikely to go down. Now let's be mean and remove his IBM card, effectively removing 8 of his 14 drives. Since he only has two drives worth of parity, is his system still online? No, we have exceeded the capability of his RAID array to recover from drive loss. If he only had this system, that makes his IBM card a SPoF, as well as his RocketRaid card. However, he has a cloud backup service, which is very reliable in terms of keeping data intact. In addition, being from the Kingdom of the Netherlands, he has fantastic 100/100 internet service, making the process of recovering from a total system loss much easier. See why RAID doesn't constitute backup? It doesn't protect you from a catastrophic event. In professional environments, lots of storage is done over specialized networks, where multiple systems can replicate data to keep it safe in the event of a single system loss. In addition, systems may have multiple storage controllers (not like RAID controllers) which allow a single system to keep operating in the event of a controller failure. These systems also run RAID to prevent against drive loss. In systems running the Z File System (ZFS) like FreeNAS or Ubuntu with ZFS installed, DIY users can eliminate SPoFs by using multiple storage controllers, and planning their volumes to reduce the risk of data loss. Something similar can be done (I believe) with FlexRAID. This article aims to provide examples for theoretical configurations, and will have some practical real-life examples as well. It also will outline the high (sometimes unreasonably high) cost of eliminating SPoF for certain configurations, and aim to identify more efficient and practical ones. Please note: There is no hardware RAID control going on here, all software RAID. When 'controllers' are mentioned, I am referring to the Intel/3rd party SATA chipsets on a motherboard, an add-in SATA controller (Host Bus Adapter), or an add-in RAID card running without RAID configured. The controllers only provide the computer with more SATA ports, and it is the software itself which controls the RAID array. First, lets start with hypothetical situations. We have a user with some drives who wants to eliminate SPoFs in his system. Since we can't remove the risk of a catastrophic failure (such as a CPU, motherboard or RAM failure), we'll ignore those for now. We can, however, reduce the risk of downtime due to a controller failure. This might be a 3rd party chipset, a RAID card (not configured for RAID) or other HBA which connects drives to the system. RAID 0 will not be considered, since there is no redundancy. Note: For clarification, RAID 5 represents single-parity RAID, or RAID Z1 (ZFS). RAID 6 represents dual-parity RAID, or RAID Z2. RAID 7 represents triple-parity RAID, or RAID Z3. Note: FlexRAID doesn't support nested RAID levels. [spoiler=Our user has two drives.] Given this, the only viable configuration is RAID 1. In a typical situation, we might hook both drives up to the same controller and call it a day. But now that controller is a SPoF! To get around this, we'll use two controllers, and set up the configuration as shown: Now, if we remove a controller, there is still an active drive that keeps the data alive! This system has removed the controllers as a SPoF. [spoiler=Our user has three drives.] With three drives, we can do either a 3-way RAID 1 mirror, or a RAID 5 configuration. Let's start with RAID 1: Remembering that we want to have at least 2 controllers, we can set up the RAID 1 in one of two ways, shown below: In this instance, we could lose any controller, and the array would still be alive. Now let's go to RAID 5: In RAID 5, a loss of more than 1 drive will kill the array. Therefore, there must be at least 3 controllers to prevent any one from becoming an SPoF, shown below: Notice that in this situation, we are using a lot of controllers given the number of drives we have. Note also that the more drives a RAID 5 contains, the more controllers we will need. We'll see this shortly. [spoiler=Our user has four drives.] We'll stop using RAID 1 at this point, since it is very costly to keep building the array. This time, our options are RAID 5, RAID 6 and RAID 10. We'll start with RAID 5, for the last time. Remembering the insight we developed last time, we'll need 4 controller for 4 drives: This really starts to get expensive, unless you are already using 4 controllers in your system (we'll talk about this during the practical examples later on). Now on to RAID 6: Since RAID 6 can sustain two drive losses, we can put two drives on each controller, so we need 2 controllers to meet our requirements: In this situation, the loss of a controller will down two drives, which the array can endure. Last is RAID 10: Using RAID 10 with four drives gives us this minimum configuration: Notice that for RAID 10, we can put one drive from each RAID 1 stripe on a single controller. As we'll see later on, this allows us to create massive RAID 10 arrays with a relatively small number of controllers. In addition, using RAID 10 gives us the same storage space as a RAID 6, but with smaller worst-case redundancy. Given four drives, the best choices look like RAID 6and RAID 10, with the trade-off being redundancy (RAID 6is better) versus speed (RAID 10 is better). [spoiler=Our user has five drives.] For this case, we can't go with RAID 5, since it would require 5 controllers, and can't do RAID 10 with an odd number of drives. However, we do have RAID 6 and RAID 7. We'll start with RAID 6: Here we need at least 3 controllers, but one controller is underutilized: For RAID 7, we get 3 drives worth of redundancy, so we can put 3 drives on each controller: In this case, we need two controllers, with one being underutilized. [spoiler=Our user has six drives.] We can now start doing some more advanced nested RAID levels. In this case, we can create RAID 10, RAID 6, RAID 7, and RAID 50 (striped RAID 5). RAID 10 follows the logical progression from the four drive configuration: RAID 6 becomes as efficient as possible, fully utilizing all controllers: RAID 7 also becomes as efficient as possible, fully utilizing both controllers: RAID 50 is possible by creating two RAID 5volumes and striping them together as a RAID 0: Notice that we have reduced the number of controllers for a single-parity solution, since we can put one drive from each stripe onto a single controller. This progression will occur later as well, when we start looking at RAID 60 and RAID 70. We can obviously do a RAID 10, but now we can also venture into the realm of RAID 70 (striped RAID 7), in addition to RAID 60. Our RAID 60 is slightly underutilized: Finally, we have our RAID 70: Our RAID 70 is relatively inefficient (using 12 drives would be better, with six drives per controller). However, it allows for increased performance in addition to providing a huge amount of redundancy for this setup. This is definitely not recommended for anything other than mission-critical data or perhaps priceless memories (wedding video/photos, important docs, etc.) In this case, we can do either a RAID 50 or RAID 7, both of which require 3 controllers: For RAID 50, we have 3 stripes of RAID 5: For RAID 7, we have 3 controllers which are all fully utilized: Both configurations give us 6 drives worth of space for 3 controllers, with the tradeoff, once again, being worst-case redundancy (RAID 7 is better) versus speed (RAID 50 is better). For configurations with more drives, we aren't going to look at RAID 7, since we'll require more controllers. With eight drives, we can jump into the realm of RAID 60 (striped RAID 6 volumes). We also have the options of RAID 10, RAID 50, and RAID 7. RAID 10 looks just like it did before: RAID 50 looks similar, but now we need more controllers since each volume has more drives: RAID 7 looks pretty similar too: RAID 60 looks pretty efficient with this number of drives. Logically progressing from RAID 50, we can have two drives per stripe on one controller, for 4 drives per controller. This lets us use only two controllers: Like we had for four drives, we now have a tradeoff between worst-case redundancy and speed between RAID 60 and RAID 10.
-
Hello, I am using FlexRAID on my Ubuntu server. I have a RAIDf cruise control/snapshot raid configuration set up and it initialized as a single folder in my filesystem. I am trying to make a second configuration in the same way, but when I try to set up a new one it forces me to select expert mode, snapshot raid and Traid infinity engine. When I create this config and add the DRUs and PPU it initializes but I can't start the array. The problem I am having is the window for the original array (see attached "original_array") is really easy to use, but the new array (see attached "new_array") is very confusing and doesn't have a button to start the array.If anyone could offer some help that would be great. Thank you
-
Hi everyone, as per another thread I recently started, I'm thinking of moving my server storage to a new computer (Or, more accurately, upgrading the CPU+MB+RAM in the same chassis). The Storage Pool itself runs off of FlexRAID - I've been digging all over their website and google, and cannot find a straight forward answer on how to do this, but I want to move the entire configuration of pooled drives over to the new parts/rebuilt OS, without having to redo all the config in FlexRAID. I've found the documents on how to transfer my license, but it doesn't talk about the pool itself, just the software license. Has anyone gone through this before that could shed some light?
-
I would like to know if I can use FlexRAID RAID-F with a 5 bay external disk enclosure that is connected via USB 3.0 my enclosure is not always connected, just when I need to access the disks , typically 3-6 hours every other day but I would like to create a pool + RAID using RAID-F for those 5 disks I guess what i am asking is , would RAID-F work for this scenario where I have a number of disks ( JBOD ) in an external enclosure that is only connected at particular times ? thanks E2001 @looney
-
After receiving a lot of positive feedback on my first tutorial I decided I should make another one. But I'm having some problems with selecting the subject, so please vote on the one you would like to see, if you miss any please tell me! Please note that I am no expert on any of these systems yet, most will be new for me, so it will take a while (at leas a month) to write as it will be a lot of new stuff for me, but that should also make the tutorial easier as I wont skip things like some of the guru's might do because they expect you to know things. :P For those who haven't seen it, this is my first tutorial: http://linustechtips.com/main/topic/33510-how-to-install-and-setup-flexraid-on-your-windows-storage-system/ PS: poll result will not be decisive, just want to get a feeling of what you guys want :P
-
Hey guys, I'm in need of more storage :D First I wanted to do a Raid 5 with a cheap 150$ 4-Port SAS Raid Controller, but then I saw some of you recommend Flexraid and it seems a lot better suited for my usage scenario. (Mostly big static media files) Here is what I have: 2x Samsung Spinpoint F1 1000GB, (HD103UJ) 2x Western Digital Caviar Green 2000GB, (WD20EARS) And these I'd buy: 2x Western Digital Red 3000GB (WD30EFRX) or Seagate Desktop HDD.15 4000GB (ST4000DM000) I still have some questions about Flexraid though and I thought you might be able to answer them: 1.) Snapshot-mode is probably the best alternative for me, right? 2.) What would you recommend how often I Update / Quick Validate / Validate / Verify? How long does it take? 3.) Is it possible to automatically send drives to sleep when they are not in use? 4.) How does Flexraid distribute new data on the drives? Does it fill up drives, or does it try to keep all drives equally full? 5.) Is the read/write performance only dependent on the drive it get's written/read from? 6.) If one of the 3TB is PPU and both of the 1TB drives fail, can I rebuild them? (Since I have 3TB parity and only 2TB failed) Thanks!
-
How to: Install and setup FlexRAID on your windows storage system. The tutorial will be spread over multiple posts so you can use the links below to navigate. What is FlexRAID? FlexRAID Tutorial system explained. FlexRAID Preparing your system. FlexRAID Software installation. FlexRAID Web UI and Licensing. FlexRAID Array/Pool creation. FlexRAID Array/Pool settings. FlexRAID Schedule setup. FlexRAID Folder sharing. FlexRAID FAQ. PDF version available >>HERE<< Disclaimer: I, Koen van der Sanden (looney), am in no way affiliated with FlexRAID, its partners or its competitors. The following tutorial/guide is presented for a educational purpose only, and no warranties are made as to its accuracy or reliability. This tutorial was made for http://www.linustechtips.com (LTT) you may share this tutorial on other websites with my express permission (PM me for this) as long as the tutorial will remain publicly available and you don’t remove the LTT watermarks.