How to setup a RAID configuration?

[Yea, i use windows, W]

I needed a ton of ultra fast storage as I am writing a AI which requires instant copy and pasting of LARGE FILES to another location and back to original position, I already have 3 NVME SSDs and my motherboard also has 3 M.2 SSD slots, they all are being detected and are already very fast, but I needed faster, So should I run them in RAID 0?

[Carbonado]

RAID 0 is taking any number of disks and merging them into one large volume. This will greatly increase speeds, as you're reading and writing from multiple disks at a time. An individual file can then use the speed and capacity of all the drives of the array. The downside to RAID 0 though is that it is NOT redundant. The loss of any individual disk will cause complete data loss. This RAID type is very much less reliable than having a single disk.

There are rarely a situation where you should use RAID 0 in a server environment. You can use it for cache or other purposes where speed is important and reliability/data loss does not matter at all. But it should not be used for anything other than that. As an example, with the 2.5% annual failure rate of drives, if you have a 6 disk RAID 0 array, you've increased your annual risk of data loss to nearly 13.5%. 

[Yea, i use windows, W]

1 minute ago, CouRageRC said:

RAID 0 is taking any number of disks and merging them into one large volume. This will greatly increase speeds, as you're reading and writing from multiple disks at a time. An individual file can then use the speed and capacity of all the drives of the array. The downside to RAID 0 though is that it is NOT redundant. The loss of any individual disk will cause complete data loss. This RAID type is very much less reliable than having a single disk.

There are rarely a situation where you should use RAID 0 in a server environment. You can use it for cache or other purposes where speed is important and reliability/data loss does not matter at all. But it should not be used for anything other than that. As an example, with the 2.5% annual failure rate of drives, if you have a 6 disk RAID 0 array, you've increased your annual risk of data loss to nearly 13.5%. 

ahh, I got the point, i dont need any actual redundant backup or anything, as my company has already made a program in which when the system is idle and nothing is going in the background and neither i am training the AI, it will automatically back up my drives to same drives, so actually i have in total 6 NVME SSDs, but in those 3 are being used as backup

[Carbonado]

1 minute ago, Yea, i use windows, W said:

ahh, I got the point, i dont need any actual redundant backup or anything, as my company has already made a program in which when the system is idle and nothing is going in the background and neither i am training the AI, it will automatically back up my drives to same drives, so actually i have in total 6 NVME SSDs, but in those 3 are being used as backup

Oh well okay in that case you can run it in RAID 0 without a headache.  

[Yea, i use windows, W]

1 minute ago, CouRageRC said:

Oh well okay in that case you can run it in RAID 0 without a headache.  

Thanks for the reply!! but I still actually dont know how to run in RAID 0, Thats why i asked "How"?

[Eigenvektor]

One thing to keep in mind, especially with NVMe: There are diminishing returns in terms of speed up adding more and more drives. At some point you will be limited by the PCIe bus and/or how fast the CPU can process the data.

 

There's a video by LTT where Linus tests a system with a ton of parallel drives and the speed up isn't as big as expected, because you run into limitations elsewhere.

[Yea, i use windows, W]

2 minutes ago, Eigenvektor said:

One thing to keep in mind, especially with NVMe: There are diminishing returns in terms of speed up adding more and more drives. At some point you will be limited by the PCIe bus and/or how fast the CPU can process the data.

 

There's a video by LTT where Linus tests a system with a ton of parallel drives and the speed up isn't as big as expected, because you run into limitations elsewhere.

Ah..... I understand but atleast it will be faster than one single NVME SSD running alone.... Also, I dont think a Threadripper will bottleneck my storage speeds  

[Carbonado]

1 minute ago, Yea, i use windows, W said:

Thanks for the reply!! but I still actually dont know how to run in RAID 0, Thats why i asked "How"?

oops sorry i did not notice that, well you can check out either linus's or Hardware Canuks's video on this 

 

[Oshino Shinobu]

Just now, Yea, i use windows, W said:

Thanks for the reply!! but I still actually dont know how to run in RAID 0, Thats why i asked "How"?

You have a few options. 

 

1. Use software "RAID" to stripe the drives together. You can do this in Windows 10, either through Disk Management or Storage Spaces. The latter is the better option, you'd just need to put the drives in a simple pool and create a vdisk out of them, which will stripe data across all three. 

 

2. Onboard RAID controller. This uses an onboard RAID controller on the motherboard to stripe the drives together. You'd have to refer to your motherboard's manual on how to do this. 

 

3. Dedicated hardware RAID. You have a dedicated PCIe device that stripes the drives together.

 

Option 1 is what I'd go for. IMO, hardware RAID is old tech now days. A lot of enterprise systems have moved away from hardware RAID to software RAID and software defined storage in general. 

 

You can read up on Storage Spaces here: https://support.microsoft.com/en-gb/help/12438/windows-10-storage-spaces

[Yea, i use windows, W]

1 minute ago, CouRageRC said:

oops sorry i did not notice that, well you can check out either linus's or Hardware Canuks's video on this 

 

Thanks!! Btw, is there any other solution to this, like if i had to make RAID 0 more secure can i do that?

[Carbonado]

2 minutes ago, Yea, i use windows, W said:

Thanks!! Btw, is there any other solution to this, like if i had to make RAID 0 more secure can i do that?

RAID 0 by itself isn't completely safe, as a small crash can ruin it, the only advice is to keep a backup of ANY files and data as they are the worst affect by a RAID 0 crash, other than that, the above solution should suffice for a good RAID 0 config.

[Yea, i use windows, W]

Just now, CouRageRC said:

RAID 0 by itself isn't completely safe, as a small crash can ruin it, the only advice is to keep a backup of ANY files and data as they are the worst affect by a RAID 0 crash, other than that, the above solution should suffice for a good RAID 0 config.

Ah, then I'm fine....  

[Carbonado]

Just now, Yea, i use windows, W said:

Ah, then I'm fine....  

Great, glad i could help!

[mahyar]

10 minutes ago, Yea, i use windows, W said:

Thanks!! Btw, is there any other solution to this, like if i had to make RAID 0 more secure can i do that?

its  expensive but raid 10 is suitable if you want speed and reliability 

 

[Yea, i use windows, W]

Just now, mahyar said:

its  expensive but raid 10 is suitable if you want speed and reliability 

 

hmmm.... What happens in RAID 10?

[mahyar]

Just now, Yea, i use windows, W said:

hmmm.... What happens in RAID 10?

x number of raid 1's in raid 0

[Dyaets]

Just as already mentioned, if you want speed from NVMe and you are fine with losing everything if a single drive fails and restoring from backups, just go with RAID 0. With any other RAID level you will use write speed. Depending on the OS, Storage Spaces for Windows or MDADM for Linux should do the job. Also, just in case here is some useful reading on different RAID levels pros and cons: https://www.starwindsoftware.com/blog/back-to-basics-raid-types

[Dyaets]

On 10/5/2020 at 12:00 AM, Oshino Shinobu said:

Dedicated hardware RAID. You have a dedicated PCIe device that stripes the drives together.

I agree on software RAID for NVMe drives but are there any real hardware RAID controllers that can collect NVMe? I mean, not RAID on chip. It's just I have never seen such.

[Oshino Shinobu]

3 minutes ago, Dyaets said:

I agree on software RAID for NVMe drives but are there any real hardware RAID controllers that can collect NVMe? I mean, not RAID on chip. It's just I have never seen such.

There are, though AFAIK they all use U.2, which is rarely used for consumer NVMe drives. You can adapt M.2 to U.2 though. 

 

Enterprise NVMe drives tend not to use M.2 from what I've seen. The likes of Intel's Gen2 Optane for example, so most of the dedicated RAID cards are U.2. I've never actually seen one in a production environment though.

[Dyaets]

1 minute ago, Oshino Shinobu said:

There are, though AFAIK they all use U.2, which is rarely used for consumer NVMe drives. You can adapt M.2 to U.2 though. 

Ahh, got you! I haven't seen those yet. Still, good info. Thank you!

[Oshino Shinobu]

Just now, Dyaets said:

Ahh, got you! I haven't seen those yet. Still, good info. Thank you!

Quick look for them turned up this for me. They're not cheap it seems. 

 

https://www.scan.co.uk/products/4-port-highpoint-ssd7120-4x-dedicated-32gbps-u2-ports-to-pcie-30x16-raid-controller

 

Hardware RAID is fading away though. Software "RAID" is generally what enterprise is moving towards from what I've seen. 

[Dyaets]

10 minutes ago, Oshino Shinobu said:

Quick look for them turned up this for me. They're not cheap it seems. 

Ouch, 400 pounds is definitely not cheap. This can be a fir for an enterprise though with a server chassis packed with U.2 NVMe drives. However, I have to agree with you, hardware RAID is getting used more rarely.