JBOD/RAID/NAS/WTF?!?!?!

[fenolitto]

Hey All,

 

I'm looking to start building in some redundancy into my current home media storage platform, and for the life of me I'm just having an extremely difficult time researching the leap from my current set up (stand alone PC) to something a bit more "server like".  Any guidance you can provide would be much appreciated.

 

First a little about me so you know what level of tech understanding you'd be dealing with in replying.  I have built about a dozen PC's so I know parts fairly well...at least from a big picture perspective.  Things like cache, bus speed, etc. are a little opaque to me, but I do get the notion of most "big ticket" components.  I understand the concepts behind RAID (0,1,5,6,10) and the way data is mirrored and striped across them.  With that said, from a hardware perspective I get a little lost.  I've never used RAID controllers, or server backplanes, etc. so while I get some of the theory, I don't necessarily understand how-when-why to use them...or if I even need to.  With that said, onto my current setup.

 

I currently run my primary PC as a media device to my TV (via HDMI through a stereo receiver).  I don't really game on it, although I have in the past.  Here is the basic parts list:

 

-       -  NZXT Source 530 Full Tower

-        - ASUS PRIME Z270-A

-        - Intel Core i5 6500

-        - ARCTIC Freezer Xtreme Rev. 2 CPU Cooler

-        - Corsair Vengeance LPX 16GB DDR4  RAM

-        - Gigabyte GeForce GTX 1060 WINDFORCE OC 6G

-        - 1 SanDisk SSD (480GB for OS and some programs)

-        - 7 WD 3TB Hard Drives (not Red)

-        - 1 WD 4TB Hard Drive (not Red)

-        - 2x Mediasonic 4 Bay HDD enclosures

-        - Running Windows 10 64-bit Home

My hard drive configuration/use is as follows:

-        - 3TB Movie Drive

-        - 3TB TV Show Drive

-        - 3TB Misc Drive (Pics, Books, etc.)

-        - 3TB Movie Drive (Manual back-up of the first drive above)

-        - 3TB TV Show Drive (Manual Back-up of the second drive above)

-        - 3TB Misc Drive (Backup….as above)

-        - 4TB Download Drive (where I park all data then distribute it to the relevant drive)

-        - 3TB Overflow Drive (for when the Movie/TV drives get full)

 

Essentially I’m running Raid 1 across 3 sets of data, with two individual drives that wouldn’t really break my heart if one or the other crashed.  Obviously I’d prefer to rope those into the recoverable structure, and that’s why I’m here.  I decided that doing manual back-ups of all those drives (across USB) to the enclosures is likely not the best way to go about accomplishing my goal.  Or maybe it is…and that’s why I need advice. 

 

Onto what I’m trying to accomplish.  I’m willing to build a completely new machine and hook it up directly to my home router if that ends up being the best way to go.  Or I’m willing to work with my current structure and expand to an additional hard drive enclosure (8 bay this time) and start filling it out with drives.  As I’m running into drive fullness issues, I think I’ll be building around or expanding with/to something like the Seagate IronFox 10TB variety, but I’m willing to listen to other ideas.  I’d like to keep all the data I currently have (approximately 10-11TB across 5 drives) in any expansion process. 

 

I guess my big questions are these:

- Should I be thinking about building a standalone server (NAS or otherwise), or would I be better served to just buy another larger enclosure?

- If I went with another enclosure, can I set it up as it's own array with RAID, and if so what software should I look at?

- If a new build would be better, can someone explain whether I would need a RAID controller to be installed or if I can just plug drives directly into the mobo?

- Lastly, since I have a significant (and growing) file storage demands, do I need to start thinking about increasing my RAM?

 

Thank you all for any/all advice and guidance you can provide.

[Nineballo]

Raid 5 would allow one drive to fail with no files lost, raid 6 would allow two failures at the same time with no files lost...

 

you can plug them into your mobo but if you are running out of Sata ports or dont have esata ports a pcie Sata/esata expansion card is an option (just make sure it supports raid) here is one it has 4 esata connections for hard drive bays or you can connect 8 hard drives directly to it.

 

you can use IRST by intel to do the raiding  here is the link and it will also tell you when a drive failure happens so you can change it out.

 

I don't think increasing your ram would help much if its only you (and your family) using the system... but you can check your usage in Task Manager to see if its high. if its over 80-90% full load you should probably think about getting another stick.

 

if i missed something or you need a better explanation put  @NineBall-uwu  somewhere in your post

[jake9000]

You can build out a cute lil DAS (direct attached storage) for relatively little. Have a look at this build

What makes it work is a 4-port SAS controller and 4 SAS --> 4x SATA cable. These 5 parts will let you hook up 16 drives to your existing computer. The rest of that build is just enclosure and power delivery. 

I have personally never heard of an Ironfox drive. If you're referring to the Ironwolfs, I can personally +1 that. I deploy them to customers and they're great. Main reason I have them is for their Synology integration, though. Consider the WD Reds as well and get whatever's on sale. 

 

Keep in mind that to induct a drive into an array, the 'new' drive will be reformatted. If you want all these drives to become a single logical volume you'll probably need to shuffle your stuff around and set your backups on fire to have enough clean drives to start the array, then gradually move data into it and induct more drives as you go. 

If you're on windows 10, Storage Spaces is easy enough. https://support.microsoft.com/en-us/help/12438/windows-10-storage-spaces 

[fenolitto]

1 hour ago, NineBall-uwu said:

Raid 5 would allow one drive to fail with no files lost, raid 6 would allow two failures at the same time with no files lost...

 

you can plug them into your mobo but if you are running out of Sata ports or dont have esata ports a pcie Sata/esata expansion card is an option (just make sure it supports raid) here is one it has 4 esata connections for hard drive bays or you can connect 8 hard drives directly to it.

 

you can use IRST by intel to do the raiding  here is the link and it will also tell you when a drive failure happens so you can change it out.

 

I don't think increasing your ram would help much if its only you (and your family) using the system... but you can check your usage in Task Manager to see if its high. if its over 80-90% full load you should probably think about getting another stick.

 

if i missed something or you need a better explanation put  @NineBall-uwu  somewhere in your post

Thank you for the reply.  That's a lot to absorb, but I'm trying.

 

I assume this is all based off the idea that going NAS is my best bet.  Correct?

[fenolitto]

18 minutes ago, jake9000 said:

You can build out a cute lil DAS (direct attached storage) for relatively little. Have a look at this build

What makes it work is a 4-port SAS controller and 4 SAS --> 4x SATA cable. These 5 parts will let you hook up 16 drives to your existing computer. The rest of that build is just enclosure and power delivery. 

That's actually pretty cool.  Might be just a bit more than I need.  Would something like this work...assuming I'm willing to just pay for the convenience of pre-built?

 

https://www.amazon.com/gp/product/B005GYDMYG/ref=ox_sc_saved_title_8?smid=A2WSTUX2A6RQ1A&psc=1

 

Or is there something to the structure of the DAS that better serves the purpose? 

 

Expanding on that, if I did use the above enclosure and just connect it to the PC I'm already using could I establish any/all of those drives in a RAID setup with software?

 

Sorry that a lot of this probably comes across as entirely noob-like.  And thank you for your help.

[jake9000]

Depends on the performance you're looking for. The DAS system I proposed will be way more performant... that said if you don't need every drive to be IOing at their maximum rate simultaneously it probably isn't going to be a problem. 

Yes though you can set up your array with one of those. or multiple of them  

[fenolitto]

3 minutes ago, jake9000 said:

Depends on the performance you're looking for. The DAS system I proposed will be way more performant... that said if you don't need every drive to be IOing at their maximum rate simultaneously it probably isn't going to be a problem. 

Yes though you can set up your array with one of those. or multiple of them  

I've currently got two 4 bay versions of that enclosure.  If I bought the larger one and started filling it with drives, is there just RAID software that runs on Windows that I'd use?  Also, since I'm not quite ready to drop $2,000 on 8x $300 drives, is there a way to expand the RAID array over time?  I've read conflicting reports about adding additional drives to existing arrays.

[jake9000]

That's a loaded question, depends a lot on the raid type and the individual hardware controller. Since you're doing it in software it's a moot point. I know for a fact that Storage Spaces, Unraid, and ZFS allow it. Those are the only three I've tried (and only Storage Spaces works with windows, unraid is an entire operating system and ZFS is a Unix filesystem.)

[fenolitto]

1 minute ago, jake9000 said:

That's a loaded question, depends a lot on the raid type and the individual hardware controller. Since you're doing it in software it's a moot point. I know for a fact that Storage Spaces, Unraid, and ZFS allow it. Those are the only three I've tried (and only Storage Spaces works with windows, unraid is an entire operating system and ZFS is a Unix filesystem.)

Okay, so it sounds like Storage Spaces it is.  Thanks for the advice.

[Nineballo]

As I said earlier Intel rapid storage tech can also raid, and is free.

 

 

my comment would work for a NAS a DAS or if you just want a lot of drives.

[dalekphalm]

On 4/7/2019 at 2:07 AM, jake9000 said:

That's a loaded question, depends a lot on the raid type and the individual hardware controller. Since you're doing it in software it's a moot point. I know for a fact that Storage Spaces, Unraid, and ZFS allow it. Those are the only three I've tried (and only Storage Spaces works with windows, unraid is an entire operating system and ZFS is a Unix filesystem.)

Please note that ZFS does not allow Array expansion - at least, not in the traditional way.

 

There are two ways to expand ZFS:

1. Replace one of the drives in the array with a larger drive. Rebuild the array. Repeat with the 2nd drive. Rebuild the array. Repeat entire process, rebuilding after each new drive, until all drives are now the newer larger capacity. Does not allow for a single drive expansion, nor does it allow for an increase in drive count.

2. Add an entirely new second array (ideally in the exact same size and configuration as the original) and span that new array with the original array. This creates what is effectively a RAID 50 or RAID 60 array, depending on whether the original array was RAIDZ1 (RAID 5 equivalent) or RAIDZ2 (RAID 6 equivalent).

 

The ZFS dev team is working on "easy" RAIDZ1 expansion, but I have not seen any updates since Summer of 2018. Once they get RAIDZ1 working, it should be easy to expand that to RAIDZ2 and RAIDZ3.

[Electronics Wizardy]

On 4/7/2019 at 3:05 AM, NineBall-uwu said:

As I said earlier Intel rapid storage tech can also raid, and is free.

 

 

my comment would work for a NAS a DAS or if you just want a lot of drives.

Id stay away from the motherboard raid, use storage spaces, it better, and free aswell.