Help me find a cheap, redundant storage solution

[7idge]

Hi,

 

So I have a windows PC currently set-up with my Plex server and some other stuff as well as storage which is shared over the network to my main PC. I am running out of storage (all the SATA ports on the motherboard are taken) because I have lots of stuff for my plex server as well as other files for other stuff. I trying to find an affordable solution to replace the storage part of this PCs use. I would like it to be redundant because currently it isn't and I am kind of worried about losing some of the stuff I have.

It would need to have at least 10TB of storage. Speed doesn't matter massively as I don't use it for anything that needs high speeds. OS doesn't matter really as long as it supports SMB.

 

Thanks,
Harry

[Radium_Angel]

4 minutes ago, iCanSpellGaming said:

10TB of storage

 

4 minutes ago, iCanSpellGaming said:

lots of stuff

 

4 minutes ago, iCanSpellGaming said:

affordable solution

Pick one....

 

a NAS is the obvious solution, but your definition of affordable and mine may be miles apart.

What's your budget?

[Windows7ge]

If you can use ~identical drives I'd look into FreeNAS. It uses SAMBA/CIFS (basically SMB). Hardware wise are you looking to build a NAS?

[Blue4130]

2 hours ago, iCanSpellGaming said:

I would like it to be redundant because currently it isn't and I am kind of worried about losing some of the stuff I have.

Then make backups, redundancy is not a backup. 

[Glenn Schultes]

I use a Qnap Tr-004

It isn’t a NAS but a storage box for 4 drives that can be hardware raid 5 configured 

works at USB 3 speed

so it’s quicker than a NAS on a 1GB connection and more affordable than anything 10GB

There is also a 2 bay version that runs at 10GB USB 3.1 gen 2 speeds

[Pixel5]

so how much do you want to spend then, to recommend anything we would need a budget

 

[Lady Fitzgerald]

14 hours ago, iCanSpellGaming said:

...I would like it to be redundant because currently it isn't and I am kind of worried about losing some of the stuff I have...

Methinks you may be conflating or confusing redundancy with backups. Redundancy will protect your data from drive failure (up to a point) and can allow you computer to keep operating if a drive should fail. However, it will not protect you from viruses and other malware, power surges frying everything inside the computer, user error (such as accidental deletion or drive formatting), natural disasters (fire, flood, wind, etc.), theft, hardware damage or fialure, etc.

 

The only way for data to be reasonably safe is for it to exist in three seperate places. Usually, this is on the computer, on an onsite external backup drive, and on an offsite external backup drive. For a drive to be a backup drive, it must be kept powered down, disconnected from the computer, and stored out of sight of the computer except while updating the backup.

[Dutch_Master]

As we have no idea of your budget, here's a suggestion:

Obtain an old(er) ATX case with 3x 5.25" bays. Fill said 5,25" bays with a hot-swap frame for 3.5" drives (like this or cheaper) Fill this frame with 2x WD Red 8TB drives and another pair of Toshiba X300 8TB drives. Then install an Asrock Fatal1ty B450 board (here) with associated RAM as preferred, and include a 256GB NVMe M.2 SSD drive for the OS. (Samsung would be ideal, but better deals can be had on Aliexpress, in that these are cheaper but not as fast. For your use case, that doesn't matter as you've already alluded to) Connect the 4x 8TB drives to the main board and install Linux (any distro will do) on the NVMe boot drive. Next, use Webmin to create an RAID6 with the storage disks. Install and configure CIFS/Samba for Win-OS accessibility. FreeNAS does almost everything the same, except it uses FreeBSD instead of Linux and its own interface instead of Webmin.

 

HTH!

[Lady Fitzgerald]

14 hours ago, Dutch_Master said:

As we have no idea of your budget, here's a suggestion:

Obtain an old(er) ATX case with 3x 5.25" bays. Fill said 5,25" bays with a hot-swap frame for 3.5" drives (like this or cheaper) Fill this frame with 2x WD Red 8TB drives and another pair of Toshiba X300 8TB drives. Then install an Asrock Fatal1ty B450 board (here) with associated RAM as preferred, and include a 256GB NVMe M.2 SSD drive for the OS. (Samsung would be ideal, but better deals can be had on Aliexpress, in that these are cheaper but not as fast. For your use case, that doesn't matter as you've already alluded to) Connect the 4x 8TB drives to the main board and install Linux (any distro will do) on the NVMe boot drive. Next, use Webmin to create an RAID6 with the storage disks. Install and configure CIFS/Samba for Win-OS accessibility. FreeNAS does almost everything the same, except it uses FreeBSD instead of Linux and its own interface instead of Webmin.

 

HTH!

Unless this rig is kept powered down, disconnected from the computer, and stored out of sight of the computer, it will not be a backup or an archive, not to mention it would be way overkill and would not include an offsite backup. Not only is RAID not a backup, it adds unnecessary complication which gives more opportunity for failure.

 

The OP would be better off to get four 6 TB or larger HDDs, use two of them onsite as explained in previous posts, and the other two kept offsite. Swapping out the onsite and offsite drives as often as practical will make sure both sets of drives are kept as up to date as possible.

[Dutch_Master]

@Lady Fitzgerald I have a feeling you're putting too much emphasis on the word 'redundant' in the OP, in a way that you associated it with "backup". My impression is that the OP is simply looking to off-load the storage in his current system onto a separate system, with the added benefit it can be made redundant-to-an-extent. My suggestion (essentially a RAID6 NAS) gives him 2x8=16TB storage with a redundancy of 2 disks. Meaning, two random disks from that array could fail and the OP would still retain his data. Is it expensive? Yes, but from his message I gather loosing his data is priceless. And by using the hot-swap frame, the OP can swap the 8TB drives for larger ones to increase his storage capacity in the future w/o having to open his machine. Moreover, when he installs a 1TB NVMe SSD (instead of the 256GB I'd originally recommended) he could run his Plex server from this machine as well, off-loading even more tasks from his current PC. Of course he could use 6TB disks over the more expensive 8TB ones I'd suggested, but experience tells me readily available storage capacity is quickly filled, even if one accounts for this phenomenon and plans on larger disks from the outset. Thus, planning even bigger makes sense as it'll take longer to fill this capacity So, my suggestion wasn't intended (nor designed) as a backup, but the redundant extra machine for his data he was/is looking for.

[Blue4130]

8 minutes ago, Dutch_Master said:

@Lady Fitzgerald I have a feeling you're putting too much emphasis on the word 'redundant' in the OP, in a way that you associated it with "backup". My impression is that the OP is simply looking to off-load the storage in his current system onto a separate system, with the added benefit it can be made redundant-to-an-extent. My suggestion (essentially a RAID6 NAS) gives him 2x8=16TB storage with a redundancy of 2 disks. Meaning, two random disks from that array could fail and the OP would still retain his data. Is it expensive? Yes, but from his message I gather loosing his data is priceless. And by using the hot-swap frame, the OP can swap the 8TB drives for larger ones to increase his storage capacity in the future w/o having to open his machine. Moreover, when he installs a 1TB NVMe SSD (instead of the 256GB I'd originally recommended) he could run his Plex server from this machine as well, off-loading even more tasks from his current PC. Of course he could use 6TB disks over the more expensive 8TB ones I'd suggested, but experience tells me readily available storage capacity is quickly filled, even if one accounts for this phenomenon and plans on larger disks from the outset. Thus, planning even bigger makes sense as it'll take longer to fill this capacity So, my suggestion wasn't intended (nor designed) as a backup, but the redundant extra machine for his data he was/is looking for.

That fixes half of his problem. Buying two and keeping one off-site fixes the other

[Lady Fitzgerald]

1 hour ago, Dutch_Master said:

@Lady Fitzgerald I have a feeling you're putting too much emphasis on the word 'redundant' in the OP, in a way that you associated it with "backup". My impression is that the OP is simply looking to off-load the storage in his current system onto a separate system, with the added benefit it can be made redundant-to-an-extent. My suggestion (essentially a RAID6 NAS) gives him 2x8=16TB storage with a redundancy of 2 disks. Meaning, two random disks from that array could fail and the OP would still retain his data. Is it expensive? Yes, but from his message I gather loosing his data is priceless. And by using the hot-swap frame, the OP can swap the 8TB drives for larger ones to increase his storage capacity in the future w/o having to open his machine. Moreover, when he installs a 1TB NVMe SSD (instead of the 256GB I'd originally recommended) he could run his Plex server from this machine as well, off-loading even more tasks from his current PC. Of course he could use 6TB disks over the more expensive 8TB ones I'd suggested, but experience tells me readily available storage capacity is quickly filled, even if one accounts for this phenomenon and plans on larger disks from the outset. Thus, planning even bigger makes sense as it'll take longer to fill this capacity So, my suggestion wasn't intended (nor designed) as a backup, but the redundant extra machine for his data he was/is looking for.

The OP specifically stated he was concerned about losing data and he needed something inexpensive. Your solution was the opposite of inexpensive and would protect his data only from drive failure. Data can be lost from far more things than just drive failure. The solution I suggested would cost dramatically less than yours and, unlike yours, will protect his data from more than just drive failure. It is also much easier to implement.

[leadeater]

Why not just buy a cheap LSI HBA off ebay cheap and 2 10TB HDDs, put the card in the PC and create a Windows Storage Pool 2-way mirror (RAID 1) with these 2 HDDs then copy all the data over, then take out all the old disks, move the 2 10TB over to motherboard SATA ports and take out the LSI HBA and keep it around (really useful to have on hand).

 

Additionally buy a 10TB external USB drive and setup backups to that, remember to actually unplug it and keep it somewhere safe.

[Lady Fitzgerald]

5 hours ago, leadeater said:

Why not just buy a cheap LSI HBA off ebay cheap and 2 10TB HDDs, put the card in the PC and create a Windows Storage Pool 2-way mirror (RAID 1) with these 2 HDDs then copy all the data over, then take out all the old disks, move the 2 10TB over to motherboard SATA ports and take out the LSI HBA and keep it around (really useful to have on hand).

 

Additionally buy a 10TB external USB drive and setup backups to that, remember to actually unplug it and keep it somewhere safe.

Instead of having to do all that fooling around inside with a temporary HBA card, then swapping the drives around, just use an external USB dock to populate the replacement drives, then swap out the drives.

 

While it may seem more convenient to have all internal storage in a single volume, it adds more failure points to the data (increasing the chances of failure) and complicates making backups. Also, unless the OP needs to have a machine that will keep on running  Keeping things simple usually is more reliable and less expensive.

[leadeater]

58 minutes ago, Lady Fitzgerald said:

Instead of having to do all that fooling around inside with a temporary HBA card, then swapping the drives around, just use an external USB dock to populate the replacement drives, then swap out the drives.

USB dock costs more than an LSI HBA, it's the cheapest method other than a double transfer using the 10TB external backup disk from the old disks to the new disks. And you make is sound way harder than what it is, it's not fooling around or swapping disks, you're taking disks out when your done copying data off them so there is no reason to keep them in the system. So if you do that there is no point to leaving the HBA in the system so if you're opening to take out unused disks take out the HBA and plug the disks in the the motherboard.

 

58 minutes ago, Lady Fitzgerald said:

While it may seem more convenient to have all internal storage in a single volume, it adds more failure points to the data (increasing the chances of failure) and complicates making backups. Also, unless the OP needs to have a machine that will keep on running  Keeping things simple usually is more reliable and less expensive.

It doesn't add any extra failure points, two independent disks is two distinct failure points with their own failure risk, it's not any safer or less complex. Two disks in a native storage pooling feature of Windows all run in software that is a core part of the OS that you can transfer between installs and entire systems is much safer and far less likely to fail. The OS would be completely non functional by the time Storage Spaces would have an issue that would cause data loss, corruption or inability to access the data.

 

And it doesn't complicate backups, you get a standard disk with a drive letter just like anything else, 100% transparent to anything you would use or have to care about, you could even use a raw device cloning tool no problem, I know I've done it.

 

Thing is Storage Spaces Two-way mirror is not limited to two disks, it's mirroring data, so you can have any number of disks greater than 1 and get 50% usable capacity but you must do it from the start. You can't add an already in use disk to a pool.

 

I've noticed you're quite resistant to drive arrays in general, which is fine as usually it doesn't make sense to use them but the technical problem you're trying to point out aren't really a thing. If they were then nobody would be using them in the enterprise, what isn't used is onboard motherboard RAID that is found on desktop motherboards, that's asking for trouble.

[Lady Fitzgerald]

4 hours ago, leadeater said:

USB dock costs more than an LSI HBA, it's the cheapest method other than a double transfer using the 10TB external backup disk from the old disks to the new disks. And you make is sound way harder than what it is, it's not fooling around or swapping disks, you're taking disks out when your done copying data off them so there is no reason to keep them in the system. So if you do that there is no point to leaving the HBA in the system so if you're opening to take out unused disks take out the HBA and plug the disks in the the motherboard...

I don't where you do your shopping but I can buy any number of good USB docks from known reputable vendors for less than I can get an LSI HBA card on Fleabay where you take a chance on how reliable the vendor may be. Fleabay used to be a good place to fairly safely find bargains but finding reliable vendors there has become more of a crap shoot lately.

 

In addition to the card, you also need to buy a set of breakout cables to be able to connect the SATA drives to a SAS port on the HBA card. All that adds up in cost and it's only for temporary use whereas a USB dock can always be useful. If the OP bought bare drives for his backup drives (bare drives are usually better quality than external drives and, since you aren't spending money for the enclosure and PSU that comes with external drives, can even cost less), he could use the dock to connect the drives to his computer.

 

Then there is the problem of whether the OP's PC has an empty PCI-e slot to plug the HBA card into that has enough available lanes to run it. There is also the issue of where to put the drives while populating them.

 

All this requires far more work than simply plugging in a drive into a dock, plugging in the dock's PSU, and connecting the dock to the computer with a USB cable.

 

I apologize for one thing. For some reason (grogginess? I plead Old Timer's Disease), I confused your suggestion with a previous one that was far more complicated more complicated than your's. However, your suggestion is still more complicated than just using a USB dock, requiring juggling internal components.

4 hours ago, leadeater said:

 

It doesn't add any extra failure points, two independent disks is two distinct failure points with their own failure risk, it's not any safer or less complex. Two disks in a native storage pooling feature of Windows all run in software that is a core part of the OS that you can transfer between installs and entire systems is much safer and far less likely to fail. The OS would be completely non functional by the time Storage Spaces would have an issue that would cause data loss, corruption or inability to access the data.

 

And it doesn't complicate backups, you get a standard disk with a drive letter just like anything else, 100% transparent to anything you would use or have to care about, you could even use a raw device cloning tool no problem, I know I've done it.

 

Thing is Storage Spaces Two-way mirror is not limited to two disks, it's mirroring data, so you can have any number of disks greater than 1 and get 50% usable capacity but you must do it from the start. You can't add an already in use disk to a pool...

Your suggestion adds a form of redundancy that really isn't necessary and adds extra disks if actual proper backup drives are used. All the redundancy will do is protect against the failure of one drive, allowing the computer to keep on running until the dead drive can be replaced and rebuilt. If something else takes out the drives or data, redundancy will not be able to recover the data. Proper backups, on the other hand, will allow recovery of lost data.

 

While being able to keep a computer running after a drive dies is desirable, most people here do not actually NEED it (it's different for business machines where redundancy in addition to backups is essential). Setting up a storage pool or RAID 1 is an additional complication (does the OP even know how to set one up?). Storage pools and RAIDs are additional failure points  can, and do fail.

 

4 hours ago, leadeater said:

...I've noticed you're quite resistant to drive arrays in general, which is fine as usually it doesn't make sense to use them but the technical problem you're trying to point out aren't really a thing. If they were then nobody would be using them in the enterprise, what isn't used is onboard motherboard RAID that is found on desktop motherboards, that's asking for trouble.

I'm not resistant to arrays in general. I've acknowledged many times that they have their uses and, in many cases, are essential. What I object to is conflating redundancy with backups. Mmost people here are using them and/or advocate using them for the wrong reasons.

 

It is an extremely (and unfortunate) common belief that redundacy is a form of backup. It is NOT. I've been involved in I don't know how many heated arguments over this with people who simply cannot (or stubbornly refuse to) grasp that and frankly, it became tiresome a long time ago so, other than to say (again) keep it simple, I'm through here.

[leadeater]

6 hours ago, Lady Fitzgerald said:

I don't where you do your shopping but I can buy any number of good USB docks from known reputable vendors for less than I can get an LSI HBA card on Fleabay where you take a chance on how reliable the vendor may be. Fleabay used to be a good place to fairly safely find bargains but finding reliable vendors there has become more of a crap shoot lately.

If you want a multi bay USB enclosure then no those are expensive. The first proposed solution in the OP appeared to be build an entire system so with that budget buying 2 internal HDDs to replace everything is already far cheaper and simpler. And when it comes to used server parts the sellers of those are highly reliable, I've never heard of anyone getting screwed buying these types of parts and products.

 

It was also requested to be redundant so my solution offered included what was asked.

 

6 hours ago, Lady Fitzgerald said:

Then there is the problem of whether the OP's PC has an empty PCI-e slot to plug the HBA card into that has enough available lanes to run it. There is also the issue of where to put the drives while populating them.

Unless it's the smallest PC possible with a single PCIe slot taken up by a GPU there is a free slot, odds are over 98% people have a free slot.

 

6 hours ago, Lady Fitzgerald said:

Storage pools and RAIDs are additional failure points  can, and do fail

Hardware RAID cards fail, RAIDs do not. The only failure point is hardware and that is common to single drives or RAIDs or what ever, so unless you ignore failures until all HDDs are dead RAID isn't going to fail and in that situation a single disk would have results in the exact same outcome. Storage Pool on the other hand has no hardware RAID card that could fail and a software error causing a problem is so outlandishly rare it's not even worth worrying about, that's just as likely as NTFS getting a software bug and destroying your data, it's never happened.

 

Thing is it's dead simple to setup a Storage Pool, it's completely GUI in the settings of Windows. The big benefit when doing so is the single volume because not everything can spread data across disks or not easily and you'll spend more time messing around with data management which is a much larger risk of data loss as the most common cause of data loss is operator error. Being able to expand the pool with more disks to get more capacity is highly useful and requires no change to existing data management.

[leadeater]

6 hours ago, Lady Fitzgerald said:

Your suggestion adds a form of redundancy that really isn't necessary and adds extra disks if actual proper backup drives are used. All the redundancy will do is protect against the failure of one drive, allowing the computer to keep on running until the dead drive can be replaced and rebuilt. If something else takes out the drives or data, redundancy will not be able to recover the data. Proper backups, on the other hand, will allow recovery of lost data.

Yes, did you somehow ignore/not see the 3rd external 10TB HDD I said to buy to backup the computer?

[Lady Fitzgerald]

32 minutes ago, leadeater said:

If you want a multi bay USB enclosure then no those are expensive. The first proposed solution in the OP appeared to be build an entire system so with that budget buying 2 internal HDDs to replace everything is already far cheaper and simpler. And when it comes to used server parts the sellers of those are highly reliable, I've never heard of anyone getting screwed buying these types of parts and products.

 

It was also requested to be redundant so my solution offered included what was asked.

 

Unless it's the smallest PC possible with a single PCIe slot taken up by a GPU there is a free slot, odds are over 98% people have a free slot.

 

Hardware RAID cards fail, RAIDs do not. The only failure point is hardware and that is common to single drives or RAIDs or what ever, so unless you ignore failures until all HDDs are dead RAID isn't going to fail and in that situation a single disk would have results in the exact same outcome. Storage Pool on the other hand has no hardware RAID card that could fail and a software error causing a problem is so outlandishly rare it's not even worth worrying about, that's just as likely as NTFS getting a software bug and destroying your data, it's never happened.

 

Thing is it's dead simple to setup a Storage Pool, it's completely GUI in the settings of Windows. The big benefit when doing so is the single volume because not everything can spread data across disks or not easily and you'll spend more time messing around with data management which is a much larger risk of data loss as the most common cause of data loss is operator error. Being able to expand the pool with more disks to get more capacity is highly useful and requires no change to existing data management.

You've been luckier than me, then. I have problems with several eBay vendors. The worst are the Chinese arbitrageurs. I had trouble with so many Chinese vendors, I will not buy from China unless I have absolutely no other choice (btw, not buying at all can be a choice).

 

Any kind of RAID can fail either due to hardware failure (the most common reason) or something in the filesystem going wonky (rare but not unheard of).

 

Setting up a storage pool might be easy for you but it won't be for everyone, especially if one hasn't done it before. Spreading data over several disks is not at all difficult. I have my data spread over four disks (the fifth one hasn't been populated yet) and it's never been a problem.  Hardware cards can go bad and software RAIDs are even more vulnerable. Also, if the volune of the RAID is large enough, backing it up gets far more complicated.

[Lady Fitzgerald]

41 minutes ago, leadeater said:

Yes, did you somehow ignore/not see the 3rd external 10TB HDD I said to buy to backup the computer?

Yes, I saw it and no, I didn't ignore it. I just addressed the unnecessary redundancy that will not adequately protect the data. Also, just one external backup drive is inadequate although it's better than none at all.

[leadeater]

12 minutes ago, Lady Fitzgerald said:

Setting up a storage pool might be easy for you but it won't be for everyone, especially if one hasn't done it before. Spreading data over several disks is not at all difficult. I have my data spread over four disks (the fifth one hasn't been populated yet) and it's never been a problem

Thing is if you're automating your Plex setup all the tools out there have single path configuration for each content type i.e. Movies, TV etc so unless you want to split data on that at some point you'll have to step in and re-balance the data across the disks.

 

As for setting it up it's literally going in to settings/control panel ticking boxes equal to number of disks and clicking next, if you can't do that you can't setup a PC which would make any advice in this topic redundant as it would be impossible to carry out. Setting up Plex is much more involved.

 

Also I've never heard of any software RAID failing and if it's the filesystem then that's the filesystem not the RAID software and would affect a single disk. Most people aren't good at root cause analysis and may have blamed a software bug when it was just a hardware device going bad, which ZFS and Storage Spaces protects against. The worst I have encountered is firmware issues on Netgear NAS's that made the NAS OS lock up and have other various issues but it's never effected the actual data on the disks. I'm not saying it's impossible for there to be a software issue but in the realm of how well those are tested and the lack of direct proof of such a failure doesn't lead to it being a concern needed to be factored over the actually probable things that would necessitate the same protection method against this extreme improbable.

 

Basically I really only object to the notion that adding in a Storage Pool or a RAID adds an extra failure point outside of a hardware RAID controller because it's about as much risk as getting hit by a meteor and what it actually does protect from far outweighs that potential event, because a disk will always fail and it's just a time question.

[Lady Fitzgerald]

1 hour ago, leadeater said:

Thing is if you're automating your Plex setup all the tools out there have single path configuration for each content type i.e. Movies, TV etc so unless you want to split data on that at some point you'll have to step in and re-balance the data across the disks...

If that is true, you just gave me a good argument not to use Plex (which I had been considering) since my music is on one drive and my movies are on two other drives.

[Poet129]

The Cloud is a great option...

[Lady Fitzgerald]

7 minutes ago, Poet129 said:

The Cloud is a great option...

[leadeater]

46 minutes ago, Lady Fitzgerald said:

If that is true, you just gave me a good argument not to use Plex (which I had been considering) since my music is on one drive and my movies are on two other drives.

Well I guess you could configure everything to one disk then once that is full change to the next disk, fill and spill.