Local file storage

Traukito · October 1, 2019

Hey everyone, this is my first post in this community.

I'm a computer entusiast, and have a little bit of knowledge wich have been real useful for my current occupation. I'm a mountainer in Chilean patagonia, and live in a place called Chiloé Island.

Recently me and some friends have embarked on a project wich involves teaching kids about mountain disciplines. We are generating a large amount of media files, mostly videos from GoPro's and photos from cellphones and cameras. We have set up a small office wich has internet access, but with a limited monthly quota. With this limitation we can't use cloud storage solutions to archive our data, so I thought maybe a solution will be to mount a server in our office so we can make a local backup system of our files, since we only need to access them from within our installations and rarely from outside.

I was thinking on setting up an old machine lying around (or buying cheap hardware) and installing FreeNas to achieve this. Since we need it only as a storage for 3 people to access the files, and probably never at the same time I think it will be enough... but I would like to know your opinion about this, or maybe a different solution. Sadly money is an issue, we are a small organization and can't afford to spend large amounts of money in this.

Any help or guidance will be welcome... thanks everyone!

Windows7ge · October 1, 2019

Well you have a few options:

You can install FreeNAS on an old machine. If this data is important though I would want to validate the reliability of the old hardware.

You could look into a pre-built NAS from companies like Synology or QNAP.

You could buy a used server off places like Ebay. This would offer better reliability than any old computer. Where you live though may be a problem.

If you go the old computer route I'd want to make sure of at least two things. That the PSU is good/quality and that you buy proper drives for the use case.

ZFS is great for protecting your data but it's not bulletproof many other things can go wrong. Hence you want to be careful in your choice of hardware.

Traukito · October 1, 2019

32 minutes ago, Windows7ge said:

If you go the old computer route I'd want to make sure of at least two things. That the PSU is good/quality and that you buy proper drives for the use case.

Thanks for your reply!

I think in my case, for the place I live the old computer option maybe the best. I'm going to see if there's anything available in my country to buy an old server... anyway I was thinking on buying new hard drives for this setup so that's covered. The new PSU is a good idea too.

Thanks!

alpha754293 · October 6, 2019

I would advise against ZFS unless you have tape backup or some other way to back up your data.

I have emails from my work with Sun Microsystems (the people that created ZFS) and also there's a video of Wendall from Level1Techs -- both of whom suggest that ZFS NEEDS to have a good backup solution for it to be really effective, because if and should it fail, it can fail in spectacular ways in which there are ZERO methods to restore and/or recover the data.

Here is the email from Sun Microsystems support:

" I called and talked to the escalation engineer for a while. This is what I learned:

We aren't going to be able to put the array back online. The zfs pool is corrupted. The only way to get the array back would be if you could remember which disk goes in which spot, and then we might be able to bring it back.
The escalation engineer repeated your scenario in the lab and it worked fine for him-but he used drives that support device IDs.
We can file an Request for Future Enhancement if you want asking that zfs be changed to deal with drives that don't have device IDs; however, zfs was designed around device IDs, this is unlikely to be implemented anytime soon, and this won't help your current problem.
At this point, the solution is to restore from backup"

This email comes directly from their kernel group/devops.

If the data is important and critical to you, I would recommend that the drives and any hardware based RAID host bus adapter be new. The host system might be a couple of years old, but you don't want to risk either having a hard drive failure (which can cause problems for you with software based RAID solutions) or the RAID controller failing (which Linus has done a couple of videos on where they have had a hardware RAID controller fail on them and they ended up having to swap drives onto a different controller which, again, without a backup, if you toast something, then it will permanently corrupt the RAID array/pool and NONE of the data would be recoverable anymore.)

So...just...be careful in regards to what you do and it will depend on how important and how valuable is the data that you are trying to store.

If it is important and valuable to you, I would spend the extra money to make sure that you have a solid hardware based solution that can tolerate/withstand multiple failures. But if the data isn't super critical/important to you, then you might be able to get away with less fault protection and accept a greater level of risk.

Thanks.

bowrilla · October 6, 2019

FreeNas or more precisely ZFS is not that easy on the hardware. It takes some resources for it to properly work. You'll need at least 8GB of memory for example. I would seriously avoid hardware raid controllers with FreeNAS. Any from of storage needs a proper backup solution, no matter which route you go. A RAID (or RAID-Z) is NEVER a replacement for a backup! Ever! So you'd basically need two systems copying the data or you might want to look into tape drivers for archiving purposes.

You can probably build a proper NAS system to run FreeNAS with around 12-16TB of net storage for around 800-900$ with all new hardware. Tape drivers are pretty expensive so expect something in the range of 2k $. A Synology NAS (or an equivalent device from a competitor) including drives will cost around 800-900$ as well for the same amount of net storrage. The cheaper the NAS the lower the performance.

Do not cheap out on the drives. Cheaper consumer drives are not made to run 24/7 and will fail a lot sooner than storage drives meant to be used in NAS.