Looking to Set Up NAS

[cflanders]

I am looking to set up a NAS on some old computer hardware, running in Raid 1 for redundancy. I already have the drives so that is not an issue but I need some kind of software to run it. I am running an old intel prebuilt system from Dell (back in the Windows Vista days) that has been upgraded to 4 GB of RAM. Also, I have a spare 16 GB SSD that can be used as a boot drive if needed. Note: Ports are not an issue neither is power draw (I have a spare power supply that I can add if necessary).

 

Edit: I have a Core 2 Duo

[Electronics Wizardy]

Do you want to upgrade in the future? Do you want a web interface?

 

Id probably go freenas here.

[cflanders]

8 minutes ago, Electronics Wizardy said:

Do you want to upgrade in the future? Do you want a web interface?

 

Id probably go freenas here.

FreeNas has a minimum 8GB RAM requirement. And my CPU can't handle something that taxing. 

[Electronics Wizardy]

3 minutes ago, cflanders said:

FreeNas has a minimum 8GB RAM requirement. And my CPU can't handle something that taxing. 

You can easily run it on less. I have ran it on 2gb just fine. The ram is just used as a read cache. And its really not cpu heavy. Just give it a shot.

[dalekphalm]

1 hour ago, cflanders said:

FreeNas has a minimum 8GB RAM requirement. And my CPU can't handle something that taxing. 

That 8GB requirement is a common misconception.

 

What size of HDD's are you planning on using for the RAID1?

[cflanders]

On 4/13/2018 at 4:16 PM, dalekphalm said:

That 8GB requirement is a common misconception.

 

What size of HDD's are you planning on using for the RAID1?

Looking at using 2 (1TB Segate HDDs) that I have laying around. Will probably upgrade to some 8tb drives later, when I have the budget. One thing I would like to look at later is being able to repair the array if one drive fails which is why I went with RAID 1, any other options you would recommend?

[dalekphalm]

1 hour ago, cflanders said:

Looking at using 2 (1TB Segate HDDs) that I have laying around. Will probably upgrade to some 8tb drives later, when I have the budget.

With mirror based arrays, your performance is largely going to be dictated by raw drive performance. In such a case, you don't need much RAM, since there are no parity calculations to store.

 

Other factors include CPU performance, but only under highly specialized circumstances (such as enabling Encryption, compression, or Deduplication, etc). What other tasks the NAS is doing will also determine CPU performance (Eg: is it just a NAS, or is it also a Plex server, or hosting web pages, or hosting VM's, etc).

 

In terms of this:

Quote

One thing I would like to look at later is being able to repair the array if one drive fails which is why I went with RAID 1, any other options you would recommend?

Can you clarify what you mean? A RAID1 Mirror (Or a Mirrored vdev, in FreeNAS) automatically protects against single drive failure, since you have two drives and the same data is copied to both. In terms of "repairing" the array, you just pull the dead drive, pop in a replacement, and let it "resliver" (rebuild) the array - this is largely the same regardless of RAID type - with the main difference being how it rebuilds the array, and what performance hit/how long it will take.

 

A RAID1 mirror will rebuild the fastest, because it's just doing a straight data dump from one drive to the other.

 

RAID1 is definitely the best way to go in terms of drive resilience, when you only have a 2-drive array. Any other suggests necessitate larger numbers of drives (eg: RAID5/RAIDZ1 has a minimum drive requirement of 3 - 2 data drives and one parity drive [nerd note: there's not actually a dedicated parity drive, since all data, including parity data, is striped across all drives, but there's an equivalent of one drive lost to parity data]. RAID6/RAIDZ2 has a minimum drive requirement of 4. So does RAID10).

 

Unfortunately, it's not easy to change from a 2-drive array to a larger one. ZFS does not support expanding arrays (though this is expected to change in specific scenarios in late 2018 with a ZFS update). Even in Hardware RAID - which some RAID Cards do support Array expansion, you cannot change RAID types. If you have a RAID5, you can make it a bigger RAID5.

 

When it comes to expanding the array down the line (especially if changing from one RAID type to another), the best solution in the vast majority of cases, is to simply backup the data elsewhere, destroy the array, and build the new larger array the way you want.

 

One last thing:

RAID does not equal backup. This is something some users (typically new to NAS's, RAID, and storage systems, etc) forget or simply don't understand.

 

They figure that since their array can survive if a single HDD crashes, that their data is safe and does not require a backup. This is dangerous thinking and gives you a false sense of security.

 

A RAID array will not protect against accidentally deleting your wedding photos. It will not protect against a virus infecting or destroying data. It will not protect against ransomware encrypting your entire NAS.

 

Make sure that you backup anything you can't afford to lose to some other system - whether that's a folder in your Google Drive/OneDrive/DropBox, whether it's a cloud backup provider such as BlackBlaze (I would say CrashCloud but they discontinued consumer services), whether it's an automated backup to an external HDD, etc.

 

Any backup system is better then no backup system. Though if you are curious, the industry standard is the "3-2-1" system.

 

The 3-2-1 system specifics a minimum of:

3 copies of relevant data

2 different mediums (A medium is the tech storing it - eg: 2 copies on HDD's, 3rd copy on an SSD)

1 offsite copy

 

This allows your data to survive pretty much anything except apocalypse level crisis events. Some definitely consider this overkill for consumers - especially on a budget, so most people can usually get by with less intense solutions.

[cflanders]

On April 13, 2018 at 2:49 PM, Electronics Wizardy said:

You can easily run it on less. I have ran it on 2gb just fine. The ram is just used as a read cache. And its really not cpu heavy. Just give it a shot.

Just of of curiosity, what version are you running? The latest version (and several versions back) say that they require 8GB of Ram with 1 additional GB of RAM per TB of storage being recommended. 

[Electronics Wizardy]

48 minutes ago, cflanders said:

Just of of curiosity, what version are you running? The latest version (and several versions back) say that they require 8GB of Ram with 1 additional GB of RAM per TB of storage being recommended. 

I have ran the latest version on 2gb of ram.

 

It don't need the ram. It doesn't do a check for 8gb+ and not boot if you don't have that much. The kernel and basic system needs like 100-200mb, the web gui needs about anount 200-500mb and the rest is used mainly as a read cache for zfs, and more is better, but you can run with very little.

 

Just try it with the hardware, it will work.

[cflanders]

3 minutes ago, Electronics Wizardy said:

I have ran the latest version on 2gb of ram.

 

It don't need the ram. It doesn't do a check for 8gb+ and not boot if you don't have that much. The kernel and basic system needs like 100-200mb, the web gui needs about anount 200-500mb and the rest is used mainly as a read cache for zfs, and more is better, but you can run with very little.

 

Just try it with the hardware, it will work.

Thanks for the clarification on if FreeNas checks. One more question since I have never done it before, how would you have a boot drive separate from the rest of your drives but still let FreeNas detect all the drives, or is it something that FreeNAS does automatically? Also, does FreeNas give you the option to have it set up the RAID or would you have to set up the raid yourself?

[dalekphalm]

1 minute ago, cflanders said:

Thanks for the clarification on if FreeNas checks. One more question since I have never done it before, how would you have a boot drive separate from the rest of your drives but still let FreeNas detect all the drives, or is it something that FreeNAS does automatically? Also, does FreeNas give you the option to have it set up the RAID or would you have to set up the raid yourself?

I would like to further add that the "1GB of RAM per 1TB addition of data" is a very misleading metric for FreeNAS.

 

People use this as a general rule of thumb, but that's not the case at all. The 1GB of RAM per 1TB of Data is only required if using Deduplication. Most people don't bother - especially in a home environment, but if you do intend on using Dedupe, then you will need to look at your RAM needs.

 

If not? Don't worry about it. 8GB of RAM is nice to have for FreeNAS, but if you can only afford 2GB or 4GB right now? You'll be fine. Add some more RAM later in the future when you can.

[Electronics Wizardy]

1 minute ago, cflanders said:

Thanks for the clarification on if FreeNas checks. One more question since I have never done it before, how would you have a boot drive separate from the rest of your drives but still let FreeNas detect all the drives, or is it something that FreeNAS does automatically? Also, does FreeNas give you the option to have it set up the RAID or would you have to set up the raid yourself?

Normally with freenas you install it on a usb drive. If uptime is a major issue, you can mirror 2 usb sticks. If the boot drives failes, you can just make a new one and import your array and your data is safe.

 

You setup the raid in freenas in the web interface, in the same place that you setup the shares and other settings.

 

You can tell what drive the boot drive is, and it won't let you add it into your main zfs array, normally the usb drive is like 16gb, and your storage drives are multiple terabytes.

[Electronics Wizardy]

Just now, dalekphalm said:

I would like to further add that the "1GB of RAM per 1TB addition of data" is a very misleading metric for FreeNAS.

 

People use this as a general rule of thumb, but that's not the case at all. The 1GB of RAM per 1TB of Data is only required if using Deduplication. Most people don't bother - especially in a home environment, but if you do intend on using Dedupe, then you will need to look at your RAM needs.

 

If not? Don't worry about it. 8GB of RAM is nice to have for FreeNAS, but if you can only afford 2GB or 4GB right now? You'll be fine. Add some more RAM later in the future when you can.

one thing to note, Id stay away from dedup even if you have the ram as it loves to get slow. So unless you are getting a lot of space savings, just leave dedup off.

[cflanders]

2 minutes ago, Electronics Wizardy said:

Normally with freenas you install it on a usb drive. If uptime is a major issue, you can mirror 2 usb sticks. If the boot drives failes, you can just make a new one and import your array and your data is safe.

 

You setup the raid in freenas in the web interface, in the same place that you setup the shares and other settings.

 

You can tell what drive the boot drive is, and it won't let you add it into your main zfs array, normally the usb drive is like 16gb, and your storage drives are multiple terabytes.

Would a 16GB SSD work instead? I would like to save my flash drives since I have the SSD laying around and the flash drives I have are all in use. 

[dalekphalm]

1 minute ago, Electronics Wizardy said:

one thing to note, Id stay away from dedup even if you have the ram as it loves to get slow. So unless you are getting a lot of space savings, just leave dedup off.

Yeah I agree - consumer grade drives are cheap, all things considered. For typical home usage, where most of the data is going to be media like pictures, music, and video - there's not likely to be a lot of duplication.

[dalekphalm]

7 minutes ago, cflanders said:

Would a 16GB SSD work instead? I would like to save my flash drives since I have the SSD laying around and the flash drives I have are all in use. 

Yes you can use any Media type to install FreeNAS, as long as it's bootable. The only reason peoiple suggest USB flash drive is because FreeNAS is so light and doesn't require much, so USB flash drives are cheap.

 

So if you have an SSD lying around? Go for it.

[jde3]

Deficiency in ram for ZFS only causes a performance hit. You can run under the recommended if needed without too much worry.. its just not optimal.

 

Do NOT use dedup. Never turn this on, on your pool. There are other ways to do this that are much more efficient than the way ZFS does it. (currently, it's being worked on)

[dalekphalm]

4 hours ago, jde3 said:

Deficiency in ram for ZFS only causes a performance hit. You can run under the recommended if needed without too much worry.. its just not optimal.

 

Do NOT use dedup. Never turn this on, on your pool. There are other ways to do this that are much more efficient than the way ZFS does it. (currently, it's being worked on)

The only place I've personally seen that uses Dedupe is at work on our backup NAS's. Even the main SAN doesn't use it, as far as I recall. On a 14TB datashare on the backup NAS, we're saving ~2TB of space. And this is being done via the backup software, not via ZFS. Not terrible savings mind you, but for home use, needing to accommodate an extra 2TB of data isn't very expensive.

 

I actually have never even bothered to use ZFS's dedupe, so I'll take your word on that.

[jde3]

If you need dedup look at what Datto has to provide, they have solved the issue a different way. https://www.datto.com/nas

[leadeater]

On 4/19/2018 at 6:35 PM, dalekphalm said:

I actually have never even bothered to use ZFS's dedupe, so I'll take your word on that.

It's a giant resource hog, never used it either. Netapp dedup works really well though and WAFL is rather similar to ZFS so I have to wonder where ZFS is falling down in that regard.

 

Have to be really careful with any inline deduplication though, if it's not being used as a backup target. There's almost always a trade off be it system resources, performance or write buffer data resiliency.

 

Netapp gets around some of the pitfalls by leveraging snapshots/monitoring changed blocks so the data footprint needing deduplication is much smaller. Also Netapp snapshots are RoW not CoW like ZFS so don't incur 2x write penalty, ZFS doesn't have SnapVault either, at least that I'm aware of so only SnapMirror (Netapp speak for both).

 

Not being a ZFS person myself I don't know all the exact capabilities of it.

[leadeater]

On 4/21/2018 at 5:26 PM, jde3 said:

If you need dedup look at what Datto has to provide, they have solved the issue a different way. https://www.datto.com/nas

It says they use ZFS dedup, what have they done differently? Just looks like a hardware ZFS appliance with some nice management interfaces over top of it with some customized software functionality like cloud sync. Not exactly dug in to it so maybe I'm missing something?

[jde3]

1 minute ago, leadeater said:

It says they use ZFS dedup, what have they done differently? Just looks like a hardware ZFS appliance with some nice management interfaces over top of it with some customized software functionality like cloud sync. Not exactly dug in to it so maybe I'm missing something?

No they use a snapshot clone method. I believe the software kinda hides this on the backend but it's a clone (writable snapshot) not a dedup hashtable like normal ZFS. Mainly I think they use it for VM hosting where much of the base image would be the same. I don't know much more than that however you'd have to ask them how it was implemented. For whatever reason also it was not pushed upstream or accepted upstream. (probably because it's done in the front end and has little to do with ZFS itself.)