Freenas Setup Questions

[Mr Technician]

Where do I start? The hardware I guess. I was originally planning to build my freenas array on an old server I had acquired which had issues with the power switch causing it to power cycle repeatedly. This would never due for a server I'll rely on. I also have an older desktop PC with a quad core xeon and 12 gb of ECC memory that I was planning to rebuild in a better case and use as the server instead. My array will be relatively small (think 6-8tb usable space for my projects and my family's files) so I think this will be enough RAM. This system did give a memory error at one point but I ran memtest for over 24 hours with no errors, so I'm guessing it was an anomaly. In the case of a hardware failure I should be able to transfer the pool to a new system, correct? I'm also trying to decide what drives to purchase. Both WD Red and Seagate ironwolf pro drives look nice but the latter has the option to include data recovery with the purchase. I could get dual 6tb drives and just do a raid 1 array but I also want to switch to 10 gigabit networking in the future so a 4 x 3tb raid 10 array is also an option. I'm open to thoughts and suggestions on this. On to the software side, I quickly setup a raid 1 array with some old drives I had and simulated a hard drive failure and resilver (the question is not if, but when a drive will fail during real world operation). I poked around the interface a bit and I generally understand it (though I'm still a little unsure of the difference between a volume and pool, they almost seem to be used interchangeably) but my main question lies in ZFS snapshots and replication. I want to make backups because raid is never a viable replacement for a proper backup strategy. Is making an encrypted ZFS snapshot on a separate drive which is then moved to an offsite location a valid option for backups? I know Freenas isn't exactly simple to use so I want to be prepared.

 

[jde3]

2 hours ago, ThatFlashCat said:

Where do I start? The hardware I guess. I was originally planning to build my freenas array on an old server I had acquired which had issues with the power switch causing it to power cycle repeatedly. This would never due for a server I'll rely on. I also have an older desktop PC with a quad core xeon and 12 gb of ECC memory that I was planning to rebuild in a better case and use as the server instead. My array will be relatively small (think 6-8tb usable space for my projects and my family's files) so I think this will be enough RAM. This system did give a memory error at one point but I ran memtest for over 24 hours with no errors, so I'm guessing it was an anomaly. In the case of a hardware failure I should be able to transfer the pool to a new system, correct? I'm also trying to decide what drives to purchase. Both WD Red and Seagate ironwolf pro drives look nice but the latter has the option to include data recovery with the purchase. I could get dual 6tb drives and just do a raid 1 array but I also want to switch to 10 gigabit networking in the future so a 4 x 3tb raid 10 array is also an option. I'm open to thoughts and suggestions on this. On to the software side, I quickly setup a raid 1 array with some old drives I had and simulated a hard drive failure and resilver (the question is not if, but when a drive will fail during real world operation). I poked around the interface a bit and I generally understand it (though I'm still a little unsure of the difference between a volume and pool, they almost seem to be used interchangeably) but my main question lies in ZFS snapshots and replication. I want to make backups because raid is never a viable replacement for a proper backup strategy. Is making an encrypted ZFS snapshot on a separate drive which is then moved to an offsite location a valid option for backups? I know Freenas isn't exactly simple to use so I want to be prepared.

 

Pool is a collection of disks.. the storage pool. It's pooled storage so it's the ZFS word for group of disks or logical volume. (FreeNAS may have a different meaning for volume, its sort of a generic term, where as pool is specific)

 

A Dataset is a word that comes from IBM. It just means a collection of data with similar properties. You can think of it as a special folder that can have different snapshot or file system properties within the same pool.

 

An example of some datasets and options on them could be:

home/user - daily snapshoted + lz4 compression

home/user/archive - weekly snapshots + lz4 compression

home/user/movies - no snapshots + no compression

tank/logs - zstd compression

tank/postgres - log only metadata - 8k block size

 

So.. for different types of data you can have filesystem options that best suit it.

 

Since you mentioned Volumes, a "zvol" is a raw ZFS block device. used for swap or VM's or there are other uses for this.. iscsi is one. It kind of functions like a partition but it shares storage with the rest of the pool unlike a partition.

[Mr Technician]

Ok thanks for the explanation! I was pretty confident on datasets though I was a bit annoyed by the automatic enabling of compression. The computer I'll be using doesn't have AES-NI unfortunately but I expect compression speeds to be fast on documents and the like which don't need high performance to access in the first place. I was going for a multiple dataset approach but it didn't occur to me that I could vary the snapshot settings with them. With these snapshots is it possible to restore just a single file to an older version using the version history controls within windows? Would it also be possible to take full snapshots and place them on another drive? (the former would be be incremental for version control, the latter full backups for offsite storage). Could you also speak to the Raid 10 vs 1 issue or the resilience of freenas when hardware failure (specifically cpu, motherboard, etc.) occurs? @jde3

[jde3]

18 hours ago, ThatFlashCat said:

Ok thanks for the explanation! I was pretty confident on datasets though I was a bit annoyed by the automatic enabling of compression. The computer I'll be using doesn't have AES-NI unfortunately but I expect compression speeds to be fast on documents and the like which don't need high performance to access in the first place. I was going for a multiple dataset approach but it didn't occur to me that I could vary the snapshot settings with them. With these snapshots is it possible to restore just a single file to an older version using the version history controls within windows? Would it also be possible to take full snapshots and place them on another drive? (the former would be be incremental for version control, the latter full backups for offsite storage). Could you also speak to the Raid 10 vs 1 issue or the resilience of freenas when hardware failure (specifically cpu, motherboard, etc.) occurs? @jde3

Ok. Uhh lets see here. I'll try to hit everything.

 

Non-disk hardware failure for ZFS is trivial. Replace the broken hardware and no problem. If you plug in the disks to another OS that can read ZFS. (FreeBSD, MacOS, Linux, Illumos, etc) then you can just import the pool. (If you are using ZFS features the OS does not support it will import read only, it may ask you to upgrade the pool as well if this is done for recovery don't do that.) - There is no special fstab or raidtab file or anything.. All the config for ZFS is in ZFS's metadata so all you need are the drives themselves.

 

LZ4 compression doesn't use AES-NI. Geli does (the disk encryption software.) And only FreeBSD based OS's can import Geli volumes. - Geli is similar to Linux's LUKS there WILL be a performance impact from this. - ZFS is getting it's own native encryption but I don't believe it's in FreeBSD yet. (It may be in ZoL 8.0 on Linux now, it had been finished for a while now. Other OS's just need to port it.) When that is done the big thing it allows is for sending encrypted snapshots across the wire, but.. we don't have this quite yet.

 

LZ4 on any half modern CPU from the last 5-10 years will always be a performance boost because the bottleneck is reading/writing the data to the disk.. and compressed data is less data to read/write. This is even true for super fast storage. CPU's are just ~1000x faster than storage. The only time you don't want to use this is when you know the data will be incompressible. (such as media files, or zip archives)

 

My observations on using ZFS on Linux would tend to say to set vm.swappiness to a low number. This is just what I do, I got nothing backing this up but it seems to help.

 

If both systems are using ZFS then all you need to send snapshots from one system to another is ssh.

zfs send tank/dataset@snapshot | ssh secondhost zfs recv tank/dataset

FreeNAS should have a gui for this I'd imagine.

 

Good practice is to add the hostname for the sending system to the dataset of the receiving system so you know where it came from. So your /home from one system does not look like /home from another. You can set up auto snapshots, and you can export them via system crontab. FreeNAS will have gui's for all this stuff but it's good to know how to do it without those gui's such as if you want ZFS on your linux desktop as well.

 

Most ZFS commands honour the -n flag, that means "not really" or a dry-run and will print out what they would do. ZFS commands don't provide a lot of feedback when they are not asked to do so. (it's a feature of good Unix programming, and it expects the user to know what they want) so it's good to get into the habit of using -n.

 

ZFS snapshots only work on datasets, you can however clone a dataset. (a clone is a writable snapshot, a new-dataset based on a old one in a different location) You can also set visible snapshots.. so you can cd into a snapshot, I haven't used that feature in a while tho so I can't remember how to us it as well. - Windows sadly can not see any ZFS features. They are still using their 1990's era filesystem and cool stuff doesn't compute there.

[Mr Technician]

 First I've got to say thank you for taking the time to write this up for me! You seem to have a lot of experience with networking and servers etc. @jde3

 

That's what I was thinking, I'm not sure how long the computer I have will last so I'll likely be switching the array to a new setup at some point. I assume I could also switch the USB drive(s) to another computer and use the same freenas config instead of transferring the array alone to a new setup.

 

I'll probably end up using LZ4 on some of my datasets then. It isn't problematic to enable compression and then disable it for a dataset, correct? How is compression negotiated if a parent dataset has compression enabled and a child dataset has it disabled? I'm running a quad core xeon at 2.8ghz so that's not going to be an issue.

 

The vm.swap setting would be for virtual machines only then.

 

I'm comfortable using SSH and the terminal provided I know what I'm entering (I run my VPN server entirely from the terminal). -n should be useful though.

 

Would it be possible to make a snapshot/clone a dataset to a drive itself rather than to another system via ssh? It doesn't surprise me that windows can't read ZFS features, though is it possible to revert to older versions of single files from within the Freenas GUI? I've also read that one can export a snapshot task that *can* be read by windows and the version history can be controlled there. This is what I'd like to do if possible.

[jde3]

1 hour ago, ThatFlashCat said:

 First I've got to say thank you for taking the time to write this up for me! You seem to have a lot of experience with networking and servers etc. @jde3

 

That's what I was thinking, I'm not sure how long the computer I have will last so I'll likely be switching the array to a new setup at some point. I assume I could also switch the USB drive(s) to another computer and use the same freenas config instead of transferring the array alone to a new setup.

 

I'll probably end up using LZ4 on some of my datasets then. It isn't problematic to enable compression and then disable it for a dataset, correct? How is compression negotiated if a parent dataset has compression enabled and a child dataset has it disabled? I'm running a quad core xeon at 2.8ghz so that's not going to be an issue.

 

The vm.swap setting would be for virtual machines only then.

 

I'm comfortable using SSH and the terminal provided I know what I'm entering (I run my VPN server entirely from the terminal). -n should be useful though.

 

Would it be possible to make a snapshot/clone a dataset to a drive itself rather than to another system via ssh? It doesn't surprise me that windows can't read ZFS features, though is it possible to revert to older versions of single files from within the Freenas GUI? I've also read that one can export a snapshot task that *can* be read by windows and the version history can be controlled there. This is what I'd like to do if possible.

LZ4 is applied to files at the time they are created, if you shut it off, new files won't have it but the old ones will. (I'd recommend setting it and forgetting it) The parent and child are not affected just whatever is in the dataset. (tho parent properties will propagate to the children if unset in the child) So if the entire pool is LZ4 all the children will be unless you set it off. (all ZFS properties work this way)

 

vm.swapiness is for ZoL, ZFS native linux. It does not need to be set on guests.

 

For ZFS send. You can do that. It's a pipe-able command.. so you can get creative with it. Visible snapshots are possible. I just don't remember how to do it.

[Mr Technician]

On 7/17/2018 at 10:58 PM, jde3 said:

LZ4 is applied to files at the time they are created, if you shut it off, new files won't have it but the old ones will. (I'd recommend setting it and forgetting it) The parent and child are not affected just whatever is in the dataset. (tho parent properties will propagate to the children if unset in the child) So if the entire pool is LZ4 all the children will be unless you set it off. (all ZFS properties work this way)

 

vm.swapiness is for ZoL, ZFS native linux. It does not need to be set on guests.

 

For ZFS send. You can do that. It's a pipe-able command.. so you can get creative with it. Visible snapshots are possible. I just don't remember how to do it.

Yeah I'll decide whether to enable or not for each dataset based on its usage. I assume windows images would compress well?

 

So I need not worry about it.

 

ok I guess I'll do more research/experiment on this. @jde3

[jde3]

3 minutes ago, ThatFlashCat said:

Yeah I'll decide whether to enable or not for each dataset based on its usage. I assume windows images would compress well?

 

So I need not worry about it.

 

ok I guess I'll do more research/experiment on this. @jde3

Yeah, I'd thing so. Lots of zero space on a disk.