Using swap space with ZFS?

SMURG · August 28, 2017

So a few years ago I built a file server/NAS based on a SoC board from Supermicro (http://www.supermicro.com/products/motherboard/celeron/x10/x10sba.cfm) using ZFS under Ubuntu, and now want to add another 4 x 3TB of storage to it. The board is pretty perfect for this sort of application, the CPU is passively cooled and has a TDP of 10W yet does everything I need it to (including plex transcoding) and it has 6 native SATA ports plus the 2 I'm drawing from mini-PCIe and an mSATA on top for 9 total of which I'm currently only using 5 (OS and 4 x 1TB in Z1). The only snag with this board is that it only supports 8GB of RAM, which isn't really going to be enough for the storage I'm going to have after I add more drives (the rule of thumb ofc being 1GB per TB of storage with ZFS). My question therefore is will ZFS use swap space to alleviate RAM issues? I can easily install an SSD for this purpose and then just run the OS using the internal USB type A (I've been thinking of moving to unRAID anyway), but obviously if it's not going to help ZFS then there's no point.

Either way I'm probably going to take the 4 x 1TB array out of ZFS and just run them separately after I've moved their 3TB contents to the new 9TB volume, but at some point I'll want to replace them with bigger drives and I won't be able to upgrade the RAM any further. 8GB for 12/9TB of storage isn't going to cause much of a problem but I'm conscious that it might if I later dump another 12/9TB on top and I don't want to replace the board if I don't have to when it's perfectly suitable aside from RAM limitations.

I've been using ZFS for years now on multiple machines so I'm pretty familiar with it at this point, I just don't know how or if it makes use of the swap space and whether I'd benefit from adding more in lieu of RAM. Hopefully somebody else on here has more experience with this than I do, any help is appreciated

Mikensan · August 28, 2017

The RAM amount is generally a recommendation and your box will continue to operate just fine more than likely. Especially if you're just on a gigabit network. For write cache you can add a ZIL/LOG disk which helps when dealing with sync writes (NFS especially). The times in which RAM because a serious issue is when you want to run VMs or use deduplication.

And yes ZFS will use swap but off the boot device IIRC which means there's very little wiggle room in most people's cases. It's FreeBSD so you'd have to look up changing the mount point for the swap partition, but FreeNAS does not like being modified outside of the GUI.

At first I thought the 8gb limit was something you read in the manual for your motherboard and assumed a bios update might work, but after looking up the CPU it is really limited to 8GB :(.

I'd just try it and see what happens. If it turns out you really need more RAM, maybe grab a new board/chip and use this old setup as an "offsite" backup.

SMURG · August 28, 2017

1 minute ago, Mikensan said:

ZFS will use swap but off the boot device IIRC which means there's very little wiggle room in most people's cases. It's FreeBSD so you'd have to look up changing the mount point for the swap partition, but FreeNAS does not like being modified outside of the GUI.

Ah okay, well that's easy enough to sort out because I don't need much space for the OS itself. The swap could be pretty big in that case and I wouldn't need to change the mount point. Doing some further reading in the ZFS documentation it talks about using L2ARC read cache as well so I was wondering if that might be a better idea?

Like you say, I've never really had too much issue with RAM before when using ZFS but yeah the CPU really is limited to 8GB, and as a side-note it's really fussy about what RAM it will take as well lol. I have managed to make systems lock up before when not using enough but you have to overdo it quite a lot. As you say I'm pretty sure I'd be fine with 8GB now but it's just a thought for the future. Thanks for the info though, if I'm sticking with Ubuntu I'll probably just increase the swap size on the boot device since I have loads spare anyway. If I go to unRAID I'll probably just use the mSATA as L2ARC since the documentation seems to suggest that helps.

Mikensan · August 28, 2017

26 minutes ago, SMURG said:

Ah okay, well that's easy enough to sort out because I don't need much space for the OS itself. The swap could be pretty big in that case and I wouldn't need to change the mount point. Doing some further reading in the ZFS documentation it talks about using L2ARC read cache as well so I was wondering if that might be a better idea?

Like you say, I've never really had too much issue with RAM before when using ZFS but yeah the CPU really is limited to 8GB, and as a side-note it's really fussy about what RAM it will take as well lol. I have managed to make systems lock up before when not using enough but you have to overdo it quite a lot. As you say I'm pretty sure I'd be fine with 8GB now but it's just a thought for the future. Thanks for the info though, if I'm sticking with Ubuntu I'll probably just increase the swap size on the boot device since I have loads spare anyway. If I go to unRAID I'll probably just use the mSATA as L2ARC since the documentation seems to suggest that helps.

L2ARC won't help very much, it's good in an environment where certain files that are accessed repeatedly gets stored there and get pushed over a large pipe. Imagine you have a couple virtual hosts connected to the NAS box via 10gb, L2ARC would help a lot. Write cache is about the only helpful thing for most home-use scenarios.

It's funny because I read ubuntu, I understood it, and as soon as I started typing... FreeNAS came to mind. Sorry about that lol.