FreeNAS -- what do I need to know?

[Jarsky]

Everything about new I read about free as makes it looks like a poorly designed, memory inefficient, unreliable, or unsuitable for home use...

 

 

It's FreeBSD with a GUI, considering your username I wouldn't expect these type of claims.

 

The memory usage isnt inefficient - it is the way ZFS works. ZFS is a redundant filesystem with a host of features that take over the functions implemented in expensive RAID cards and SAN level hardware - and runs it in software - hence the memory requirement. There is a minimum memory requirement of around 4GB, but then as the array size grows, the memory/storage ratio becomes less.

As far as reliability, it's only as unreliable as you configure it. if you're a power user who knows how to properly configure an array and scrub files, etc...it will be perfectly reliable.

It's also most certainly designed for home use, hence the ease of use GUI, with a plugin framework for torrents, plex, etc....

[79wjd]

Transcoding doesn't take too much memory, the memory requirement is mostly for ZFS.

As for CPU,  those are all going to be much the same as they're all 2 threads - but for the small price difference, may as well get the G3258.

 

Transcoding up to 1080p should be perfectly fine with any of those CPU's with something like Plex - though if you run the mediascanner library update while playing something HD at the same time, then you will probably get some stutter. The mediascanner on Plex can be pretty aggresive. I have a 24thread server, and it can launch a handful of instances that use upwards of 30-40% for short bursts.

So what happens if I run out of memory? I saw several people saying it could lead to data corruption --is that true or just bullshit?

If the g3258 wouldn't impact performance much, then maybe I'll just get the celeron.

[Jarsky]

You wont get data corruption, but you may have issues such as performance problems and functional problems like array imports, scrubbing, deduplication, etc...

FreeNAS does reserve some memory for additional functions though.

 

For everything to run smoothly and for full support, the recommended memory requirement is 8GB - you can get by with less for a basic small nas (e.g 3 x 3TB disks in raidz-1)

[79wjd]

You wont get data corruption, but you may have issues such as performance problems and functional problems like array imports, scrubbing, deduplication, etc...

FreeNAS does reserve some memory for additional functions though.

 

For everything to run smoothly and for full support, the recommended memory requirement is 8GB - you can get by with less for a basic small nas (e.g 3 x 3TB disks in raidz-1)

So 4gb should be workable for a 4-1-1 drive array with no redundancy where 1-3 systems will be streaming from it (or say 2 streaming and 1 transcoding).

[Jarsky]

I'm not really sure how you're planning on setting it up with a 4-1-1 array. That sounds to me like concatenated jbod - so not using ZFS. I'm not sure if FreeNAS still supports this - I thought they dropped everything but ZFS? or are you planning to use all the disks individually (plain jbod)?

 

If you want to use a concatenated jbod then you're going to need something more like OMV, FlexRAID, unRAID or Windows Storage Spaces.

They all use operating systems that are compatible with the likes of Plex Server for transcoding/dlna functionality.

 

But 4GB probably wouldn't be enough for a raidz and plex - since plex will want around 1GB for full 1080p stream transcoding. Just reading the video files over the network (without encoding) is very low resource though.

[linuxfan66]

It's FreeBSD with a GUI, considering your username I wouldn't expect these type of claims.

 

The memory usage isnt inefficient - it is the way ZFS works. ZFS is a redundant filesystem with a host of features that take over the functions implemented in expensive RAID cards and SAN level hardware - and runs it in software - hence the memory requirement. There is a minimum memory requirement of around 4GB, but then as the array size grows, the memory/storage ratio becomes less.

As far as reliability, it's only as unreliable as you configure it. if you're a power user who knows how to properly configure an array and scrub files, etc...it will be perfectly reliable.

It's also most certainly designed for home use, hence the ease of use GUI, with a plugin framework for torrents, plex, etc....

Well on biased towards freebsd and i was only making claims about what other have said. here is my context. I have seen multiple people say you need a gigabyte of ram per f terabyte storage or your data will corrupt. When i read that it sounds utterly rediculous to me as design premise in any os. Maybe they were exaggerating the risk, because im not biased against freebsd, i just use it myself as i ahve never found a good reason to use it. It just i have never seen a linux/windows/mac user say low memory inherently corrupted their storage. I was not saying that statement from a position of expertise, rather a position of freebsd layman. I see your point ZFS is doing that but im not ZFS experienced so dont quite get how it handles memory.

[Eniqmatic]

Well on biased towards freebsd and i was only making claims about what other have said. here is my context. I have seen multiple people say you need a gigabyte of ram per f terabyte storage or your data will corrupt. When i read that it sounds utterly rediculous to me as design premise in any os. Maybe they were exaggerating the risk, because im not biased against freebsd, i just use it myself as i ahve never found a good reason to use it. It just i have never seen a linux/windows/mac user say low memory inherently corrupted their storage. I was not saying that statement from a position of expertise, rather a position of freebsd layman. I see your point ZFS is doing that but im not ZFS experienced so dont quite get how it handles memory.

 

8GB is the minimum, then a recommended 1GB per TB after that as you say. However you hit a point where most people feel there is no longer a point of adding more. 16GB is the normal for most people. As far as I'm aware, you won't suffer corruption from running less than recommended memory, just performance. 

 

I started explaining the whole ZFS thing, then decided that these quotes from Wikipedia explain some of the key features quite well:

 

"One major feature that distinguishes ZFS from other file systems is that ZFS is designed with a focus on data integrity."

 

 

 

it is designed to protect the user's data on disk against silent data corruption caused by bit rot,current spikes, bugs in disk firmware, phantom writes (the previous write did not make it to disk), misdirected reads/writes (the disk accesses the wrong block), DMA parity errors between the array and server memory or from the driver (since the checksum validates data inside the array), driver errors (data winds up in the wrong buffer inside the kernel), accidental overwrites (such as swapping to a live file system)

 

 

Data integrity is a high priority in ZFS because recent research shows that none of the currently widespread file systems—​such as UFS, Ext,^[8] XFS, JFS, or NTFS—​nor hardware RAID provide sufficient protection against such problems (hardware RAID has some issues with data integrity).^{[9][10][11][12]} Initial research indicates that ZFS protects data better than earlier efforts.

 

So there you have it. Designed with data integrity in mind. The reason it requires so much RAM is that it uses ECC RAM to check each block of data and correct it.

 

If you say you aren't experienced in FreeNAS and ZFS, why make these sorts of claims?

[alpenwasser]

So what happens if I run out of memory? I saw several people saying it could lead to data corruption --is that true or just bullshit?

If the g3258 wouldn't impact performance much, then maybe I'll just get the celeron.

 

 

You wont get data corruption, but you may have issues such as performance problems and functional problems like array imports, scrubbing, deduplication, etc...

FreeNAS does reserve some memory for additional functions though.

 

For everything to run smoothly and for full support, the recommended memory requirement is 8GB - you can get by with less for a basic small nas (e.g 3 x 3TB disks in raidz-1)

 

 

I'm not really sure how you're planning on setting it up with a 4-1-1 array. That sounds to me like concatenated jbod - so not using ZFS. I'm not sure if FreeNAS still supports this - I thought they dropped everything but ZFS? or are you planning to use all the disks individually (plain jbod)?

Yeah, according to their documentation, 9.3 is ZFS-only.

 

Well on biased towards freebsd and i was only making claims about what other have said. here is my context. I have seen multiple people say you need a gigabyte of ram per f terabyte storage or your data will corrupt. When i read that it sounds utterly rediculous to me as design premise in any os. Maybe they were exaggerating the risk, because im not biased against freebsd, i just use it myself as i ahve never found a good reason to use it. It just i have never seen a linux/windows/mac user say low memory inherently corrupted their storage. I was not saying that statement from a position of expertise, rather a position of freebsd layman. I see your point ZFS is doing that but im not ZFS experienced so dont quite get how it handles memory.

About the RAM thing: There is lots of half-assed advice going on around on

the internet about that. Partially, as far as I can tell, that is because ZFS

itself has changed a bit and in the olden days the situation was a bit different

(though I'm not 100% sure how accurate that really is, so take it with a grain

of salt maybe), and part is because some people just confuse different advice

for different scenarios sometimes (see dedupe below).

The FreeNAS hardware guide (from the makers of FreeNAS, so I hope they'd

know) says this on the topic:

 

How Much RAM is needed?

FreeNAS requires 8 GB of RAM for the base configuration. If you are using

plugins and/or jails, 12 GB is a better starting point. There’s a lot of advice

about how RAM hungry ZFS is, how it requires massive amounts of RAM, an oft

quoted number is 1GB RAM per TB of storage. The reality is, it’s complicated.

ZFS does require a base level of RAM to be stable, and the amount of RAM it

needs to be stable does grow with the size of the storage. 8GB of RAM will get

you through the 24TB range. Beyond that 16GB is a safer minimum, and once you

get past 100TB of storage, 32GB is recommended. However, that’s just to satisfy

the stability side of things. ZFS performance lives and dies by its caching.

There are no good guidelines for how much cache a given storage size with a

given number of simultaneous users will need. You can have a 2TB array with 3

users that needs 1GB of cache, and a 500TB array with 50 users that need 8GB

of cache. Neither of those scenarios are likely, but they are possible. The

optimal cache size for an array tends to increase with the size of the array,

but outside of that guidance, the only thing we can recommend is to measure

and observe as you go. FreeNAS includes tools in the GUI and the command line

to see cache utilization. If your cache hit ratio is below 90%, you will see

performance improvements by adding cache to the system in the form of RAM or SSD

L2ARC (dedicated read cache devices in the pool).

source: http://www.freenas.org/whats-new/2015/02/a-complete-guide-to-freenas-hardware-design-part-i-purpose-and-best-practices.html

Now, speaking from personal experience: I've never run FreeNAS, but I have

run ZFS on Linux for about two years now, and one year of that was on a machine

with 4 GB of RAM.

ZFS will gobble up lots of RAM, but it will free it if other processes need

it. What happened on my machine when ZFS got low on RAM is that I had significant

speed drops in file transfers, sometimes down to almost a standstill. However,

once I freed up some RAM again, it just continued as if nothing had happened

with zero issues. No system freeze, no crashes, and no data corruption. I should

note though that I did not run my system on a ZFS partition, things might of

course look a bit different then.

Regarding the corruption: ZFS can do deduplication of data. Dedupe requires

a crapload of RAM and indeed I have read at least one story on [H] from a poster

who lost his entire pool to corruption after enabling dedupe and running out

of memory. As a normal consumer, do not enable dedupe, unless you really seriously

know what you're doing. Most of the time you won't gain anything by that anyway.

The memory requirements for dedupe do scale much more aggressively with increasing

capacity than if you're running with out it, which is probably where the oft-quoted

"1GB of RAM per TB of storage" thing comes from. But even that is according to

Oracle's documentation on the subject not quite the proper procedure:

http://www.oracle.com/technetwork/articles/servers-storage-admin/o11-113-size-zfs-dedup-1354231.html

They also only talk about slowing down, not actual corruption.

And the FreeBSD doc mentions that in practice you might need 5 GB per 1 TB of storage

for dedupe: https://wiki.freebsd.org/ZFSTuningGuide

(Yes, I know that at this point OpenZFS is not the same zs Oracle's ZFS, but dedupe

was there before the split afaik, so I'll assume that its implementation and requirements

are similar.)

But for me, even when running practically out of RAM, I never had issues. Now,

I realise that memory management between Linux and FreeBSD is not quite the same,

however as best as I can tell, ZFS does not differ significantly between the two

platforms; there's apparently basically a wrapper for Linux which sort-of

helps ZFS cope with Linux' RAM management (the SPL -- Solaris Portability Layer).

Lastly, something to keep in mind: ZFS was not designed for home use originally.

It was designed as a solution for large-scale datacenter deployment, running on

serious hardware for serious purposes. Yes, it can absolutely do a great job

in a small setup, which is why I use it, but there are simply points where its

heritage will shine through. Memory requirements (both capacity and ECC) are

one aspect of that, the fact that you can't expand an existing parity vdev with

more drives is another (you don't really buy a few drives when you upgrade a

datacenter, you buy and upgrade in bulk, so there's no point in this feature),

as is the fact that you cannot shrink a zpool.

ZFS is like any other tool: If you use it for a job for which it is suited and

fulfill its requirements with regards to environment it can do great things,

but use it in an unsuitable scenario and it will not be a satisfying experience.

But that's true for other things as well; the best hammer is useless as a scalpel

for performing heart surgery after all.

[linuxfan66]

 

If you say you aren't experienced in FreeNAS and ZFS, why make these sorts of claims?

 

 

 

 

 

Lastly, something to keep in mind: ZFS was not designed for home use originally.

It was designed as a solution for large-scale datacenter deployment, running on

serious hardware for serious purposes. Yes, it can absolutely do a great job

in a small setup, which is why I use it, but there are simply points where its

heritage will shine through. Memory requirements (both capacity and ECC) are

one aspect of that, the fact that you can't expand an existing parity vdev with

more drives is another (you don't really buy a few drives when you upgrade a

datacenter, you buy and upgrade in bulk, so there's no point in this feature),

as is the fact that you cannot shrink a zpool.

ZFS is like any other tool: If you use it for a job for which it is suited and

fulfill its requirements with regards to environment it can do great things,

but use it in an unsuitable scenario and it will not be a satisfying experience.

But that's true for other things as well; the best hammer is useless as a scalpel

for performing heart surgery after all.

 

I hadnt seen alot of contextful advice that made freenas look good in any context, and gave some objective advice that clearly explained and dispelled myths that were way to easy to pick up about about it.  My philosophy if im wrong feel free correct me.

[alpenwasser]

I hadnt seen alot of contextful advice that made freenas look good in any context, and gave some objective advice that clearly explained and dispelled myths that were way to easy to pick up about about it.  My philosophy if im wrong feel free correct me.

From what I've heard from those who've used FreeNAS in for a while in small setups they

seem mostly pretty happy with it. ZFS has its recommended requirements and caveats,

but if you fulfill those, it's a pretty powerful tool.

It honestly wouldn't be what I'd recommend to run on old consumer hardware as I said

somewhere else in this thread, but as long as OP is aware of the caveats and can make

an informed decision about possible sources of trouble I won't scream at him not to

do it.

Something not based on ZFS would IMHO be the more suitable option for old consumer

hardware, maybe OpenMediaVault or Amahi or w/e, I've heard good things about those,

but there are other choices which would be fine I'm sure.

I haven't really come across any credible info which suggests FreeNAS is inherently

bad or unsuitable (after all, the company behind it is heavily targeting it at business

users as well, and business doesn't seem to be going to badly as they've been in it for

a while now and don't seem in immediate threat of bankruptcy) , but as said, ZFS has

its recommended requirements, such is life.

The same requirements are true if you use ZFS on any other platform though. On my

server I use Arch + ZFS, has been working pretty great. But that machine was

specifically built for that purpose (server-pull CPUs, RAM, M/B), not based on

old consumer hardware (it's also not quite as old as OP's platform, it's a dual-

socket LGA1366 server, link in signature is APOLLO if interested).

But as said, as long as OP can make an informed decision and is aware of possible

caveats that's fine in my book, even if the solution he decides to go with is in

the end maybe not what I'd personally recommend or run.

[Eniqmatic]

I hadnt seen alot of contextful advice that made freenas look good in any context, and gave some objective advice that clearly explained and dispelled myths that were way to easy to pick up about about it.  My philosophy if im wrong feel free correct me.

 

FreeNAS is hugely popular, even more so in the past year or so it seems to be everywhere. I'm sure this wouldn't be the case if it wasn't a good product. 

 

@alpenwasser has put it perfectly above so I will refer you to that:

 

 

 

ZFS is like any other tool: If you use it for a job for which it is suited and

fulfill its requirements with regards to environment it can do great things,

but use it in an unsuitable scenario and it will not be a satisfying experience.

[Eniqmatic]

@alpenwasser props for a excellent answer!

[alpenwasser]

@alpenwasser props for a excellent answer!

Glad to be of service.

[79wjd]

Alright, well hopefully I'll be able to get $350-400 from my current system and then pick up a celeron g1820 + gigabyte b85m gaming 3 board + 8gb ram + 4tb red.

[Eniqmatic]

Alright, well hopefully I'll be able to get $350-400 from my current system and then pick up a celeron g1820 + gigabyte b85m gaming 3 board + 8gb ram + 4tb red.

Whats the total cost of that?

[79wjd]

Whats the total cost of that?

About $300.

But hopefully I'll be able to get a nice amount from my current system due it being in an area-51 alx R6.

[79wjd]

@Eniqmatic 

Any idea what kind of hardware I would need (and what kind of internet speeds I would need) in order to be able to stream offsite? 

[Eniqmatic]

Stream plex? You can set various bandwidth for devices that aren't local (aka VPN or offsite). However when I'm on my VPN, I stream to it using the max bandwidth settings and looking at the firewall logs it only seems to need a couple of mbps upload.

[79wjd]

Stream plex? You can set various bandwidth for devices that aren't local (aka VPN or offsite). However when I'm on my VPN, I stream to it using the max bandwidth settings and looking at the firewall logs it only seems to need a couple of mbps upload.

Would it be through plex (could I not just stream it through vlc) -- would I be able to get away with a celeron (1 local and 1 off site stream)?

[Eniqmatic]

Hard to say, if not transcoding then probably, if you are then 1 will probably be your limit.

[79wjd]

Hard to say, if not transcoding then probably, if you are then 1 will probably be your limit.

Can I just stream to a different network without transcoding (and would that be more bandwidth intensive)? Would something like an 860k (or my current q6700) be better?

The only reason I don't want to stick with my current q6700 is because I'd likely need a raid/data controller card to utilize a/multiple 4tb drive/s. Plus, if I could sell the entire q6700 system I can cover the complete cost of the new hardware and the 4tb red without spending any money.

[79wjd]

Would this be compatible with a freenas server?

http://m.newegg.com/Product/index?itemnumber=16-124-064

I don't care about the raid functionality, I just need a controller that supports 4tb hdds, which this does.

Edit: by the looks of it, it appears that it might not be compatible as it uses a marvel controller.

[Jarsky]

Would this be compatible with a freenas server?

http://m.newegg.com/Product/index?itemnumber=16-124-064

I don't care about the raid functionality, I just need a controller that supports 4tb hdds, which this does.

Edit: by the looks of it, it appears that it might not be compatible as it uses a marvel controller.

 

That card shows it uses the Marvell 88SE9215 chipset

 

The build of FreeBSD which FreeNAS 9.2 uses, shows that it is supported in ahci.c

{0x91201b4b, 0x00, "Marvell 88SE912x", AHCI_Q_EDGEIS|AHCI_Q_NOBSYRES}, {0x91231b4b, 0x11, "Marvell 88SE912x", AHCI_Q_NOBSYRES|AHCI_Q_ALTSIG}, {0x91231b4b, 0x00, "Marvell 88SE912x", AHCI_Q_EDGEIS|AHCI_Q_SATA2|AHCI_Q_NOBSYRES}, {0x91251b4b, 0x00, "Marvell 88SE9125", AHCI_Q_NOBSYRES}, {0x91281b4b, 0x00, "Marvell 88SE9128", AHCI_Q_NOBSYRES|AHCI_Q_ALTSIG}, {0x91301b4b, 0x00, "Marvell 88SE9130", AHCI_Q_NOBSYRES|AHCI_Q_ALTSIG}, {0x91721b4b, 0x00, "Marvell 88SE9172", AHCI_Q_NOBSYRES}, {0x91821b4b, 0x00, "Marvell 88SE9182", AHCI_Q_NOBSYRES}, {0x92201b4b, 0x00, "Marvell 88SE9220", AHCI_Q_NOBSYRES|AHCI_Q_ALTSIG}, {0x92301b4b, 0x00, "Marvell 88SE9230", AHCI_Q_NOBSYRES|AHCI_Q_ALTSIG}, {0x92351b4b, 0x00, "Marvell 88SE9235", AHCI_Q_NOBSYRES},

[Godlygamer23]

1. Yes, you must put it on its own drive. The most popular option is an 8GB flash drive. Mine is on a small SSD, which is faster and will last much longer, but you can really use any drive.

For the record, my flash drive almost never gets accessed.

[79wjd]

@Jarsky Thanks 

 

@Eniqmatic 

Would you be able to help me run through the setup process? Specifically, Being prompted for a username/password when trying to modify/write anything to the server and setting up a way to access it from off the network. Also, any idea if there is a way to force the system to see if it's on, and if it's not then power on (or try to start up/restart once a day -- to make sure it comes back online automatically after a power outage or something of that nature). 

 

To access the server from a different network would I have to create a DNS server (free through optimum Boost, which I have) and use that as a static-ip equivalent?