ECC vs. Non-ECC memory in FreeNAS

[79wjd]

So from my (very limited) understanding, FreeNAS caches data you're using from storage into memory and then saves whatever's in memory back to storage -- and that's where ECC memory comes in -- it prevents corrupted data from being written back to storage. 

 

So I was wondering, is there a way to disable memory to storage write back? I'm wondering because I'd rather not spend $300 to upgrade my NAS to ECC memory when it's mainly just a plex server. Since I never actually write to my NAS (except when I'm adding more content), ECC memory shouldn't be required (if my understanding of it is correct and IFF it's possible to prevent ZFS from writing what's in memory back to storage). 

[Altecice]

If you are using ZFS you need to be using ECC else you may as well not be using ZFS. Its really as simple as that.

/thread

[79wjd]

If you are using ZFS you need to be using ECC else you may as well not be using ZFS. Its really as simple as that.

/thread

Well, I'm also using FreeNAS because I like the OS itself, not necessarily because it's ZFS. And I don't really think it's worthwhile to spend $300 just to ensure that a movie or tv show every once in a while doesn't get corrupted. So if there would be a way to turn my storage into read only, then that corruption shouldn't be possible?

[KemoKa]

RAM gets a load of errors because it's RAM - Every so often, a cosmic ray is gonna fly out of nowhere and zap off an electron from one of the memory gates. Over time, this causes bitrot.

This is what ECC memory is made to prevent. Normally it doesn't matter to consumers because it doesn't happen often enough, and the memory empties when the power is cut. But in the case of ZFS, where it's constantly checking the drives for errors, you need error correction.

[Altecice]

Well, I'm also using FreeNAS because I like the OS itself, not necessarily because it's ZFS. And I don't really think it's worthwhile to spend $300 just to ensure that a movie or tv show every once in a while doesn't get corrupted. So if there would be a way to turn my storage into read only, then that corruption shouldn't be possible?

 

A better option for you would probably swap Hypervisor to something like ESXI and use a Windows Server 2012 R2 (you can get the keys very cheap)

[79wjd]

RAM gets a load of errors because it's RAM - Every so often, a cosmic ray is gonna fly out of nowhere and zap off an electron from one of the memory gates. Over time, this causes bitrot.

This is what ECC memory is made to prevent. Normally it doesn't matter to consumers because it doesn't happen often enough, and the memory empties when the power is cut. But in the case of ZFS, where it's constantly checking the drives for errors, you need error correction.

A better option for you would probably swap Hypervisor to something like ESXI and use a Windows Server 2012 R2 (you can get the keys very cheap)

Well, what's the likelihood (a guess) of a single movie getting corrupted and is there a possibility that the entire drive gets corrupted? 

 

Like I said, there's nothing super critical on it, so I have a hard to rationalizing spending $300.

[KemoKa]

Well, what's the likelihood (a guess) of a single movie getting corrupted and is there a possibility that the entire drive gets corrupted? 

 

Like I said, there's nothing super critical on it, so I have a hard to rationalizing spending $300.

Not saying you have to or that you should - it's just that normally the idea is that if you're crazy enough to use FreeNAS, then the assumption is that you want to protect your data at all costs. Not that it has to be used that way, but if used with that in mind, having ECC memory is an absolute must.

A movie would probably not be subject to any significant bitrot that you would notice...

[79wjd]

Not saying you have to or that you should - it's just that normally the idea is that if you're crazy enough to use FreeNAS, then the assumption is that you want to protect your data at all costs. Not that it has to be used that way, but if used with that in mind, having ECC memory is an absolute must.

A movie would probably not be subject to any significant bitrot that you would notice...

Yeah, I understand why ECC is as critical as it is, I'm not denying that. Don't get me wrong, I'd love to do a proper rebuild of my server -- as it is I'm running three 4tb WD Reds in JBOD and two random 1tb drives in RAID1 off a Pentium G3250, an H81 board, and a CX500m in a Source 210. I'd like nothing more than to pick up an R4/R5, 5 8tb Seagate Archival drives in RaidZ2, a nice new supermicro X10 motherboard and a Xeon 1220. I just don't feel like spending $1800~. 

 

And why wouldn't a movie be subject to bitrot (what about a picture?)? Is it because bitrot would affect a pixel here and there and thus in a movie you'll never notice those random corrupted pixels? 

[KemoKa]

And why wouldn't a movie be subject to bitrot (what about a picture?)? Is it because bitrot would affect a pixel here and there and thus in a movie you'll never notice those random corrupted pixels? 

pretty much exactly that. Probably not even that bad.

[79wjd]

pretty much exactly that. Probably not even that bad.

Alright, well, I probably won't be upgrading to ECC then for now...as much as I might like to.

[alpenwasser]

Okay, it's been a while since I looked into the details of ZFS, but off the top of my head, this comes to mind:

ZFS checksums each block (unless you disable checksumming). When you read data, the blocks are verified against their checksums as they're being read, and if an error is encountered, ZFS will try to fix it on the fly from parity.

On top of that, you can manually scrub the pool, meaning the entire pool is read and verified against its checksums. Same here, when an error is encountered, ZFS will try to fix the block from parity, if available.

If during the checksum verification you have an error in memory (either the checksum or the data), you will get a mismatch between checksum and data despite the data on the disk still being fine. ZFS will then try to "fix" this. How precisely this works I haven't looked into to be honest, but the consensus is generally that in such a case, ZFS can end up corrupting your data instead of fixing it.

As long as we're just talking about files of comparatively small importance, this might not be much of an issue. But, as far as I recall, if this corruption occurs in the file system's metadata, you can have your entire pool smoked. And unlike other file systems where you have tools like fsck to fix broken file systems, ZFS doesn't really have anything like that available (there is some magic which can be worked, but it's highly complex and not really intended for the end user afaik).

In the end, this is a personal decision. I run ZFS on a machine without ECC RAM just like a normal filesystem, but I do have ECC on my server. Now, I've read that some people advise not to scrub your pool without ECC, or even disable checksums. Both of these have their upsides and downsides. Upside: Decreases the risk of corrupting data due to memory error. Downside: As long as you don't have a memory error, scrubbing and checksumming would still be pretty damn neat and do their job. Problem is, you just can't know if/when you'll have a RAM error. Note that "Disabling checksums is NOT a recommended practice.", to quote the ZFS manual.

[braneopbru]

Yeah, I understand why ECC is as critical as it is, I'm not denying that. Don't get me wrong, I'd love to do a proper rebuild of my server -- as it is I'm running three 4tb WD Reds in JBOD and two random 1tb drives in RAID1 off a Pentium G3250, an H81 board, and a CX500m in a Source 210. I'd like nothing more than to pick up an R4/R5, 5 8tb Seagate Archival drives in RaidZ2, a nice new supermicro X10 motherboard and a Xeon 1220. I just don't feel like spending $1800~. 

 

And why wouldn't a movie be subject to bitrot (what about a picture?)? Is it because bitrot would affect a pixel here and there and thus in a movie you'll never notice those random corrupted pixels? 

No need to spend $1800. I just bought two Supermicro X9-SCM boards from two different people. One came with a Xeon 1220 and 16Gigs ECC for $250, the other came with a Pentium G630t and 8gigs ECC for $130.

 

eBay and forum marketplaces are your friends.

[79wjd]

No need to spend $1800. I just bought two Supermicro X9-SCM boards from two different people. One came with a Xeon 1220 and 16Gigs ECC for $250, the other came with a Pentium G630t and 8gigs ECC for $130.

 

eBay and forum marketplaces are your friends.

I'm not a fan of used parts. The board + 16gb of RAM would cost me $260 and the CPU I have is good enough for what I'm doing. $1800 was for a full rebuild including a nicer case, better cpu, 5 8tb seagate drives, and a new psu. Either way, it's money I don't really want to be spending. 

[braneopbru]

I'm not a fan of used parts. The board + 16gb of RAM would cost me $260 and the CPU I have is good enough for what I'm doing. $1800 was for a full rebuild including a nicer case, better cpu, 5 8tb seagate drives, and a new psu. Either way, it's money I don't really want to be spending. 

I would say that if you are not storing any important data on the NAS, don't worry about it. In the absolute worst case scenario that your entire pool is corrupted, the worst that could happen is you having to re download all your movies.

[79wjd]

I would say that if you are not storing any important data on the NAS, don't worry about it. In the absolute worst case scenario that your entire pool is corrupted, the worst that could happen is you having to re download all your movies.

Which is still a nauseating thought.

 

$260 to pick up 16gb ECC memory and a compatible board - the (hopefully) $50-$60 I get from selling my H81 board and 8gb of RAM. $200 to upgrade to a proper ECC setup....I won't lie, it is tempting. 

[alpenwasser]

I'm not a fan of used parts. The board + 16gb of RAM would cost me $260 and the CPU I have is good enough for what I'm doing. $1800 was for a full rebuild including a nicer case, better cpu, 5 8tb seagate drives, and a new psu. Either way, it's money I don't really want to be spending.

Buying server pull parts from a reputable source should be fine (aside from them being not the newest shizzle anymore). I've made pretty good experiences with server pulls (CPU, M/B, RAM). Then again, if you're only $200 away from a proper ECC setup with new parts, might also be worth a thought.

In the end though, it's your data and your money, we can't really make this decision for you.

[79wjd]

Buying server pull parts from a reputable source should be fine (aside from them being not the newest shizzle anymore). I've made pretty good experiences with server pulls (CPU, M/B, RAM). Then again, if you're only $200 away from a proper ECC setup with new parts, might also be worth a thought.

In the end though, it's your data and your money, we can't really make this decision for you.

What would be a reliable source? I mean, it is still $200 for a Pentium based ECC setup where I already have the Pentium -- i'd definitely consider selling the pentium as then I could sell a whole system (and probably by able to charge a slight premium). Part of the problem is I also don't really know what to look for and/or how to find good used parts. 

 

 

I could probably get around $50-$60 for the motherboard and ram, but I could also probably sell it as a whole system: G3250, H81 board, 8gb Ram, 250gb HDD, Cooler Master N200 or Centiruon 532, some random PSU that came with the 532, DVD reader, DVD burner, with Linux, Windows XP, or possibly windows 7 (I'd have to use a school license) installed and get close to $250....which is probably a better idea in the end. And then either buy a Skylake Pentium setup for $350 (or a Skylake i3 for $410?) OR go with a Haswell Pentium for $325 or a Haswell i3 for $370 OR buy used...