Jump to content

Non ECC with ZFS

adaq
2 minutes ago, techmattr said:

Read their own FAQ. Their FAQ is littered with disconnect issues that would 100% cause data corruption. http://www.sybausa.com/faq.php?type=1 

This is normal expansion card boiler plate FAQ stuff and it's not indicative of any hardware issues.

 

You seem to be making stuff up.

Link to comment
Share on other sites

Link to post
Share on other sites

2 minutes ago, djdwosk97 said:

Wait wait wait wait wait.....explain to me how an unsupported NIC is "definitely putting your data at risk". 

 

And, no, they aren't the same price. There is about a $100-$150 difference to step up to ECC/server-grade hardware. You gain other features as well (IPMI, Intel NIC, potentially more memory slots, more sata ports, lower power draw, better quality, etc...), but it's definitely not the same price. 

So using hardware directly related to transferring your data on a system that doesn't have correct drivers for that hardware isn't a risk? Since when?

 

There are plenty of systems on eBay for $150 or less. Here's the first result I got: http://www.ebay.com/itm/SuperMicro-X8SIA-F-Server-Motherboard-I-O-Shield-32G-Memory-X3450-CPU-Fan-/262740915896?hash=item3d2c9436b8:g:WeMAAOSwOyJX6Sey

Link to comment
Share on other sites

Link to post
Share on other sites

2 minutes ago, AshleyAshes said:

This is normal expansion card boiler plate FAQ stuff and it's not indicative of any hardware issues.

 

You seem to be making stuff up.

Okay.

Link to comment
Share on other sites

Link to post
Share on other sites

1 minute ago, techmattr said:

So using hardware directly related to transferring your data on a system that doesn't have correct drivers for that hardware isn't a risk? Since when?

 

There are plenty of systems on eBay for $150 or less. Here's the first result I got: http://www.ebay.com/itm/SuperMicro-X8SIA-F-Server-Motherboard-I-O-Shield-32G-Memory-X3450-CPU-Fan-/262740915896?hash=item3d2c9436b8:g:WeMAAOSwOyJX6Sey

You're acting like the NIC is going to blow up. Worst case scenario it has poor transfer speeds. And let's pretend that it just stops working.....your data will still exist in perfect condition either on the server or on the computer -- wherever the source files are/were located. 

 

Not everyone likes to buy used -- most people don't. It's not a fair comparison to compare a new system to a used one. 

PSU Tier List | CoC

Gaming Build | FreeNAS Server

Spoiler

i5-4690k || Seidon 240m || GTX780 ACX || MSI Z97s SLI Plus || 8GB 2400mhz || 250GB 840 Evo || 1TB WD Blue || H440 (Black/Blue) || Windows 10 Pro || Dell P2414H & BenQ XL2411Z || Ducky Shine Mini || Logitech G502 Proteus Core

Spoiler

FreeNAS 9.3 - Stable || Xeon E3 1230v2 || Supermicro X9SCM-F || 32GB Crucial ECC DDR3 || 3x4TB WD Red (JBOD) || SYBA SI-PEX40064 sata controller || Corsair CX500m || NZXT Source 210.

Link to comment
Share on other sites

Link to post
Share on other sites

5 minutes ago, djdwosk97 said:

You're acting like the NIC is going to blow up. Worst case scenario it has poor transfer speeds. And let's pretend that it just stops working.....your data will still exist in perfect condition either on the server or on the computer -- wherever the source files are/were located. 

 

Not everyone likes to buy used -- most people don't. It's not a fair comparison to compare a new system to a used one. 

Most people who build their own servers buy used. Who says it isn't fair? You? Okay. I don't care if you don't think it's fair. It's correct.

 

I've only been doing DIY servers for 25+ years, working in Enterprise data centers for 15+ years and helping people on various bulletin boards, mailing lists, blogs and forums for 20 years.... but who am I to argue. :dry:

Link to comment
Share on other sites

Link to post
Share on other sites

6 minutes ago, techmattr said:

Most people who build their own servers buy used. Who says it isn't fair? You? Okay. I don't care if you don't think it's fair. It's correct.

 

I've only been doing DIY servers for 25+ years, working in Enterprise data centers for 15+ years and helping people on various bulletin boards, mailing lists, blogs and forums for 20 years.... but who am I to argue. :dry:

Considering the nonsense you're spewing, I can't imagine how drunk someone would have to be to hire you. But congratulations on finding that someone. 

PSU Tier List | CoC

Gaming Build | FreeNAS Server

Spoiler

i5-4690k || Seidon 240m || GTX780 ACX || MSI Z97s SLI Plus || 8GB 2400mhz || 250GB 840 Evo || 1TB WD Blue || H440 (Black/Blue) || Windows 10 Pro || Dell P2414H & BenQ XL2411Z || Ducky Shine Mini || Logitech G502 Proteus Core

Spoiler

FreeNAS 9.3 - Stable || Xeon E3 1230v2 || Supermicro X9SCM-F || 32GB Crucial ECC DDR3 || 3x4TB WD Red (JBOD) || SYBA SI-PEX40064 sata controller || Corsair CX500m || NZXT Source 210.

Link to comment
Share on other sites

Link to post
Share on other sites

1 minute ago, djdwosk97 said:

Considering the nonsense you're spewing I can't imagine how drunk someone would have to be to hire you. 

Yeah. Doing things correctly is some nonsense. 

Link to comment
Share on other sites

Link to post
Share on other sites

1 hour ago, techmattr said:

It's not rare. It's actually incredibly common. Just that most people wouldn't know. Especially if you have RAM with a bad IC, damaged in shipping or installation.... or a host of other different issues. If it boots to the OS and is mainly stable then you'd never know it was producing erroneous writes until you go to open the file it corrupted. ECC RAM isn't that much more than non-ECC RAM and in many cases it's the same price. There is no point in not having it. If your data is important then it is absolutely a must have. 

Obviously it isn't important for machine that's just used for gaming or something like that but we're in the "Servers and NAS" forum.

RAM with a bad IC? So you're talking about running defective RAM.

 

I don't know about you, but I tend to test the stuff I run w/ Memtest, which would pick up on a module with a damaged IC, or other issues.

 

Yes we're in the "Servers and NAS" forum, but we're talking about a home build, not a production business server. In a business environment, you should use ECC, regardless of how small the risk is, because your business depends on that data being valid, and also it can help with uptime. But in a personal/home environment, running ZFS does not require ECC RAM. It's nice to have, certainly, but not necessary by any means.

For Sale: Meraki Bundle

 

iPhone Xr 128 GB Product Red - HP Spectre x360 13" (i5 - 8 GB RAM - 256 GB SSD) - HP ZBook 15v G5 15" (i7-8850H - 16 GB RAM - 512 GB SSD - NVIDIA Quadro P600)

 

Link to comment
Share on other sites

Link to post
Share on other sites

21 minutes ago, dalekphalm said:

RAM with a bad IC? So you're talking about running defective RAM.

 

I don't know about you, but I tend to test the stuff I run w/ Memtest, which would pick up on a module with a damaged IC, or other issues.

 

Yes we're in the "Servers and NAS" forum, but we're talking about a home build, not a production business server. In a business environment, you should use ECC, regardless of how small the risk is, because your business depends on that data being valid, and also it can help with uptime. But in a personal/home environment, running ZFS does not require ECC RAM. It's nice to have, certainly, but not necessary by any means.

RAM can have all sorts of issues that don't show up in any memory test. Ask anyone who has ever run memory tests against an HP or Dell server that just threw a bunch of un-correctable memory errors. Almost every time those tests come back clean. They don't show anything. At best they give you a little piece of mind when testing new memory that was just delivered.

 

As far as a home build... I care about my personal data. As I said, it's implied that if someone is taking the steps to build their own NAS that they care about their data as well. It doesn't make any sense whatsoever to bypass the most important check when writing that data. If you don't care enough about your data to skip a check to see if the write was valid then why store it at all? Or why not just forego the home build and use odrive?

Link to comment
Share on other sites

Link to post
Share on other sites

19 minutes ago, techmattr said:

RAM can have all sorts of issues that don't show up in any memory test. Ask anyone who has ever run memory tests against an HP or Dell server that just threw a bunch of un-correctable memory errors. Almost every time those tests come back clean. They don't show anything. At best they give you a little piece of mind when testing new memory that was just delivered.

 

As far as a home build... I care about my personal data. As I said, it's implied that if someone is taking the steps to build their own NAS that they care about their data as well. It doesn't make any sense whatsoever to bypass the most important check when writing that data. If you don't care enough about your data to skip a check to see if the write was valid then why store it at all? Or why not just forego the home build and use odrive?

Do you have any sources to back up your claim that they happen all the time?

 

There doesn't seem to be a lot of solid info one way or the other, but I found info on a Google Study that indicated that a single-bit error occurred on average every 25,000 to 70,000 FIT (Failures per 1 billion device hours):

http://www.intelligentmemory.com/support/faq/ecc-dram/how-often-do-ecc-correctable-single-bit-errors-occur-and-how-about-double-multi-bit-errors.php

 

So what that means is that for 1 billion hours of operation (This would likely be per DIMM module), there were (on the high end) 70,000 single bit errors. That means that per DIMM, there were 14,285.7 hours between individual single bit failures. That's over a year and a half before a single bit error. I'm also making the assumption, worst case, that this is per DIMM, and not per machine (Since it is per device hours, and the device could be the entire computer/server, or it could be the DIMM).

 

Now, for the ENTIRE Google Data Center, that worked out to a single error every 14 and 40 hours. But they would have been using multiple thousands (or tens of thousands) of DIMM modules.

 

The study also goes on to say that for multiple bit errors to occur at once, would happen on average every 48,000 years.

 

So, certainly, as we've been saying all along, there is some risk. But it's pretty small. Environmental conditions, and stress/load, also have an impact in how often these happen.

 

If you have any other sources that can show different results, I would be more than happy to read them.

For Sale: Meraki Bundle

 

iPhone Xr 128 GB Product Red - HP Spectre x360 13" (i5 - 8 GB RAM - 256 GB SSD) - HP ZBook 15v G5 15" (i7-8850H - 16 GB RAM - 512 GB SSD - NVIDIA Quadro P600)

 

Link to comment
Share on other sites

Link to post
Share on other sites

30 minutes ago, techmattr said:

As I said, it's implied that if someone is taking the steps to build their own NAS that they care about their data as well.

I'd say the majority building a home NAS are trying to store a growing hoard of pirated media, actually.

Link to comment
Share on other sites

Link to post
Share on other sites

3 minutes ago, AshleyAshes said:

I'd say the majority building a home NAS are trying to store a growing hoard of pirated media, actually.

Presumably they still care about the data being stored. However, a system based on a consumer platform is very unlikely to die and take all your data with it. 

PSU Tier List | CoC

Gaming Build | FreeNAS Server

Spoiler

i5-4690k || Seidon 240m || GTX780 ACX || MSI Z97s SLI Plus || 8GB 2400mhz || 250GB 840 Evo || 1TB WD Blue || H440 (Black/Blue) || Windows 10 Pro || Dell P2414H & BenQ XL2411Z || Ducky Shine Mini || Logitech G502 Proteus Core

Spoiler

FreeNAS 9.3 - Stable || Xeon E3 1230v2 || Supermicro X9SCM-F || 32GB Crucial ECC DDR3 || 3x4TB WD Red (JBOD) || SYBA SI-PEX40064 sata controller || Corsair CX500m || NZXT Source 210.

Link to comment
Share on other sites

Link to post
Share on other sites

1 minute ago, djdwosk97 said:

Presumably they still care about the data being stored. However, a system based on a consumer platform is very unlikely to die and take all your data with it. 

And if it does die, the data isn't that valuable.  This is why my NAS uses a snapshot system.  It'd bi inconvenient if I somehow lost everything but it wouldn't be like losing the novel I wrote over 5 years or family photos going back 10 generations.

Link to comment
Share on other sites

Link to post
Share on other sites

Just to double up - a business cares about downtime and thus invests more money into their equipment. More money doesn't promise you won't lose data - a backup plan does. Overspending on a NAS does nothing to protect your data, only increase uptime. A corrupted file just means now you have to call IT to restore the file, now you are eating IT resources and have an employee twiddling their thumbs.. A corrupted file should never mean "oh shit it's gone forever."

 

ECC or not, server-grade or home-grade, none of it matters without a backup. So you (tech) can preach to "buy enterprise hardware and if they care they'd spend extra money on the NAS" - but honestly I'd tell the user to split the difference and go with a consumer build + backup if they care about their data. Backup can be something as simple as an external hard drive or cloud. Use multiple backups so if data is corrupted then at least you'll have other sources of the files. Backup > highend NAS. ECC + Server boards and ECC are not a guarantee.

 

 

 

 

Link to comment
Share on other sites

Link to post
Share on other sites

1 hour ago, Mikensan said:

Just to double up - a business cares about downtime and thus invests more money into their equipment. More money doesn't promise you won't lose data - a backup plan does. Overspending on a NAS does nothing to protect your data, only increase uptime. A corrupted file just means now you have to call IT to restore the file, now you are eating IT resources and have an employee twiddling their thumbs.. A corrupted file should never mean "oh shit it's gone forever."

 

ECC or not, server-grade or home-grade, none of it matters without a backup. So you (tech) can preach to "buy enterprise hardware and if they care they'd spend extra money on the NAS" - but honestly I'd tell the user to split the difference and go with a consumer build + backup if they care about their data. Backup can be something as simple as an external hard drive or cloud. Use multiple backups so if data is corrupted then at least you'll have other sources of the files. Backup > highend NAS. ECC + Server boards and ECC are not a guarantee.

 

 

 

 

What if they backup the corrupted file? :o 

Link to comment
Share on other sites

Link to post
Share on other sites

1 minute ago, alex75871 said:

What if they backup the corrupted file? :o 

According to my research, up above, that's an incredibly unlikely thing to happen, assuming all hardware is in good working order.

 

But yes, if you backup a corrupted file, then the Hash/Checksum will be useless.

For Sale: Meraki Bundle

 

iPhone Xr 128 GB Product Red - HP Spectre x360 13" (i5 - 8 GB RAM - 256 GB SSD) - HP ZBook 15v G5 15" (i7-8850H - 16 GB RAM - 512 GB SSD - NVIDIA Quadro P600)

 

Link to comment
Share on other sites

Link to post
Share on other sites

11 minutes ago, alex75871 said:

What if they backup the corrupted file? :o 

Then you restore from a previous backup set. I keep monthlies for a year.

And yearlys indefinitely.

Link to comment
Share on other sites

Link to post
Share on other sites

6 minutes ago, Mikensan said:

Then you restore from a previous backup set. I keep monthlies for a year.

And yearlys indefinitely.

I think what he's implying is that the original file was corrupted upon first write. A possible occurrence from a Single-Bit RAM error (Something ECC would correct), but very VERY incredibly unlikely to happen.

For Sale: Meraki Bundle

 

iPhone Xr 128 GB Product Red - HP Spectre x360 13" (i5 - 8 GB RAM - 256 GB SSD) - HP ZBook 15v G5 15" (i7-8850H - 16 GB RAM - 512 GB SSD - NVIDIA Quadro P600)

 

Link to comment
Share on other sites

Link to post
Share on other sites

I think a lot of this is psychological.  There's 'comfort' in thinking you protected yourself against 'extreme scenarios'.  As Mikensan says, double up, do backups, but few discussions about a Home NAS involve also having a backup server.

 

The three largest threats are usually ignored:

 

1) Electrical failure that results in every drive in the box dying at once.

2) User error.  Let's admit it, the user is always the weakest link.  "Yeah, I just deleted that junk folder."  *six months later*  "Shit, I think that important thing was in that Junk Folder for some reason..."

3) Software Error.  As a example, when removing a series from SickRage can delete a series when I remove it from it's tracking.  Normally I'm just removing the series from SickRage while keeping the files on the drive.  However, since the program is ready and willing to delete most of the media files it's tracking, a mistake or malicious action in one commit could leave the program deleting every media file without ever telling the user.

Link to comment
Share on other sites

Link to post
Share on other sites

19 hours ago, dalekphalm said:

I think what he's implying is that the original file was corrupted upon first write. A possible occurrence from a Single-Bit RAM error (Something ECC would correct), but very VERY incredibly unlikely to happen.

Ah I see, that would certainly pose a problem.

 

Thinking further.. I suppose you could extend the same theory to the workstation that's uploading the file, as it would not have ECC memory in 9/10 scenarios and a flipped bit is just as bad in windows. I guess if one is truly paranoid, ECC for the whole house! :-D

Link to comment
Share on other sites

Link to post
Share on other sites

2 hours ago, Mikensan said:

Ah I see, that would certainly pose a problem.

 

Thinking further.. I suppose you could extend the same theory to the workstation that's uploading the file, as it would not have ECC memory in 9/10 scenarios and a flipped bit is just as bad in windows. I guess if one is truly paranoid, ECC for the whole house! :-D

Yes in a network storage device (A NAS, or a File Server w/ a shared drive), if the workstation writing the file has a single-bit memory error, nothing can prevent that aside from said workstation having ECC.

 

But again, it's pretty unlikely. So you have to take the risk, the level of said risk, and balance it vs the cost of avoiding said risk. You need to do a risk/benefit assessment.

For Sale: Meraki Bundle

 

iPhone Xr 128 GB Product Red - HP Spectre x360 13" (i5 - 8 GB RAM - 256 GB SSD) - HP ZBook 15v G5 15" (i7-8850H - 16 GB RAM - 512 GB SSD - NVIDIA Quadro P600)

 

Link to comment
Share on other sites

Link to post
Share on other sites

is it really necessary to have ECC ram?with free nas 

 

who has run into issues?

Link to comment
Share on other sites

Link to post
Share on other sites

1 minute ago, Unrealcpu said:

is it really necessary to have ECC ram?with free nas 

 

who has run into issues?

I would suggest going back and reading the entire thread. Your question should be more or less already answered.

 

Short answer: Necessary? No. Recommended? Yes.

 

It also largely depends on what kind of files you're using on the NAS, and how important those files are.

For Sale: Meraki Bundle

 

iPhone Xr 128 GB Product Red - HP Spectre x360 13" (i5 - 8 GB RAM - 256 GB SSD) - HP ZBook 15v G5 15" (i7-8850H - 16 GB RAM - 512 GB SSD - NVIDIA Quadro P600)

 

Link to comment
Share on other sites

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×