Getting truenas on an Old Server

[Maske_]

Hi;

I had a lot of problems with loosing data, and decided to convert an cheap old rackmount server of E-Bay into a Nas. (I also want to put my familys DVD-colection on there).the Problem is that i can not find the Bootmenue. and therefor can not use my trounas Bootstick. Is it possible that the Server does not support booting from an USB? Or can it be that the OS needs storage other than the internal Bootdrive in it? Or am i simply to dumb to find the Bootmenue in the Bios?

On an other note. Do you have any recomendations for inexpensive HDD? im a Student with a verry tight budget (i can not Pay more than 100Euros on the Prodgeckt but i would need at least 3TB of sorage)

Sorry for the dumb Questions 

[aramini]

26 minutes ago, Maske_ said:

Hi;

I had a lot of problems with loosing data, and decided to convert an cheap old rackmount server of E-Bay into a Nas. (I also want to put my familys DVD-colection on there).the Problem is that i can not find the Bootmenue. and therefor can not use my trounas Bootstick. Is it possible that the Server does not support booting from an USB? Or can it be that the OS needs storage other than the internal Bootdrive in it? Or am i simply to dumb to find the Bootmenue in the Bios?

On an other note. Do you have any recomendations for inexpensive HDD? im a Student with a verry tight budget (i can not Pay more than 100Euros on the Prodgeckt but i would need at least 3TB of sorage)

Sorry for the dumb Questions 

As to the server. What make/model is it? There should be info on the web about it. Depending on the brand it could be F2, F4, F11, Del. You'd need to look it up based on the server model. Some older systems did not support booting from USB (like circa 2000 - 2005). It might just be disabled in the BIOS by default for security since it is a server.

 

I am looking in to doing a project like this as well soon. I think if you don't want to lose data, you should think very carefully about your build and how you set up the vdevs and storage pools. There is lots of information on TrueNas forums. For example.. this is old information (FreeNAS is now TrueNAS Core) but some of the concepts are still relevant.

 

https://www.truenas.com/community/threads/slideshow-explaining-vdev-zpool-zil-and-l2arc-for-noobs.7775/

 

Also this guide for newbies:

 

https://www.truenas.com/community/resources/introduction-to-zfs.111/

 

I read both of those and they have some good info, things to avoid, etc. They recommend ECC RAM and also if you go too cheap, you might lose data anyway. I don't know the configuration you plan to do. For RAIDZ2 you can lose 2 drives before data loss but it requires at least 3 drives 1 for storage and 2 for parity. So you only get 1/3 the storage in that config. (read the resources in those links to learn about vdevs, storage pools, recommended setups, etc).

 

100 Euros seems like a very tight budget if you plan on building something reliable. A lot of those renewed servers on eBay/Amazon do not come with any storage at all, just drive trays. A lot of the forums and info I read about TrueNAS recommend Western Digital Red NAS CMR drives. But depending on how many you need and how you plan to end up with the 3TB, I think it is going to be more than 100 Euros. If you go to cheap on the hardware, you might have more headaches/problems for yourself than you are solving.

 

 

[NelizMastr]

We’ll really need to know what model of server you bought. Could you tell us that, at least?

 

€100 is not a lot of money and buys you 4TB at best with no redundancy, if it accepts your drives at all (HP servers go to 100% fan speed without drives with supported temp sensors).

[jde3]

I'm going to say something here that is probably going to be highly contested but.. ECC makes no difference to ZFS.

 

An error in memory will cause all filesystems to write a bad block, not just ZFS.

Quote

"There’s nothing special about ZFS that requires/encourages the use of ECC RAM more so than any other filesystem." -Matthew Ahrens (Cofounder of ZFS at Sun Microsystems and current ZFS developer at Delphix) -- Ars Technica

That pretty much says it.. it's just a myth people repeat over and over.. "Oh ZFS takes extreme amount of ram and ECC!!!!"
.. not true. ZFS uses ram for performance, nothing more.

 

It was propagated by enterprise deployment guides at Sun and critics.. but you don't really need to worry about ram with ZFS. (unless you are trying to hit performance benchmarks or enterprise data integrity requirements)

 

If you are on cheep flaky hardware, ZFS is the best thing you could use. It goes far further ensuring the data is accurate than anything else.

[aramini]

On 10/8/2021 at 2:36 PM, jde3 said:

I'm going to say something here that is probably going to be highly contested but.. ECC makes no difference to ZFS.

 

An error in memory will cause all filesystems to write a bad block, not just ZFS.

That pretty much says it.. it's just a myth people repeat over and over.. "Oh ZFS takes extreme amount of ram and ECC!!!!"
.. not true. ZFS uses ram for performance, nothing more.

 

It was propagated by enterprise deployment guides at Sun and critics.. but you don't really need to worry about ram with ZFS. (unless you are trying to hit performance benchmarks or enterprise data integrity requirements)

 

If you are on cheep flaky hardware, ZFS is the best thing you could use. It goes far further ensuring the data is accurate than anything else.

Theoretically ECC can protect against a 1-bit error (hence the parity bit) due to bad RAM or a "cosmic ray" or whatever. If bad (corrupted data) in RAM is written to disk, how is ZFS (or any other file system for that matter) going to know how to "fix" that? Sure it could correct it to match the originally written (albeit corrupted data due to bit flip in RAM). Garbage data, correctly written, is still garbage data.

 

That said, yeah ECC is not necessary, especially for a home NAS. But could possibly add another point of protection.

 

My main point to the OP was that there are considerations (based on what some have said on the TrueNAS forums) one must make if they want a reliable system that doesn't lose data. I was trying to set their expectations to be more realistic.

 

Trying to cheap out on hardware (expecting a robust system for 100 Euro) doesn't seem realistic to me, regardless of whether it has ECC RAM or not. You could barely buy 1x 3TB drive for 100 Euro, and that means there's no redundancy. Might as well slap that drive in your main system and utilize a good backup strategy. 3x 1TB is going to cost closer to $300 but that would only give you 1 TB storage in a RAIDZ 2 configuration. Almost pointless for what the OP wants/expects.

[jde3]

20 hours ago, aramini said:

how is ZFS (or any other file system for that matter) going to know how to "fix" that?

 

 

Because it separately calculates a checksum of it on write.. That checksum is verified on read. Can it do it in the case of a garbage write? maybe.. depends when the bit flipped and the skill of Sun's legendary software engineering teams.

[aramini]

On 10/11/2021 at 7:21 AM, jde3 said:

Because it separately calculates a checksum of it on write.. That checksum is verified on read. Can it do it in the case of a garbage write? maybe.. depends when the bit flipped and the skill of Sun's legendary software engineering teams.

ZFS separately calculates a checksum of the data on write, yes. But a checksum of erroneous data (data that was corrupted in RAM to begin with) would just ensure that the data written to disk matches the erroneous data the RAM passed to disk/ZFS. So if it was erroneous in a different way (didn't match the original, corrupted data in RAM that was written to disk) due to bit rot or whatever, ZFS would "heal" it back to the original (erroneous) data that was written?

 

It's not magic, it's logical parity and checksums based on data in/data out. That's the purpose of ECC RAM; to checksum the data in RAM.

 

https://www.onlogic.com/company/io-hub/what-is-ecc-memory-and-how-does-it-protect-your-data/

 

https://www.atlantic.net/hipaa-compliant-hosting/ecc-vs-non-ecc-memory-critical-financial-medical-businesses/

 

ZFS (file system level) would be checksuming data written to and read from disk. If it was bad in RAM to begin with, I don't see how the file system (no matter how robust) will be aware of that and know what the original data in RAM should have been. I wouldn't trust bad RAM, nor would I trust any file system that alters data that was in RAM (on write).

 

An oversimplified example here...

 

If the data in RAM is supposed to be

"10001111", but the RAM is bad/unreliable (and unbuffered non-ECC) and so "10001110" is in RAM and is written to disk, all ZFS can do is verify what was written is "10001110" which from its perspective is the correct data. But the data is erroneous because it was changed in RAM and should have been "10001111" and written to disk as "10001111".

 

Or maybe I'm missing something here, as, admittedly,  I've not studied or looked into the deeper details of how ZFS works. I do have an Enterprise Storage Systems background however, where parity, ECC, and checksums, are used in pretty much everything at the Enterprise level of hardware RAID arrays, RAID controllers, redundant (mirrored) cache, RAM, etc.

 

Granted this is a home user, so ECC is probably overkill. I'm not arguing that the more expensive ECC RAM is an absolute must in a non-mission‐critical, home server.

 

But I'm also not 100% sold that ZFS can address errors in RAM (which is what ECC RAM is for) without further research/explanation of how ZFS would know what the data in RAM should be before it is written to disk from RAM. Unless the ZFS file system can communicate directly with CPU cache or something, but I highly doubt that. Again, I'm not an expert on ZFS and am new to TrueNAS myself.

 

My goal was just to give the OP some considerations, as they said they did not want to lose data.

[aramini]

On 10/8/2021 at 2:36 PM, jde3 said:

(unless you are trying to hit performance benchmarks or enterprise data integrity requirements)

Due to the checksuming, ECC RAM is actually slightly slower than unbuffered non-ECC RAM. Albeit not necessarily perceivable by humans, but be seen in a benchmark and can affect database performance by 1-2%. Though the integrity of the data in mission-critical systems is often higher priority than performance (again we are talking 1 or 2% here).

[aramini]

Here's a  study done by the CS Department at University of Wisconsin-Madison:

http://research.cs.wisc.edu/adsl/Publications/zfs-corruption-fast10.pdf

 

And the full quote of Matthew Ahrens' post is...

 

Quote

There's nothing special about ZFS that requires/encourages the use of ECC RAM more so than any other filesystem. If you use UFS, EXT, NTFS, btrfs, etc without ECC RAM, you are just as much at risk as if you used ZFS without ECC RAM. Actually, ZFS can mitigate this risk to some degree if you enable the unsupported ZFS_DEBUG_MODIFY flag (zfs_flags=0x10). This will checksum the data while at rest in memory, and verify it before writing to disk, thus reducing the window of vulnerability from a memory error.

I would simply say: if you love your data, use ECC RAM. Additionally, use a filesystem that checksums your data, such as ZFS.

(emphasis mine)

 

So Ahrens isn't anti-ECC RAM. But lets be honest about what he is saying, which is that you are at no more risk using non-ECC RAM with ZFS than you are with any other file system. He's not saying there's no risk. His last statement is to the contrary of that.

 

I want to be clear too that the scenarios I was talking about are RAM bit flips, not disk bit flips.

 

I am not saying that you cannot use ZFS without ECC RAM or that non-ECC RAM is more dangerous with ZFS or absolutely necessary or anything like that. But if one was really concerned about the highest level of data integrity, they might want to consider using ECC RAM, if it is supported by their system and fits in their budget.

 

My guess would be that most people using TrueNAS core are probably not running mission-critical Enterprise level servers.

 

I myself am planning on using TrueNAS core with an old system I have laying around that has non-ECC RAM. As I am going to be using it as a low-traffic storage server to store my local backups (personal home stuff, not Enterprise or business data), in addition to off-site (cloud) backups.

 

[jde3]

"anti-ECC" that is a new term to me.. is that real? are people out there anti-ECC?? What does that even mean.. lol

Ok.. So my understanding of how it works from reading the ZFS design specification on it decades ago is that the checksum is calculated and written to a different filesystem block. (it's not appended it's written to the parent in the tree of blocks and that parent and so on.) Does this help with memory corruption, well it might or it might not depends when the bit in memory flips. Operationally it's doing more than one I/O's to the disk. (it has to work it's way back up the tree of blocks to the root)

 

End note / tl;dr:
ECC is good.

Use it if you can.

ECC is not necessary in any form or fashion for ZFS.

[aramini]

3 hours ago, jde3 said:

"anti-ECC" that is a new term to me.. is that real? are people out there anti-ECC?? What does that even mean.. lol

"Anti" meaning against. Nobody I know is.

 

On 10/10/2021 at 10:41 AM, aramini said:

I'm going to say something here that is probably going to be highly contested but.. ECC makes no difference to ZFS.

They serve similar but different purposes was my point.

On 10/8/2021 at 2:36 PM, jde3 said:

That pretty much says it.. it's just a myth people repeat over and over.. "Oh ZFS takes extreme amount of ram and ECC!!!!"
.. not true. ZFS uses ram for performance, nothing more.

I never said anything like that. I was talking about guidelines. If someone wants to use TrueNAS Core with 32MB of RAM go ahead and try, and good luck! (yes I'm being facetious ).

 

The general rules that I've read (in the TrueNAS docs) is at least 8GB of RAM baseline, with an additional GB per TB of storage. Granted that may not make sense for everyone, especially a home user on a budget who isn't super concerned about performance.

 

 

 

[jde3]

9 minutes ago, aramini said:

"Anti" meaning against. Nobody I know is.

 

They serve similar but different purposes was my point.

I never said anything like that. I was talking about guidelines. If someone wants to use TrueNAS Core with 32MB of RAM go ahead and try, and good luck! (yes I'm being facetious ).

 

The general rules that I've read (in the TrueNAS docs) is at least 8GB of RAM baseline, with an additional GB per TB of storage. Granted that may not make sense for everyone, especially a home user on a budget who isn't super concerned about performance.

 

 

 

TrueNAS has their own for their own software but I can tell you I've used ZFS since it's release in Solaris 10 and I've used it with as little as 128MB of ram without issue. (idk if 32M will hold the FreeBSD kernel and the ZFS module lol. but it may)

ZFS uses ram for cache and so it's for performance only. (yes, it may preform like a dog but it will work)