TrueNAS metadata vdev

[D1398342003]

Hi, I'm fairly new to TrueNAS.  I know enough to get myself in trouble, that is.  My current system isn't great on data redundancy with only one drive, but I'm going to get more drives as time goes on and expand things.  Yes, I know three is one and one is none.  Cash is tight and it's made of stuff I already have, and none of the data is mission critical or unique.

 

I rescued a 240gig SSD from an old system, and I was thinking of trying it out as a Metadata drive.  Given that I only have the one SSD, TrueNAS cautions against using it as a Metadata drive. 

 

If I do make a Metadata vdev, does TrueNAS remove all metadata records from the non-metadata vdev?  In essence making that metadata vdev the only metadata record on the NAS?  Is there some option to preserve the original vdev's metadata and use that in case of a failure?

 

I understand if there isn't, I just need some clarity on the subject.  The various videos and articles I saw were unclear on some of the specifics.

[Needfuldoer]

IIRC, if you lose the metadata VDEV, you lose the pool. So if your SSD dies, you'll experience data loss even if your spinning drives are fine. I wouldn't trust a lone SSD like that, even if it was an SLC datacenter drive rated for multiple full drive writes per day.

 

I'd set up the SSD as its own device in its own pool, and use it for the databases for media applications like Plex, Subsonic, and Jellyfin. Since that's just episode descriptions and thumbnails (the actual media lives on your array), it's just an inconvenience if the drive dies, not a catastrophe. (Keep anything you actually care about backed up to the drive array, and always remember that RAID is not a backup.)

[LIGISTX]

9 hours ago, D1398342003 said:

Hi, I'm fairly new to TrueNAS.  I know enough to get myself in trouble, that is.  My current system isn't great on data redundancy with only one drive, but I'm going to get more drives as time goes on and expand things.  Yes, I know three is one and one is none.  Cash is tight and it's made of stuff I already have, and none of the data is mission critical or unique.

 

I rescued a 240gig SSD from an old system, and I was thinking of trying it out as a Metadata drive.  Given that I only have the one SSD, TrueNAS cautions against using it as a Metadata drive. 

 

If I do make a Metadata vdev, does TrueNAS remove all metadata records from the non-metadata vdev?  In essence making that metadata vdev the only metadata record on the NAS?  Is there some option to preserve the original vdev's metadata and use that in case of a failure?

 

I understand if there isn't, I just need some clarity on the subject.  The various videos and articles I saw were unclear on some of the specifics.

You know you can’t add drives to existing vdevs, correct? Using ZFS as a “I will add more drives over time” is a very inefficient idea. Unraid may be a better option as you can slowly grow your overall pool, with redundancy. A vdev is forever locked to the amount of drives originally in said vdev, and each vdev needs its own redundancy…

 

3 hours ago, Needfuldoer said:

IIRC, if you lose the metadata VDEV, you lose the pool. So if your SSD dies, you'll experience data loss even if your spinning drives are fine. I wouldn't trust a lone SSD like that, even if it was an SLC datacenter drive rated for multiple full drive writes per day.

 

I'd set up the SSD as its own device in its own pool, and use it for the databases for media applications like Plex, Subsonic, and Jellyfin. Since that's just episode descriptions and thumbnails (the actual media lives ion your array), it's just an inconvenience if the drive dies, not a catastrophe. (Keep anything you actually care about backed up to the drive array, and always remember that RAID is not a backup.)

This is correct. Metadata will be stored solely on the metadata drive. If said metadata vdev is lost, your pool is done.

[Jarsky]

13 hours ago, D1398342003 said:

Hi, I'm fairly new to TrueNAS.  I know enough to get myself in trouble, that is.  My current system isn't great on data redundancy with only one drive, but I'm going to get more drives as time goes on and expand things. 

You cant "expand" a VDEV without destroying the data, except perhaps a Mirror. But if you want to build something more balanced like a RAIDZ1 or RAIDZ2, then you need at least 3 drives, and to build a new VDEV, add it to the pool, then move your data across and destroy the existing one.

 

13 hours ago, D1398342003 said:

 

Yes, I know three is one and one is none.  Cash is tight and it's made of stuff I already have, and none of the data is mission critical or unique.

 

I rescued a 240gig SSD from an old system, and I was thinking of trying it out as a Metadata drive.  Given that I only have the one SSD, TrueNAS cautions against using it as a Metadata drive. 

You've been watching to much LTT to be dangerous. Don't use a dedicated Metadata drive. There is absolutely no need or benefit with such a small system. Even if you had a 20 disk array, there would be very little benefit for 99% of cases. Its benefit beside faster file listing is also faster deduplication. 

 

Instead you could use that for L2ARC or SLOG, but this level of cache is quite unnecessary for most home users. 

 

You're probably best just to leave the SSD in its own separate pool. Say you have something like Plex, then chuck your AppData on it, or if you run any other Dockers or VM's use it to host those (dont forget to back it up as well). Ideally you'd want at least a mirror with SSD's as well. They can still die.  

 

13 hours ago, D1398342003 said:

If I do make a Metadata vdev, does TrueNAS remove all metadata records from the non-metadata vdev?  In essence making that metadata vdev the only metadata record on the NAS?  Is there some option to preserve the original vdev's metadata and use that in case of a failure?

 

I understand if there isn't, I just need some clarity on the subject.  The various videos and articles I saw were unclear on some of the specifics.

 

Im pretty certain that isn't a thing with special vdev's. You're setting yourself up with a SPoF (Single Point of Failure) by doing it, regardless of how you build out your storage vdev's. You really need to have redundancy in your special vdev. 

[D1398342003]

I didn't know that you can't add drives to a vdev after making it.  That's some need to know info there.  Does that mean that a metadata drive has to exist before the data vdev is made?

 

I'll look into Unraid, see what's up with it.

[Jarsky]

3 hours ago, D1398342003 said:

I didn't know that you can't add drives to a vdev after making it.  That's some need to know info there. 

I believe it can only be done with a basic (single drive) vdev using attach but yeah for any sort of redundant array or multiple vdev's it's a no. You have to create additional vdev's which you can add to the same zpool. 

 

3 hours ago, D1398342003 said:

Does that mean that a metadata drive has to exist before the data vdev is made?

No you can add a special vdev to an existing zpool. Theres an article on Level1Tech (Wendell's site) about it: https://forum.level1techs.com/t/zfs-metadata-special-device-z/159954

Removing it is the issue, or if it fails. 

 

3 hours ago, D1398342003 said:

I'll look into Unraid, see what's up with it.

I find UnRAID quite a handy solution for a lot of things and theres a large community that you can ask. It's very much community supported like Ubiquiti

[D1398342003]

My probable storage expansion (when money allows for it) would be two SSDs as a mirror for thumbnails and the like with three 8 or 16 tb nas drives with one as parity for the bulk stuff. I don't have that much on the NAS at the moment, and a three drive array with an SSD for non-video files would be entirely acceptable for my use case.

 

Thanks for the advice guys, it's been a lot of help.

[LIGISTX]

2 hours ago, D1398342003 said:

My probable storage expansion (when money allows for it) would be two SSDs as a mirror for thumbnails and the like with three 8 or 16 tb nas drives with one as parity for the bulk stuff. I don't have that much on the NAS at the moment, and a three drive array with an SSD for non-video files would be entirely acceptable for my use case.

 

Thanks for the advice guys, it's been a lot of help.

You probably don’t need SSD’s in the array, or as an array. I am not entirely sure how unraid works, but typically all containers and VM’s unless otherwise specified would be installed on the host’s boot media - that should be an SSD. As such all of the Plex metadata and database would live on the boot SSD. 

[Jarsky]

1 hour ago, LIGISTX said:

You probably don’t need SSD’s in the array, or as an array. I am not entirely sure how unraid works, but typically all containers and VM’s unless otherwise specified would be installed on the host’s boot media - that should be an SSD. As such all of the Plex metadata and database would live on the boot SSD. 

 

UnRAID boots off a USB key, it uses the UID of the key as part of the license. 

OP can setup a "Cache Pool" with mirrored SSD's for this purpose. You can use it independant of the Array and do not actually have to use it as a cache target for the main array. 

OP could also install the ZFS Plugin and use CLI to setup the SSD's as a ZFS mirror, seperate of UnRAID's array tools and GUI. 

 

IIRC the UnRAID team want to have ZFS as a feature eventually in UnRAID, but I believe its a target for UnRAID 7 if they do.

It could be quite some time until we see it as an alternative option to the FUSE system UnRAID currently uses.