NAS solution that supports PLEX + GPU transcoding + Bitrot Protection + Mixed Storage + 1 Drive Parity (or 2) ?

[DaveBetTech]

Is there a NAS solution that supports the following requirements:

 

1.      Redundancy: 1 drive parity. Could start off w/five 10 TB drives with 40 TB effective, then add a 40 TB drive which brings effective capacity to 50 TB (40TB is now baseline), then adding a second 40 TB drive changes effective capacity to 90TB. Over the last couple of decades, drives get bigger and it's been a super pain to add storage. I want something flexible and easy. NOTE: Yes, I know, 40 TB isn't available now or in the near future.

2.      Changing Parity: Allows for future capability to change parity from 1 to 2 drives. It's fine if the array goes offline, just as long as no data is lost.

3.      Bit Rot Protection: Need checksums and periodic scrubbing.

4.      PLEX APP SUPPORT: Plex works on this NAS (could be out of the box or just an OS dedicated to NAS)

5.      PLEX GPU SUPPORT: Would want to add a GPU and the OS and PLEX needs to support. Will stick to probably an Nvidia 3060 or 4060, or whatever is a reasonable price.

 

Windows mostly satisfies the above, if you use MS Storage Spaces w/ReFS and integrity streams, but super burned after they took away ReFS from Pro/Workstation and only allow it for Enterprise. That's too fickle and long term support is a big concern. Also, I've had MS Storage Spaces act really finicky, like requiring a reboot for drives to be removed or added, or just not explaining why a drive can't be added.

 

If I understand ZFS properly, it can't really add mixed storage sizes and use them effectively. Unraid would work, but I don't believe they handle bitrot self healing, so it's manual. Integrity streams was manual w/MS Storage Spaces (& poorly supported), so wanted to shy away from anything like that.

 

My reseach thus far shows there isn't a single-device-solution. The configuration would be a dedicated PLEX server where the metadata is local to the PLEX server, but all the media is on a Synology NAS. They would be connected via a 10 Gbe connection using iSCSI. I'd just use SHR w/1 drive parity initially & BTRFS along with periodic scrubbing setup in DSM. I'd probably also use an NVMe read-cache + pin BTRFS metadata to it. Having the PLEX metadata local + 10 Gbe iSCSI + NVMe array cache should alleviate concerns about latency, since PLEX likes it's storage local.

 

I'd prefer to use a single system for both storage and PLEX w/GPU transcoding, but not seeing any good solutions. I dislike Storage Spaces w/ReFS because of how much of a PITA it's been to use. Does anyone know about a NAS system that might be able to satisfy my requirements? I have done a lot of research, and I don't think it's possible, so this is my 'hail mary'.

[Electronics Wizardy]

If you want a prebuilt nas, synology units with a iGPU check all these boxes.

 

For a diy solution with your own os, I'd probably go unraid here personally. Yes it does support checksumming, but I'd say thats generally not a big issues as the drives them selves checksum all the data, and read checksum errors are very rare in checksummed configs.

 

You can do this with BTRFS on linux, but raid 5 and 6 aren't really stable yet. Technically Ceph on a single node would also check all those boxes for storage. 

[DaveBetTech]

@Electronics Wizardy Thank you for replying. I'm aware of the units with an integrated GPU. Unfortunately they won't meet my needs since some of the files that would need transcoding on the fly are over 20 GB, and sometimes transcoding is needed for 2-3 external clients.

 

Regarding Unraid, all of the information I found was that any kind of checking was done manually. There was no periodic scrubbing. It used an addon Dynamix file integrity plugin to do this. I would assume ZFS does this, but then ZFS doesn't have the flexibility for storage that SHR does, which is what I'm looking for.

 

I can't do RAID 4 & 6 because they are static and inflexible. You can't just YOLO and add another drive. I do like how Unraid allows for some of the files to be safe if a drive fails above the parity level, because of the individual file systems means (there was an old Windows Home Media server product that had very similar functionality & it was awesome, just very limited capability & discontinued). That's great, but the bitrot protection using Unraid (not ZFS) is manual and doesn't seem very robust from what I could dig up on it.

 

It seems like my Synology SHR-1 on BTRFS connected via 10 Gbe using iSCSI to a PLEX server hosting meta data & the transcoding GPU seems like the best option. Was just really hoping there'd be a all-in-one option that meets those five bulleted requirements.

[Electronics Wizardy]

7 minutes ago, DaveBetTech said:

Thank you for replying. I'm aware of the units with an integrated GPU. Unfortunately they won't meet my needs since some of the files that would need transcoding on the fly are over 20 GB, and sometimes transcoding is needed for 2-3 external clients.

The iGPUs should be able to handle a few streams at the same time, so I don't see this being an issue here. 

 

8 minutes ago, DaveBetTech said:

 

Regarding Unraid, all of the information I found was that any kind of checking was done manually. There was no periodic scrubbing. It used an addon Dynamix file integrity plugin to do this. I would assume ZFS does this, but then ZFS doesn't have the flexibility for storage that SHR does, which is what I'm looking for.

I wouldn't worry about checksumming for media here. Your drives, sata, and basically every other part of the system is checksummed anyways, and filesystems like ZFS very rartly show checksum errors in files when there aren't other issues. 

 

9 minutes ago, DaveBetTech said:

 

I can't do RAID 4 & 6 because they are static and inflexible. You can't just YOLO and add another drive. I do like how Unraid allows for some of the files to be safe if a drive fails above the parity level, because of the individual file systems means. That's great, but the bitrot protection is manual and doesn't seem very robust from what I could dig up on it.

Most RAID 5/6 systems Ive seen allow adding an additional drive of the same size later on. 

 

 

[DaveBetTech]

RE: The iGPUs should be able to handle a few streams at the same time, so I don't see this being an issue here. 

 

DS423+ & DS1520+ is all they have. I've investigated this before. Unless something has changed, this won't cut it for what I'm wanting to do. I have multiple >30GB x265 files in 4k 10bit HDR. I often have 2-3 people streaming simultaneously. The streams are on 4k TVs and some might be remote or on site. I want the highest quality stream too. I can do this great in Windows w/a GPU currently, but I dislike ReFS a lot.

 

A discrete GPU just seemed like the easiest setup. If I wanted addnl capability in the future, I could upgrade it. If I choose a meteor lake iGPU, it could probably mostly handle it gracefully, but it'd be a pain to upgrade.

 

---

RE: I wouldn't worry about checksumming for media here. Your drives, sata, and basically every other part of the system is checksummed anyways, and filesystems like ZFS very rartly show checksum errors in files when there aren't other issues. 

 

Regarding checksumming, I mean, I'd be spending a lot of cash specifically because I want this feature. If I don't get it, I'd just keep it setup with existing ReFS with bitrot protection. This is one of my hard requirements I was looking for.

 

---

RE: Most RAID 5/6 systems Ive seen allow adding an additional drive of the same size later on.

 

Unless I'm mistaken, traditional RAID 5/6 cannot drives added, unless you're replacing a failed one. RAID-like configurations that mimic RAID 5/6 can of course, like SHR, BeyondRaid, etc, if that's what you're referring to.

 

I stayed away from using RAID 5/6 language & used the generic term of 1 drive parity to try and not mix that up. I just want 1 drive parity, bitrot protection, self healing and periodic scrubbing, from the storage perspective.

[Jarsky]

33 minutes ago, DaveBetTech said:

Is there a NAS solution that supports the following requirements:

 

1.      Redundancy: 1 drive parity. Could start off w/five 10 TB drives with 40 TB effective, then add a 40 TB drive which brings effective capacity to 50 TB (40TB is now baseline), then adding a second 40 TB drive changes effective capacity to 90TB. Over the last couple of decades, drives get bigger and it's been a super pain to add storage. I want something flexible and easy. NOTE: Yes, I know, 40 TB isn't available now or in the near future.

Your most traditional and popular storage solutions do not work like this. 

You could maybe do this with something like Synology SHR or a Raid-on-Filesystem like SnapRAID. 

 

But the likes of your UnRAID, RAID, RockStor, StorageSpaces, MDADM, etc...some you can use mixed size drives however your parity disk needs to be your largest disk, and/or the size of your virtual disk will only use the space of your smallest disk. 

 

Example:
If you use UnRAID or Rockstor and have 5x10TB Disks in "RAID5" and add a 40TB Disk, that disk must be your parity, then you can move the old 10TB to storage increasing to 50TB usable. If you add 2 x 40TB disks though, one can be parity the other pool; which would increase it to 70TB (if you're discarding 2 drives)

If you use Linux RAID (MD) or Hardware RAID and have 5x10TB Disks in "RAID5", you must upgrade all 5 disks to 40TB before you can expand the Virtual Disk and the filesystem. 

 

33 minutes ago, DaveBetTech said:

2.      Changing Parity: Allows for future capability to change parity from 1 to 2 drives. It's fine if the array goes offline, just as long as no data is lost.

Changing Parity between 1-2 Disk Parity isnt an issue on a number of solutions out there.

Linux RAID, UnRAID, RockStor (BTRFS), Hardware RAID (LSI) allow you to do this. 

Other solutions like Storage Spaces and ZFS require you to destroy the pool and create a new pool configuration. 

 

33 minutes ago, DaveBetTech said:

3.      Bit Rot Protection: Need checksums and periodic scrubbing.

Bitrot is somewhat overhyped and really not as big an issue with more modern controllers/drives; they do all their own CRC etc...

But if you feel this is needed, then ReFS w Storage Spaces and ZFS are your main 2 options.

Both of these solutions have their drawbacks as above.

 

33 minutes ago, DaveBetTech said:

4.      PLEX APP SUPPORT: Plex works on this NAS (could be out of the box or just an OS dedicated to NAS)

5.      PLEX GPU SUPPORT: Would want to add a GPU and the OS and PLEX needs to support. Will stick to probably an Nvidia 3060 or 4060, or whatever is a reasonable price.

If you must have an Nvidia GPU then this rules out Synology (though they do have SoC's that can do hardware transcoding in the higher up tier of NAS units). 

 

 

 

Ultimately the conclusion that you came to is correct; there isnt a single solution that can do ALL of the above. 

You would be best to pick what is most important to you out of your main points and make a compromise. 

 

If the ability to expand one disk at a time is the most important to you and to add additional parity later, then I would pick UnRAID.

You can increase 1 at a time (starting with the parity), and easily run the Plex App in docker and passthrough the Nvidia GPU (UnRAID comes with the Nvidia Driver). 

 

If bitrot protection is more important, then I would pick TrueNAS Scale. 

But keep in mind you can only expand a VDEV once all disks have been replaced. And/or you must built an equal VDEV to the existing one to add it to the pool (e.g you have a 5 disk RAIDZ1; then your new VDEV must be a 5 disk RAIDZ1). 

[DaveBetTech]

@Jarsky Thanks for your comprehensive response. I only mentioned Nvidia because they and Quicksync were the initial supported transcoding offerings by PLEX, plus if it uses the hardware for transcoding on the GPU, Nvidia's quality is better. Also, I hate troubleshooting one-more-thing, since it eats up so much time, so figured I'd just go with the most compatible.

 

Yeah, I know bitrot protection is overhyped, and have looked at the white papers, but it's just one more layer of protection I'd like. There will be some non media info on my NAS that I'm a little paranoid about corruption even with redundancy. I've dealt with it before, so just making sure it's all infallible.

 

Looks like the separate PLEX server separate but connected directly to a NAS is the best way to go. I guess it'll make it easy to upgrade either, plus for support, Synology should be around for awhile.

 

Thanks again.