Best redundancy setup for new Plex server...

[Gerr]

I am building a new all in one server for my home to take care of multiple functions.  It MUST run Windows 2016 Server Essentials, so need help figuring out what the best redundancy options I have for all the hard drives.

 

Functions:

Plex

NVR(Blue Iris)

DVD/Blu-Ray Ripping & Transcoding(Handbrake)

Two NAS, one for me and one for the wife.

Client backups

 

Hardware:

Xeon E3-1231v3

32GB DDR3-1600 ECC RAM

Asus P9D-WS C226 ATX mobo

GT 710 video card(no iGPU on CPU)

450W Seasonic Gold PSU

LSI 9210-8i HBA card (allows 8 additional SATA ports)

Blu-Ray optical drive

Define R4 case

Two 2-slot external USB/eSATA HDD enclosures.

 

Drives and my initial thoughts on redundancy setup...

3x3TB HGST NAS - Plex media using software RAID-5

1x4TB HGST NAS - Client & OS backup drive, no redundancy needed.

4x3TB WD Red - Two in software RAID-1 for my NAS with the other two in one of the external enclosures in RAID-0 as a backup for NAS.

2x3TB WD Green - in the other external enclosures in RAID-0 as a backup for Plex media.

1x3TB WD Purple - single drive for NVR storage, no redundancy needed.

2x1TB WD Red - Wife's NAS setup in software RAID-1.

2x500GB WD Re4 - Mirrored OS drive using motherboard RAID-1.

1x250GB MX200 SSD - single SSD for Plex client & transcoding, no redundancy needed.

 

How would you setup redundancy if you had this equipment?  All drives are already owned.  My other option would be to buy a hardware RAID card like an LSI 9260, but nothing were is mission critical, just a home server.

 

Ideas/thoughts?

[ARikozuM]

RAID 5 is my go-to when I'm doing anything for my servers, but the surveillance uses RAID 1. 

[Oshino Shinobu]

RAID 0 for backup drives is a terrible, terrible idea. If you lose one of the drives, the backup is gone. If something goes wrong with the RAID 0 array, the backup is gone. If the enclosure dies, the backup is gone.

 

Do you already have these drives or are you buying them all for this? If you're buying them, go for as large a drive as you can for the backups. Something like Seagate's archive drives will work fine. If you're not, just don't put them in any RAID. If you have to combine them somehow, use JBOD. RAID 0 shouldn't be used in any area of this, it's not a good idea. 

 

I wouldn't advise using RAID 5 with only 3 drives, it's not great. I'd stick to 4 drives minimum. 

 

Do you actually need redundancy for this? While it may be a pain to restore things, I would focus on having a very robust backup solution rather than redundancy. I assume it's not exactly critical if you have some downtime on your Plex server in the event a drive fails. I'd say the NVR is the one place you really want redundancy, yet it's one of the only places without it. It's important to understand the difference between redundancy and protecting data. Redundancy doesn't protect data, it protects from down time, so you should implement it in places where you can't afford to have the system to go down. If the Plex server goes down, you can't watch media until you restore it. If the NVR's storage goes down, it can no longer record, so if something happens during that time, you won't have any footage, so you're kind of buggered. 

[ArtificialBlood]

A raid 5 would be sweet but don't you need 4 disk not 3? If I needed redundancy for a plex server I would focus on making the most of the disk space for my budget. I would use raid 1 and make sure the read performance for the drive is at least good enough for what I need. 

[Oshino Shinobu]

1 minute ago, ArtificialBlood said:

A raid 5 would be sweet but don't you need 4 disk not 3? If I needed redundancy for a plex server I would focus on making the most of the disk space for my budget. I would use raid 1 and make sure the read performance for the drive is at least good enough for what I need. 

3 is the minimum requirement for RAID 5, but it's generally advised not to implement it below 4 drives in order to get the most out of it. 

[brwainer]

You’re running Windows Server 2016 - why not throw all the drives (other than the OS boot) into Storage Spaces, and set your redundancy / lack-thereof at the virtual drive level? If you do go this route, then I recommend against using Simple (raid-0 equivalent) volumes as you do have a higher risk of losing the virtual disk than if you just used a single drive directly, just like Raid 0. And if you want to use Parity volumes, then I recommend setting a 5 or 10GB write-back cache, which will take up some space on the SSD.

[gonzap50]

Why do you have to run Windows Server? You could just run FreeNAS and setup different permissions for you and your wife. You can run your Plex server using the FreeNAS plugin too.

[Gerr]

I need Windows for a couple reasons.  First is that the best NVR program is Blue Iris and it's Windows only.  No other similar product on Linux or FreeNAS even comes close.  Second is that I will be using this server to learn Active Directory and I don't want to run multiple servers, my electric bill is high enough.  Yes I would prefer ZFS, but I just can't without giving up both those options.  Also, I don't know Window's Storage Spaces and ReFS.  I would love to use them to create something similar to a RaidZ1 volume, not sure if I can.  Any tips there would be helpful.

 

Remember that this is a HOME server, so you can't use redundancy & backup logic that you would use in a business.  I have to use RAID or something similar to create larger volumes as I mainly have 3TB drives.  I don't want a ton of mirrored 3TB volumes, sometimes I need 6TB or more.  For example, how am I supposed to backup the 3TB NAS to a 3TB backup, especially if I want revision history, ie multiple backups so I can recover a file several versions old if I wanted.  Not about to buy 6TB drives, I want to use what I have.  Also, what's wrong with RAID-0 as a backup?  if it fails, all I lose are the backups and I can replace a failed drive within 24 hours and make an immediate backup.  Considering that I also use redundancy, to lose my data, I would need both the backup and redundancy to fail, and the likely hood of that is extremely low.  This is why I want both redundancy and backups, but because I use both, I won't need high levels of either.  Remember that I already own all these drives, so I am stuck using them, so to create larger volumes, I have to us RAID. 

 

I can buy a couple things, but not a lot.  If I had to buy a full hardware RAID card or another 3 TB drive, I can, but don't want to dump a lot into this.  I could switch the RAID-5 to RAID-10 by just buying a 4th HDD and that would be cheaper than buying a hardware RAD card.  I just would prefer to use the hardware I have if possible and if I had to buy something, keep it minimal.

 

Basically these are the questions I want answered...

 

1.  I like ZFS, but can't use it here, so is there any solution, either within Windows Server 2016 or a separate installed product, that will allow me to do something similar to ZFS's RaidZ1?  Multiple volumes of this would be ideal.

 

2.  When I do have to use RAID-1, which solution would be best, motherboard RAID, Windows software RAID, or LSI software RAID through the HBA?  I would prefer if I didn't have to buy a hardware RAID card if possible.

 

3.  Reconsidering what a HOME SERVER needs, are my spots for backup and redundancy solid choices or should I change?

[brwainer]

4 hours ago, Gerr said:

1.  I like ZFS, but can't use it here, so is there any solution, either within Windows Server 2016 or a separate installed product, that will allow me to do something similar to ZFS's RaidZ1?  Multiple volumes of this would be ideal.

 

2.  When I do have to use RAID-1, which solution would be best, motherboard RAID, Windows software RAID, or LSI software RAID through the HBA?  I would prefer if I didn't have to buy a hardware RAID card if possible.

 

3.  Reconsidering what a HOME SERVER needs, are my spots for backup and redundancy solid choices or should I change?

1. Storage Spaces. Use Parity volumes for anything where you want redundancy but don't need a lot of write speed (Plex's media files, for example), Mirror for anything where write speed is important (NAS), and Simple for anything that can be lost without concern (backups). Use ReFS for everything unless there is a conflict - for example, Plex uses certain NTFS features that aren't available on ReFS for its metadata - so while you can store the videos/audio/etc on a ReFS volume, the actual Plex library directory has to be on an NTFS drive. Otherwise, you'll get things like missing thumbnails and other artwork, missing video titles, etc.

2. Storage Spaces. See above.

3. See above. You can use the builtin "Windows Server Backup" (may have to enable it via Add Roles and Features, on the Features page), or you may look into other backup tools. Anything that works on regular windows will work here too.

 

Storage Spaces operates very differently than ZFS. With ZFS, and basically all other types of RAID, you set the level of redundancy when you add drives into the pool/array. With Storage Spaces, you dump all your disks into a single Pool, and then whenyou make a Virtual Disk you select the redundancy. Virtual Disks are split up into 256MB slabs.

• If you choose Simple, then the slabs will just be spread out across your disks, like RAID0.
• If you choose Mirror, then each slab will be written to at least 2 disks (or even 3 if you set it up for 3 copies), and these will also be spread out across your disks, so its sort-of like RAID10 (you can control this behavior at creation time by manually defining the number of "copies" (RAID1) and "columns" (RAID0) that will be used to make the Mirror space (RAID10)).
• If you choose Parity, the 256MB slab will be distributed across 3+ disks using standard parity calculations, consuming the same amount of space as it would if it were RAID5 (you can also make this into RAID6 by setting the redundancy to a higher number than the default)

You can make as many virtual disks as you want, and you can choose for each of then whether to use Thick provisioning (all space is pre-allocated) or Thin provisioning (256MB slabs are allocated as they are needed, until you run out of space or the size you defined for the virtual disk is reached).

 

EDIT: A huge benefit of Storage Spaces is that, given sufficient capacity, you can usually repair from a drive failure without even having to replace the failed drive - the slab structure means that the system just has to rebuild enough slabs on appropriate disks in order to achieve the required redundancy again. The one "gotcha" with this involves Mirror disks - let's say you start with 10 drives in your pool at the time the Mirror disk is created. By default, the wizard will create as many columns as it can - so in this case, your new disk will use 2 copies and 5 columns, using all 10 disks. This is great for speed, but when one disks dies, based on the requirement that was set out for this disk, it won't be able to properly rebuild until you have 10 healthy drives again. My understanding is that it will first repair the copies required, but since two of the columns are now on the same disk, it will keep showing you that the disk is Degraded/Unhealthy until it can fully repair across 10 disks. For this reason, I like to manually define my number of columns, which as of the last time I checked can only be done by making the virtual disk via Powershell. I can give Powershell advice and commands if you want.

[Gerr]

Problem is that I don't have 10 of the same drives.  Instead, I bought the drives for specific uses.  The 7200rpm HGST drives were to be my fast Plex drives, the WD Red's to be data drives, the WD Green drives for backups, and the WD Purple drive for the NVR.  If I throw all of them into a single pool, I lose the hardware specific features they bring to the table.

[brwainer]

28 minutes ago, Gerr said:

Problem is that I don't have 10 of the same drives.  Instead, I bought the drives for specific uses.  The 7200rpm HGST drives were to be my fast Plex drives, the WD Red's to be data drives, the WD Green drives for backups, and the WD Purple drive for the NVR.  If I throw all of them into a single pool, I lose the hardware specific features they bring to the table.

That is the beauty of storage spaces - they don’t all have to be the same type, acapcity, etc. it will even let you throw in USB-attached drives, although that is not recommended for anything but consumer / DAS use.

 

EDIT: I do see the special-ness of the Red and Purple drives, the purple especially could be used on its own. The Red drives are actually even more suited to use for Storage Spaces than the Greens/etc.

 

EDIT2: even if you don’t end up putting all the drives in a single pool, there is no limit to the pools that you can create. I would suggest using storage spaces for everything that is not just a single drive operating on its own. Even with just 2 drives in a pool, Storage Spaces and ReFS adds a lot of safety against bitrot and other corruption, like ZFS does, which traditional hardware and software RAID don’t give you

[Gerr]

So I could setup Storage Spaces with multiple pools like this...?

 

Pool #1 = Parity over 3 7200rpm 3TB drives for Plex media pool.

Pool #2 = Mirror over 2 WD 3TB Red drives for my NAS.

Pool #3 = Mirror over 2 WD Red 1TB drives for my wife's NAS.

Pool #4 = Mirror over 2 WD Purple 3TB drives for NVR(would have to buy a 2nd WD Purple).

 

Can I format my single drives that are not OS drives ReFS or would NTFS be better for that?

[brwainer]

1 hour ago, Gerr said:

So I could setup Storage Spaces with multiple pools like this...?

 

Pool #1 = Parity over 3 7200rpm 3TB drives for Plex media pool.

Pool #2 = Mirror over 2 WD 3TB Red drives for my NAS.

Pool #3 = Mirror over 2 WD Red 1TB drives for my wife's NAS.

Pool #4 = Mirror over 2 WD Purple 3TB drives for NVR(would have to buy a 2nd WD Purple).

 

Can I format my single drives that are not OS drives ReFS or would NTFS be better for that?

Yes that would work, you could also put all 4 Red drives into a single pool and seperate the NAS sizes via the virtual disk sizes, or put them onto the same large volume and use folder quotas if you’re really concerned about reserving more space for yourself. The benefit of this is that until you’re actually using more than 1.5TB between the two of you, any of those 4 disks could fail and you’d be able to rebuild and be safe - and if it was a 1TB that failed and you put in a 3TB or larger, you would then have additional space available for the both of you. At the end of the day, you’re just looking at two SMB shares on a single server, not really seperate NASs (since so far you haven’t mentioned used HyperV to make a bunch of VMs) so you can use the quota features available via SMB and the filesystem in place of hardware limitations that place restrictions on how you can manage the server.

 

For the Purple drive(s), whether you get a second one and put them into storage spaces or not is up to you - I don’t know how likely a disk failure and a break-in are likely to coincide, but I assume its a tiny chance. On the other hand, I do surveillance system operation at work, and it seems almost half the time we need footage, it is often when there is a camera or system issue.

 

When you create a pool, you manually select the drives to add to it. When a new drive is inserted, it is not automatically grabbed by Storage Spaces, you have to manually add it to a pool.

 

You can format individual drives as ReFS, but the benefit is limited since it won’t have another disk to grab data from in case of corruption (Unlike ZFS which is a filesystem and disk subsystem in one, Microsoft split the roles into Storage Spaces and ReFS). I would feel fine using NTFS for those. Especially remember what I said about Plex’s metadata - by default that resides on the boot disk, but if you move it then it has to be to another NTFS disk.

[FuzzyLobster]

Did you ever get the software raid thing figured out? If you ended up using software raid 10 how are the speeds?

[Gerr]

I ended up researching Windows ReFS and deciding that while not as good as ZFS, it was good enough for my needs, so going an all Windows 2016 Server Essentials build.