storage pool Custom Plex Server/NAS & Microsoft Storage Pools

[ObiWanMonk]

Hey Guys im looking for a little bit of an idea really and before you say it... yes i know this is overkill but its been made to be overkill for a reason which is to make sure the system is never stressed and all been well never causes an issue I shall run you through the specs first and then explain:

 

Case: PCI Case 5U 19" Rackmount 8001 with 4x - Silverstone 2x 5.25" Device bay to 3x 3.5"

PSU: 1000W Corsair HX1000i (Linked to a UPS)

Motherboard:  Supermicro MBD-X10SRA-O

CPU: Intel Xeon E5-2620 - Socket 2011-3

Cooler: Corsair Hydro Series H80i V2

RAM: 2x 32GB Samsung DDR4, PC4-19200 (2400)

Cache: Silverstone SST-ECM20 2 Port M.2 Expansion Card + 2x 250GB Samsung 960 Evo NVMe M.2

Storage: 10x 6TB Seagate ST6000NM0024 Nearline V.4 Enterprise

NIC: 2 Port Intel X540-T2 10 Gbps

 

The backstory basics of this is that there is an internal 10Gb Network for the main computers on the network with everywhere wired with CAT6a given the property is in the process of being renovated so only made sense to run the cables while i could allbe it most of the devices on the network will be running at 1gb/s its there to future proof. Plex and the NAS run as normal with an external link directly in via VPN from the Work server to my system with a 300Mb/s download 30mb/s up connection to deal with that which is plenty.

 

Ok to the gritty point... This system is mainly going to be used as a Plex Server, NAS and a Backup for my companies offsite. As you can see the drives are the highly rated Seagate enterprise drives that Linus raved about a while ago and i have never had any issues with this capacity of drive however when it comes down to how to run this beast i am a little lost. So far the current ideas are either running UnRaid as the back bone which is by far the most professional way of doing it, the other which i have just stumbled over after all this time is Microsoft Storage pools which appear to work in a similar way to UnRaid's Storage pools. Has anybody had any experience pitting these two against each other and the differences in performance between the two?

UnRaid makes things a little more complex down the road as obviously i could just use a copy of Windows 10 Pro to run the Storage pools however i kind of get the feeling that the UnRaid server with a 10GBe NIC in there will do the job just as well, but ties the rest of the system up and i don't really want to run VM's on the Storage array for my needs. Windows 10 Pro would be the most convenient way but that means nothing in technology if there is a better way! anybody can help i would be really grateful!

 

Sorry for this post sounding rushed, im just leaving the office so need to get out haha!!!!

 

 

 

[Electronics Wizardy]

9 minutes ago, ObiWanMonk said:

either running UnRaid as the back bone which is by far the most professional way of doing i

Id stay away. 

 

For windows server, do you need AD, SMB, or other windows features that works best on windows.

 

Id probably run freenas or centos

 

 

For the build

 

Are you going to rack mount this? If you are, get a case with hotswap built in, like the norco's

 

Id get 4 dimms, cheaper and faster

 

For cpu id get a e5 1650 v4 or e5 1620 v4. There clocked much higher, and for many network protocals, you need clock speed, not cores.

 

Id get a small noctua cooler, not a water cooler, cools better and is more reliable.

 

Id probably not get the cache drives for now, and just use a raid, Cache won't help for what you seem to be using this for. 

 

What are you going to use for boot? PXE? Id probably get a small usb or sd card.

[SCHISCHKA]

TLDR: They will both do the job, use what you are familiar with.

50 minutes ago, ObiWanMonk said:

Has anybody had any experience pitting these two against each other and the differences in performance between the two?

I don't think there will be a performance difference when the limiting factor is your drive and network speed. Consider the features, data integrity & security each provides. With that much ECC RAM have you considered the ZFS filesystem & FreeNAS?

43 minutes ago, ObiWanMonk said:

Cache: Silverstone SST-ECM20 2 Port M.2 Expansion Card + 2x 250GB Samsung 960 Evo NVMe M.2

Im on the fence whether a NAS actually needs cache. Most of the data that goes through my NAS goes through once and doesn't need caching. Anything I fetch frequently I just download. Same goes for Plex, I don't see a cache benefiting media streaming.

27 minutes ago, ObiWanMonk said:

Windows 10 Pro

Windows server it has more admin tools which make it more secure. if you are familiar with it you may as well save yourself the trouble of learning a new OS. Your business probably already has paid for security services that you may as well use here. I want to scare you with the word ransomware. I run a different operating system from my clients because I want to make it difficult for ransomware distributors to steal from me.

35 minutes ago, ObiWanMonk said:

VM's on the Storage array

I didn't know UnRAID or Windows runs VMs for storage. I know they do a virtual storage in the way the disks are presented to the user but I didn't  think they use virtualisation in the sense of VT-x

[Mikensan]

Don't let the ideology of "professional" stop you from making a choice. I don't foresee unRaid being overly popular for a majority of businesses though. I will say the nice thing about it, is the whole "turn key" experience. It certainly fills a need in the NAS community, and personally I think it'd be fine as a cold storage solution or even a NAS solution in very small environments. 

 

If you aren't in this as a learning experience and just want it to work - I'd go with Windows. Much easier to troubleshoot if you're already a windows shop and permissions are just that much easier. You will want to have AD setup if not already. 

 

Another similar solution to Storage Spaces or unRaid would be FlexRAID, quite a few here use it as well. 

[samiscool51]

in theory it should work, just make a back up of the data before attempting to try this, you may lose data

also storage pools are slow at first but they will speed up, so long as no-one uploads a large file (1gb or more) it should be fine

[brwainer]

6 hours ago, ObiWanMonk said:

Cache: Silverstone SST-ECM20 2 Port M.2 Expansion Card + 2x 250GB Samsung 960 Evo NVMe M.2

This adapter only supports 1 NVMe M.2 card - the other M.2 card is Sata only.

[harry4742]

If you use Storage pools and consider windows, then go with the Server version, mostly beause of Deduplication, it can save you a ton of space (350GB saved on 600GB of Backups) and with storage pools and thin provisioning + ntfs you can do lots of fun stuff

 

[brwainer]

4 hours ago, harry4742 said:

If you use Storage pools and consider windows, then go with the Server version, mostly beause of Deduplication, it can save you a ton of space (350GB saved on 600GB of Backups) and with storage pools and thin provisioning + ntfs you can do lots of fun stuff

 

Storage pools on Windows 10 has all the same features, you just have to use Powershell more because the UI is much more limited. But yes for some workloads dedup can be awesome.

[leadeater]

11 hours ago, brwainer said:

Storage pools on Windows 10 has all the same features, you just have to use Powershell more because the UI is much more limited. But yes for some workloads dedup can be awesome.

My steam library dedups around 25%-30%, ~300GB savings, which is actually very good considering it's primary storage content and all on SSD. Compare that to the 75%-90% you get when using it as backup storage repository the Windows option is a very decent option.

 

One of the positives for this type of solution for small deployments is the dedup is not inline, it's after the fact, so resource requirements are orders of magnitude smaller than ZFS.

[brwainer]

39 minutes ago, leadeater said:

My steam library dedups around 25%-30%, ~300GB savings, which is actually very good considering it's primary storage content and all on SSD. Compare that to the 75%-90% you get when using it as backup storage repository the Windows option is a very decent option.

 

One of the positives for this type of solution for small deployments is the dedup is not inline, it's after the fact, so resource requirements are orders of magnitude smaller than ZFS.

That's true and those numbers are greater than my tests had found... but I can't get over the requirement for NTFS 

[leadeater]

2 hours ago, brwainer said:

That's true and those numbers are greater than my tests had found... but I can't get over the requirement for NTFS 

Yea I can't believe ReFS doesn't have dedup from the very outset, however it does have other features which allows it to compete heavily with dedup. Check out Veeam's overview of what they can do with Server 2016 and Veeam 9.5.

 

https://www.veeam.com/blog/advanced-refs-integration-coming-veeam-availability-suite.html

 

Quote

One of the hardest parts of making a backup, and in particular a synthetic full backup, is the transformation associated with moving blocks between different backups files in the incremental chain. Version 9.5 leverages the fast cloning capability in ReFS API to create synthetic backups without moving the data blocks between files, and instead, references backup file blocks already present on the volume. This means all manipulations associated with synthetic full backups are limited to metadata updates and require no actual I/O operations on backed up data.

Quote

I have some good news for you here too: The spaceless full backup technology brought by the advanced ReFS integration allows multiple full backups on disk to share the same data blocks. Obviously, this means a significant reduction in overall storage capacity requirements for the backup repository, providing storage efficiency rivaling that of some deduplication storage systems, but without the performance impact associated with using those (data rehydration). And what’s best, all you need is a general purpose Windows Server 2016 with a bunch of internal or direct-attached disks — same as the backup server itself uses! How cool is that?

This I think is pretty damn awesome.

 

[brwainer]

1 hour ago, leadeater said:

Yea I can't believe ReFS doesn't have dedup from the very outset, however it does have other features which allows it to compete heavily with dedup. Check out Veeam's overview of what they can do with Server 2016 and Veeam 9.5.

 

https://www.veeam.com/blog/advanced-refs-integration-coming-veeam-availability-suite.html

 

This I think is pretty damn awesome.

 

Yes I am seriously looking forward to this, and might even look into Veeam Endpoint Backup to replace File History, for the dedup savings while still being on REFS.