Server 2012 r2 Storage Pools/Storage Spaces for NAS

[SamDaBos]

Hiya

 

I have a home server running Server 2012 R2. At the moment it has a jumbled mix of HDD's which I want to sort out. I have set my sights on WD Blue 5400RPM drives however I'm not sure what to do with them and therefore which size to get. These new drives will pretty much just be for videos (ie movies and TV shows). After reading into my options, I decided I want to try the Storage Pools feature because (as I understand at the moment) I can add drives later on. This is where I would like clarification, if I used the parity option, started off with 3 drives and filled them, can I bung in another one and tell Windows to spread files across the 4 drives? (with all the drives being exactly the same model).

 

If this is true and it will allow me to easily expand the 'array' (do you call it that in Storage Pools?), that is what I am leaning towards. However the other option is the mirror. So I would either get 3x 1TB drives with parity, or 2x 2TB drives and mirror them. Both give me the same redundancy and storage space. What happens if I added a third drive to the mirror? I'm guessing nothing because I read that to go above 2 drives, you have to jump to 5. 

 

The cost between the two options is pretty much the same (£120 plus the SATA controller I need) but I can't decide which is more suitable. Parity writing I hear is God awful but pretty much the only time I will be writing to the disks is when I add new videos, the rest of the time Plex will be reading them.

 

What are the advantages and disadvantages of going each way, and if you were me, which would you lean towards?

 

EDIT: After reading a bit more, I have come across someone who has used an SSD as a cache. Now I don't have a spare SSD but now I'm wondering if I can use a regular HDD (say an unused 160GB one) as my cache. So when I want to write something to the storage pool, the data gets written to the 160GB HDD at full speed, then in the background windows writes it to the array and removes it from the 160GB drive when complete.

 

Any help is greatly appreciated, thanks  

[Xploda]

I can't recommend using storage spaces under Windows. I have tried a similar setup on 3x 4TB Seagate Barracuda, and even though it works, performance was absolutely horrible. I never got around to figuring out why, but I was getting results like 30MB/s read speeds and 10MB/s write. Maybe it was a bug which they have fixed in the meantime, but I instantly switched over to mdadm on Ubuntu, on which I am receiving the read and write performance that is to be expected from these drives.

 

Not only that, but in my opinion, Windows gives you very little control over your array of drives from the GUI. You would probably have to dive into PowerShell for doing more advanced operations.

 

On topic, the parity option is the equivalent of a RAID5 setup. Double parity would be the equivalent of RAID6 (minimum 5 drives). A mirror with more drives is possible, but this is called a three way mirror in storage spaces (also minimum 5 drives required). As far as choice of drive capacity, I think it is best to use the largest drives available and within your budget (that's at least what I did, 3x 4TB instead of 4x 3TB). Less but bigger drives means less power draw, but more importantly, less SATA ports in use, which means it will be easier to expand. Especially with current developments such as 4K, it wouldn't hurt to invest into better upgradability.

 

About your performance concerns. Yes, RAID5 could take a while to rebuild, because your system has to calculate the parity. I have done this in the past, and it took approximately 10 hours to rebuild the array on a system with a Pentium G630 and 4GB of DDR3. During this time, you will be running in a degraded state, meaning that another drive failure will make you lose all of the data. If you add another drive, I think the OS would be smart enough to relocate the data as required by the changed setup. Mirror should be a bit faster, because it would just be a matter of directly copying the data. Mirroring data would use more space than the parity bits on a RAID5.

 

If you're mostly going to read data from it, I would be leaning towards a RAID5 setup. Read speeds are excellent, write speed a little less so.

 

In any case, I would recommend to set up something like a virtual machine for testing purposes. You can add as many virtual drives as you want (make them like 100MB each and imagine they are larger  ), which allows you to test the exact behavior of each of the options. You can test the redundancy by removing or unmounting your virtual drives. That's at least how I figured out storage spaces before using it on my actual hardware. You can simulate some of the scenarios that could happen in real life.

[SamDaBos]

Quote

50 minutes ago, Xploda said:

-Snippy snip snip-

 

Yeah, I have a load of other stuff running on the server otherwise I'd use some Linux based OS. Doing some more searching, I've found that   Dell PERC 5i Raid cards are available pretty cheaply, with battery backups. This is now making me want to go for a hardware based RAID 5 setup, this card also supports RAID expansion too...

 

Thoughts?

[cgtechuk]

I don't recommend this approach as its risky and its not portable should the OS have a crap out. Rather than doing this method I always recommend getting a hardware card / sata controller and going for a JBOD configuration as this would work well or if the machine was powerful enough then use something like UNRAID to host a storage system that is a lot more stable than windows and still have a Windows OS on there too.

 

Anything software wise is poor performance, a dedicated hardware solution is much better as it takes it off the CPU, Depends what you are looking for really.

[SamDaBos]

Just now, cgtechuk said:

----

Ok. For maximum budgetness is the Dell PERC 5i raid card any good? After finding one of these can be had for under £30, this is the way I'd like to go now. So if it is, in your opinion, 2x 2TB or 3x 1TB?

[cgtechuk]

8 minutes ago, SamDaBos said:

Ok. For maximum budgetness is the Dell PERC 5i raid card any good? After finding one of these can be had for under £30, this is the way I'd like to go now. So if it is, in your opinion, 2x 2TB or 3x 1TB?

Totally depends, Less risk with 3 x 1TB but then you lose expandability, I currently have in a hp microserver 4 x 2TB and a custom machine 4 drives in JBOD totalling ~ 20TB 

 

If you are in the UK have you seen this little GEM? 

 

http://www.hotukdeals.com/deals/hp-proliant-microserver-gen8-celeron-dual-core-4gb-110-after-55-cashback-165-serverplus-2396185

 

Basically £110 for a Microserver that has 4 drive bays good to go, Totally silent usage, Built in JBOD / Raid, ILO for headless use etc 2 x 1GB NICs. I use this one and its a good machine.

 

For the Perc Card you would need to have a google as I am not 100% sure they work with desktops very well, The 4i used to be pretty compatible with desktop machines but the HP P410i doesnt like non HP machines at all.

 

 

[brwainer]

In general there are mixed opinions about Storgaes Spaces, however everyone agrees that Parity pools are TERRIBLE if you don't set a huge writeback cache or set up Journal drives (which dedicates the whole disk to writeback caching I believe) - either way you *need* 2 SSDs for this purpose (the SSDs function in RAID1 to protect data before it gets put onto the hard drives). Without SSDs, or even with SSDs, most people use Mirror pools.

 

People compare Storage Spaces Mirror to RAID 1 but that's not true at all. It is much cooler than that.

 

If you start with just 2 drives, then yes it behaves like RAID 1. But if you have 4 or more drives and set it for a 2 drive mirror, it behaves like RAID 10. 

 

The important thing is the relationship between "NumberIfCopies" and "NumberOfColumns". Number of Copies is self explanatory, a 2 copy virtual disk uses 2 disks, and a 3 copy virtual disk requires 5 drives for reasons I don't know (I assume they have some testing that showed only having 3 or 4 drives doesn't work out well). Number Of Columns turns a RAID 1 into a RAID 10. A 2 copy 2 column virtual disk will use 4 drives. 2x3 uses 6, etc. I have no idea what a 3 copy 2 column virtual disk requires.

 

The setting for number of columns can only be set when the virtual disk is created. If you use the Server Manager UI, it defaults to as many columns you can support up to 4. If you have enough drives, you can manually set the number of columns by creating the disk in Powershell. 

 

When you start with 2 disks, yoir data is equally spread between the two. Whenever you add a drive, the data is not rebalanced automatically, but there is a tricky way to make it rebalance by retiring one of the disks that is full, repairing a virtual disk or two, and setting the now partially empty disk back to Auto-select (this wouldn't work if you only had one virtual disk, or only one that's mainly been used). When you get to 4 disks and want to convert to a 2x2 virtual disk, you have to create the new disk then copy your data over and delete the old one.

[Xploda]

19 minutes ago, cgtechuk said:

I don't recommend this approach as its risky and its not portable should the OS have a crap out. Rather than doing this method I always recommend getting a hardware card / sata controller and going for a JBOD configuration as this would work well or if the machine was powerful enough then use something like UNRAID to host a storage system that is a lot more stable than windows and still have a Windows OS on there too.

 

Anything software wise is poor performance, a dedicated hardware solution is much better as it takes it off the CPU, Depends what you are looking for really.

And what if the hardware breaks down? I've always heard in the past that it could make recovering the data a bit more difficult, as you would need (almost) the exact same model of card to be able to pull it off. Is that still a valid point today? That's why I personally opted for a software RAID solution, but it would be nice to know for some point in the future  .

[SamDaBos]

5 minutes ago, cgtechuk said:

---

If I could, I would, but I can't so I shan't... My dad won't want to pay for that and it's electricity. He's happy with my current server because I've set it up for everyone to use (and have set up programs for them too), but he won't be happy with another thing running (the amount of electricity I use at the moment... yeah... ahem... and that's before my siblings attack the power meter). 

After my googling, it seems people have used it with Server 2012 R2 without hickups so I think it looks good. I think I'll just get 2x 2TB drives. If I need more storage for videos than that then I'll cross that bridge when I get to it (probably have left home by then anyway so I'll be able to do what I like).

 

13 minutes ago, brwainer said:

---

Well that's parity out the window for me then. I still have documents, music, photos etc stored on this server too so I think I'm going to have a look at setting up a mirrored pool for that. Can unmatched sized disks be used together in one pool?

[cgtechuk]

Just now, Xploda said:

And what if the hardware breaks down? I've always heard in the past that it could make recovering the data a bit more difficult, as you would need (almost) the exact same model of card to be able to pull it off. Is that still a valid point today? That's why I personally opted for a software RAID solution, but it would be nice to know for some point in the future  .

Yes I have been there lol, 

It totally depends on the card , Here is an example,

 

I have an Old ( 7 or 8 Y/O HP Server using a P400i Raid card, machine died, unplugged the disk and plugged it into a B210 Controller on a brand new machine and it found it set it up and was running in about 1 minute without even as much as a reboot, Where the problems come is if the hardware craps out and starts writing crap to the disks. 

A cheap card wont give you these functions as a direct card would be needed but most HP / Dell. / LSI solutions write the config to a part of the HD so its readable to nearly 90% of other cards in that product line / manufacturer

[cgtechuk]

Just now, SamDaBos said:

If I could, I would, but I can't so I shan't... My dad won't want to pay for that and it's electricity. He's happy with my current server because I've set it up for everyone to use (and have set up programs for them too), but he won't be happy with another thing running (the amount of electricity I use at the moment... yeah... ahem... and that's before my siblings attack the power meter). 

After my googling, it seems people have used it with Server 2012 R2 without hickups so I think it looks good. I think I'll just get 2x 2TB drives. If I need more storage for videos than that then I'll cross that bridge when I get to it (probably have left home by then anyway so I'll be able to do what I like).

 

Well that's parity out the window for me then. I still have documents, music, photos etc stored on this server too so I think I'm going to have a look at setting up a mirrored pool for that. Can unmatched sized disks be used together in one pool?

Haha been there mate, I know how that works lol. Except when I moved out my parents house I became that very guy moaning about electricity costs lol. If it helps the microserver uses 50watts under load but yeah I hear ya.

 

Be careful with the card, its not the OS thats the problem but the BIOS usually doesnt like the Option ROM included on them but the Dell is just a rebadged LSI so you should be good.

[brwainer]

47 minutes ago, SamDaBos said:

If I could, I would, but I can't so I shan't... My dad won't want to pay for that and it's electricity. He's happy with my current server because I've set it up for everyone to use (and have set up programs for them too), but he won't be happy with another thing running (the amount of electricity I use at the moment... yeah... ahem... and that's before my siblings attack the power meter). 

After my googling, it seems people have used it with Server 2012 R2 without hickups so I think it looks good. I think I'll just get 2x 2TB drives. If I need more storage for videos than that then I'll cross that bridge when I get to it (probably have left home by then anyway so I'll be able to do what I like).

 

Well that's parity out the window for me then. I still have documents, music, photos etc stored on this server too so I think I'm going to have a look at setting up a mirrored pool for that. Can unmatched sized disks be used together in one pool?

Yes I have used mixed sizes. What happens is when you created a Fixed virtual it allocates the space equally across the available drives, eually being calculated by the percentage the drive is already filled. When you create a Thin virtual disk is does the same thung but only allocating as space is filled. So if you have a 1TB drive and two 500GB drives and make a 500GB mirror, it will use half the capacity of each of the drives. Where it gets messed up is when one of your drives is full and you add a new drive, since it doesn't have a normal way to reallocate, but you can use the process I said above to trick it.

[leadeater]

I currently use Storage Spaces under Server 2012 R2 and it works great. As has already been pointed out don't use any parity configurations performances is horrible. Also note that the mirror options are nothing like traditional mirrors, it means mirror data blocks not disks. This is why you can have many disks in mirror configurations not just two, like RAID 1.

 

Also in regards to adding disk to an existing pool there is a limitation that is fixed in Server 2016, you can't re-balance data evenly across the disks once the new ones are added. Only new data is spread on to the new disks, Server 2016 has added a re-balance option to solve this issue.

 

Currently I run two disk pools, one 4x Samsung 850 Pro 512GB for my Steam library and one 2x Samsung 840 Pro 512GB + 5x 3TB HDD with Auto Tier enabled. The SSD only pool is using simple mode and the other pool is using 2-way mirror, I don't care about data on the SSD only pool just speed.

[leadeater]

3 hours ago, brwainer said:

The important thing is the relationship between "NumberIfCopies" and "NumberOfColumns". Number of Copies is self explanatory, a 2 copy virtual disk uses 2 disks, and a 3 copy virtual disk requires 5 drives for reasons I don't know (I assume they have some testing that showed only having 3 or 4 drives doesn't work out well). Number Of Columns turns a RAID 1 into a RAID 10. A 2 copy 2 column virtual disk will use 4 drives. 2x3 uses 6, etc. I have no idea what a 3 copy 2 column virtual disk requires.

Columns is essentially stripe width. Number of copies sets the amount of copies for data blocks. When you add disks to a pool you must add the number of disks by Columns x Copies to actually gain usable space (for mirror spaces). Columns directly effects the performance but also effects the amount of disks you need to add to expand the pool so never set this higher than you actually need to. Anything bulk storage related I set mine to two, I also always SSD cache so I easily get away with two.

 

Three-way mirror is resilient to two disks failures. If you go find a graphic or draw it out yourself you will find the only way to survive two disks failing with the rule of 3 copies of data blocks is 5 disks. I disagree with the names they have used because it is easily confused with what it actually means, Type 'Mirror' Protection '1 Disk' or '2 Disk' is just hard to market and display in a GUI.

 

One of the Microsoft Ignite videos has an animated graphic that shows all this stuff very well, was ages ago so I don't remember which one.

 

[brwainer]

42 minutes ago, leadeater said:

I currently use Storage Spaces under Server 2012 R2 and it works great. As has already been pointed out don't use any parity configurations performances is horrible. Also note that the mirror options are nothing like traditional mirrors, it means mirror data blocks not disks. This is why you can have many disks in mirror configurations not just two, like RAID 1.

 

Also in regards to adding disk to an existing pool there is a limitation that is fixed in Server 2016, you can't re-balance data evenly across the disks once the new ones are added. Only new data is spread on to the new disks, Server 2016 has added a re-balance option to solve this issue.

 

Currently I run two disk pools, one 4x Samsung 850 Pro 512GB for my Steam library and one 2x Samsung 840 Pro 512GB + 5x 3TB HDD with Auto Tier enabled. The SSD only pool is using simple mode and the other pool is using 2-way mirror, I don't care about data on the SSD only pool just speed.

Glad to know 2016 fixes the lack of auto rebalancing, I'll be really glad to forget about my tricky powershell workaround. Just curious, why are you using seperate pools?  I have a single pool with 4 HDDs and 4 SSDs, some virtual disks are auto tiering, but I have one virtual disk that is SSDs only and one that is HDDs only. In Powershell it doesn't complain if you do "-StorageTiers $SSD -StorageTierSizes 100GB" - it just makes a virtual disk using only a single tier.

[leadeater]

Just now, brwainer said:

Glad to know 2016 fixes the lack of auto rebalancing, I'll be really glad to forget about my tricky powershell workaround. Just curious, why are you using seperate pools?  I have a single pool with 4 HDDs and 4 SSDs, some virtual disks are auto tiering, but I have one virtual disk that is SSDs only and one that is HDDs only. In Powershell it doesn't complain if you do "-StorageTiers $SSD -StorageTierSizes 100GB" - it just makes a virtual disk using only a single tier.

No real reason. Just wanted separation between data I don't care about, VHDX file mounted on my desktop via SMB3 multichannel 10Gb for steam library, and important data I iSCSI present to my ESXi servers. Also I wanted stupid speed so column width of 4 on the SSD only and column width of 2 on the other.

 

It's only temporary anyway until I can be bothered to migrated to Server 2016. Going to move to Storage Spaces Direct but I need to get 4 servers for this (or cheat with VMs) and 8 SSDs.

[brwainer]

8 minutes ago, leadeater said:

No real reason. Just wanted separation between data I don't care about, VHDX file mounted on my desktop via SMB3 multichannel 10Gb for steam library, and important data I iSCSI present to my ESXi servers. Also I wanted stupid speed so column width of 4 on the SSD only and column width of 2 on the other.

 

It's only temporary anyway until I can be bothered to migrated to Server 2016. Going to move to Storage Spaces Direct but I need to get 4 servers for this (or cheat with VMs) and 8 SSDs.

I was really excited about storage spaces direct until they said 4 servers... 2 or 3 I could live with. I'm not comfortable tricking it using VMs. But I'm still looking forward to 2016.

[leadeater]

4 minutes ago, brwainer said:

I was really excited about storage spaces direct until they said 4 servers... 2 or 3 I could live with. I'm not comfortable tricking it using VMs. But I'm still looking forward to 2016.

Yep I was also going AWESOME!!..... wait, bloody hell. I'd only be happy tricking it with a 2 physical server setup but I'll cross that bridge when I get to it. I had the same issue with VSAN but with that you could override the limit and run it on a single server to try it out.