Raid 5 in Windows 10 - capacity 10% lower than expected

[chnapo]

Hi guys, I am using 4 HDDs (WD Red 4TB) to create Raid 5 in Windows 10. When formated in windows, each drive displays about 3.63 TB which is OK (TB vs TiB - from now on, by TB I will refer to TB in Windows). When creating Raid 5, I would expect that my capacity will be n-1 or close to n-1, in my case n = 14.5 TB and n-1 therefore about 10.8 TB. But after creating the storage pool, the capacity is only 9.67 TB, so I lost more than 1TB, but where and why?

[leadeater]

Edit: Hmm actually never mind, you're right there is a discrepancy. 

[chnapo]

25 minutes ago, leadeater said:

Edit: Hmm actually never mind, you're right there is a discrepancy. 

Excuse me, I did not understand this one

 

EDIT: understood

[leadeater]

2 minutes ago, chnapo said:

EDIT: understood

Yes, I did the classic not read the post properly and then post nothing of use to you at all, oops lol.

 

However I cannot think of a reason, I use Storage Spaces a lot myself and I even have WD 4TB HDDs, I'll give my systems and check and run the numbers and see if I have the correct pool capacities or not. Mine is a bit more complicated as I also have SSDs in the pool and tiering configured but I'll see if I can get some useful info.

[manikyath]

i'd have to experiment with this a bit to uderstand what's going wrong here..

 

for the sake of clarity in the topic, the issue that appears to exist here is, aside from the TB /TiB argument, you expect that if one disk formats in windows as 'N' storage size, you'd expect a RAID5 of 4 disks to format to roughly 'N*3' storage size, whereas here we see about 12% less size in the formatted raid pool, than in the formatted individual disks.

[Electronics Wizardy]

What is the number of columns set to? This feels like the number of columns is set to 3(which I think is the default for the gui) so then the parity is 1/3 of the total space instead of 1/4 as expected. I think that would explain that gap in storage space.

[chnapo]

3 minutes ago, manikyath said:

i'd have to experiment with this a bit to uderstand what's going wrong here..

 

for the sake of clarity in the topic, the issue that appears to exist here is, aside from the TB /TiB argument, you expect that if one disk formats in windows as 'N' storage size, you'd expect a RAID5 of 4 disks to format to roughly 'N*3' storage size, whereas here we see about 12% less size in the formatted raid pool, than in the formatted individual disks.

Yes, if single disk is N and I have 4 disks, with RAID 5 I would expect roughly 3*N storage size or within 1-3 % from that number, definitely not 12% less.

[chnapo]

2 minutes ago, Electronics Wizardy said:

What is the number of columns set to? This feels like the number of columns is set to 3(which I think is the default for the gui) so then the parity is 1/3 of the total space instead of 1/4 as expected. I think that would explain that gap in storage space.

That would roughly add up, but I can´t find the setting for number of columns. I only get to choose JBOD/stripe/mirror/parity, drive letter and name and then the final capacity, which allows me to actually input more, but then notes that I will have to add more HDDs when it starts filling up.

[Electronics Wizardy]

2 minutes ago, chnapo said:

That would roughly add up, but I can´t find the setting for number of columns. I only get to choose JBOD/stripe/mirror/parity, drive letter and name and then the final capacity, which allows me to actually input more, but then notes that I will have to add more HDDs when it starts filling up.

You have to make the virtual disk in powershell to set the number of columns. The gui isn't great for storge spaces as there are lots of hidden features.

 

So use a commnd like new-virtualdisk -numberofcolumns 4 -storagepoolfriendlyname STORAGEPOOLNAMEHERE

[dalekphalm]

30 minutes ago, Electronics Wizardy said:

What is the number of columns set to? This feels like the number of columns is set to 3(which I think is the default for the gui) so then the parity is 1/3 of the total space instead of 1/4 as expected. I think that would explain that gap in storage space.

Good catch - the math lines up perfectly with your theory.

 

3.63 TiB *4 = 14.52 TiB

 

14.52 TiB / 3 = 4.84 TiB (1/3rd of the total volume)

 

4.84 TiB * 2 = 9.68 TiB (2/3rd of the total volume available)

[chnapo]

42 minutes ago, Electronics Wizardy said:

You have to make the virtual disk in powershell to set the number of columns. The gui isn't great for storge spaces as there are lots of hidden features.

 

So use a commnd like new-virtualdisk -numberofcolumns 4 -storagepoolfriendlyname STORAGEPOOLNAMEHERE

Thanks, this was definitely the case. I was able to create RAID 5 array and volume on it that has 10.9 TB - which is as expected. Thanks a lot guys for awesome help! For reference for people who may have as little skill as I have, I used commands from this forum thread: https://social.technet.microsoft.com/Forums/ie/en-US/435f642a-a850-4d94-9521-92d9012bf20d/windows-10-storage-space-parity-calculation?forum=win10itprogeneral

EDIT: I have to add that it is really stupidly designed in the GUI, because everybody who uses more than 3 drives in RAID 5 and heavily prefers GUI over powershell will encounter the same issue. There are people who are DIY capable enough to want RAID 5 and not pro enough to like working in power shell. Also most of the tutorials lead to the GUI and there are next to 0 tutorials on how to do it in power shell.

[Nick7]

Is RAID5 performance (writes) with SS still abysmal with just HDD's?

I know it was basically unusable before... haven't tried it in a long time tho.

[Electronics Wizardy]

7 hours ago, Nick7 said:

Is RAID5 performance (writes) with SS still abysmal with just HDD's?

I know it was basically unusable before... haven't tried it in a long time tho.

Yea its still not great. There is some tweaking you can do, but I wouldn't expect over about 100mB/s write speeds in a setup like this.  Seems to be a design limit of parity in storage spaces. You get better speeds with more columns and larger arrays though, and you can add caching ssds to speed up writes.

[leadeater]

On 2/18/2022 at 4:12 AM, Electronics Wizardy said:

You have to make the virtual disk in powershell to set the number of columns. The gui isn't great for storge spaces as there are lots of hidden features.

 

On 2/18/2022 at 4:37 AM, dalekphalm said:

Good catch - the math lines up perfectly with your theory.

 

Ah nice, I thought the issue was the pool capacity and not the virtual disk capacity created in the pool. Nice that it was solved.

 

Note to everyone else Storage Spaces Pool capacity and Virtual Disk capacity are different things, a pool can have many virtual disks of any different configuration (support by the pool ofc) that will consume different amounts of pool capacity.

 

Had me stumped because I was thinking it was purely pool capacity issue 

 

7 hours ago, Nick7 said:

Is RAID5 performance (writes) with SS still abysmal with just HDD's?

 

Yes, Storage Spaces Parity must be used along side a Journal SSD or add data SSDs in to the Pool and create a Multi-Resilient Virtual Disk with an SSD tier and an HDD tier, these tiers can have different resiliency and column settings. I use Multi-Resilient Virtual Disk and it works very well.

 

On 2/18/2022 at 4:45 AM, chnapo said:

I have to add that it is really stupidly designed in the GUI, because everybody who uses more than 3 drives in RAID 5 and heavily prefers GUI over powershell will encounter the same issue. There are people who are DIY capable enough to want RAID 5 and not pro enough to like working in power shell. Also most of the tutorials lead to the GUI and there are next to 0 tutorials on how to do it in power shell.

The reality is Storage Spaces is a server usage designed Windows feature and should really only be configured and used via PowerShell. In Windows 10 (even Server) you can use the GUI however you have to live with the defaults. The defaults are like that for a reason, just scaling column size with number of disks isn't actually a good idea and has other factors to think about. The default column sizes are based on resiliency type used and minimum number of disks to meet that resiliency for Parity types. I think from memory Simple and Two-Way/Three-Way Mirror do scale to the number of disks in the Pool divided by 2/3 (Don't quote me on that).

[Electronics Wizardy]

1 minute ago, leadeater said:

Yes, Storage Spaces Parity must be used along side a Journal SSD or add data SSDs in to the Pool and create a Multi-Resilient Virtual Disk with an SSD tier and an HDD, these tiers can have different resiliency and column settings. I use Multi-Resilient Virtual Disk and it works very well.

 

How do you do you set this up?  Can this be done on a single system with storage spaces or do you need storage spaces direct? Is this just using tiering? I think they added the ability to do mirror and parity tiers in 2022.

[leadeater]

5 minutes ago, Electronics Wizardy said:

How do you do you set this up?  Can this be done on a single system with storage spaces or do you need storage spaces direct? Is this just using tiering? I think they added the ability to do mirror and parity tiers in 2022.

Single server, it's been possible since Server 2016. You're "supposed" to only use this and be able to configure it using Storage Spaces Direct (S2D) but you can just use PowerShell and create one just fine.

 

Edit:

https://charbelnemnom.com/how-to-create-a-multi-resilient-volume-with-refs-on-standalone-server-in-windows-server-2016-ws2016-hyperv-storagespaces/

[Nick7]

Yes, tiering works even with Windows 10 with SS.

From what I remember, it knows 'hot spots' and accorind to them it shuffles data between tiers. For that is uses 'defrag', so it's not immediate. I believe it was '-g' option?

[Electronics Wizardy]

42 minutes ago, Nick7 said:

Yes, tiering works even with Windows 10 with SS.

From what I remember, it knows 'hot spots' and accorind to them it shuffles data between tiers. For that is uses 'defrag', so it's not immediate. I believe it was '-g' option?

You can also use refs that does realtime tier optimization in storage spaces, only ntfs has that lmit where it runs nightly. I use the optimize-disk -tieroptimize to do this in powershell.

[leadeater]

1 hour ago, Electronics Wizardy said:

You can also use refs that does realtime tier optimization in storage spaces, only ntfs has that lmit where it runs nightly. I use the optimize-disk -tieroptimize to do this in powershell.

@Nick7 ReFS is a mandatory requirement for Multi-Resilient Virtual Disks to work, hence it is no longer supported in Windows 10 and Microsoft removed ReFS from standard desktop editions of Windows. You'd have to partition the virtual disk outside of the Windows install and then import the Storage Spaces pool along with the virtual disks to use it in Windows 10 today, I believe it'll still address and use an existing ReFS volume you just cannot create a new one. Not tried but I remember that was supposed to be the situation, as to not break everyone who were already using it.

 

Windows 10 would only support legacy tiering which isn't very good compared to ReFS and Multi-Resilient Virtual Disk

[Nick7]

@leadeater Ah, OK... guess I'll have to test ReFS a bit to see how it behaves