Windows Server storage pool size

[KingCollins]

Hey guys, I have a noob question, if you see my screenshot below, you can see from a storage pool in a test machine I have running Windows Server 2016. I have a 2TB and 1.5TB HDD in a storage pool, raid 0. 

Capacity does say 3.18TB, but the final storage pool capacity is 2.73TB.

I know there is some sort of calculation doing on regarding the pool, but is it always as bad as 300GB?

 

Please do not roast, I am noob, rush B

 

[Olaf6541]

Hi,

The 3.18TB is the total capacity of the two drives, but since you're using raid0 you can only use 2x1.36=2.73TB so the difference is the unused space on the 2TB drive.

[BloodKnight7]

3 hours ago, Olaf6541 said:

Hi,

The 3.18TB is the total capacity of the two drives, but since you're using raid0 you can only use 2x1.36=2.73TB so the difference is the unused space on the 2TB drive.

 

That's correct! A RAID 0 can be created with disks of differing sizes, but the storage space added to the array by each disk is limited to the size of the smallest disk. For example, if a 100 GB disk is striped together with a 350 GB disk, the size of the array will be 200 GB (100 GB × 2).

[KingCollins]

Wow, thanks for the explanation guys, makes total sense!
Is there any way possible to create a storage pool in Windows server without having to use Raid? 

 

For just a bunch of disks? JBOD 

[BloodKnight7]

6 hours ago, KingCollins said:

Wow, thanks for the explanation guys, makes total sense!
Is there any way possible to create a storage pool in Windows server without having to use Raid? 

 

For just a bunch of disks? JBOD 

Just keep them separate. With JBOD you have no idea where files are stored on your drives. Is it on one or the other? If a drive fails, which files or parts of files are gone? Are the files spread across the slowest parts of the drives? 
 

[Electronics Wizardy]

On 5/16/2018 at 9:43 AM, Olaf6541 said:

Hi,

The 3.18TB is the total capacity of the two drives, but since you're using raid0 you can only use 2x1.36=2.73TB so the difference is the unused space on the 2TB drive.

nope, this isn't how storage spaces works, it will fill mis matched drives.

 

You can change the size of the volume, do that. The volume size can be bigger or smaller than the pool size and isn't related at all.

[BloodKnight7]

7 hours ago, Electronics Wizardy said:

nope, this isn't how storage spaces works, it will fill mis matched drives.

 

You can change the size of the volume, do that. The volume size can be bigger or smaller than the pool size and isn't related at all.

Yeah man... but for Storage spaces, you need at least 3 drives.... our friend here was asking about a 2 disk RAID 0 configuration. Personally, I dislike storage spaces because of 1) performance is low 2)when people get they can use any hardware indiscriminately, which leads to them using very crappy one. 3) If Storage spaces fails, there is no way to recover data. 

[Electronics Wizardy]

33 minutes ago, BloodKnight7 said:

Yeah man... but for Storage spaces, you need at least 3 drives.... our friend here was asking about a 2 disk RAID 0 configuration. Personally, I dislike storage spaces because of 1) performance is low 2)when people get they can use any hardware indiscriminately, which leads to them using very crappy one. 3) If Storage spaces fails, there is no way to recover data. 

you can run storage spaces with less than 3 drives. And you can run a config with 2 different sized drves and both will fill.

 

1. It can be good, all depends on config. Storage spaces was really made for the enterprise where it does very well

2. Yep, you need good hardware for it to work well.

3. Shouldn't be an issue backups. But yea recovery is harder with any raid or pooling configuration.

[BloodKnight7]

14 minutes ago, Electronics Wizardy said:

you can run storage spaces with less than 3 drives. And you can run a config with 2 different sized drves and both will fill.

 

1. It can be good, all depends on config. Storage spaces was really made for the enterprise where it does very well

2. Yep, you need good hardware for it to work well.

3. Shouldn't be an issue backups. But yea recovery is harder with any raid or pooling configuration.

Requirements

 

https://support.microsoft.com/en-us/help/12438/windows-10-storage-spaces

 

Windows+2drives=3

[Blebekblebek]

Does it have to be in RAID0?

I mean if you just want to POOL it, why not just do that?

You can JBOD it or combine with storage spaces windows manager.

 

it's raid 0 afterall there's no redundant with that option.

[leadeater]

23 hours ago, BloodKnight7 said:

Personally, I dislike storage spaces because of 1) performance is low

This is a virtual disk mounted on my workstation to my server over a dual 10Gb SMB3 Multi-Channel connection to a 6 SSD 3 HDD Multi-Resilient virtual disk so it is being network limited as you can see (2GB/s). SSDs are in two-way simple and HDDs are single parity, tiering is in use. Yes this is virtual disk inception, VHDX file on a SMB share hosted on a virtual disk in a Storage Spaces pool.

 

Going to have to disagree with the slow part. You can configure it in a manor that is slow but Storage Spaces itself is not slow.

 

23 hours ago, BloodKnight7 said:

2)when people get they can use any hardware indiscriminately

Exactly like FreeNAS and ZFS, you can do both badly, in fact you can do anything badly if you don't know what you're doing so I don't see this as a specific negative point only for Storage Spaces.

 

23 hours ago, BloodKnight7 said:

If Storage spaces fails, there is no way to recover data. 

Yes you can, but only if you wrote down every important setting or can remember them so you can tell recovery software stripe widths and chunk sizes, unless it's a 2 HDD mirror setup then recovery is trivial. If it's a system failure then you can move the disks to another system and it'll automatically detect it then you just set it's state to online, very simple and done this a couple of times now (migration rather than failure).

http://www.storage-spaces-recovery.com/library/case3-recovery.aspx

 

Btw those above requirements are the same for recovering a traditional hardware RAID array.

 

 

23 hours ago, BloodKnight7 said:

but for Storage spaces, you need at least 3 drives.... our friend here was asking about a 2 disk RAID 0 configuration.

In the scenario for the OP question the OS disk can be ignored, you cannot use an OS disk in Storage Spaces at all so the two disks shown are for data only. The minimum for Storage Spaces is dependent on configuration but 2 is the absolute minimum.

 

On 5/18/2018 at 7:39 AM, Electronics Wizardy said:

nope, this isn't how storage spaces works, it will fill mis matched drives.

 

You can change the size of the volume, do that. The volume size can be bigger or smaller than the pool size and isn't related at all.

@BloodKnight7 While not really applicable to the OP question being ask as the number of disks in the pool is too low you can have different sized disks in a pool and it is possible to fill them all. However this depends on stripe width configured and that must be smaller than the total number of disks in the pool to be even possible. It's easier to create a virtual disk to it's maximum size then create another virtual disk in the same pool with the residual pool capacity left over due to the smaller disks limiting the maximum size of the first virtual disk without manually configuring stripe width to work around it.

 

Basically it is possible to have a single virtual disk fill all disks in a pool even when they are different sizes but it's hard to achieve and gets even harder as the number of disks decreases.

 

I've been using Storage Spaces for about 3 years now so I'm quite accustomed to using it and seen it evolve over time and understand it's limitations and how to work around them. It's something often not considered with talking about Storage Spaces versus say FreeNAS, it's not really one technology is better than the other its more one is more well understood and there is a wealth of knowledge to draw on where the other does not have that.

 

Storage Spaces is in fact more flexible with disk configurations than ZFS, other software RAIDs and hardware RAID which actually make it well suited for home users as well but that flexibility is a doubled edged sword, many possibilities also includes bad ones as well as good ones. Sometimes ridged requires are a good thing.

[Electronics Wizardy]

1 hour ago, leadeater said:

've been using Storage Spaces for about 3 years now so I'm quite accustomed to using it and seen it evolve over time and understand it's limitations and how to work around them. It's something often not considered with talking about Storage Spaces versus say FreeNAS, it's not really one technology is better than the other its more one is more well understood and there is a wealth of knowledge to draw on where the other does not have that.

 

Storage Spaces is in fact more flexible with disk configurations than ZFS, other software RAIDs and hardware RAID which actually make it well suited for home users as well but that flexibility is a doubled edged sword, many possibilities also includes bad ones as well as good ones. Sometimes ridged requires are a good thing.

Ive been diving into storage spaces a bit more lately, and its a nice flexible system. Its not perfect(nothing is though). But how it lets you just have a pool and setup whatever arrays you want on it is nice. 

 

Also you can do storage spaces tiering on windows 10(you need powershell, and I was using ent, idk about home and pro)

 

My personall guess is that in the future all volumes in windows will use storage spaces, which will be nice(the whole ntfs and partition system has been a thing for long time)

[leadeater]

1 hour ago, Electronics Wizardy said:

Also you can do storage spaces tiering on windows 10(you need powershell, and I was using ent, idk about home and pro)

Even on Windows Server 2016 I configure it all using PowerShell, I just feel it's safer to do it that way as it will be exactly how you wanted it rather than interpreted through GUI options you selected. Been bitten in the past with sub optimal things like column size (stripe width) and chunk size, column size has a rather big impact on how you add disks to the system to expand it and performance when doing so. Would be very nice to be able to change it after the fact and then run a re-balance.

 

Edit:

I would say limit your column size to 4 at home and 8 for larger server setups. I say no bigger than 8 since that is a typical backplane number for HDDs in a server, groups of 8 going back to a HBA/RAID card.

[KingCollins]

Wow looks like my thread got alot of attention/feedback! 

Just so everyone knows, I am a noob when it comes to Windows Server, this is my first time using it and I am only using it because Plex Media Server seems to transcode my files a little more brisk compared to FreeBSD or Linux. I was using FreeNAS before and I quite enjoyed it! I am only learning  

 

19 hours ago, leadeater said:

In the scenario for the OP question the OS disk can be ignored, you cannot use an OS disk in Storage Spaces at all so the two disks shown are for data only. The minimum for Storage Spaces is dependent on configuration but 2 is the absolute minimum.

Yup you are absolutely right, I have a separate SAS drive for my OS.

 

19 hours ago, leadeater said:

Going to have to disagree with the slow part. You can configure it in a manor that is slow but Storage Spaces itself is not slow.

 

Am I right in saying that the performance is quite good? I personally have maxed out (almost)  my gigabit home network connection transferring files from an SSD in my gaming PC to my server. Write speeds were 900-950 Mbits/s to my storage pool which is mechanical disks. I think that is fantastic?

 

There is nothing important on this server, only some movies and TV shows, "IF IT DIES, IT DIES" (insert meme).

 

Was just looking to buy some more storage and was wondering if I just added a 4Tb HDD to the pool, would I loose a lot of space.

 

Thanks again guys for the feedback and lesson  

[leadeater]

3 hours ago, KingCollins said:

Write speeds were 900-950 Mbits/s to my storage pool which is mechanical disks. I think that is fantastic?

Probably a little bit of ram caching going on there, it's in the too fast realm for the number of disks but hey we want ram caching . Generic not fast HDDs I calculate at 80MB/s-100MB/s per disk in an array and for faster ones 120MB/s-150MB/s per disk in an array, smaller arrays do get more effective performance as it gets harder to get all the disks working together as the number goes up but those numbers are fairly safe to use as a sanity check for performance when you're testing.

 

If you want to see performance without any caching going on copy a file way bigger than the ram in your server and client.

[BloodKnight7]

My friend @leadeater

Lots of valid points in your post but let's compare apples with apples. OP here has 2 Mechanical drives, no SSDs.

 

1)Performance wise I have not tried tiered parity recently. If you are doing NTFS and you have enough SSD's you can get decent performance. you have to be careful as you will get double/triple writes to your SSD's this way. One to the WBC and again to the SSD tier (if you tier) and the Parity journal. In my opinion not very efficient and guaranteed to eat the life of your SSD way faster. Curious note here, the OP has 2 mechanical hard drives, no SSDs. (at least from his screenshot he is using WD Caviar drives SATA 3.0 if I'm not mistaken)

 

I am not saying it doesn't work, my point is that Storage spaces is not the most efficient solution in the market, let alone faster/better than a good RAID 0. (Again for the use presented in this very specific scenario (2 mechanical SATA 3.0 drives)

 

2) Yeah, agreed, but other solutions are not marketed as the panacea giving the users a false idea of oh yes put any storage device in Storage spaces, it will rock!. Storage Spaces is marketed as the "breakthrough/game changer" as most of Microsoft products. This is NOT the case. There are lots of best practices to apply/configure it's not an open and use product (maybe in home use it is, not in an enterprise environment).

 

3)  The inherent complexity of Storage Spaces is why the development of data recovery algorithms for failed Storage Spaces will be difficult. Try sending drives to companies like Ontrack or any of your preference, and you will not like the quotes... the reasons why are:

a) The internal complexity of Storage Spaces will surely affect data processing time during the recovery. For example, when recovering data from a deleted NTFS volume, data recovery software can avoid reading the entire disk and create a folder tree within, say, ten minutes. In the case of a deleted Storage Spaces pool, however, you need to read all the data from all the disks that were members of the pool. Depending on the size and number of disks this could take several days.

b) Multiple drive storage pools are further divided into elements called "slabs" that Storage Spaces use to mirror data so that at least two (and optionally three) copies are maintained. The number of slabs in a volume may be very large. While a typical JBOD consisting of three or maybe five drives can be assembled manually, there can be thousands or dozens of thousands of slabs in Storage Spaces. This makes manual recovery too expensive (time=$) for practical use against standard scenarios.

 

Again the main point being, Storage spaces makes it way too hard/costly for a standard or new user to recover from any failure. I will give you props on the software you recommend, I haven't used it, but for only $300 bucks its worth the shot!

 

4) The reason I never ignore the OS file is because of its considered part of the "footprint" or overhead of the solution. There must always be a Windows drive in the system because that's where the Storage spaces service lives! Other solutions don't have this "requirement" as they can be run from USB, SD card, Micro SDXC, etc...

 

Agree with your conclusions/closing points: Storage spaces as a "free" (included in the windows license) solution have its benefits, but it's not for everyone, either home or enterprise. Its a good starting point depending on your resources available (My point here being the false sense of you can use anything with Storage spaces marketing BS), but just as Hyper-V (which is "free" as well) it has a lot of shortcomings that with no doubt will be kept to be solved while new versions appear.

 

[leadeater]

3 hours ago, BloodKnight7 said:

If you are doing NTFS and you have enough SSD's you can get decent performance. you have to be careful as you will get double/triple writes to your SSD's this way. One to the WBC and again to the SSD tier (if you tier) and the Parity journal. In my opinion not very efficient and guaranteed to eat the life of your SSD way faster. Curious note here, the OP has 2 mechanical hard drives, no SSDs. (at least from his screenshot he is using WD Caviar drives SATA 3.0 if I'm not mistaken)

If you use ReFS which means Multi-Resilient then you don't have to worry about write amplification much. All writes and modifications go to the fast tier then get moved to slow if required. The old tiering and using NTFS isn't the way it shouldn't be done anymore. As I mentioned though ReFS got removed from desktop Windows, really sucks Microsoft did that.

 

Point really is only pure parity is terribly slow, you don't have to use that. If you use mirror with HDDs or SSDs performance will be excellent, of course you lose a ton of space.

 

3 hours ago, BloodKnight7 said:

Yeah, agreed, but other solutions are not marketed as the panacea giving the users a false idea of oh yes put any storage device in Storage spaces

I've not seen much marketing for Storage Spaces like this at all, in fact I see far more of that for ZFS and FreeNAS to be honest. It's hard not to fall over someone touting FreeNAS as the best thing to use.

 

[BloodKnight7]

Just a couple for the more tech savy, most of them are supported in windows (the ones with *):

-Stablebit*: https://stablebit.com*

-Snapraid*: https://www.snapraid.it

-uNRaid https://lime-technology.com/

-FlexRaid* http://www.flexraid.com/

 

Those are kinda the best cherrypicks from the bunch. There are a lot of opensource solutions but most of them are just too time consuming to set up.