Picking a tiered storage solution!

[Mr.Humble]

Hi there!

Recently I set up a Parity Storage space consisting of 4 1TB + one 2TB HDD, for a total of 3TB usable capacity, in my dad's PC and mapped it as a network drive to the rest of our devices. The objective was to have a central backup storage location and share media files in the household.

 

Aside from a lower-than-expected capacity, it works as intended, but the write speed s u c k s.

 

I have a spare 500GB SSD that I want to combine with it to accelerate it and create a small tiered solution.

 

Question is, what would be the best way about it?

 

Option 1: Tiered Storage Spaces

 

The machine is Windows 10 Pro, so it would require Powershell wizardry, but it can be done.

 

I would have to offload around 2TB of data first and dismantle the current Storage Space (I assume), but I have the spare capacity on external drives.

 

Option 2: Primocache

 

The idea is to combine the "single drive" - Parity Storage Space - and the SSD. I'm not sure it would work, but it could be easier and without hassling with Powershell. Paid solution dependent on software

 

Option 3: Stablebit Drivepool

 

Same as Primocache, possibly more robust in terms of setting up the drive pool from scratch. Paid solution dependent on software

 

Option 4: StoreMI

 

Right now the PC is based on Intel Z77, but a Ryzen 3600 upgrade is planned for around the end of the year.

 

What would be your recommendation?

[Windows7ge]

Storage Spaces using parity the performance is terrible (although I've been told in Windows Server 2019 there has been some improvement) You'd be forced to use RAID10 (two way mirrors) here if you want the performance you're looking for.

[Electronics Wizardy]

3 minutes ago, Windows7ge said:

If you want to keep Windows as the host OS what I would probably try here at least to see how well it works is take advantage of WSL and install ZFS on a Debian distro of Linux creating a raidz1 in bash. I have never tried this before using WSL but in theory it's do-able.

I don't think there is a easy way to do this over than passing disks to vm, then running zfs in there and making a network share, I wouldn't do that.

 

3 minutes ago, Windows7ge said:

Storage Spaces using parity the performance is terrible (although I've been told in Windows Server 2019 there has been some improvement) You'd be forced to use RAID10 (two way mirrors) if you want the performance you're looking for.

New versions are much  better, and if you have a large ssd tier it will not be a issue for most uses.

 

29 minutes ago, Mr.Humble said:

side from a lower-than-expected capacity, it works as intended, but the write speed s u c k s.

Set number of collums to 5 and you should get the space you need.

 

What version of windows are you running?

 

Try chaning the is power protected in storage spaces, it should make it much faster for writes.

[Windows7ge]

3 minutes ago, Electronics Wizardy said:

I don't think there is a easy way to do this over than passing disks to vm, then running zfs in there and making a network share, I wouldn't do that.

I don't actually know how or if Linux registers the disks on the host. I do see you can get outside the Linux folder structure into your Windows C:\ drive which makes me think it's potentially possible but as I said I haven't tried it. I may just have to even if the results are terrible.

 

The reason I offered that suggestion was to potentially improve the parity performance of his RAID5 on Windows without telling him to switch to Linux or to buy a RAID card.

[Electronics Wizardy]

Just now, Windows7ge said:

I don't actually know how or if Linux registers the disks on the host. I do see you can get outside the Linux folder structure into your Windows C:\ drive which makes me think it's potentially possible but as I said I haven't tried it. I may just have to even if the results are terrible.

WSL 1 doesn't show drives to linux, so you can't just make a zfs array. I haven't used WSL2, but it seems to just be a vm with some more intergration, so no direct drive support. So you would need a vm with passthrough. Id use storage spaces over passed through zfs here.

[Windows7ge]

1 minute ago, Electronics Wizardy said:

WSL 1 doesn't show drives to linux, so you can't just make a zfs array. I haven't used WSL2, but it seems to just be a vm with some more intergration, so no direct drive support. So you would need a vm with passthrough. Id use storage spaces over passed through zfs here.

It was just an idea. If you know it was a terrible one then I'll drop it.

[Electronics Wizardy]

Just now, Windows7ge said:

It was just an idea. If you know it was a terrible one then I'll drop it.

Since you can't rut it natively in wsl or windows(easily at least), Your stuck with vms, and at that point, id just get a deticated nas unless op wants to do setup with vms.

[Windows7ge]

6 minutes ago, Electronics Wizardy said:

Since you can't rut it natively in wsl or windows(easily at least), Your stuck with vms, and at that point, id just get a deticated nas unless op wants to do setup with vms.

I wouldn't recommend going that far either. The VM software itself would add overhead. The WSL idea was to have it run "native" that is unless I'm mistaken and WSL is actually Linux running in some type of VM or container to begin with.

[Mr.Humble]

24 minutes ago, Electronics Wizardy said:

Set number of collums to 5 and you should get the space you need.

I'll look into that, currently the Storage Space is set up using the GUI.

 

The total capacity including the parity is around 4,5TB, so it appears it only uses half of the 2TB drive but otherwise it's roughly on point (1TB ~ 931GB as shown etc.) 

28 minutes ago, Electronics Wizardy said:

What version of windows are you running?

Windows 10 Pro 1903

34 minutes ago, Electronics Wizardy said:

Try chaning the is power protected in storage spaces, it should make it much faster for writes.

I assume that needs to be changed in Powershell? I can try it, currently (can post diskmark later) the sequential writes are about 30MB/s writing from my SSD powered PC over Gigabit network.

2 minutes ago, Electronics Wizardy said:

Since you can't rut it natively in wsl or windows(easily at least), Your stuck with vms, and at that point, id just get a deticated nas unless op wants to do setup with vms.

not on that PC

 

I don't have the authority to switch the OS on that machine, the "NAS" needs to stay a part of Windows. I know it's not the best way to go about it but I don't want to spend money on another device and I can't run the Windows as a VM.

 

40 minutes ago, Electronics Wizardy said:

New versions are much  better, and if you have a large ssd tier it will not be a issue for most uses.

Do you think that's my best course of action? Laboring through a Powershell setup and setting it up as a Tiered Storage Space with 1 SSD and 5 HDDs in Parity?

I don't strictly require it to be parity but I would like it to be as capacity efficient as possible.

[Mr.Humble]

2 minutes ago, Windows7ge said:

The WSL idea was to have it run "native" that is unless I'm mistaken and WSL is actually Linux running in some type of VM or container to begin with.

No idea about that, and I have zero experience with bash and very little with actual Linux. The other day I was super happy and pleased about myself for managing to install a program that wasn't in the package manager so... 

[Electronics Wizardy]

7 minutes ago, Mr.Humble said:

I don't have the authority to switch the OS on that machine, the "NAS" needs to stay a part of Windows. I know it's not the best way to go about it but I don't want to spend money on another device and I can't run the Windows as a VM.

 

Can you run anouther os as a vm and use that as the nas? WIndows would still be the host os.

 

8 minutes ago, Mr.Humble said:

Do you think that's my best course of action? Laboring through a Powershell setup and setting it up as a Tiered Storage Space with 1 SSD and 5 HDDs in Parity?

I don't strictly require it to be parity but I would like it to be as capacity efficient as possible.

Probably, but you gotta know know powershell with storage spaces, as the gui sucks.

 

8 minutes ago, Mr.Humble said:

The total capacity including the parity is around 4,5TB, so it appears it only uses half of the 2TB drive but otherwise it's roughly on point (1TB ~ 931GB as shown etc.) 

do you mean 4.5TiB? that still seems wrong, the max you should get from that array with parity is 4tb.

 

9 minutes ago, Mr.Humble said:

I assume that needs to be changed in Powershell? I can try it, currently (can post diskmark later) the sequential writes are about 30MB/s writing from my SSD powered PC over Gigabit network.

What are speeds on the local system. Don't use the network to test file share speeds.

[Mr.Humble]

22 minutes ago, Electronics Wizardy said:

Can you run anouther os as a vm and use that as the nas? WIndows would still be the host os.

The PC has a 4 core Intel i5 and is used as a host for VMware so I doubt it.

23 minutes ago, Electronics Wizardy said:

Probably, but you gotta know know powershell with storage spaces, as the gui sucks.

This and one other thread on this would be my guide. I know the GUI leaves a lot to be desired - literally, I desire more settings - but for the initial setup of "just" a Parity Space it was good practice.

 

25 minutes ago, Electronics Wizardy said:

do you mean 4.5TiB? that still seems wrong, the max you should get from that array with parity is 4tb.

Okay the numbers before are a bit of a fake news, the GUI shows max. capacity of the array at 5.45TB, and the Space is currently 3TB (4.5TB with Parity).

27 minutes ago, Electronics Wizardy said:

What are speeds on the local system. Don't use the network to test file share speeds.

Of course, I'm not that ignorant  Here's the diskmark result:

Spoiler

 

[Electronics Wizardy]

3 minutes ago, Mr.Humble said:

Okay the numbers before are a bit of a fake news, the GUI shows max. capacity of the array at 5.45TB, and the Space is currently 3TB (4.5TB with Parity).

If you set number of collumns right, then you can get 4tb usable

 

3 minutes ago, Mr.Humble said:

The PC has a 4 core Intel i5 and is used as a host for VMware so I doubt it.

How much ram? Cpu won't really matter here. It won't use much more cpu than just doing it on the host.

 

Id personally use hyper-v here.

[Windows7ge]

56 minutes ago, Mr.Humble said:

No idea about that, and I have zero experience with bash and very little with actual Linux. The other day I was super happy and pleased about myself for managing to install a program that wasn't in the package manager so... 

Well, if you need something that's user friendly and Windows based Storage Spaces would still be your best option. You just need to tweak your config for better performance. I have also been told though that Tiered storage has it's own issues. Unless those have also been addressed recently.

[SSD Sean]

You could also use StableBit DrivePool. With it you can pool multiple drives into what looks like a single volume, even with various sizes. You set up duplication, which is like mirroring, but it is based on folders, so it’s more flexible. You get full disk performance. No need for tiered storage, tho I think it has the option. 

 

https://stablebit.com/

 

 

[Mr.Humble]

I ended up going with a tiered storage space - 450GB SSD Simple Tier, 3700GB Parity 5 columm HDD tier.

Initially I set up a storage pool in the GUI, then moved over to Powershell to set up the columms, tiers, and the virtual disk. I was reffering to the topics below and some guides on Microsoft website.

Performance of the previous Parity Space:

 

performance of the Tiered storage space: