These Servers are TOO EXPENSIVE

[AcquaCow]

The main issue with using Storage Spaces to "tier" your data and automatically do anything with it, is that Storage Spaces does not do that.

Yes, you can put SSDs and HDDs together in Storage Spaces, but in order to "tier", you have to essentially run a scheduled script to move specific files up into the SSD "tier."

 

This is fine for write once, read many, assuming you can wait till the script runs to promote the files, but if you are doing constant write/read operations, you have zero control of where those new writes land. There is no write caching and then reading from that cache (currently, that I'm aware of) in Storage Spaces Direct.

 

Example: https://richardjgreen.net/pin-a-file-or-vhd-to-a-storage-space-tier/

 

I fought with this a lot, hoping to use all my leftover Fusion-io hardware with Storage Spaces to make a killer home NAS, but it performed worse than my FreeNAS VM, so I finally just settled on a large FreeNAS build instead and re-purposed the Fusion-io gear for hypervisor datastores.

 

-- Dave

 

[Rudolf Meier]

5 minutes ago, AcquaCow said:

I fought with this a lot, hoping to use all my leftover Fusion-io hardware with Storage Spaces to make a killer home NAS, but it performed worse than my FreeNAS VM...

That was also my finding when I tried to combine SSDs with my HDD-RAID. I also tried to accelerate a raid with an LSI controller and cache cade... but guess what? They don't support NVMe because "it is faster than hdds" ... that's really what they sayed ... so the only option was to combine a "normal" SSD (so, those that have a transfer rate of about 500-700 MB/s) with my HDD-RAID (with a transfer rate of 600-800 MB/s) ... and... sure, lower latency, but I didn't think that's useful. That's why I switched over to Intel CAS... it seems to work pretty well, even with slow SSDs (at least in the tests I made... I'm waiting for some cables but when I get them I will use it in our new servers with NVMe disks). One interesting thing here is, that it also uses RAM as reading-cache. The only problem I can imagine to run into is that my SSDs could be too small. But, we will see... I'm trying it with 1 TB of SSD space per 12 TB of (usable) HDD space.

[leadeater]

4 hours ago, AcquaCow said:

The main issue with using Storage Spaces to "tier" your data and automatically do anything with it, is that Storage Spaces does not do that.

It does do that, you must either use Windows Server GUI to configure actual Storage Spaces Tiering or use PowerShell if using non Windows Server (if Desktop version supports tiering). However PowerShell is always the preferred configuration method for Storage Spaces.

 

Just throwing SSDs and HDDs in to a Storage Spaces Pool and creating a virtual disk is not enough, that isn't tiering. Neither is putting SSDs in to a Pool and setting them to Jounral mode, that isn't tiering either.

 

This is how you configure tiering using the newer ReFS true dynamic tiering, no schedule task that moves hot and cold blocks around:

#Setting storage pool to Power Protected
Set-StoragePool -FriendlyName StoragePool1 -IsPowerProtected $true


#Creating Storage Tiers
$PerformanceTier = New-StorageTier -FriendlyName ProjectData_SSD_Mirror -MediaType SSD -StoragePoolFriendlyName StoragePool1 -ResiliencySettingName Mirror -NumberOfDataCopies 2 -NumberOfColumns 2
$CapacityTier = New-StorageTier -FriendlyName ProjectData_HDD_Parity -MediaType HDD -StoragePoolFriendlyName StoragePool1 -ResiliencySettingName Parity -NumberOfDataCopies 1 -NumberOfColumns 3
 

#Creating Virtual Disk
New-VirtualDisk -StoragePoolFriendlyName StoragePool1 -FriendlyName Test -StorageTiers $PerformanceTier, $CapacityTier -StorageTierSizes 1GB, 10GB 


#Create Volume, use ReFS, get disk number for new virtual disk
New-Volume -DiskNumber [DiskNumber] -FriendlyName ProjectData -FileSystem ReFS -AccessPath P:

Note: Above method done this way because there is a bug creating tiers using other methods where tier settings are not honored. 

 

Even the older way using NTFS is still tiering but it's not dynamic and relies on the scheduled optimization task to move the data between the tiers but any sort of manual file picking is not required, that is completely optional feature that in general is not recommended as it consumes hot tier capacity needlessly.

 

Also like to point out that Storage Spaces tiering is not file level, it's block/chunk level so it does not move files between tiers it moves data.

[Marvin_Nor]

 

9 hours ago, leadeater said:

#Creating Storage Tiers $PerformanceTier = New-StorageTier -FriendlyName ProjectData_SSD_Mirror -MediaType SSD -StoragePoolFriendlyName StoragePool1 -ResiliencySettingName Mirror -NumberOfDataCopies 2 -NumberOfColumns 2 $CapacityTier = New-StorageTier -FriendlyName ProjectData_HDD_Parity -MediaType HDD -StoragePoolFriendlyName StoragePool1 -ResiliencySettingName Parity -NumberOfDataCopies 1 -NumberOfColumns 3

This also introduces resiliency tiering, parity is intended for archive storage in Storage Spaces / S2D. Can cause some latency, the parity in Storage Spaces is greatly impacted by CPU speeds.

Resiliency tiering works on 2016, but it's not all that good for big workloads. Heard it's actually working wonders in 2019, but have no personal experience with it. 

9 hours ago, leadeater said:

Even the older way using NTFS is still tiering but it's not dynamic and relies on the scheduled optimization task to move the data between the tiers but any sort of manual file picking is not required, that is completely optional feature that in general is not recommended as it consumes hot tier capacity needlessly.

Yeah, I think the only reason to pin files is in VDI environments.

[leadeater]

11 minutes ago, Marvin_Nor said:

This also introduces resiliency tiering, parity is intended for archive storage in Storage Spaces / S2D. Can cause some latency, the parity in Storage Spaces is greatly impacted by CPU speeds.

Then change the slow tier to mirror . That's just an example, but honestly get the fast tier large enough and having the slow tier parity won't matter a lot. When it does is when you shouldn't be tiering in the first place, or tiering between really fast and fast.

 

Storage Spaces is also tuned to not eat up CPU resources so Parity is just all round crap period, you must use a Journal device in the pool or Tiering else don't even bother. That's my advice anyway, who wants write performance of less than 80MB/s.

 

11 minutes ago, Marvin_Nor said:

Heard it's actually working wonders in 2019, but have no personal experience with it.

Personally haven't heard much about 2019, went looking very early and wasn't much. Got any links? Might look in to it since I'm super eager for Storage Spaces improvements.

[Marvin_Nor]

On 3/8/2019 at 1:01 PM, leadeater said:

Storage Spaces is also tuned to not eat up CPU resources so Parity is just all round crap period, you must use a Journal device in the pool or Tiering else don't even bother. That's my advice anyway, who wants write performance of less than 80MB/s.

Haha. pretty much. 

 

On 3/8/2019 at 1:01 PM, leadeater said:

Personally haven't heard much about 2019, went looking very early and wasn't much. Got any links? Might look in to it since I'm super eager for Storage Spaces improvements.

There's a lot of it coming from Ignite, and some of the firms we work with also states that 2019 is better.

Here's a link that summarizes some of the 2019 improvements.

https://blogs.technet.microsoft.com/filecab/2018/06/27/windows-server-summit-recap/

 

Of these five, I am personally mostly interested in point three.

 

Here's some good Ignite sessions too:

https://myignite.techcommunity.microsoft.com/sessions/65880

 

[AndyNotts]

I used to work in the video production industry and built a in-house NAS for a couple of edit suites but that was many years ago. We only had to work with HD video and smaller files though. I'm in the biotech industry now, genomics, and the files are slightly bigger 800GB. We're pushing data out to cloud storage and putting the compute in the same datacentre. Our 3 workloads are CAD design, chip design and dna sequencing. We still developing the systems and there are some similarities in the way we process dna data to video editors but without the visualisation. 

 

So my suggestion is look into Azure Files Premium and the cloud compute offerings they have with GPU. I saw a link here on cloud gaming https://medium.com/microsoftazure/a-beginners-guide-to-cloud-gaming-c3e8a5915440 it might get you started but at least it's an idea for a couple of videos.

 

FYI also swaps cost from CAPEX to OPEX and should save some of those $$$

 

 

 

[AcquaCow]

On 3/7/2019 at 8:58 PM, leadeater said:

It does do that, you must either use Windows Server GUI to configure actual Storage Spaces Tiering or use PowerShell if using non Windows Server (if Desktop version supports tiering). However PowerShell is always the preferred configuration method for Storage Spaces.

 

Just throwing SSDs and HDDs in to a Storage Spaces Pool and creating a virtual disk is not enough, that isn't tiering. Neither is putting SSDs in to a Pool and setting them to Jounral mode, that isn't tiering either.

 

This is how you configure tiering using the newer ReFS true dynamic tiering, no schedule task that moves hot and cold blocks around:

#Setting storage pool to Power Protected
Set-StoragePool -FriendlyName StoragePool1 -IsPowerProtected $true


#Creating Storage Tiers
$PerformanceTier = New-StorageTier -FriendlyName ProjectData_SSD_Mirror -MediaType SSD -StoragePoolFriendlyName StoragePool1 -ResiliencySettingName Mirror -NumberOfDataCopies 2 -NumberOfColumns 2
$CapacityTier = New-StorageTier -FriendlyName ProjectData_HDD_Parity -MediaType HDD -StoragePoolFriendlyName StoragePool1 -ResiliencySettingName Parity -NumberOfDataCopies 1 -NumberOfColumns 3
 

#Creating Virtual Disk
New-VirtualDisk -StoragePoolFriendlyName StoragePool1 -FriendlyName Test -StorageTiers $PerformanceTier, $CapacityTier -StorageTierSizes 1GB, 10GB 


#Create Volume, use ReFS, get disk number for new virtual disk
New-Volume -DiskNumber [DiskNumber] -FriendlyName ProjectData -FileSystem ReFS -AccessPath P:

Note: Above method done this way because there is a bug creating tiers using other methods where tier settings are not honored. 

 

Even the older way using NTFS is still tiering but it's not dynamic and relies on the scheduled optimization task to move the data between the tiers but any sort of manual file picking is not required, that is completely optional feature that in general is not recommended as it consumes hot tier capacity needlessly.

 

Also like to point out that Storage Spaces tiering is not file level, it's block/chunk level so it does not move files between tiers it moves data.

This is the last thing I have in my notes from 2017 when I last dug into this:

Quote

#List all disks that can be pooled and output in table format (format-table)
Get-PhysicalDisk -CanPool $True | ft FriendlyName,OperationalStatus,Size,MediaType

$pd = (Get-PhysicalDisk -CanPool $True | Where MediaType -NE UnSpecified)
New-StoragePool -PhysicalDisks $pd –StorageSubSystemFriendlyName “Windows Storage*” -FriendlyName “StoragePool”

$ssdTier = New-StorageTier -StoragePoolFriendlyName "StoragePool" -FriendlyName SSDTier -MediaType SSD
$hddTier = New-StorageTier -StoragePoolFriendlyName "StoragePool" -FriendlyName HDDTier -MediaType HDD

New-VirtualDisk -StoragePoolFriendlyName "StoragePool" -FriendlyName TieredSpace -StorageTiers $ssdtier, $hddtier -StorageTierSizes 1090GB, 7260GB -ResiliencySettingName Mirror -WriteCacheSize 1GB

Get-VirtualDisk TieredSpace | Get-Disk | Initialize-Disk -PartitionStyle GPT
Get-VirtualDisk TieredSpace | Get-Disk | New-Partition -DriveLetter “E” -UseMaximumSize
Initialize-Volume -DriveLetter “E” -FileSystem REFS -Confirm:$false

 

There are a few differences, but back then the tiering really didn't exist in the true sense of the term. 

 

[Results45]

Is there gonna be a video update on resolving LMG's storage issue considering some of the aforementioned ideas/solutions?

 

Also, how affordable (or un-affordable) would it be for LMG to utilize an auto-tiering service provider like QNAP if they choose to wait on buying another storinators' worth of "high-tier" NVMe storage?

[wtflinus]

On 3/4/2019 at 8:42 PM, unijab said:

With your resources Linus.

 

 

I believe you could get ahold of some of those optane dimm modules... ? (no not that optane memory mess intel released first)

edit... the official name is: optane dc persistent memory

 

Use those as a "shit ton" of RAM for ZFS

HOly SH*T.. With those resources and he is testing "enterprise" tech on windows? LMAO.. No wonder he can't get anything done. 

 

Don't get me wrong, love the guy, the things I've seen so far (you go gurl! <3), but ye.. I mean .. get serious. Windows.. 90s? I guess it's his way of hiding he doesn't know much then, just a pretty talker eh? Ah well.