Hey Linus,
I work with Storage Spaces and Storage Spaces Direct at my work place.
If you'd like some help with setting up a Storage Spaces or S2D (Storage Spaces Direct) environment, let me know.
There are a lot of documented best practices, but there are also some field best practices regarding how to setup Storage Spaces for good performance. You could run into huge bottlenecks when it's not configured as it should be, or using wrong hardware.
There is also some hardware requirements you should meet, especially for disks, so that you don't run into throttle issues. This is due to disk firmware in most cases (been cases where people have seen their performance increase by four times due to a firmware upgrade on disks). Some issues related to disk can be the disk sizes, if you under budget the cache tier, you'll run into performance issues. Rule of thumb is that cache should be 15% of cold data in size (before mirroring/parity).
Also, ReFS vs NTFS is something to look into. ReFS uses Block Cache (in memory cache) ontop of the Cache Tier (SSD or NVMe).
If you're going to invest in new hardware, a recommendation would be 2 x S2D servers (as a minimum) setup in a 2-way mirror.
Also, to shed some light on the way Storage Spaces work with cache: All writes are done to the fastest tier first; NVMe > SSD > HDD.
NVMe will work as write cache for SSD and HDD.
NVMe will only work as read cache for HDD, it's not a read cache for SSD.
SSD will work as both write and read cache for HDD.
Data will be cached based on a usage algorithm. So it will only shuffle files down from cache when they're rarely used, or when a file on the slow tier is consistently used more than a file that is in the cache tier. This also means all new written files will be cached for a while. The cache will by default also shuffle out files when it's filled up by 70%, so that it got free room for new writes.
Storage Spaces / S2D doesn't really tier data either, unless you have a three tier setup; NVMe, SSD and HDD. With only two "tiers" one will be cache and the other will be storage.
In a three tier solution, NVMe will be cache, SSD will be fast tier and HDD will be cold tier.