Cache a sata SSD with 16GB optane (primocache)

[ianm_ozzy]

Hi all

 

So my main machine has 2x 256GB nvme for sul boot windows 10/linux and a 1/2 TB nvme for a cache drive.

I  will be upgrading to windows 11, but the drive is working but out of warranty.

 

So I think I may get the crucial  MX500 sata drive for windows 11 -  500GB.  Model no: CT500MX500SSD1

Apparently they have much better write endurance and warranty lengths, compared  nvme of similar prices.

Also it apparently has DRAM cache & a decent slc cache setup.

 

The drive with windows 10 nvme will be put into a server, specifically as an L2arc in truenas scale.

When it fails, no data loss!

 

I have some cheap 16GB intel optanes + some pciex1 adapters. they are cheap.

 

 I did find this interesting:

 At 5.13, it shows  the crystaldiskmark results.

 

The Q1T1 random 4k reads are 159.43MB/s.

That is better than most nand nvmve drives it seems.

 

I already have a primocache personal license.

 

So on windows 11, could have:

1/2TB nvme to cache my games on the NAS ISCSI - doing already.

Sata SSD for windows 11 OS only, cached with 16GB intel optane -  4k block size  & read only cache.

 

I would like to know who else is doing the optane caching in this way for the OS drive?

What is the cache hit rate?

 

 Thanks

 

 

 

 

 

[kokosnh]

Don't do it... 

 

[ianm_ozzy]

1 hour ago, kokosnh said:

Don't do it... 

 

You need to do a bit better than that if trying is dissuade me.

As I see it, will be extremely responsive.

The only question is if the cache hit rate is high enough.

 

 

 

[kokosnh]

Good SSD NVMe are fast enough

 

There are test, look it up. 

Doing optane cache for NVMe SSD is not good for performance, and uneven, as the driver is to small. 

 

And I say that as somebody with optane 905p 960GB OS drive. 

[ianm_ozzy]

43 minutes ago, kokosnh said:

Good SSD NVMe are fast enough

 

There are test, look it up. 

Doing optane cache for NVMe SSD is not good for performance, and uneven, as the driver is to small. 

 

And I say that as somebody with optane 905p 960GB OS drive. 

Here is something from my post:

 

What tests exactly are you talking about?

I have looked up some crystaldiskmrk results for some nvme  nand SSDs

 

So: The Q1T1 random 4k reads are 159.43MB/s.

 

As I understand that is the bottleneck for an OS.

 

Most  are less than that - unless paying much more - which I am not willing to do.

 

I did ask these:

 

'I would like to know who else is doing the optane caching in this way for the OS drive?

What is the cache hit rate?'

 

 

If 16GB optane is too small, could add system memory as cache also.

 

Are you using primocache at all?

If not your replies are of little use to me.

 

 

[kokosnh]

7 hours ago, ianm_ozzy said:

So: The Q1T1 random 4k reads are 159.43MB/s.

 

As I understand that is the bottleneck for an OS.

I literaly have optane 905p 960GB, hooked up directly to the CPU M.2 slot as OS drive, and I get Q1T1 random 4k read of 304 MB/s


You can't get really much faster than that

I also have PNY CS3140 2TB (Phison E18 + 96l TLC micron), that I get Q1T1 random 4k read of 68MB/s off (the best ones, like Samsung 990 pro 2TB  go up to 102MB/s)https://www.tomshardware.com/reviews/pny-xlr8-cs3140-ssd-review/2

It turned off fast startup in windows, I clone my OS, and measured how long the OS takes to start up.

They literally start up the same exactly to the second, I tested it multiple times, measured since POST and everything, it literally takes the same time.  

Yes, the Q1T1 random 4k read, and latency is the final bottleneck, for the OS, but we have literally so fast SSD, that they wait for other things to load, and the difference between them, are very little. 

This is what I meant by saying the modern good NVME SSD are fast enough. 
 

[ianm_ozzy]

7 hours ago, kokosnh said:

I literaly have optane 905p 960GB, hooked up directly to the CPU M.2 slot as OS drive, and I get Q1T1 random 4k read of 304 MB/s


You can't get really much faster than that

I also have PNY CS3140 2TB (Phison E18 + 96l TLC micron), that I get Q1T1 random 4k read of 68MB/s off (the best ones, like Samsung 990 pro 2TB  go up to 102MB/s)https://www.tomshardware.com/reviews/pny-xlr8-cs3140-ssd-review/2

It turned off fast startup in windows, I clone my OS, and measured how long the OS takes to start up.

They literally start up the same exactly to the second, I tested it multiple times, measured since POST and everything, it literally takes the same time.  

Yes, the Q1T1 random 4k read, and latency is the final bottleneck, for the OS, but we have literally so fast SSD, that they wait for other things to load, and the difference between them, are very little. 

This is what I meant by saying the modern good NVME SSD are fast enough. 
 

 

Again I have trouble seeing the relevence of your answer.

 

 

 

 

I ask again do you use or ever used the primocache software?

 

I am not fussed about boot times.

It seems there is a few seconds difference between a sata SSD & nvme.

I watched a few youtube videos, and they seem consistent.

 

A 990 pro 2TB as boot drive - that seems extremely wasteful.

 

 The Sata SSD I have in mind - MX500 - DRAM cache + 180GB TBW value.

Similar prices nvme drives are highly dubious. Lower TBW vales, no DRAM cache  and  crystaldiskmark  results are very elusive.

 

Maybe a 980 pro for twice the price, which I expect would work well.

 

Or the MX500 + 16Gb optane as a cache eliminating the random 4k bottleneck  I could have (16GB read cache + maybe around 2GB write cache using memory)

 

The samsung 980 pro 4k random Q1T1 is  89.95MS/s and the optane 16GB is 160Mb/s

 

Oh and this is useful.

 

https://forum.romexsoftware.com/en-us/viewtopic.php?p=16853#p16853

 

 

 

 

 

 

 

[LIGISTX]

Don’t cache an nvme ssd with Optane. You don’t gain anything for any type of typical workload. 
 

If you have a very specific use case or need where you think it would be helpful, then you probably already know why you would need it and what it would do for you. If you are asking if you need to do it, you don’t. 
 

The reason for this is a normal NVME drive already has very high bandwidth, and very low latency. Adding more steps to this (caching software which then needs to be queried, and upon a successful hit it would result in the Optane providing the data, and upon a miss the nvme would provide the data), is just adding latency to the loop and will almost certainly cause increased overall latency in almost all cases. 
 

If you REALLY want to use Optane simply because you have it laying around, use a ~64 GB Optane drive as your windows boot device, and then use an nvme drive for programs and such. This would at least fully separate all of the “busy work” of the OS and shove that onto Optane, and allow your nvme drives to have much less random writes and reads hitting them. But even this… you wouldn’t be able to tell any difference in day to day use. Nvme is already so fast and capable, you are worrying about things that just don’t matter.  

[ianm_ozzy]

On 4/18/2024 at 3:33 AM, LIGISTX said:

Don’t cache an nvme ssd with Optane. You don’t gain anything for any type of typical workload. 
 

If you have a very specific use case or need where you think it would be helpful, then you probably already know why you would need it and what it would do for you. If you are asking if you need to do it, you don’t. 
 

The reason for this is a normal NVME drive already has very high bandwidth, and very low latency. Adding more steps to this (caching software which then needs to be queried, and upon a successful hit it would result in the Optane providing the data, and upon a miss the nvme would provide the data), is just adding latency to the loop and will almost certainly cause increased overall latency in almost all cases. 
 

If you REALLY want to use Optane simply because you have it laying around, use a ~64 GB Optane drive as your windows boot device, and then use an nvme drive for programs and such. This would at least fully separate all of the “busy work” of the OS and shove that onto Optane, and allow your nvme drives to have much less random writes and reads hitting them. But even this… you wouldn’t be able to tell any difference in day to day use. Nvme is already so fast and capable, you are worrying about things that just don’t matter.  

Cache an nvme SSD with optane?

When did I say I wish to to that?

I am thinking of using an inexpensive, but good quality  sata SSD cached with a  super cheap 16GB optane.

 

For the Q1T1 random 4k reads, seems to make a lot of sense.

 

So by going by  a standard crystaldiskmark result  for reads comparing them:

 

Q32T1 - about 2 x the number, but almost pointless in a windows OS drive.

Q8T8 & Q32T1 - about the same.

Q1T1 - about 4x the speed from about 40Mb/s to 160MB/s (faster than most NAND SSDs)

It seems about 2/3 of windows I/O is random 4k, so makes sense to me.

 

A windows OS, only, where I use windows only for gaming , where games are all on another drive, also makes sense to me.

 

The cache hit rate will be  the most important value, but primocache is exceptionally good at maximizing that.

 

 

 

 

[LIGISTX]

9 hours ago, ianm_ozzy said:

Cache an nvme SSD with optane?

When did I say I wish to to that?

I am thinking of using an inexpensive, but good quality  sata SSD cached with a  super cheap 16GB optane.

 

For the Q1T1 random 4k reads, seems to make a lot of sense.

 

So by going by  a standard crystaldiskmark result  for reads comparing them:

 

Q32T1 - about 2 x the number, but almost pointless in a windows OS drive.

Q8T8 & Q32T1 - about the same.

Q1T1 - about 4x the speed from about 40Mb/s to 160MB/s (faster than most NAND SSDs)

It seems about 2/3 of windows I/O is random 4k, so makes sense to me.

 

A windows OS, only, where I use windows only for gaming , where games are all on another drive, also makes sense to me.

 

The cache hit rate will be  the most important value, but primocache is exceptionally good at maximizing that.

 

 

 

 

Sorry, read the post wrong, didn’t notice the SATA SSD would be for boot and what you are caching. 
 

I still doubt it’s worth it for the same reasons I identified. But you can give it a try I suppose. 

[ianm_ozzy]

On 4/20/2024 at 2:10 AM, LIGISTX said:

Sorry, read the post wrong, didn’t notice the SATA SSD would be for boot and what you are caching. 
 

I still doubt it’s worth it for the same reasons I identified. But you can give it a try I suppose. 

I forgot about an 'unused' 256GB basic sata SSD I have in  my server.

I could not cut it.

Using it as a boot drive & for virtual machines was a lot to ask for a use in proxmox

I guess write speeds & trim were an issue.

It is the craptastical Teamgoup CX 256MB.

The random 4k reads are about 20MB/s.

I will use that it think, and the 16GB optane cache will certainly speed it up.

So very little  cost, but need to change out drives.