Topping up on storage - Intel optane news

[williamcll]

While the CPU branch continues to struggle, Intel's Storage development has made steady progress with their recent announcement including new PCIe 4.0 SSDs and PLC drives. They've announced a few devices recently.

Quote

Compared to the first-gen Optane DC P4800X, Intel says the 5800X offers an impressive 3X performance improvement in random read/write workloads, peaking at 1.5 million 4K random IOPS, and 3X more sequential performance at a peak of 7.2 GB/s. The drive also delivers up to 4.6 million IOPS in random 512B workloads (useful for certain types of caching workloads) and up to 1.8 million IOPS in mixed workloads. Additionally, the drive offers up to 67% more write endurance than the first-gen P4800X, and now peaks at a staggering 100 drive writes per day (DWPD). That also makes it the most endurant SSD on the market. For perspective, there are very few 'regular' NAND-based SSDs that offer even 10DWPD of endurance, meaning the P5800X is truly in a class of its own.

 For the Consumer, there's the 670p:
 

Quote

The SSD 670p still leverages the PCIe 3.0 interface, which will inevitably lead to reduced throughput compared to competing PCIe 4.0 SSDs, but it comes armed with a next-gen Intel SSD controller that the company hasn't fully detailed yet. We do know that the SSD comes in 512GB, 1TB, and 2TB flavors and supports Pyrite 2.0 security, end-to-end data protection, and endurance ratings of 150 terabytes written (TBW) per 512GB of capacity. That puts the range-topping 2TB model at an acceptable endurance of 600 TBW, matching the previous-gen 665p. Intel refined the 670p's dynamic cache, which helps boost both endurance and performance by using an adaptive amount of SLC-programmed flash to absorb data headed to the underlying QLC flash, by increasing the amount of available cache when the drive is full. Intel has improved the cache capacity by 11% when the drive is more than 35% full. As before, the drive keeps a pre-programmed amount of static cache regardless of drive fill, as outlined in the final slide in the above album. 

 

For regular Optane drives, there's the updated H20:

Quote

Starting with the Intel Optane Memory H20, which is an update to the Optane Memory H10 announced at CES 2019. Like its predecessor, it combines Intel's high-speed Optane Memory with traditional flash on a single PCIe M.2 module. It is aimed at slim and light notebooks and ultra-compact PCs which would not otherwise have space for multiple storage tiers. It will be marketed for performance and productivity as well as gaming and content creation. The two types of flash storage have their own controllers and split four PCIe 3.0 lanes between them. They do not directly interact with each other, but are tiered into a single logical drive using Intel's software. The Optane Memory caches important data to accelerate reads and writes while the NAND flash is used for storage as usual.

For other enterprise users, their big ruler design is also now made to work with PCIe 4.0

Quote

The first two SSD announcements are updates to Intel's datacenter SSDs using 3D NAND. The new D7-P5510 uses 144L 3D TLC NAND and is the successor to the D7-P5500 which uses 96L TLC. Since the P5500 was an OEM-only product rather than widely distributed through the channel, the P5510 will also serve as the successor to the P4510 for the portion of the customer base. Intel has not announced a 144L replacement for the D7-P5600, the higher-overprovisioning counterpart to the P5500.

Using Intel's 144L QLC NAND is the new D5-P5316 SSDs in 15.36 TB and 30.72 TB capacities in either U.2 or E1.L form factors. The E1.L version allows Intel to achieve the original goal of the "Ruler" form factor by enabling 1PB of storage in a 1U server. Since the P5316 is replacing the older P4326 (64L QLC and PCIe gen3), it's a much more substantial upgrade over its predecessor than the TLC-based P5510 is. Aside from the introduction of Intel's third-generation enterprise NVMe SSD controller to their QLC product line, the most important change the P5316 brings is a major shift in how the Flash Translation Layer works. The P5316 enables a 16x reduction in DRAM by changing the SSD's Flash Translation Layer to work with a granularity of 64kB rather than 4kB.

 Both the TLC-based P5510 and QLC-based P5316 use the same controller platform as Intel's other P5000-series SSDs announced earlier this year. Those drives introduced Intel's third-generation enterprise NVMe SSD controller, their first supporting PCIe 4.0. The new 144L drives don't really push performance or feature set any further, but will be more widely available and should be cheaper than the 96L drives. The TLC-based P5510 has already been sampled to customers for qualification and will be shipping for revenue by the end of this year. The QLC-based P5316 is sampling and will be available in the first half of 2021.

If someone can get me photos of their new persistent DIMMs that would be nice.
Source: https://www.anandtech.com/show/16318/intel-announces-new-wave-of-optane-and-3d-nand-ssds

https://www.tomshardware.com/news/intel-debuts-worlds-fastest-ssd-the-pcie-40-optane-ssd-p5800x

https://gadgets.ndtv.com/laptops/news/intel-optane-memory-h20-ssd-670p-data-centre-ssds-announced-qlc-144-layer-persistent-memory-2339916

Thoughts: It's great, but the 600 series has always seen as a last option budget when cheaper drives like Micron, Verbatim and Crucial aren't available. That U.2 Drive however is something that I would love to get my hands on.

[Mling]

RIP magnetic storage. You were fun. Especially when you came in 10k rpm and high performance drives that sound like jack hammers.

[porina]

11 minutes ago, williamcll said:

Thoughts: It's great, but the 600 series has always seen as a last option budget when cheaper drives like Micron, Verbatim and Crucial aren't available.

Might vary with local market, but where I am the 600p was for a long time by far the cheapest NVMe SSD per-capacity from a brand you've ever heard of, helped in part it was one of the first QLC drives to be widely available. Obviously not a drive for those seeking particularly write performance, but quite often it was cheaper than SATA drives even. The later ones in series didn't seem to make as much impact as the competition increased.

 

I'm still curious about Optane options but unless pricing moves a lot closer to flash SSDs I'm not sure I'll get another one.

[StDragon]

3 hours ago, comander said:

The optane advances are what's exciting to me. 

 

This stuff is "close enough" to RAM on latency (think 200ns vs 50ns) for a lot of things. If 80% of your operations are in DRAM and having 4x as much Optane as DRAM allows you to avoid the HUGE hit of going to storage, then you're ahead a fair amount. 

Too bad it's expensive though.

 

For large SANs, it's better than NVDIMM for cache buffering in the event of loss of power (redundant circuits, UPS, and generators should be used, but accidents do happens) 

[WhitetailAni]

15 hours ago, Mling said:

RIP magnetic storage. You were fun. Especially when you came in 10k rpm and high performance drives that sound like jack hammers.

No, 3600 RPM drives that sounded like a chainsaw!

[Granular]

23 hours ago, Mling said:

RIP magnetic storage. You were fun.

X-point doesn't compete with magnetic storage in any segment, NAND does. And even NAND doesn't compete with magnetic tape.

 

[Roswell]

300 write cycles before failure sounds abysmal.

[Dylanc1500]

I think the 3.2TB P5800X will be my next purchase. Then we wait for the consumer variant to come up, and hopefully releasing a HHHL version.

[Rohith_Kumar_Sp]

On 12/18/2020 at 10:04 PM, Mling said:

RIP magnetic storage. You were fun. Especially when you came in 10k rpm and high performance drives that sound like jack hammers.

 

[RejZoR]

On 12/18/2020 at 5:34 PM, Mling said:

RIP magnetic storage. You were fun. Especially when you came in 10k rpm and high performance drives that sound like jack hammers.

Not quite. For main storage, sure, I've been on SSD for many years. Used one since when Intel X25M was a thing. But for bulk storage, HDD's are still a thing. 8TB HDD for I don't know what it was, 150€ ? 5400 RPM variant is quiet and still fast enough for archiving or bulk transfers from the internets. There is no way you can get SSD of that capacity for such price.

[Drama Lama]

At least one of their sections is doing well

[Kisai]

On 12/18/2020 at 3:55 PM, comander said:

The optane advances are what's exciting to me. 

 

This stuff is "close enough" to RAM on latency (think 200ns vs 50ns) for a lot of things. If 80% of your operations are in DRAM and having 4x as much Optane as DRAM allows you to avoid the HUGE hit of going to storage, then you're ahead a fair amount. 

200ns is like, 386-era memory. So that's a still 30+ years away

 

I question the durability of Optane to be used in this way. Persistent memory may eventually come along where volatile memory (eg DDR4, L1/L2/L3 cache) requires an explicit "reset" to clear it and reboot the computer.

 

However there's quite a few OS obstacles that need to be changed to get there, like removing the dependency on page files, live-patching instead of rebooting, temporary files not being scattered all over the drive, and such.

[porina]

9 hours ago, Kisai said:

I question the durability of Optane to be used in this way. Persistent memory may eventually come along where volatile memory (eg DDR4, L1/L2/L3 cache) requires an explicit "reset" to clear it and reboot the computer.

 

However there's quite a few OS obstacles that need to be changed to get there, like removing the dependency on page files, live-patching instead of rebooting, temporary files not being scattered all over the drive, and such.

I had wondered in the past if Optane could be a unified tier combining the roles currently played by ram and flash. OS changes like those mentioned would be required to optimise for it so maybe not something Windows would run any time soon but more targeted use cases. I don't see it as a high performance ram-substitute though. Advantage would be simplicity, since I'd assume Optane replacing flash will cost more than the savings of Optane replacing ram.

 

I also tried to look up Optane and ram cycle life. Didn't get a good number for either although I'm sure I saw some in the past. Intel says ">10x flash" for Optane, which is a pretty low bar and I'm sure it was practically much more. If you divide the P4800X endurance by capacity, I make that over 100,000 cycles, but may include over-provisioning. I'm pretty sure I saw a ram cycle life in the past, but the number was essentially so high it was practically infinite.

[Kisai]

16 hours ago, porina said:

I had wondered in the past if Optane could be a unified tier combining the roles currently played by ram and flash.

That's how Windows CE worked before flash memory became cheap enough. Everything ran from device memory, only the OS was on the device flash memory, and couldn't be updated. Even some late Windows Mobile 2003SE devices had like just enough flash memory on board to back up the RAM, but if you didn't use it when you replaced the batteries (or let the batteries run to zero) the device was effectively restored to factory settings.

 

In theory, yes, it was a much nicer, clean design, everything was instant-on, but power requirements of needing to last more than a day on battery favors flash memory and reloading things from flash if the battery dies rather than needlessly losing the users data. So at best, the way to fix this on phones, tablets and laptops is to have the system RAM and a dedicated equal amount of "optane" type solid state memory on the device, allow over-provisioning of the system memory to use the optane memory instead of "paging" to disk, and when the system needs to suspend/sleep compress the system memory and write it to the optane memory and go into a zero power state.

 

[leadeater]

On 12/21/2020 at 2:27 PM, Kisai said:

like removing the dependency on page files

You know you can turn page file off right? I've run Windows without one way back in Windows 2000 Professional, was great if you had enough ram due to SSDs not being a thing.

 

On 12/21/2020 at 2:27 PM, Kisai said:

200ns is like, 386-era memory. So that's a still 30+ years away

These and Optane DIMMs for that matter are replacements for RAM disks and data persistence in memory not for system memory. These latencies are so much faster than other persistent storage that it is enough to replace the use cases for RAM disks and persistent memory resident data, safer too.

 

They are also good targets for high end enterprise storage controllers that currently use NVDIMMs and NVMe read cache devices, Optane can replace both of these and for NVDIMM at larger capacities while retaining overall storage system performance. I'd rather have 1.5TB of Optane DIMM than 0.5TB NVDIMM.

[DavidC2]

On 12/21/2020 at 3:41 AM, porina said:

I also tried to look up Optane and ram cycle life. Didn't get a good number for either although I'm sure I saw some in the past. Intel says ">10x flash" for Optane, which is a pretty low bar and I'm sure it was practically much more. If you divide the P4800X endurance by capacity, I make that over 100,000 cycles, but may include over-provisioning. I'm pretty sure I saw a ram cycle life in the past, but the number was essentially so high it was practically infinite.

The Optane DIMM modules are rated for 5 year 24/7 usage at maximum write speeds, and Intel engineers say it'll last much beyond that.

 

The Optane PMEM 200 series 256GB is rated at 497PBW, which equals to over 1.9 million cycles, which is equal to 24/7 operation at its maximum 3.15GB/s write speed for 5 years. That's nearly 1100 DWPD.

 

The 10X endurance and latency comparisons are for the Optane SSDs. The read latency for the DIMM modules are several hundred times lower and endurance even compared to most enterprise SSDs are 100x better. Against client it does come close to their original claims of 1000x more endurance and 1000x lower latency.

 

And this is for the first gen Optane technology. Considering how the P5800X is the first device with the second generation tech, and how much the performance and endurance has improved, I expect great things for the Optane PMEM 300 which should be the first DIMM version to come with second-gen XPoint.

 

The only issue with the P4800X/900/905P was that it had sequential numbers not saturating the PCIe 3.0 bus. The P5800X does not have the problem.

[Kisai]

20 hours ago, leadeater said:

You know you can turn page file off right? I've run Windows without one way back in Windows 2000 Professional, was great if you had enough ram due to SSDs not being a thing.

I've been running my systems with page file off for a decade. The only times it's ever been a problem is when doing things (like video editing, hundred+ layer photoshop files and such), debugging C++ programs is a particularly angry use case where the page file is necessary.

 

Quote

 

These and Optane DIMMs for that matter are replacements for RAM disks and data persistence in memory not for system memory. These latencies are so much faster than other persistent storage that it is enough to replace the use cases for RAM disks and persistent memory resident data, safer too.

 

They are also good targets for high end enterprise storage controllers that currently use NVDIMMs and NVMe read cache devices, Optane can replace both of these and for NVDIMM at larger capacities while retaining overall storage system performance. I'd rather have 1.5TB of Optane DIMM than 0.5TB NVDIMM.

Yep, they have use cases where you want zero downtime, but I still don't see these being viable as "replacement for system memory", and when they're placed in DIMM slots, this is just a work-around for getting more bandwidth, as plugging these into M2 PCIe 4.0 x 4 lanes has a peak bandwidth that is too slow to utilize it efficiently.

[leadeater]

2 hours ago, Kisai said:

Yep, they have use cases where you want zero downtime, but I still don't see these being viable as "replacement for system memory", and when they're placed in DIMM slots, this is just a work-around for getting more bandwidth, as plugging these into M2 PCIe 4.0 x 4 lanes has a peak bandwidth that is too slow to utilize it efficiently.

By placing it in the DIMM slots you get massively more bandwidth and they can operate in memory mode or storage mode. When in memory mode they are used as a bulk memory layer and the DDR4 memory is used as a cache for it. It's not a replacement for system memory, it's a mistake to try and think of it like that.

 

When you are using it in memory mode you are doing so because your workload required the entire dataset to be in memory or it will not run, that does not actually mean that entire dataset is actually active and it is so much faster to not have to go through another subsystem layer to pull data in to memory when it is already in memory, just that the memory pages are in a slower memory and can be raised in to faster memory as needed more quickly and efficiently than any other way.

 

Additional to that not everything actually requires extremely low latency, some just want raw sustained bandwidth.