Apple Patent May Hint at the Future of Its Chips: A Multi-Level Hybrid Memory Subsystem

[Lightwreather]

Disclaimer: I do not know if someone else has done this, I searched through the forum and I have not found this, also I believe this qualifies as tech news, correct me if I'm wrong.

Summary

Apple has patented a hybrid memory subsystem that comprises at least two types of memory: a high-bandwidth low-density type of DRAM as well as a low-bandwidth high-density type of DRAM. The patent may provide a glimpse of how Apple sees the future of its system-on-chips.

One of the schematics filed in the patent.

 

Quotes

Quote

Apple has patented a hybrid memory subsystem that comprises at least two types of memory: a high-bandwidth low-density type of DRAM as well as a low-bandwidth high-density type of DRAM. The patent may provide a glimpse of how Apple sees the future of its system-on-chips. Naturally, the new patent will fuel speculation that we could see newer versions of Apple's M1 chip come with the new memory design, but this type of implementation could be used in several different types of chips.

Patents don't always manifest as products on the market, but given the feedback we've received from patent lawyer Kerry Creeron, it appears that Apple has gone to great pains, and expense, to patent this new technique in a wide array of jurisdictions around the world, implying that it could be used in Apple's products worldwide. 

HBM2 and HBM2E types of memory provide very high bandwidth, but these memory devices cost a lot, consume a lot, and cannot be upgraded by the end-user. In contrast, building a high-capacity memory subsystem with enough bandwidth for a high-end GPU using conventional or GDDR-type memory isn't always possible, or feasible. In a bid to combine the best of both worlds, Apple has patented a hybrid memory subsystem combining HBM-like and DDR-like types of DRAM."With two types of DRAM forming the memory system, one of which may be optimized for bandwidth and the other of which may be optimized for capacity, the goals of bandwidth increase and capacity increase may both be realized, in some embodiments," a description by Apple reads. "[Some implementations of hybrid memory subsystems may] to implement energy efficiency improvements, which may provide a highly energy efficient memory solution that is also high performance and high bandwidth.""This patent was filed in a wide array of jurisdictions, including the EP (European Patent Org.), U.S., China, and Japan," said Kerry Creeron, an attorney with the firm of Banner & Witcoff. "Such a wide filing strategy is expensive and typically only makes sense if IP protection is important."

 

My thoughts

This is an rather interesting development. Now I'm still very much a noob when it comes to tech and especially RAM, but it seems to me that this kinnd of memory system might be more efficient and it seems like CPU cache just on the SOC shared by the CPU and GPU with a slightly larger capacities. But whatever the cse maybe, It does make apple and their Apple Silicon macs worth watching.

 

Sources

https://www.tomshardware.com/news/apple-patent-hybrid-memory-subsystem-m1-processor

https://patents.google.com/patent/US10573368B2/

[YourRandomForumGuy]

IMHO, that's awesome and all, but I feel Apple has hugely affected stock of anything based on silicon by giving a massive amount of work to TSMC.

[Orangeator]

Very interesting idea, would there be some kind of cache for the difference in speed between the two types of memory.

 

Edit: Now that I think about it, would the HBM just serve almost like a super highspeed cache for the DDR memory?

[Vishera]

This move is just to cut costs,Apple wants to integrate the cheapest slowest RAM out there into their products.

1 hour ago, J-from-Nucleon said:

high-bandwidth low-density type of DRAM

The expensive stuff,good stuff.

1 hour ago, J-from-Nucleon said:

low-bandwidth high-density type of DRAM

The cheap,good for nothing stuff.

 

It's like using 3200MHz CL16 for applications that require speed alongside 2133MHz CL22 for idle applications.

It's worse than what Apple use now,and not to mention that the prices of Apple products are already expensive with huge profit margins.

 

What if you need all the RAM you have but also need high speed?

Apple are introducing new problems that we have never seen before.

 

That's definitely a bad thing.

[Nineshadow]

35 minutes ago, Vishera said:

This move is just to cut costs,Apple wants to integrate the cheapest slowest RAM out there into their products.

The expensive stuff,good stuff.

The cheap,good for nothing stuff.

 

It's like using 3200MHz CL16 for applications that require speed alongside 2133MHz CL22 for idle applications.

It's worse than what Apple use now,and not to mention that the prices of Apple products are already expensive with huge profit margins.

 

What if you need all the RAM you have but also need high speed?

Apple are introducing new problems that we have never seen before.

 

That's definitely a bad thing.

It depends on the execution. I didn't read all the specifics of the patent exactly.

 

Another somewhat similar examples are QLC SSDs. QLC store 4 bits of information per cell and is much slower. Most drives offer an SLC cache for decent performance. Are QLC SSDs bad? Not necessarily, they provide a larger storage space at a really low price.

[Vishera]

4 minutes ago, Nineshadow said:

Are QLC SSDs bad?

YES,They are.

But for a different reason (endurance).

 

It's bad because you have less "high performance" RAM to use,

There is also the fact that Apple are simply downgrading the RAM into a stupid hybrid system,and all for the sake of cost cutting.

[leadeater]

46 minutes ago, Vishera said:

What if you need all the RAM you have but also need high speed?

Apple are introducing new problems that we have never seen before.

 

That's definitely a bad thing.

Well it's working for RDNA2 with it's on-die cache along with GDDR6. Really depends on what is actually being proposed and planned, this patient is super broad and AMD already has similar patient and products in the market that use them, not sure how that affects each others patient though. Last thing we need is these two companies getting in to a patient war with each other.

 

If Apple wants much bigger SoCs, either or both CPU and GPU wise, they are going to need much higher bandwidth to support it and memory types like HBM and GDDR have rather bad latency so you wouldn't actually want only these as it'll affect CPU performance. The larger the memory chip, capacity wise, the higher the access latency and the same is also generally true of higher bandwidth wider memory bus memory too.

 

It's really not so bad of an idea, but that swings many ways and execution of this will entirely make it good or bad.

[Vishera]

8 minutes ago, leadeater said:

Well it's working for RDNA2 with it's on-die cache along with GDDR6. Really depends on what is actually being proposed and planned, this patient is super broad and AMD already has similar patient and products in the market that use them, not sure how that affects each others patient though. Last thing we need is these two companies getting in to a patient war with each other.

 

If Apple wants much bigger SoCs, either or both CPU and GPU wise, they are going to need much higher bandwidth to support it and memory types like HBM and GDDR have rather bad latency so you wouldn't actually want only these as it'll affect CPU performance. The larger the memory chip, capacity wise, the higher the access latency and the same is also generally true of higher bandwidth wider memory bus memory too.

 

It's really not so bad of an idea, but that swings many ways and execution of this will entirely make it good or bad.

It's a good point your are making,so you are saying that in addition to the RAM they use now,they can make it better by replacing some of it with something better.

And that's why you say that it can be good or bad depending on the execution.

 

They can use your example which is the good,or my example which is the bad.

 

I hope that they will use better RAM,and not worse.

[porina]

My first reaction to this was, there's nothing new? Then I looked deeper at the patent and they do limit the scope in the body than the summary would imply.

 

Having different performing "ram" solutions is nothing new. I forget if it was the current or previous generation xbox that had two regions of ram with different performance, I'd guess something to do with the quantity not all having equal connectivity. By that measure, similar could be said for PCs with unequal capacities in ram channels, or even the GTX 970 thing. Then there's Intel's Optane Persistent Memory DIMMs, which are added in addition to SDRAM.

 

However none of the above fit within the scope of Apple's Patent. They state that the two types of DRAM chip are different performance characteristics. So same ram connected differently rules out the xbox, unbalanced ram channels and GTX 970 scenarios. Optane as RAM would be a different type, but still doesn't fit within Apple's patent for two reasons I can see. One, xpoint I don't think is counted in the DRAM family even if it can be used in a similar way. Taking the example to the extreme, you could use a page file on a hard disk as "RAM", but it still isn't DRAM. The other is in the physical construction, but I really suck at interpreting patent speak so this is much less clear to me.

 

The patent also makes specific comparisons between the two types of ram used, implying it was written with two existing types in mind, without saying what they are. They refer to first and second types of ram. Following are quotes from the patent

• a second memory array in the second type of DRAM is less dense than a first memory array in the first type of DRAM
• accesses to the second memory array in the second type of DRAM are lower in energy consumption than accesses to the first memory array in the first type of DRAM
• the second type of DRAM includes fewer memory cells per word line than the first type of DRAM
• the second type of DRAM is 6×-8× less dense than the first type of DRAM
• the second type of DRAM is 4×-16× less dense than the first type of DRAM

The last two lines might seem to say the same thing with different numbers, were taken from different parts of the text and may refer to different possible implementations.

 

BTW since many have mentioned HMB, I personally doubt that is applicable here. It really isn't suited to general CPU usage scenarios. It is high bandwidth but also higher latency. To get the most of that bandwidth you need to have large sustained transfers. However, if the the SoC includes some kind of GPU, then having that HBM could be a big boost.

 

This also makes me wonder, how many types of RAM are there?

SRAM

DRAM

HBM

I don't know if GDDR is different enough to be considered their own category from DRAM. Any other types I might have overlooked?

[leadeater]

1 hour ago, porina said:

GTX 970 scenarios

I don't know, the 970 to me seems like it fits in just the same. It had two different DRAM types connected to the same GPU die. Just neither were a cache for the other, the last little bit was more a spill over area in an attempt to prevent even worse performance loss than using shared virtual memory pages in system memory which sucks even harder.

 

1 hour ago, porina said:

I don't know if GDDR is different enough to be considered their own category from DRAM.

Yes, the only similarity between the two is they both use double data rate. The signaling and data handling between the two are entirely different.

 

1 hour ago, porina said:

The patent also makes specific comparisons between the two types of ram used, implying it was written with two existing types in mind, without saying what they are. They refer to first and second types of ram. Following are quotes from the patent

If I were to take a guess now with zero information I would be placing bets on LPDDR and DDR, right now and it even looks like future LPDDR memory is faster but it costs more. LPDDR4X is about 40% faster than DDR4 3200. 5th generation ram looks to be even more in favor of LPDDR.

 

[Spindel]

hmmm...

 

So FusionDrive but for RAM.

[porina]

1 hour ago, leadeater said:

I don't know, the 970 to me seems like it fits in just the same. It had two different DRAM types connected to the same GPU die. Just neither were a cache for the other, the last little bit was more a spill over area in an attempt to prevent even worse performance loss than using shared virtual memory pages in system memory which sucks even harder.

To my understanding all the ram on the 970 was the same. The performance difference was due to limited bandwidth connectivity to that last bit due to the way the GPU was partitioned.

 

1 hour ago, leadeater said:

If I were to take a guess now with zero information I would be placing bets on LPDDR and DDR, right now and it even looks like future LPDDR memory is faster but it costs more. LPDDR4X is about 40% faster than DDR4 3200. 5th generation ram looks to be even more in favor of LPDDR.

I had forgotten about LPDDR since it only sometimes appears in some laptops. Presuming it is also different enough from what we know as ram, makes me wonder what the difference is to allow it to work faster, and why aren't we using it more widely? Cost?

[Bombastinator]

9 hours ago, Vishera said:

This move is just to cut costs,Apple wants to integrate the cheapest slowest RAM out there into their products.

The expensive stuff,good stuff.

The cheap,good for nothing stuff.

 

It's like using 3200MHz CL16 for applications that require speed alongside 2133MHz CL22 for idle applications.

It's worse than what Apple use now,and not to mention that the prices of Apple products are already expensive with huge profit margins.

 

What if you need all the RAM you have but also need high speed?

Apple are introducing new problems that we have never seen before.

 

That's definitely a bad thing.

Depends on how they implement it but that does seem to be a possibility.

[StDragon]

So the TLDR is this:

 

CPU has L1, L2, and L3 cache (memory).

RAM would traditionally be L4. But now Apple is segmenting that further into L4, L5

SSD would be L6 for the page file.

 

Is that about right?

[Sauron]

So just off-chip cache...?

 

It's not a bad idea but it just lowers the platform's modularity even more - desktop macs may have to come with soldered or proprietary memory just like mobile macs.

[Guest]

"Wow, another amazing technical acheievement by Apple."

Spends £2k on a new top spec macbook.

"Let's play some games!"

Loads up Geforce Now

[Bombastinator]

A lot of assumptions seem to being made about whether this is a way to make things cheaper and crap, or a way to make things better.  Both are based on assumptions about which way Apple jumps on this one.  I’m willing to take “it could go either way and makes things more complex by effectively adding another “L” but without knowing which way they are going to jump its hard to pass judgement.  Suffice it they could jump in either direction. 

[Spindel]

15 hours ago, StDragon said:

So the TLDR is this:

 

CPU has L1, L2, and L3 cache (memory).

RAM would traditionally be L4. But now Apple is segmenting that further into L4, L5

SSD would be L6 for the page file.

 

Is that about right?

In essence yes.