Is upgradeable memory finished?

[Thomas A. Fine]

Apple m* chips and the new AMD Strix Halo both seem to tell us that you can make a system faster if you can get the memory closer to the CPU.  (And the GPU also).  And so they're putting this chips into boards where the memory is permanent.

Is anybody working on removable ram packages and motherboard designs that let you have memory modules that can be put very close to the CPU with very precise timings?

 

I'm imagining memory upgrades in packages the size of micro SD cards that slot into very tiny high-pin-density connectors, maybe even 2D connectors (like CPU sockets).

Even if it's possible to develop such standards, is it worth it?  As in, would it gain us one or two more generations, and then things would start to be packed so tightly it just wouldn't matter, because it would no longer be possible to use this standard?

Another way of asking this same question is, will the future most powerful systems be SOCs, because of the gains of packing chips that need very fast transfer speeds very close to each other?

[porina]

It will come down to what is the end goal for the system. The other advantage of on package ram is smaller footprint. This matters in smaller form factors, but not if you're not space constrained. Possibly slightly lower power consumption e.g. it was cited as a feature of Lunar Lake.

 

As for the performance, to me there doesn't seem to be anything that special. Most of the benefit comes from having a wider bus. For a very long time, a typical desktop running dual channel will be 128-bit width. Perf then scales with clock speed.

Strix Halo is a bit better by having 256-bit width, matching Intel HEDT systems and their quad channel ram. Also I don't know if AMD have changed things with the CPU cores in Halo. If it is still connected by some variation of Infinity Fabric, similar to the chiplet desktop chips, then that bus will constrain it from fully using the bandwidth available. Only the GPU part can make full use of it . And even then, the Halo GPU needs to be kept in perspective. While it has high bandwidth from a CPU perspective, it is a lower range GPU level.

 

Apple M chips take this idea even further. Their highest bandwidth model is equivalent to 16 channel ram which is quite insane. You can have it if you want, but you will be paying for it.

 

Strix Halo is only "faster" than a similar conventional configuration in very specific cases, where the GPU benefits from a larger pool of ram than is easily attainable. But keep in mind the GPU is not very high performance otherwise, and the CPU part offers nothing new. We still need a large amount of silicon to implement this, so there is no cost advantage either. 

 

The problem isn't that desktops can't have this level of performance, but that no one makes it. All the technology is already available to do it. We need to return to the old days of affordable HEDT-like platforms. Unfortunately AMD killed affordable HEDT by adding more cores to consumer offerings, where what I feel is needed was the other platform benefits like quad channel ram, and more PCIe lanes to a lesser degree. The mainstream didn't "need" this. I've long wondered what if we got a split in the consumer space, where consumer tier offerings would go to the 3/5 tiers, and a HEDT-like offering returned for 7/9 tiers. Especially with the price AMD demands these days the platform deserves to be better. 

 

There is a possible solution that is kinda in between all this. CAMM2 is a 128-bit module, or equivalent to dual channel. A desktop system with 2 or 4 of these could scale up bandwidth nicely. Also they enable faster LPDDR over conventional DDR.