NVIDIA Made a CPU.. I’m Holding It. - Computex 2023

ImorallySourcedElectrons · June 18, 2023

14 hours ago, igormp said:

I'd understand if we were talking about a 5120-bit bus, but 256-512 is far from "very wide".

No, 256 bit is already quite wide from an electronics perspective, it means you got to wait for all those signals to stabilize before you can clock in the data, which is one of the reasons we like to keep bus widths down (clocking faster with a smaller bus is often advantageous for throughput), it also makes for one hell of a routing and timing challenge. Many of those ultra wide busses you hear about in marketing materials are in reality multiple smaller busses or just a whole bunch of serial links in parallel. But even then, adding I/O quite quickly expands die area, which really comes back to bite you once you're making a wafer floorplan (smaller dies = more dies per wafer = more profit), not to mention the decreased yield during bumping as the pin count increases (= less profit). It also quite heavily complicates some of the signal integrity issues once you go to a circuit board level, but not getting into that anymore on this forum before people start citing Xilinx presentations again that they don't actually understand. The only way to get around many of those issues is to limit the bus width and never going off the die, or at the very least off the interposer, where you can run high density interconnects relatively cheaply. Which are exactly the tactics that Apple applied, but the latter comes at the cost of losing modularity and having to have a SKU per memory size. This is an optimization most manufacturers will be quite unwilling to make due to volume concerns.

But as to how Apple really approached it:

(Image source: https://www.tomshardware.com/news/apple-uses-cowos-s-to-build-m1-ultra)

First of all, please note they min-maxed the living daylights out of this one to save some money on their mask set, I wouldn't even be surprised if they managed to limit the difference between the Pro, Max and Ultra to a single metal layer.

But what I actually wanted to point out is that what you're seeing here is how they did their RAM, the M1 Ultra doesn't really have a 1024 bit memory bus, it looks like a bunch of 128 bit blocks strapped together. There are some advantages to taking this approach, especially if you combine it with programmable phase shifters, since you'd be able to ship in data throughout the clock cycle - which could lead to a profound advantage in some pipelined architectures if you get the timing right. And because you can spread it out if you work like that, you can avoid some of the potential signal integrity and routing issues as well. Which is to say, Apple is probably also just running a bunch of 64 or 128 bit busses in parallel internally on their chip to ship around data, and in reality they probably have multiple memory controllers speaking on those busses and the real value in what they developed is figuring out which data is accessible where.

igormp · June 18, 2023

10 hours ago, ImorallySourcedElectrons said:

No, 256 bit is already quite wide from an electronics perspective, it means you got to wait for all those signals to stabilize before you can clock in the data, which is one of the reasons we like to keep bus widths down

10 hours ago, ImorallySourcedElectrons said:

But what I actually wanted to point out is that what you're seeing here is how they did their RAM, the M1 Ultra doesn't really have a 1024 bit memory bus, it looks like a bunch of 128 bit blocks strapped together. There are some advantages to taking this approach, especially if you combine it with programmable phase shifters, since you'd be able to ship in data throughout the clock cycle - which could lead to a profound advantage in some pipelined architectures if you get the timing right. And because you can spread it out if you work like that, you can avoid some of the potential signal integrity and routing issues as well. Which is to say, Apple is probably also just running a bunch of 64 or 128 bit busses in parallel internally on their chip to ship around data, and in reality they probably have multiple memory controllers speaking on those busses and the real value in what they developed is figuring out which data is accessible where.

I never said a single, really large bus, let's refresh what I said:

On 6/16/2023 at 2:04 PM, igormp said:

[...]mobile devices, mostly of which already have soldered LPDDR RAM, should just go for a different memory controller that has more channels in order to give more bandwidth to the CPU/iGPU.[...]

Multiple 32~64 bit controllers is a non issue and we have been able to do so for years already.

We're also not talking about microcontrollers, the complexity of doing such thing on the scale of a consumer general-purpose CPU is pretty insignificant.

ImorallySourcedElectrons · July 11, 2023

On 6/18/2023 at 10:42 PM, igormp said:

I never said a single, really large bus, let's refresh what I said:

Multiple 32~64 bit controllers is a non issue and we have been able to do so for years already.

We're also not talking about microcontrollers, the complexity of doing such thing on the scale of a consumer general-purpose CPU is pretty insignificant.

Which is a questionable tactic given how they've been promoting it, because you're still limited by the internal bus to actually transfer the data anywhere. By this logic I can make a 2056 bit memory bus and then hook it up to a single 8 bit bus...