HBM vs DDR......conclusion?

[Longbow1]

SO after reading all about HBM and DDR and the pros and cons........Just to make you i am making the right conclusion..........the latency issue with HBM are the only issue holding it back from mainstream use?  Obviously cost, but as its adopted, the cost will come down.  They only issues is speed.  Am i correctly identifying the current issue with HBM implementation?

[mariushm]

No, not only latency...

 

HBM is designed for wide bus (1024 bit wide, 2048 bit wide is in the works) so a DDR5 style stick with HBM memory would have something like 3000-4000 contacts instead of 288 contacts a DDR5 stick has. 

 

The signals use very low power so the chips have to be very close to the actual memory controller - that's why we had for example the HBM memory and the gpu chip on top of an interposer chip (see picture below), the interposer chip is like an actual silicon chip but only has hundreds of "wires", multiples layers of copper traces inside the silicon die to create the connections between the HBM chips and the gpu or cpu chip. 

A HBM chip would not work on an actual ddr stick, the distance between the chip and the hbm chips would be too long, and the signals would be distorted too much due to distance. 

The interposer adds cost... when Fury cards were made, that basic cheap interposer still used to cost around 30$ to make. 

 

It's also a price issue... HBM is made by stacking multiple chips... they have a base "controller" die that has the connections to pins / traces and some "drivers" (signal amplifiers) and then there's 4/8/12 dies stacked on top of it to get 8 / 16 / 32 GB of memory by stacking 1/2/4 GB chips 

Often there's some failure stacking them, something doesn't work right, so they end  up selling chips with 8 stacked dies as chips with 4 dies, with half of the dies disabled, at a lower price. So that also keeps the price higher than regular memory chips.

 

 

[YoungBlade]

Since you put this in the CPU, Motherboards, and Memory sub-forum, I'm going to assume you're talking about HBM for CPUs, which is something that has never been seen on the consumer market.

 

In addition to the cost, which matters a lot for consumer electronics and should not be hand-waved away, HBM typically goes on the same package as the CPU. This reduces configurablity, which is very important for both DIY and OEM markets in the consumer space.

 

Imagine if you had to choose how much RAM you want when you buy a CPU. The 14900K comes in a 128GB, 64GB, and 32GB version. And the cost of the memory also needs to be built into the cost of the CPU. So suddenly the top-of-the-line i9 and R9 CPUs cost $1200 because they include HBM memory.

 

You now rely on Intel and AMD to give you reasonable memory configurations with their processors. If you need 128GB for video editing, but your software doesn't scale beyond 6 cores, you may just have to buy an i9 anyway because it's the only one that has 128GB on package.

 

Now, in the server space, HBM can be used as a sort of caching layer, rather than as the only memory option, so it could be that you get both, but including an additional memory controller to handle the possibility of also wanting DDR5 memory on top will again add cost. The socket size is going to be giant, so motherboard prices would go up even more, and they have the engineering headache of figuring out how to fit the socket and RAM slots on a standard ATX board width. ITX would almost certainly only be able to use the on package HBM.

 

And all of this is to get more bandwidth on a consumer platform where half the CPUs won't even benefit from it. You only see HBM included on CPUs with many cores, but Intel still even sells 2 core CPUs without HT. Imagine buying a Celeron with HBM. There's no way it will cost $50.

 

So no, it's not just about latency.

[Mark Kaine]

22 hours ago, YoungBlade said:

Since you put this in the CPU, Motherboards, and Memory sub-forum, I'm going to assume you're talking about HBM for CPUs, which is something that has never been seen on the consumer market.

 

In addition to the cost, which matters a lot for consumer electronics and should not be hand-waved away, HBM typically goes on the same package as the CPU. This reduces configurablity, which is very important for both DIY and OEM markets in the consumer space.

 

Imagine if you had to choose how much RAM you want when you buy a CPU. The 14900K comes in a 128GB, 64GB, and 32GB version. And the cost of the memory also needs to be built into the cost of the CPU. So suddenly the top-of-the-line i9 and R9 CPUs cost $1200 because they include HBM memory.

 

You now rely on Intel and AMD to give you reasonable memory configurations with their processors. If you need 128GB for video editing, but your software doesn't scale beyond 6 cores, you may just have to buy an i9 anyway because it's the only one that has 128GB on package.

 

Now, in the server space, HBM can be used as a sort of caching layer, rather than as the only memory option, so it could be that you get both, but including an additional memory controller to handle the possibility of also wanting DDR5 memory on top will again add cost. The socket size is going to be giant, so motherboard prices would go up even more, and they have the engineering headache of figuring out how to fit the socket and RAM slots on a standard ATX board width. ITX would almost certainly only be able to use the on package HBM.

 

And all of this is to get more bandwidth on a consumer platform where half the CPUs won't even benefit from it. You only see HBM included on CPUs with many cores, but Intel still even sells 2 core CPUs without HT. Imagine buying a Celeron with HBM. There's no way it will cost $50.

 

So no, it's not just about latency.

on the other hand, ram on chip (?) would allow for a lot smaller, compact builds.

 

this reminds me of having unified ram in pcs... it would be awesome,  but everyone seems to think thats not possible (yet comsoles can do it no problem! )

[YoungBlade]

14 minutes ago, Mark Kaine said:

on the other hand, ram on chip (?) would allow for a lot smaller, compact builds.

 

this reminds me of having unified ram in pcs... it would be awesome,  but everyone seems to think thats not possible (yet comsoles can do it no problem! )

It would have advantages for space saving, which is why plenty of laptops and some mini PCs solder RAM to the board. It would also have latency advantages compared to socketed RAM.

 

However, when it comes to configurablity, it would be terrible. You'd have to buy a whole new CPU or motherboard just to upgrade the RAM, which is a common mid-life upgrade for a lot of computers.

[Mark Kaine]

39 minutes ago, YoungBlade said:

It would have advantages for space saving, which is why plenty of laptops and some mini PCs solder RAM to the board. It would also have latency advantages compared to socketed RAM.

 

However, when it comes to configurablity, it would be terrible. You'd have to buy a whole new CPU or motherboard just to upgrade the RAM, which is a common mid-life upgrade for a lot of computers.

yeah... upgrade-ability, price,  modularity... all things that probably don't sound ideal for avg customers,  but to me... this is all i want, small, minimal latency box  that plays all games,that i don't have to bother "upgrading"*... give me a gpu on board too while we're at it! 

 

(basically a console without all the associated console typical shenanigans, like being locked down, cant run random software, paid online,  modding is difficult...)

 

 

*ps: you could still "upgrade" tho, just less modular...