ddr DDRDDR5 - Multi-ranked DIMMS to push DDR beyond 17,000 MT/s

[igormp]

1 hour ago, porina said:

That's probably not done for cost reasons. We'd need even more pins on the CPU socket and mobo layout would get more complicated across the whole range.

That and product segmentation. One of the main differentiators between mainstream and HEDT has always been more memory channels and PCIe lanes.

1 hour ago, porina said:

I was thinking about technical reasons other than "it isn't currently supported". Obviously if it were to take off they'll put the support in. This method doesn't sound like it needs extra connectivity between CPU and memory. The improvement is from hitting the link harder.

There are none. The same cost and segmentation reasons apply in this case.

[porina]

41 minutes ago, leadeater said:

Yep but MRDIMM is going to come with a tradeoff to memory latency and when you factor it against DDR5-8000 which is already a thing MRDIMM-8800 doesn't make a whole lot of sense when effective performance would be lower than that of DDR5-8000. Of course not everything can or will be able to actually function at DDR5-8000 but it won't be that long until new products can.

Might be time for someone to do a test, if it doesn't already exist, to see how much each of latency and bandwidth scale. Main difference is this will be in DDR5 era. It is my suspicion for most cases, if it scales with ram performance at all, latency < bandwidth.

 

Also the difference between DDR5-8000 and QDR-8800 as I think they were going to call it, would be that the latter is a standard configuration and the former would only be reached when overclocked to 11. I did go back and check what eventual DDR5 speeds might be offered, and it seems up to 8400 may be defined but how long will it take to get there?

[leadeater]

5 hours ago, porina said:

and it seems up to 8400 may be defined but how long will it take to get there?

Ram kits on the market? Probably months given 8000 is already here. Products working at those speeds? Likely as soon as the next generation of products that come out. Certainly far far sooner than anything with QDR-8800 support that is for sure.

 

5 hours ago, porina said:

if it scales with ram performance at all, latency < bandwidth.

Do remember that the effective latency is only equivalent to DDR5-4400 with added buffer latency. While the total bandwidth is 8800 actual memory operations are only happening at 4400 but in parallel. The increased bandwidth will only be most effective in situations where bandwidth itself is most important. When comparing memory technology in their more mature states the latency tends to not deviate too much between the best possible lower bandwidth and the best possible high bandwidth, latency is roughly the same since the various factors offset each other.

 

If this is anything like FBDIMM in the past, since both these are buffered, it'll be adding about 5ns of latency.

 

Quote

Fun facts on latency:
From the introduction of DDR memory all the way to the launch of DDR5, standard JEDEC memory’s true memory latency has stayed consistent in the range of 13 to 16 ns. Standard JEDEC memory’s system latency has stayed consistent in the range of 90 to 100 ns.

https://www.crucial.com/articles/about-memory/everything-about-ddr5-ram

 

 

 

Basically confining yourself to JDEC standards means you'll be subjecting yourself to nearly twice the latency of what we've gotten with "XMP" with DDR3, DDR4 & DDR5. I really don't see QDR being offered outside of JDEC configurations since it's honestly for servers rather than desktops.

[porina]

36 minutes ago, leadeater said:

Ram kits on the market? Probably months given 8000 is already here. Products working at those speeds? Likely as soon as the next generation of products that come out. Certainly far far sooner than anything with QDR-8800 support that is for sure.

Again I'm not talking about extreme overclocked ram kits. I'm talking about 8000 being a standard offering. Right now I don't see JEDEC modules beyond 5600 on offer. Probably a side question, is this chicken and egg? If no CPU supports JEDEC above 5600, mem makers wont sell it?

 

IMO these parallel rank approaches don't sound like they'll add that much cost overhead so adding CPU support, and selective mobo support is potentially viable. The slide photo that is being referenced even question the need for DDR6 if we go this route. I do hope we get more details from that presentation one way or other.

 

We already have high end OC DDR4 kits that are higher speed than low end JEDEC DDR5, but the DDR5 will be a lot cheaper and more reliable. That's what I'd hope QDR would bring.

 

36 minutes ago, leadeater said:

Basically confining yourself to JDEC standards means you'll be subjecting yourself to nearly twice the latency of what we've gotten with "XMP" with DDR3, DDR4 & DDR5. I really don't see QDR being offered outside of JDEC configurations since it's honestly for servers rather than desktops.

I don't see latency, even JEDEC latency, being a major problem. Bandwidth shortage definitely is a problem. And if it does get offered in consumer CPUs it'll almost certainly be to  some JEDEC standard. Someone else can worry if you can XMP it.

 

I'm not afraid to do ram testing, although I'm limited to DDR4 and I don't have high speed modules any more. Still, I can still easily test up to 3200, JEDEC and XMP configurations.

[leadeater]

2 hours ago, porina said:

Again I'm not talking about extreme overclocked ram kits. I'm talking about 8000 being a standard offering. Right now I don't see JEDEC modules beyond 5600 on offer. Probably a side question, is this chicken and egg? If no CPU supports JEDEC above 5600, mem makers wont sell it?

Why does this even matter? Even SI systems don't use JDEC so I see no reason to restrict to this when that's not what actually gets used. We should be keeping the comparisons against practical real life, which means XMP is on and the restriction there is what is practically usable which today DDR5-8000 is not. This doesn't take away from the fact DDR5-8000 will be usable before it's a JDEC standard.

 

So to answer your question, yes absolutely memory makers will make it. Like I've said consistently DDR5-8000 is already a retail product right now, you can buy it and it's not JDEC, nothing "XMP" is JDEC. https://www.anandtech.com/show/18765/gskill-launches-ddr5-8000-c38-48gb-memory-kit

 

I have for example DDR3-2400, that was never JDEC and it was easily obtainable and anything at the time could run it. Official DDR3 JDEC stopped at 2133 and that came WAY after XMP kits were doing it.

 

2 hours ago, porina said:

IMO these parallel rank approaches don't sound like they'll add that much cost overhead so adding CPU support, and selective mobo support is potentially viable.

It doesn't add much cost if any to the motherboard. It adds cost to the DIMM and it adds cost in the IMC in the CPU. LRDIMM like for like capacity costs more the RDIMM and like for like RDIMM costs more than UDIMM. Since this is going to be fully buffering memory accesses it'll be more cost than RDIMM and likely a little more than LRDIMM. But the cost is mainly in the design, testing and validation. It would really make the most sense on desktop if DDR was entirely dropped for QDR so there isn't a double of QVL requirements. You'd be essentially doubling what needs to be tested, roughly speaking supporting both.

 

2 hours ago, porina said:

I don't see latency, even JEDEC latency, being a major problem. Bandwidth shortage definitely is a problem. And if it does get offered in consumer CPUs it'll almost certainly be to  some JEDEC standard. Someone else can worry if you can XMP it.

It absolutely is for games. DDR4 low latency on Alder Lake had better actual performance than many different DDR5 transfer rates and CL combination for some games. If you equalize for latency then both perform the same.

 

Latency might not matter for what you are doing much but there are a lot of things where it does matter, more than bandwidth. Obviously the ideal is to scale evenly so bandwidth goes up and latency remains the same but that's not always possible. It certainly won't be if QDR/MRDIMM only comes in JDEC standards of the time then XMP kits are going to thoroughly beat them in anything that moderately cares about latency over bandwidth.

 

That's all within reason since if you drop bandwidth too much then latency will never overcome that.

 

MRDIMM/MCRDIMM make lots of sense in servers where latency already isn't competitive with gaming desktops and the difference will only be the slight extra latency the buffering will add.

 

Also remember JDEC also defines the timings and DDR5-5600 comes in JDEC and non-JDEC and one is significantly better than the other, if you have the choice you then get the non standard "XMP" kit that operates at near half the latency. Other than cost, which is small typically, I don't see why that wouldn't be the choice for a desktop system. 

 

Edit:

QDR without XMP, which I will guarantee won't be able to get as high in transfer rates and low in timing as DDR (per induvial channel, sub channel), will be near the bottom.

Spoiler

 

It doesn't take much to not be completely garbage here but that is 100% reliant on if XMP will ever be a thing for MRDIMM/MCRDIMM which I personally

doubt. I just don't see it coming to desktop until DDR is committed to be dropped outright.

[ouroesa]

DDRDDR? QDR. I swear, these news article titles are getting closer and closer to Youtube clickbait bullshit.

[igormp]

3 hours ago, porina said:

The slide photo that is being referenced even question the need for DDR6 if we go this route.

That doesn't make much sense, you can still apply this same idea to the future DDR6 standard to get double the bandwidth with faster modules.

[Brooksie359]

13 hours ago, leadeater said:

This doesn't increase pin count. All this is doing is sending 4 signals per cycle instead of 2. Maybe the traces need to be slightly better for signal integrity but everything else is unchanged motherboard wise. Only the DIMMs and CPUs will change.

 

And it's 100% happening, Intel already have real technical demo hardware with it and it's part of the product roadmap. This is a server thing so far, that's it. Desktop in future? Maybe, not yet sure it makes sense cost or performance wise. More to this than just increasing bandwidth. 

 

LRDIMMs never made it to desktop platform, never will. MRDIMM/MCRDIMM could well be the same.

I don't know. On one hand this seems like a niche usecase for desktop users as alot of people who would need this would be running server grade hardware. Also I would imagine that Intel and AMD might want to lock this feature for server grade hardware like they have with other features in the past to promote server grade hardware that has higher margins. 

[starsmine]

1 hour ago, ouroesa said:

DDRDDR? QDR. I swear, these news article titles are getting closer and closer to Youtube clickbait bullshit.

well its not true QDR. 
Its just Double data rate Double data rate.

which achieves the same thing, but through different method.
Like Duel channel doubles your data rate, you wouldn't call that QDR either. 

[williamcll]

Latency has been AMD's bottleneck, why would they do this?

[porina]

9 hours ago, leadeater said:

Why does this even matter? Even SI systems don't use JDEC so I see no reason to restrict to this when that's not what actually gets used. We should be keeping the comparisons against practical real life, which means XMP is on and the restriction there is what is practically usable which today DDR5-8000 is not. This doesn't take away from the fact DDR5-8000 will be usable before it's a JDEC standard.

Cost mainly. XMP8000 if you can buy it in near term will be fantastically expensive relative to more common lower speed grades, and I'd question if it will be usable without high end mobos. Gaming desktops might use XMP but outside that niche pretty much everyone else uses JEDEC ram. Also please put that missing "E" in there. Who's JDEC?

 

We can draw parallels with DDR4. Around 2015 when DDR4 hit mainstream, standard supported speed was 2133 on Skylake. You can certainly buy 3200+ XMP modules although there was quite a price hike once you get above 3000. I still have a 4x4GB 3333 kit kit from that year, although more targeted at older X99. JEDEC 3200 was first supported 4 years later in 2019 by Zen 2 if I'm not mistaken, with the XMP "sweet spot" moving to 3600. That is, XMP3600 ram started to cost similar to XMP 3200 ram.

 

DDR5 is roughly double the speed of DDR4. We had Alder Lake come out in 2021 at 4800 speed. If this follows a similar speed ramp to DDR4, we might have 6400 standard for the late 2024/2025 generation, with affordable XMP around 7200. Today, looking at one supplier, the cheapest 7000 ram is 73% more expensive than cheapest 6000 ram, with 6400 ram hardly any more than 6000. Similarly looking back at DDR4, looking at two suppliers cheapest 4000 2x8GB kits are about 60% more expensive than cheapest 3600 kits, with an insignificant price drop to 3200.

 

9 hours ago, leadeater said:

Also remember JDEC also defines the timings and DDR5-5600 comes in JDEC and non-JDEC and one is significantly better than the other, if you have the choice you then get the non standard "XMP" kit that operates at near half the latency. Other than cost, which is small typically, I don't see why that wouldn't be the choice for a desktop system. 

There's a world of difference between lower speed XMP and the top end XMP. It isn't that difficult for module manufacturers to squeeze more performance out at similar or not much above JEDEC speeds, like overclockers do. Once you get significantly above JEDEC speeds, the difficulty increases, as does pricing to buy it.

 

6 hours ago, igormp said:

That doesn't make much sense, you can still apply this same idea to the future DDR6 standard to get double the bandwidth with faster modules.

I agree, just echoing what the slide says. I'm not the one presenting at a memory conference. Maybe if the economics work out for it, it could delay the development and introduction of DDR6 as it reduces the pressure for it.

[leadeater]

50 minutes ago, porina said:

Also please put that missing "E" in there. Who's JDEC?

lol I know, I noticed yesterday but couldn't be bothered to fix them 

 

50 minutes ago, porina said:

Cost mainly. XMP8000 if you can buy it in near term will be fantastically expensive relative to more common lower speed grades, and I'd question if it will be usable without high end mobos. Gaming desktops might use XMP but outside that niche pretty much everyone else uses JEDEC ram.

Sure but that is also what the K and KS CPUs are targeted at and even SI/OEM actual gaming systems come with XMP enabled so it's just as relevant as strictly JEDEC speeds and timings are. You can buy a low tier gaming system from Dell.com and it comes with XMP enabled (unless QA missed it and it wasn't done).

 

DDR5-5600 JEDEC is CL40/46/50 where if you buy anything DIY from G.Skill/Corsair etc it'll be CL28 or 32 for the lesser but better kits. CL56 vs CL28 at the same transfer rate is a significant latency difference at a not insurmountable cost.

 

The cheapest 16GB module on newegg is $48 DDR5-4800 CL40 (which is JEDEC official B specification) and to go from that to something a lot better: $60 DDR5-5600 CL28, $59 DDR5-5600 CL30, $57.50 DDR5-5600 CL36, $55.50 DDR5-5600 CL40 (JEDEC official A specification).

https://www.anandtech.com/show/16143/insights-into-ddr5-subtimings-and-latencies

 

So we aren't talking huge price differences for significant latency reductions, and this hold true for faster transfer rates. There are lots around this $60 per module price for 16GB capacity. Yes $159 per module options exist but that's also DDR5-7800 CL36.

 

You can get 16GB DDR5-6000 CL30 for $62.50 per module and 16GB DDR5-6400 CL32 for $79.50 per module. So yes above 6000 the price does increase quite a bit but XMP right now is more than affordable with vastly better latency options and they are only going to get cheaper while JEDEC official is still going to be absent for a while.

 

50 minutes ago, porina said:

There's a world of difference between lower speed XMP and the top end XMP.

As above there is still a world of difference between well priced XMP and JEDEC. I just don't see your point, it's not that costly and they aren't that underused in the market.

[porina]

2 minutes ago, leadeater said:

As above there is still a world of difference between well priced XMP and JEDEC. I just don't see your point, it's not that costly and they aren't that underused in the market.

You do seem to be arguing something other than what I am. My point? Basically I want faster (speed) ram without insane pricing. Talk of latency at lower speed modules is irrelevant to that. Saying XMP 8000 ram exists now doesn't help. It will take many years for ram to mature enough for XMP 8000 ram to be commonly affordable. JEDEC speeds would have to be much closer to 8000 before that happens. I completely agree that if I were building a DDR5 system today, for sure I'd get some kind of mid speed grade XMP, not JEDEC modules, just like many did with DDR4. But the dream is affordable higher speeds sooner than just letting DDR5 mature naturally. I don't feel it is an instant "not a chance in hell" level of writing it off this "QDR" reaching consumer tier. I'm not saying it is likely to happen, just that I hope it does. Could be years away if ever.

 

BTW in the gaming laptop space, XMP does seem much rarer than desktops. I've never owned a gaming laptop that supported XMP. I'm sure they do exist, since I see XMP modules offered on sale, but these are probably limited to highest end models.

[leadeater]

1 hour ago, porina said:

You do seem to be arguing something other than what I am. My point? Basically I want faster (speed) ram without insane pricing. Talk of latency at lower speed modules is irrelevant to that.

It's not because my point was that you only get aggregated higher bandwidth between the DIMM itself and the CPU, it's multiplexed. Inside you're only getting DDR5-4400 at JEDEC timings. That's by no means any good for games.

 

Literally everything Sk Hynix, Renesas, Intel and AMD are talking about for this technology has the word "server" in it so when it comes out it'll be on server platforms only, if it ever comes to desktop and that's a big if it'll be competing with mature very fast low latency DDR5 which is my point. You're not going to get much advantage when MRDIMM and MRDIMM are 100% going to cost more than regular DDR5. So while good XMP will cost more, not talking the highest end, MRDIMM/MCRDIMM is not going to be bottom end pricing either. This is why I'm saying it's a factor because if you have $XYZ to spend MRDIMM/MCRDIMM may not be the best option at that price point.

 

 

New custom Renesas data buffers are required for group of 4 DRAM modules and a new multiplex timer is required. Each DRAM module also requires to be wired to the MDB rather than sharing the same link for the pairs. This isn't going to be cheap. When these companies are talking about cost it's all relative to server DIMM pricing.

 

1 hour ago, porina said:

Saying XMP 8000 ram exists now doesn't help. It will take many years for ram to mature enough for XMP 8000 ram to be commonly affordable.

I don't think so, it'll be a year since that's how it's tracked for 6000 and is tracking for 6400 etc.

 

Edit:

Just for a cost measure for you I just brought 24 of these for work

Quote

HPE SmartMemory RAM Module for Server, Desktop PC - 16 GB (1 x 16GB) - DDR4-3200/PC4-25600 DDR4 SDRAM - 3200 MHz - CL22 - 1.20 V - ECC - Registered - 288-pin - DIMM

These are about, lower than but hey can't say exactly, $315 USD per module. This is the price point I am expecting for MRDIMM/MCRDIMM. This is the same cost as DDR5-8000 is right now.

/Edit

 

1 hour ago, porina said:

BTW in the gaming laptop space, XMP does seem much rarer than desktops. I've never owned a gaming laptop that supported XMP. I'm sure they do exist, since I see XMP modules offered on sale, but these are probably limited to highest end models.

HP/Dell/Lenovo/Acer/Asus/MSI all support XMP on most of their laptops. What they ship with can however be hard to find out. A lot of laptops also use LPDDR now too.

[porina]

1 hour ago, leadeater said:

It's not because my point was that you only get aggregated higher bandwidth between the DIMM itself and the CPU, it's multiplexed. Inside you're only getting DDR5-4400 at JEDEC timings. That's by no means any good for games.

I'll await more details before judging the latency. At best you're looking at half a clock cycle extra for that 2nd additional rank to transfer. Maybe more depending on how the buffering is handled.

 

1 hour ago, leadeater said:

if it ever comes to desktop and that's a big if it'll be competing with mature very fast low latency DDR5 which is my point. You're not going to get much advantage when MRDIMM and MRDIMM are 100% going to cost more than regular DDR5.

But these MR/MCR will also progress as DDR5 matures. Fundamentally it'll offer up to double the bandwidth of whatever is current. It'll be a cost value trade off. It doesn't have to be the cheapest.

 

1 hour ago, leadeater said:

HP/Dell/Lenovo/Acer/Asus/MSI all support XMP on most of their laptops. What they ship with can however be hard to find out. A lot of laptops also use LPDDR now too.

I'm going to have to call the "citation needed" game here. I've owned an MSI 970 laptop, Asus 1050 laptop, and currently Lenovo 3070 laptop. Ok, the Asus was a low end one, but the other two are at least mid range. None of these support XMP. I have no doubt they do exist, but I have to question "most". I suspect it'll be strictly high end only (HK, HX tier CPUs maybe?). Outside of gaming laptops, I'd be surprised to see XMP at all. I'm going to guess also the LPDDR is more a thin/light target than gaming. Maybe some smaller form factor gaming laptops could fall into that.

[leadeater]

38 minutes ago, porina said:

I'm going to have to call the "citation needed" game here. I've owned an MSI 970 laptop, Asus 1050 laptop, and currently Lenovo 3070 laptop.

They're buried in option menus or you need you need to activate advance option, or do a BIOS update. It really does depend on what exactly you have even from the same brand

https://www.nosware.com/do-laptops-have-xmp/8397/

 

Personally I've only got HP laptops. I have dealt with Lenovo a lot in the past though, some Dell but not a lot.

38 minutes ago, porina said:

None of these support XMP. I have no doubt they do exist, but I have to question "most". I suspect it'll be strictly high end only (HK, HX tier CPUs maybe?).

I think that's the difficult point, you do need a CPU installed that supports it. The platform might but you can't utilize XMP if the CPU doesn't allow it. Not much different to desktop though except it's more a chipset thing. Same for laptops but you have little ability to choose so you have to get something that has a HK/HX possible option to be more confident the platform is going to support it.

 

38 minutes ago, porina said:

I'm going to guess also the LPDDR is more a thin/light target than gaming. Maybe some smaller form factor gaming laptops could fall into that.

Surprisingly not, lots of laptop use LPDDR5 now. I think that's mostly due to LPDDR5 officially supporting 6400 MT/s and also gives more space for cooling which helps with larger GPUs. You'll see quite a few gaming laptops with SO-DIMM only running DDR5-4800 for example while a comparative LPDDR5 option does a little better because LPDDR5 can or will be 6400.

 

This one is LPDDR5-6000, just ah, don't look at the price.

 

[porina]

14 minutes ago, leadeater said:

I think that's the difficult point, you do need a CPU installed that supports it. The platform might but you can't utilize XMP if the CPU doesn't allow it. Not much different to desktop though.

It's the other way around on desktops isn't it? Pretty much all CPUs support it (at least the ones of interest to most on this forum), and if you can use XMP or not comes down to chipset choice. Mobile might work differently due to the chipset side.

 

On the laptops I've owned, I've certainly gone through all the options available in bios. No XMP. They all, unsurprisingly, came with JEDEC ram appropriate to its age. The first was Skylake at 2133, 2nd was Kaby Lake at 2400, last is a Zen 3 with 3200.

 

14 minutes ago, leadeater said:

This one is LPDDR5-6000, just ah, don't look at the price.

I looked at the price. The cost delta between the 4080 and 4090 versions of that laptop in UK is more than my 3070 laptop cost when I got it!

 

That's what I'd class as a thin laptop though. I guess I can't call it light at that size. It's maximum thickness is 18.57mm. My more mainstream gamer laptop peaks at 25.75mm (+38%). I think that's due to the screen hinge and feet though, so excluding those it might be closer. The soldered nature makes sense to get the physical size down in use cases like these.

[igormp]

4 hours ago, porina said:

I agree, just echoing what the slide says. I'm not the one presenting at a memory conference. Maybe if the economics work out for it, it could delay the development and introduction of DDR6 as it reduces the pressure for it.

Yeah, I'm aware, not disagreeing with you (not your point in the first place), but rather to this take in a general way. 

Anyhow, I still doubt it'd delay DDR6 in any way, current CPUs are bottlenecked hard by memory bandwidth, and, unless everyone decides to go for an M1-like idea (just like amd did with their MI300 or Intel with hbm on top of the die) and make it work for really large amounts of ram, I don't think they'll ever stop chasing faster memories. 

[leadeater]

2 hours ago, porina said:

It's the other way around on desktops isn't it? Pretty much all CPUs support it (at least the ones of interest to most on this forum), and if you can use XMP or not comes down to chipset choice. Mobile might work differently due to the chipset side.

Yea, there may be some odd ball CPUs that don't support it on desktop but nothing anyone would realistically come across. It's just about the chipset, so if you have a H610 you aren't getting XMP far as I know, but you can pair a 13900K with one. That's not a "choice" you get for laptops.

 

2 hours ago, porina said:

I looked at the price. The cost delta between the 4080 and 4090 versions of that laptop in UK is more than my 3070 laptop cost when I got it!

I warned you not to look haha. But there are much less costly ones running LPDDR5 but usually 12th Gen and running LPDDR5-5200.

 

2 hours ago, porina said:

That's what I'd class as a thin laptop though. I guess I can't call it light at that size. It's maximum thickness is 18.57mm. My more mainstream gamer laptop peaks at 25.75mm (+38%).

That type of thickness is quite typical of new designs, I also class it as thin but options not thin are sadly less common. Thicker gaming laptops like to market M.2 and memory expansion for those so even though LPDDR5 would offer better performance, buyers I doubt want that. Also last I checked none of the 12th/13th Gen HX mobile CPUs supported LPDDR5 at all.

[ouroesa]

20 hours ago, starsmine said:

well its not true QDR. 
Its just Double data rate Double data rate.

which achieves the same thing, but through different method.
Like Duel channel doubles your data rate, you wouldn't call that QDR either. 

It's not. It's more like dual channel on a single dimm. Not DDRDDR

Theoretically, dual-channel configurations double the memory bandwidth when compared to single-channel configurations. This should not be confused with double data rate (DDR) memory, which doubles the usage of DRAM bus by transferring data both on the rising and falling edges of the memory bus clock signals.