Ryzen's high cache latency (probably) explained.

[Notional]

So a user on the r/AMD subreddit called tuxubuntu, made a great post about the very high (primarily L3) cache latencies of the Ryzen CPU. As always, take this with a grain of salt.

 

Quote

Why is the L3 performance so bad?

This is where things get weird. If you have been reading the architectural details of Ryzen, you'll know that the 8c/16t chips have 2 CCXs on them, each containing 4 cores. You'll also know that Ryzen's L3 cache is not a true general-purpose cache. It's a victim cache.

 

The Victim Cache

A victim cache is a specialized type of cache. The way it works is that any bytes that are going to be evicted from the lower level are put into the victim cache. Then it generally works as a normal cache, until data needs to be pulled from it. Then, the data in the lower level cache and the data in the victim cache are swapped.

It should be noted that some of Intel's Haswell product line also implemented an L4 victim cache that was shared by the CPU and iGPU, called Crystal Well. For Skylake, Intel canned the victim cache concept as it is quite complicated compared to a normal cache.

 

 

 

 

 

Apparently, the L3 cache on the Ryzen CPU is not a "standard" cache, but rather what is known as a victim cache. Due to the way the Ryzen CPU is built, by having 2x CCX (that is 2 4 core modules) essentially copy/pasted on the die, it means the victim cache is split in 2, one for each CCX module:

 

Full Ryzen 7 8 core die:

 

1 CCX module:

 

The issue then arises when a core/thread from one CCX module tries to access data from the victim L3 cache from the other CCX module:

Quote

Ryzen's architecture is such that if a thread on one CCX needs to access the cache in the other CCX, it needs to talk through a bus system that goes through the memory controller. The bandwidth of this interconnection is only 22GB/s, about the speed of DDR3-1600. SLOW. This introduces two possible problematic scenarios:

 

1

So imagine one core from the left CCX try to read date from the mid L3 cache block in the other CCX module on the right.

 

So what does this mean? Crappy performance forever? Well not necessarily. One of the fundamental tasks of an operating system is task scheduler. That is the part that of the OS that controls all the threads made by software, and applies them to the different cores on the CPU. What Windows seems to be doing at this point, is to just spread threads out to whatever core, despite the needs of the L3 cached data by these threads. A quick fix would be to patch the scheduler to assort the threads in a better way suited for the Ryzen 6 and 8 core SKU's. 

 

Quote

This problem can partially be remedied in software. It is likely that AMD and Microsoft are working on scheduling adaptations to prevent the occurrence of thread walking across CCXs as much as possible. Additionally, it's been reported that one of the optimizations coming in Windows 10's Game Mode is to move threads less. Also of note is the fact that when Crystal Well was released (Intel's foray into victim caches), Linux kernel and driver updates were able to provide double the performance that it was displaying prior to the software upgrades.

 

 

 

As things stand right now, Windows are essentially mishandling/scheduling the threads on Ryzen, and this could be solved to a high extent (although not completely). Thing is, we don't exactly know how Windows treats Ryzen: Like a normal Intel CPU or like an old AMD bulldozer based CPU, however, a solution seems to be possible.

 

This could also explain the very odd performance behaviour of Watch Dogs 2. A game that can utilize a lot of threads yet performs poorly on Ryzen:

 

Spoiler

 

 

 

It is possible for a piece of software to override the OS scheduler, in deciding where each thread goes. As this game is made with the 6/7 jaguar cores in PS4 and XBOne in mind, as well as standard Intel cores, it is highly likely (though unconfirmed), that this game places threads in a way, that handles data on the victim cache in a way that causes a lot of cross traffic between the 2 CCX's, thus creating huge latencies at the cost of performance.

 

With AMD's recent announcement of coop with Bethesda for upcoming games, both for VEGA, but also Ryzen, it is my understanding, that these games will utilize not only VEGA/Vulkan specific tech, but also be optimized for multithreaded applications on the CPU side, as well as preventing this issue (one would assume).

 

Either way, with the Asus boards seemingly crapping the performance of both Ryzen and DDR4 memory giving wrong performance impressions and a non-supporting scheduler in Windows for this entirely new architecture, the performance of Ryzen has not had its final say yet.

 

Source:

https://www.reddit.com/r/Amd/comments/5x7oaq/ryzens_memory_latency_problem_a_discussion_of/

 

Full post:

Spoiler

 

 

 

 

[captain cactus]

The architecture is so new that even without proper code optimisation it's on-par or better than a 6900K. So it will only get better. That's nice. Also nice is that this will mean the 1600x and 1500X launches will be smoother since these bugs will probably be ironed out by then. 

[Simon771]

And that's exactly why I will be waiting another few months to see how well it will perform when there will be new drivers, new updates, BIOS updates, firmware updates, Windows optimisations, game optimisations, app optimisations bla bla bla

 

Intel is as optimised as it can get. While Ryzen is barely supported right now, so there is still a lot of room for improvements and optimisations.

 

Spoiler

Yes I know Intel CPUs can be optimised even further. Anything can be optimised further. It's 2017 and we can split a single atom on 10 pieces and eat them. So anything is possible.

 

[Organized]

Well, this explanation seems quite logical for me. I hope that a fix/customization of the windows scheduler will do the trick. The I also hope that the Linux kernel will get this fix aswell.

[sof006]

5 minutes ago, Simon771 said:

And that's exactly why I will be waiting another few months to see how well it will perform when there will be new drivers, new updates, BIOS updates, firmware updates, Windows optimisations, game optimisations, app optimisations bla bla bla

 

Intel is as optimised as it can get. While Ryzen is barely supported right now, so there is still a lot of room for improvements and optimisations.

 

  Reveal hidden contents

Yes I know Intel CPUs can be optimised even further. Anything can be optimised further. It's 2017 and we can split a single atom on 10 pieces and eat them. So anything is possible.

 

Agreed, Ryzen is a completely new architecture. Intel has been refining the same tech for years thus the issues aren't there. AMD is using a new method/approach to getting improvements in performance that Windows and games don't have yet. Once support is added for Ryzen then we should see the difference.

[Simon771]

4 minutes ago, Organized said:

Well, this explanation seems quite logical for me. I hope that a fix/customization of the windows scheduler will do the trick. The I also hope that the Linux kernel will get this fix aswell.

I'm pretty sure updates for both OS will come sooner or later.

When they see how many Ryzen CPUs have been sold, they will be kinda forced to do something about that.

[MageTank]

Cache aside, I just don't think I can touch Ryzen until they at least explain why their IMC is so weak. Their IMC was my #1 concern several months ago, and it's proven to be the biggest issue thus far. We've seen double the latency compared to Intel, and terrible bandwidth efficiency (80-90%) for something that is only dual channel. 

 

If they can fix that IMC, there is no doubt that the gaming performance will improve. With this many threads, you are going to need fast memory to keep them fed. We've seen that with Intel's X99 platform, and we've seen it (to some extent) when comparing 2133 JEDEC to 2933 on Zen. Judging by how awful that latency is, I am confident in placing my bets entirely on RRSR. Since we can't really see the subtimings, I can't be certain, but it would make the most sense. 

[Notional]

So apparently AIDA64 results are not necessarily correct.

[leadeater]

20 minutes ago, Notional said:

So apparently AIDA64 results are not necessarily correct.

That is actually disappointing, I'd rather them be true and then AMD and/or software developers including OS developers work on fixes. The most obvious thing of course is to make sure as little cross CCX L3 cache operations are happening as possible, this should have a visible performance improvement if not already optimized.

 

If it's just a reporting error and the values are not accurate then I can't really see any big improvements coming through any time soon or ever until Zen+.

[Notional]

1 minute ago, leadeater said:

That is actually disappointing, I'd rather them be true and then AMD and/or software developers including OS developers work on fixes. The most obvious thing of course is to make sure as little cross CCX L3 cache operations are happening as possible, this should have a visible performance improvement if not already optimized.

 

If it's just a reporting error and the values are not accurate then I can't really see any big improvements coming through any time soon or ever until Zen+.

 

Well even if the numbers in AIDA are not correct, the performance is, so there is obviously some issues with cross L3 access between the CCX's. I do believe Microsoft will release better scheduler support for Ryzen.

However, victim cache or not, I do wonder why the 2 CCX's aren't vertical and connected on the die instead. For both L3 cache parts to be connected and talking to each other, would have prevented all this crap.

ZEN+ should solve this I hope.

[porina]

I'm not familiar with a victim cache, but this does not sound good at all to me and my compute workloads. Because the L2/L3 don't duplicate data between them, what happens if more than one core needs to act on the same data set? You have to load separately and can't pull from a common shared L3?

 

The two parts of L3 cache was also a concern, and if the stated 22GB/s bandwidth between them is true, it is really crap. Crystal Well as L4 cache has stated 50MB/s of bandwidth for comparison. Crystal Well is kinda redundant with the availability of high speed DDR4 modules, but it helps in DDR3 era and/or low performance ram environments e.g. SFF low power systems. So in Ryzen's case of multi-thread compute, it seems better to treat the 8 core as if it were two separate 4 cores. With team blue at least you can play with the entire L3 cache on a single data set to help mitigate ram bandwidth limitations.

 

If my Ryzen order ever arrives I will aim to manually test that.

[leadeater]

6 minutes ago, Notional said:

However, victim cache or not, I do wonder why the 2 CCX's aren't vertical and connected on the die instead. For both L3 cache parts to be connected and talking to each other, would have prevented all this crap.

ZEN+ should solve this I hope.

The CCX is by design for use in the server platform to allow them to offer various different core count SKUs. On the higher core count processors from Intel they actually have a similar grouping of cores, in a slightly different way and the L3 cache isn't as separated.

 

 

The CCX L3 cache is connected on die but not in the way you're thinking.

[Notional]

Just now, leadeater said:

The CCX is by design for use in the server platform to allow them to offer various different core count SKUs. On the higher core count processors from Intel they actually have a similar grouping of cores, in a slightly different way and the L3 cache isn't as separated.

 

 

Yeah it makes sense for scalability, but if the interconnect between the various victim caches are really slow, instead of "full cache speed", then it's really a problem if threads needs cross ccx cache data. Maybe making Windows think it's a dual quad core CPU system would have fixed it. Idk., but there are certainly things to improve for Zen+, whether it's better L3 cache interconnects, scheduling in OS', etc.

[randomhkkid]

I guess when Ryzen 5 comes out we'll be able to verify this as there will only be one CCX and 1 pool of L3 cache. Anecdotally I wonder if the 6 core CPUs are just 8 cores with 2 cores fused off.

[TidaLWaveZ]

They should know better than to copy/paste hardware.

[Trixanity]

This sounds weird. How can you make an interconnect on die with only 22 GB/s bandwidth?

 

Also, didn't AMD use their GMI interconnect with 100 GB/s bandwidth?

 

Something's fishy. That's a massive bottleneck and AMD engineers would know about this. They did claim no obvious bottleneck. However this is an obvious bottleneck if true.

 

It sounds far-fetched to me though.

[Sparviero]

Thank you so much. Much better than bombastic definitive declarations based on first impressions.

[BuckGup]

So basically AMD is ready but no one else is making AMD look not ready.

[Valentyn]

3 minutes ago, SamStrecker said:

So basically AMD is ready but no one else is making AMD look not ready.

From a german review site, after they kept having issues with their review kit and motherboard.
So there's hopefully room for proper improvement still.

 

Quote

 

The MSI board was delivered with BIOS version 113, until last Friday a new one appeared. 

Version 117, which is still up-to-date, improved speed and stability. If we were still able to count on sporadic Bluescreens with the older UEFI, the board is currently stable. Much more important, however, is the drastically higher performance in games and the real pack with 7-Zip. The release notes include, among other things, a fixed problem with the memory act and its timing as well as the voltage.
Compared to the original bios, the new UEFI increases the image rate in our game course between plus 4 and plus 26 percent, on the average even plus 17 percent!

 

[Guest Kloaked]

There's also this: 

 

[ashypanda]

3 hours ago, randomhkkid said:

I guess when Ryzen 5 comes out we'll be able to verify this as there will only be one CCX and 1 pool of L3 cache. Anecdotally I wonder if the 6 core CPUs are just 8 cores with 2 cores fused off.

that's possible as the 6 core R5's have the same 16MB of cache, and the way AMD usually works you could get an R7 really cheap by unlocking the 2 extra cores.

[Mira Yurizaki]

So the gist of it is, the 8-core Ryzen acts more like two 4-core processors and allegedly they talk between each other via a slow enough bus.

 

Granted fixing how Windows (and other OSes) handle scheduling, the fact that the two are supposedly connected to each other via a slow bus irks me. How come this didn't bite Intel in the rear when they released the two die Pentium D Smithefield or the two die Core 2 Quad Conroe?

[Guest Kloaked]

48 minutes ago, M.Yurizaki said:

So the gist of it is, the 8-core Ryzen acts more like two 4-core processors and allegedly they talk between each other via a slow enough bus.

On paper sure, but real-world scenarios they are good (if not better) workstation CPUs compared to the 6800k and 6900k. However, it's been implied (at least in my opinion and I'm open to being wrong, which I really don't care either way anyways) that these would also replace i7s for gaming and they don't. They do minimum frame rates better than i5s which is arguably more important, but i7s are doing what they do better.

[Prysin]

8 hours ago, MageTank said:

Cache aside, I just don't think I can touch Ryzen until they at least explain why their IMC is so weak. Their IMC was my #1 concern several months ago, and it's proven to be the biggest issue thus far. We've seen double the latency compared to Intel, and terrible bandwidth efficiency (80-90%) for something that is only dual channel. 

 

If they can fix that IMC, there is no doubt that the gaming performance will improve. With this many threads, you are going to need fast memory to keep them fed. We've seen that with Intel's X99 platform, and we've seen it (to some extent) when comparing 2133 JEDEC to 2933 on Zen. Judging by how awful that latency is, I am confident in placing my bets entirely on RRSR. Since we can't really see the subtimings, I can't be certain, but it would make the most sense. 

Told you the IMC would be shit. This is what happens when they modify the bristol ridge IMC.

 

 

It is clear that Ryzen is a failure, which I've been saying since November last year. 

When will people listen ?

[Guest Kloaked]

Just now, Prysin said:

Told you the IMC would be shit. This is what happens when they modify the bristol ridge IMC.

 

 

It is clear that Ryzen is a failure, which I've been saying since November last year. 

When will people listen ?

I don't think it's a failure per-se, but I do see your point since many workloads that would use more than four cores on the CPU could easily be offloaded to a cheaper GPU and get done even quicker.

 

What AMD did was dramatically increase their IPC in comparison to Bulldozer, but Intel is still king.