AMD Zen Architecture Deep Dive

[DocSwag]

It seems that quite a bit of info has come out about the zen architecture thanks to David Kanter, who was the same guy who figured out that nvidia's Maxwell and Pascal GPU architectures are utilizing tile based rasterization. You can check out his full deep dive here.

 

Now I won't pretend to even understand half of it, but some of those of you out there with EE, CE, CS, or any other major that required extensive knowledge about CPU architecture may find it quite interesting.

 

Quote

Although the Zen front end is quite conventional for a high-performance x86 processor, it’s a big change for AMD. It is the company’s first processor with a micro-op cache, which first appeared in the Intel Sandy Bridge microarchitecture; this feature improves performance and saves power.

The instruction stream is tracked in 64-byte windows—the same granularity as branch prediction—although instruction fetches are 32 bytes wide. Branch prediction is dynamic, using three different mechanisms. For conditional branches, a perceptron-based predictor checks the two-level branch target buffer (BTB). AMD withheld the size and associativity of the L1 and L2 BTB, but similar to Jaguar, each line can hold two branches if the first one is predicted as not taken. Predictions from the L2 BTB will likely stall fetches for 1–3 cycles.

Once Zen detects a dynamic indirect branch, it will move that branch to a special indirect target array (ITA) that is 512 entries and direct mapped. Tracking of call and return pairs is through a 32-entry return address stack, which is statically partitioned and can recover from incorrect speculation. Most other branch-prediction resources are shared, and the processor can algorithmically prioritize one thread in response to events. For example, when one thread recovers from a branch misprediction, the recovering thread receives priority.

Unlike previous generations, the instruction TLB is part of the branch-predictor block, which enables more-aggressive prefetching and provides a physical address earlier in the pipeline. The ITLB is a three-level structure that AMD starts numbering at zero. The L0 and L1 ITLBs are fully associative with 8 and 64 entries that can cache 4KB, 2MB, and 1GB page mappings. The L2 ITLB is 512 entries and four-way set associative; it doesn’t map 1GB pages. Hits in the L1 and L2 ITLBs inject several delay cycles into the fetch pipeline. Moving the TLBs into the branch-prediction pipeline enables better prefetching (e.g., using physical addresses), but AMD declined to reveal any prefetching details.

Quote

Once a physical address for the instruction pointer is determined, it goes into a request queue for the conventional instruction cache and is used to probe the microtags for the new micro-op cache. The microtags indicate whether a given address is present and, if so, which cache way is predicted to contain the desired micro-ops.

 

AMD withheld details of the micro-op-cache organization. A micro-op-cache hit can read out the entire line in one cycle, although most lines are only partially packed, so the cache will typically sustain a throughput lower than the theoretical limit. Given that the core was designed to sustain six micro-ops per clock and that the micro-op-cache lines are partially packed, the lines must be at least six micro-ops—probably more. A micro-op-cache hit also reduces the instruction-pipeline length by two stages and cancels the L1 instruction-cache request, saving power.

Quote

The Zen decoder reads up to four x86 instructions and can emit up to eight micro-ops, but it typically emits four micro-ops, since most instructions map to one micro-op. The decoder includes a dedicated stack engine that can eliminate PUSH and POP instructions, along with a memory file to track the actual data dependencies. Apparently, AMD removed the stack engine from the Bulldozer and Excavator cores—one of many unfortunate architectural mistakes. The decoders can fuse compare and jump for branches as well.

 

After decoding, micro-ops go into a 72-entry micro-op queue, which is where the conventional fetch-and-decode path and micro-op cache converge. The micro-op queue is statically partitioned between threads.

Quote

The dispatcher can send up to six micro-ops to the integer side, which is much wider than the four-wide Bulldozer family. Zen tracks all micro-ops through a 192-entry retirement queue and physical register files. Register moves are resolved by changing the register mapping rather than by executing a micro-op, although AMD declined to indicate whether the renamer can zero out registers through remapping. The core state is check pointed, enabling faster recovery from branch mispredictions and other pipeline flushes. The retirement queue is statically partitioned and can retire up to eight micro-ops per cycle from a single thread in each cycle.

 

Integer operations must also allocate 1 out of 168 physical registers and a scheduler entry. The registers are competitively shared with algorithmic priority. Zen has four integer schedulers, each with 14 entries. Each scheduler fans out to an ALU, which can execute basic integer operations including shifts. Two-operand LEA (load effective address) instructions decode into ADD micro-ops and can use any ALU pipeline, while three-operand versions employ the address-generation unit (AGU) instead. In addition, ALUs 0 and 3 execute branches, ALU 1 performs multiplication, and ALU 2 handles division.

Quote

Memory micro-ops are more complex, requiring registers as well as entries in both a scheduler and the load or store queue. Accesses are tracked in a 72-entry load queue and a 44-entry store queue until they become globally observable. Unlike most other resources, the store buffer is statically partitioned between threads. Memory requests can enter either of the two 14-entry schedulers. The memory pipelines are 128 bits wide, so a 256-bit access (e.g., AVX) takes two slots in the register file, memory queue, and schedulers. Load requests can be reordered around other loads and stores with known addresses and can be speculatively moved ahead of stores with unknown addresses. Gather instructions are microcoded, similar to the previous-generation design, Excavator.

 

Each scheduler issues memory-address micro-ops to an address-generation unit, which calculates the virtual address. The virtual address is translated in the DTLB, which has two levels. The L1 DTLB, which contains 64 fully associative entries, caches all page sizes. The L2 DTLB, which caches 4KB and 2MB mappings, holds 1.5K entries and is six-way set associative. Zen includes two hardware page-table walkers to service all TLB misses (instruction and data for both threads).

Quote

Another big change in the memory hierarchy is a private L2 cache, compared with the high-latency shared L2 cache in the Bulldozer family. The 512KB L2 cache is eight-way set associative and is inclusive of the L1 data, L1 instruction, and micro-op caches, dramatically reducing snoop traffic to the other per-core caches. The L2 services requests from the L1 caches using a 32-byte bus to each one. The data and L2 caches include prefetchers, but AMD disclosed no further details.

 

A group of four cores and a shared L3 cache form a CPU complex. The L3 cache for a CPU complex is a mostly exclusive victim cache for the private L2 caches and is implemented as four slices. This 8MB cache is 16-way set associative, and data is spread across it using address interleaving on low-order bits rather than using a hash function. This approach should improve locality at the cost of creating address conflicts. The L3 cache can send 32 bytes per clock to each L2 cache in the CPU cluster. Larger chips will have multiple clusters, which will communicate through a coherent fabric that’s 32 bytes wide. AMD withheld additional details about the L3 cache and overall SoC architecture.

Quote

As with most AMD processors, floating-point (FP) and SIMD execution is separate from integer and memory operations. Although Zen supports up to AVX2, the entire FP pipeline (and memory hierarchy) is optimized for 128-bit operations; 256-bit instructions require twice the resources (e.g., registers and scheduler entries). Zen can dispatch up to four micro-ops to the FP half of the back end, which includes its own renaming and scheduling resources.

 

FP and SIMD micro-ops are tracked in the 192-entry retirement queue but are renamed onto a bank of 160x128-bit physical registers (which are shared between threads with prioritization). Operations also allocate from two schedulers, which can hold a total of 96 micro-ops. The first scheduling queue (SQ) can send operations to four FP execution units. The second scheduling queue (NSQ) cannot issue micro-ops; it simply holds FP micro-ops until SQ entries are free. The NSQ ensures that operations containing an FP micro-op (e.g., register-memory computation and FP stores) can be scheduled on the integer side, even if the 96-entry scheduler is full. It prevents the FP back end from causing dispatch stalls in the processor. SIMD micro-ops use the same scheduler and execution units as FP micro-ops. The individual scheduler sizes are undisclosed.

Quote

AMD has withheld Zen’s physical-design details, such as frequency. But we believe the FO4 depth of each stage is similar to Excavator’s, so it should achieve similar clock speeds in the same process. The 14nm FinFET process, however, provides a combination of higher frequency and greater efficiency. We expect the Zen core will ship at roughly 3.2GHz; the peak frequency (i.e., accounting for dynamic voltage and frequency scaling) is difficult to estimate.

 

AMD has disclosed little useful performance data, but it provided guidance that Zen’s IPC is 40% better on SPECint_rate2006 than Excavator’s, which is in turn 15% better than Steamroller’s. Unfortunately, reliable SPEC CPU scores for AMD processors are nearly impossible to find, as previous products were highly uncompetitive and the company refused to submit results. In fact, Intel generated the best reported SPECint_rate score for a recent AMD processor: 90.3 (base) for a quad-core A10-7850K—a 3.7GHz (4.0GHz turbo) chip with four Steamroller cores—running Windows using ICC 14.0.

 

Figure 2 shows our estimates for Excavator and Zen. First, we recalculated the A10-7850K’s benchmark score without libquantum, which ICC has cracked, resulting in an adjusted score of 81.4. Increasing that number by 15% for an Excavator-based design should yield 93.6, and a 40% boost from moving to Zen yields 131. We further expect that using a compiler optimized for Zen (instead of Intel’s compiler) would boost performance by another 10% to 144.

Quote

Good For Servers But Not HPC

 

The exact range for Zen-based PC processors will depend on the frequency. Intel’s PC processors clock the CPU cores at 2–4GHz. Because Zen has a slight IPC deficit, it will need a higher frequency, all other things being equal, to match a Skylake core. The CPU is just one of several components in a PC processor, however—it’s clearly the most important, but graphics, media, image processing, and display interfaces are all critical for a well-rounded product. Any of these factors (along with lower prices) can sway OEM and end-user buying decisions.

 

To maximize multicore throughput, most Intel server processors operate at 2–3GHz, which is quite feasible for Zen. But servers require more than just low-power cores (e.g., 22 cores in 135W): they need a high-bandwidth low-latency L3 cache and fabric, coherent links, memory and I/O controllers, power management, and excellent overall integration. The new Zen core is a necessary but insufficient condition for server success. AMD has experience with many of these components and certainly understands what is necessary, but its disappearance from the mainstream server market means it must do more work to refresh its server products. The company is declining to discuss these other platform factors.

 

The Zen core does have some limitations that make it less suitable for scientific computing, which accounts for 15–20% of the server market. It sacrifices floating-point and SIMD throughput to reduce area and power—important metrics for this segment. As Table 1 illustrates, Zen offers more FP flexibility than Sandy Bridge and will deliver much better performance on SSE code. Haswell and Skylake, however, provide twice the flops per clock using AVX FMA instructions and, more importantly, twice the cache bandwidth to feed the FP and SIMD execution units.

Quote

AMD Reboots With Zen

 

Almost every company has produced one or two subpar architectures: Intel had the P4 and Itanium, IBM had the Power6 and Cell, ARM had the Cortex-A8, Sun had the UltraSparc V, and Nvidia had the NV30. After winning plenty of battles with the K8 microarchitecture, AMD’s Waterloo was the Bulldozer line, which all but ended the company’s presence in the lucrative server market as well as in midrange client systems. After five difficult years, the Zen core is slated to reset the competitive landscape and reinvigorate AMD’s product line.

 

On the basis of our estimates, the 14nm Zen core should offer performance somewhere between that of Intel’s Ivy Bridge and Haswell generations on integer workloads. Although Zen-based processors cannot rival the latest Skylake core in high-end clients, AMD’s eight-core Summit Ridge chip should be a credible contender for midrange desktops. The company will thus have a shot at PC designs that were previously out of reach, expanding its market share and increasing average selling prices. Future Zen-based notebook processors should be similarly compelling, although they won’t arrive until late 2017.

As well, if you're interested, David Kanter also did an interview with PCPer about the zen architecture which you can check out here:

I think that all this info arose from ISSCC. I've found it quite interesting even if I didn't understand all of it. It seems though that zen won't be very suitable for HPC; perhaps they did this to save die space in order to maximize profits of consumer and server CPUs. As well, it seems zen ipc would be between Ivy bridge and haswell. I'm honestly not surprised, there have only been a limited amount of official benchmarks from AMD and since they were the ones showing them off I wouldn't be surprised if they were cherry picked. However, even if it's only ivy bridge IPC, zen still could very well shake up the whole CPU market.

[tlink]

i like this, its very thorough even though i don´t understand it all to its full extend. i can´t wait for the overclocking and benchmark results.

[BuckGup]

That's disappointing a bit. Well it's an improvement and if AMD can get back into the server business they can finally MAKE MONEY

[Misanthrope]

This is what we've known for a while. Also it isn't as negative: Sure the chips won't have a chance in IPC vs the 7700k but many people would take a very slight deficit by going to a 4.4ghz 6 core 12 thread chip that's 3/4 the price would be extremely competitive to sacrifice 5 to 10% less IPC.

 

 

[Memories4K]

*Slams fist on table*
YES, YES, THAT'S ALL GOOD AND DANDY BUT I WANT TO KNOW ABOUT THE REAL IMPORTANT STUFF.
HOW MANY 'EFF-PEE-S-ES' (FPSs) WILL I GET IN HUNIEPOP?

[Orangeator]

This is still all useless if I cannot buy one at the launch... AMD better have enough product to sale, because if I see an out of stock sign on Newegg I am going to be mad. Lol.

[MageTank]

59 minutes ago, Misanthrope said:

This is what we've known for a while. Also it isn't as negative: Sure the chips won't have a chance in IPC vs the 7700k but many people would take a very slight deficit by going to a 4.4ghz 6 core 12 thread chip that's 3/4 the price would be extremely competitive to sacrifice 5 to 10% less IPC.

 

 

How do we know it won't stand a chance in IPC? If Zen can achieve Haswell's IPC (clock for clock) then it'd only be roughly 5% behind Kaby (clock for clock). 5% isn't that much. 

 

We only saw roughly a 2-3% boost going from Haswell to Broadwell, and only 2-3% more going from Broadwell to Skylake. Skylake to Kaby yielded zero IPC gains. If AMD was not kidding, and actually achieved Haswell IPC, Zen will be fine. The only issues (or should I say, unknown variables) are it's potential ability to overclock, it's IMC quality, and it's real world thermal/power constraints. 

 

The biggest hurdle Zen has at the moment, is the extremely high overclocking Kabylake offers. People seem overly obsessed with the "5ghz" number. Kaby seems to be quite capable of achieving that. If Zen faces a 5-10% IPC deficit as you say, then another 10% core clock deficit on top of that might not bode too well for gamers. Though, you are right when it comes to potential cost. If Zen is cheap enough, that performance deficit won't really matter. Besides, CPU clock speed shouldn't be that big of a deal these days for gamers. Get a better monitor and make your GPU the bottleneck, lol. 

[Misanthrope]

16 minutes ago, MageTank said:

How do we know it won't stand a chance in IPC? If Zen can achieve Haswell's IPC (clock for clock) then it'd only be roughly 5% behind Kaby (clock for clock). 5% isn't that much. 

Well we don't know of course so conceded on that. But what we've seen is that this numbers about IPC could be accurate and we've seen the chips can probably overclock to 4.4 or 4.5 whereas Kaby can easily to 5 ghz. 

 

This can all change and again we of course don't know until we have benchmarks and a big sample of people overclocking the customer chips (yes I am NOT gonna trust youtuber reviews since they probably are already benchmarking and overclocking as we speak under NDA but probably very cherry picked engineering samples) but assuming that your average Ryzen chip will be 400 to 500mhz behind the intel then yes they'll be stronger on IPC. 5% to 10% isn't much granted, but it is significant.

 

And one last point is that none of this matters in the long term since having 6 cores 12 threads with 10% lower IPC is an excellent compromise that would trade in maybe 4 or 5 FPS in games vs a very substancial jump for people encoding, multitasking, etc.

[samcool55]

51 minutes ago, Memories4K said:

*Slams fist on table*
YES, YES, THAT'S ALL GOOD AND DANDY BUT I WANT TO KNOW ABOUT THE REAL IMPORTANT STUFF.
HOW MANY 'EFF-PEE-S-ES' (FPSs) WILL I GET IN HUNIEPOP?

69, i mean why would you even want a different FPS number in huniepop? It's absolutely perfect for that game

[Okjoek]

Too technical for me so I'll just wait for application benchmarks. Still hyped!

[Bananasplit_00]

i find these deep dives interesting even though i dont understand some parts to the full extent nor what they mean for preformance seeing as i dont know what Intels CPUs are doing in the same areas but this is looking good but not as good as i hoped, they are saying Ivy brige IPC, Ivy bridge isnt compleatly obselete but its still not too great(cant remember if Ivy Brige is 2xxx or 3xxxx, if its the later im compleatly fine with this tbh). hope the day 0 benchmarks show great results and overclocks because honestly i am really hyped right now. i hate it because im going to be dissapointed but i cant help it. If the priceing we have seen this far is real(there have been leeked Sweden prices that perfectly line up with the other ones iv seen) Zen is going to drop the entry into the gameing world waaaaay down, $150 instead of $350 for an unlocked quad core is insaine value. If Ryzen overclocks good aswell(like @Misanthrope said im not either trusting youtubers OCs here at all.) then im all for selling basically the entire LAN PC and getting a R9 Fury and an R7 1700X lol

[DocSwag]

2 minutes ago, Bananasplit_00 said:

i find these deep dives interesting even though i dont understand some parts to the full extent nor what they mean for preformance seeing as i dont know what Intels CPUs are doing in the same areas but this is looking good but not as good as i hoped, they are saying Ivy brige IPC, Ivy bridge isnt compleatly obselete but its still not too great(cant remember if Ivy Brige is 2xxx or 3xxxx, if its the later im compleatly fine with this tbh). hope the day 0 benchmarks show great results and overclocks because honestly i am really hyped right now. i hate it because im going to be dissapointed but i cant help it. If the priceing we have seen this far is real(there have been leeked Sweden prices that perfectly line up with the other ones iv seen) Zen is going to drop the entry into the gameing world waaaaay down, $150 instead of $350 for an unlocked quad core is insaine value. If Ryzen overclocks good aswell(like @Misanthrope said im not either trusting youtubers OCs here at all.) then im all for selling basically the entire LAN PC and getting a R9 Fury and an R7 1700X lol

The thing to remember though is that they've said zen is bad for HPC where I think integer performance is important and they're comparing the integer ipc performance so perhaps the FPU performance or something else will make up for it. But then again I'm just guessing  

[Dietrichw]

17 minutes ago, Bananasplit_00 said:

i find these deep dives interesting even though i dont understand some parts to the full extent nor what they mean for preformance seeing as i dont know what Intels CPUs are doing in the same areas but this is looking good but not as good as i hoped, they are saying Ivy brige IPC, Ivy bridge isnt compleatly obselete but its still not too great(cant remember if Ivy Brige is 2xxx or 3xxxx, if its the later im compleatly fine with this tbh). hope the day 0 benchmarks show great results and overclocks because honestly i am really hyped right now. i hate it because im going to be dissapointed but i cant help it. If the priceing we have seen this far is real(there have been leeked Sweden prices that perfectly line up with the other ones iv seen) Zen is going to drop the entry into the gameing world waaaaay down, $150 instead of $350 for an unlocked quad core is insaine value. If Ryzen overclocks good aswell(like @Misanthrope said im not either trusting youtubers OCs here at all.) then im all for selling basically the entire LAN PC and getting a R9 Fury and an R7 1700X lol

The comparison to Ivy bridge is based on the 40% IPC gain that was first claimed  (This projection is from August 29, 2016). AMD has since claimed that they have exceeded 40%, which if true will help.

http://www.linleygroup.com/mpr/article.php?id=11666

[Bananasplit_00]

8 minutes ago, DocSwag said:

The thing to remember though is that they've said zen is bad for HPC where I think integer performance is important and they're comparing the integer ipc performance so perhaps the FPU performance or something else will make up for it. But then again I'm just guessing  

yah well as long as it can outpreform my I5 4690K at 4.7GHz fps wise im getting either the 1600x or the 1700x, just hope my NH-D14 will be enough to keep it cool after i push a decent OC

[3DOSH]

All this talk of IPC means nothing, IPC+Clocks is what matters and PRICE, and the benchmarks for each and single workload.

If its 5-15 % slower in gaming but its 100-200$ cheaper than the Intel counter part I'll take that any day.
 

 

[N3v3r3nding_N3wb]

57 minutes ago, Bananasplit_00 said:

getting a R9 Fury and an R7 1700X lol

That is exactly my plan, except I've already got the Fury.

[Bananasplit_00]

1 minute ago, N3v3r3nding_N3wb said:

That is exactly my plan, except I've already got the Fury.

 

you even got the same Fury as im planning on getting(well depends on what will be the cheapest one when i get it probably), can you unlock that one? you can unlock some R9 Furys to become basically Fury Xs instead if you didnt know

[Vode]

3 minutes ago, N3v3r3nding_N3wb said:

That is exactly my plan, except I've already got the Fury.

Haha nice. Got one as well.

 

 

 

 

[Doobeedoo]

Well, what is known and speculated so far, IPC looks to be more than decent really. Also as far as clocks if Ryzen can OC really then that's great.

[Techstorm970]

3 hours ago, Memories4K said:

*Slams fist on table*
YES, YES, THAT'S ALL GOOD AND DANDY BUT I WANT TO KNOW ABOUT THE REAL IMPORTANT STUFF.
HOW MANY 'EFF-PEE-S-ES' (FPSs) WILL I GET IN HUNIEPOP?

 

[Techstorm970]

2 hours ago, samcool55 said:

69, i mean why would you even want a different FPS number in huniepop? It's absolutely perfect for that game

 

[Vode]

Damn this article is a bit over my head lol but it's still just so interesting!

 

 

[Scruffy90]

4 hours ago, SamStrecker said:

That's disappointing a bit. Well it's an improvement and if AMD can get back into the server business they can finally MAKE MONEY

I assume they have been making money thanks to ps4 and xb1

[BuckGup]

1 minute ago, Scruffy90 said:

I assume they have been making money thanks to ps4 and xb1

On there earnings report I think they were still loosing money. Loosing a lot less but still loosing

[N3v3r3nding_N3wb]

49 minutes ago, Bananasplit_00 said:

you even got the same Fury as im planning on getting(well depends on what will be the cheapest one when i get it probably), can you unlock that one? you can unlock some R9 Furys to become basically Fury Xs instead if you didnt know

Yeah, I got it when it was only like $240 (It is up to $260 now on Newegg).  I have no idea.  I have heard that before, but I've never explored it.  I have precisely 0 experience with building computers, just a whole lot of research, so I probably am not going to try anything advanced like that.