Anandtech release their deep dive on A14 cpus microarc.

[hishnash]

Summary

Every year Anandtech do a deep dive into the new A* cpu microarc and this year is not change. They have some very good things to day about the new chips, they even consider that apples claim the M1 chip have the fastest cpu core out of all cpus might well be true.  

 

These A14 Firestorm (the Big) cores have some very impressive specs when vs Intel/AMD
* 8-wide decode block vs 4-wide decoder for modern

* 630 out of order instruction buffer vs Intel’s Sunny Cove and Willow Cove cores with only 352 ROB

... and the list goes on a another extreme metric is the floating point throughput at 4 times the per-cycle throughput of Intel CPUs and previous AMD CPUs

 

 

Quotes

Quote

We expect the M1 core to be faster than what we’re going to be presenting today with the A14, so Apple’s claim of having the fastest CPU core in the world certainly seems extremely plausible.

Quote

What really defines Apple’s Firestorm CPU core from other designs in the industry is just the sheer width of the microarchitecture. Featuring an 8-wide decode block, Apple’s Firestorm is by far the current widest commercialised design in the industry. IBM’s upcoming P10 Core in the POWER10 is the only other official design that’s expected to come to market with such a wide decoder design, following Samsung’s cancellation of their own M6 core which also was described as being design with such a wide design.

My thoughts

This does put the fire under other cpu makes (and not just mobile cpu makers) to keep in innovating. Apple is achieving desktop class performance at a fraction of the power draw of the rest of the industry, and while gamers might not think this is important reducing the per-core power draw of a cpu enables you to add more cores to your system. Now we have 3 cpu vendors pushing each-other and this will help push improvements in all vendors, it is good apple is coming into this space as it will ensure AMD do not just become lazy like intel did.

 

Sources

https://www.anandtech.com/show/16226/apple-silicon-m1-a14-deep-dive

[Letgomyleghoe]

jesus christ...

[CarlBar]

 

I suspect there's info here thats not being communicated well or is being outright misrepresented. But i don't know enough about the benchmark in question to even begin to kknow where it might be.

[Curufinwe_wins]

Obviously it isn't so much that Anandtech is doing anything wrong here, but between a 14 year old benchmark and then the jokes that are geekbench and similar... I really think they have been putting cart before the horse claiming performance leads in anything. Like literally spec 2006 is older than Cinebench R10. Geekbench is a worst of both worlds program in being inconsistent and bursty. https://www.eetimes.com/new-geekbench-5-improves-tests-but-adds-bias/

 

Torvalds can bitch about AVX (and SVE2) but if modern coding best practices weren't massively parallelized, no one would be trying to add more cores either. Fact of the matter is that when properly implemented continued refinement of SIMD extensions are like 90%+ of the improvements cpus (on all ends) have made in the last decade, and for codes that support it, can literally offer multifold speedups. Far in excess of the power draw increase. 

 

I would like to see some proper benchmarks of SVE2 equivalent code on these chips before making sweeping generalizations.

 

Regardless, definitely true that A14 is class of ARM, and also almost certainly true that in the right cases, it is dramatically more power efficient than x86. 

[Fnige]

whatever libquantum is looks like it likes many cores

[Belgarathian]

3 minutes ago, SlimyPython said:

whatever libquantum is looks like it likes many cores

More information can be found here https://en.wikipedia.org/wiki/SPECint

[LAwLz]

5 hours ago, Curufinwe_wins said:

Obviously it isn't so much that Anandtech is doing anything wrong here, but between a 14 year old benchmark and then the jokes that are geekbench and similar...

Saying that SPEC2006 is a "14 year old benchmark" as some kind of reason why for it is irrelevant is like saying we shouldn't teach plus and minus in schools because they are over 4000 years old. "It's old" does not mean "it's bad" or "it's irrelevant".

 

Also, can you please explain why you think GeekBench is a joke? I would be very interested in knowing the technical details of why you personally believe it is a joke.

Because the article you linked does not say Geekbench is a bad CPU benchmark. The criticism the article you linked of GeekBench is stuff like "why doesn't it benchmark battery life as well", not "it gives inaccurate results for comparing CPU performance", which is what matters in this test.

 

  

5 hours ago, Curufinwe_wins said:

I would like to see some proper benchmarks of SVE2 equivalent code on these chips before making sweeping generalizations.

Ehm, maybe SVE2 isn't included because it isn't supported by any processors yet?

But if SVE2 was included then the performance numbers for the M1 would just be even higher. So I am not sure what your point is.

 

 

Edit:

Hey, looks like Andrei actually addressed people like you in the the written article and he thinks the same way I do about these tests:

Quote

There’s been a lot of criticism about more common benchmark suites such as GeekBench, but frankly I've found these concerns or arguments to be quite unfounded. The only factual differences between workloads in SPEC and workloads in GB5 is that the latter has less outlier tests which are memory-heavy, meaning it’s more of a CPU benchmark whereas SPEC has more tendency towards CPU+DRAM.

 

The fact that Apple does well in both workloads is evidence that they have an extremely well-balanced microarchitecture, and that Apple Silicon will be able to scale up to “desktop workloads” in terms of performance without much issue.

 

The reason why Linus Torvalds criticized GeekBench (version 4 at the time but it applies to GeekBench 5 as well) was because the benchmarks could fit in the cache and therefore did not put much stress on the memory. This is not true for SPEC.

I got a feeling that most people criticizing GeekBench probably think "Torvalds don't like it and since it shows Apple being good, which I don't want, I will say it is a bad benchmark too!".

Because for some reason people have a really hard time understanding and admitting that the ARM cores Apple have been pushing out for several years now have been absolutely amazing in terms of performance. People still seem to have this (incorrect) idea that ARM cores are power efficient because they are only capable of low performance. That is not the case as we can see in these benchmarks.

[Tenelia]

28 minutes ago, LAwLz said:

-snip-

Exactly. Considering power bracket, price point, long-term support, I'm much more confident with swapping over from my 2012 MBP. Since 2012, I simply have not met any compelling options from any Windows laptop manufacturer. But then again, I might still stay on the 2012 because I've only just swapped out the batteries last year...

[leadeater]

1 hour ago, LAwLz said:

Saying that SPEC2006 is a "14 year old benchmark" as some kind of reason why for it is irrelevant is like saying we shouldn't teach plus and minus in schools because they are over 4000 years old. "It's old" does not mean "it's bad" or "it's irrelevant".

But it's also not an exceptionally good benchmark is it when almost everything you run today will have AVX2 in it... At least I'm pretty sure Anandtech compiles SPEC2006 with AVX2 flag set so there is that.

 

Like SPEC2017 exists and SPEC officially considers SPEC2006 retired.

 

But it's a really good benchmark if it wanted to know how ~10 year old software runs, which might be useful to at least a few people.

 

Edit:

Also on Windows the benchmarks are run using Windows Subsystem for Linux which is how Anandtech are running them on the x86 CPUs.

[Spindel]

A thing that I'm curious about is how open the NE is to use in general computing. 

I mean if programmers have access to it in the API there might become some clever ways to boost performance in general computing (for said applications) if they can tap in to the NE. If it is possible to tap in to the computing power of the NE I don't think any synthetic benchmarks will show the performance increase of such actions. 

 

Just a thought while discussing the performances of future M-series of Macs. 

[hishnash]

1 minute ago, Spindel said:

If it is possible to tap in to the computing power of the NE I don't think any synthetic benchmarks will show the performance increase of such actions. 

Yes as a dev you can use the NE for some tasks but you need to remember it is a very focused compute set of T-matrix low precision FP compute cores. Regular benchmarks will not use this but some tasks could opt to use this (even tasks that are not NN based), most of these tasks would normally be evaluated on a GPU but if you can move the to a NE then you can free up GPU time.. just makes your application a lot more complex.. at least with unified memory between all fo these you don't need to worry about copying values around and the latency of when a value is available for the GPU to use. I am interested in how devs start to use this ability to dispatch over to the gpu to do small task were before the overhead of context switching to a discrete gpu was not worth the faster compute.

[RejZoR]

Seeing how tightly they integrated all the subsystems, I find these claims entirely plausible. What Apple did is essentially stuffing not only GPU into the CPU but also Neural processor and RAM. This last one in particular skews my perception of how big the M1 actually is since you can't make it as tiny as it looks on the slideshows and have 16GB of RAM built in. That memory takes some big surface estate. Unless it's HBM style approach and you could technically build that right into the chip itself instead of placing it next to the CPU chip like AMD did with GPU's. This also means that if Apple has 2 SKU's of M1, one with 8GB RAM and another with 16GB, we're talking about two physically different M1 chips.

 

Thing is, biggest bottleneck on traditional devices is the interconnections between subsystems. Bandwidth between CPU and RAM, CPU and GPU and RAM when tasks are shuffled through both RAM and VRAM. Then there is storage that runs on separate SATA route through chipset which has own interconnect with the CPU. Or latest NVMe storage that goes through PCIe lanes to the CPU and RAM. Imagine these subsystems as large cities and interconnects between them are highways. You can only have so much traffic go through those highways to reach other cities. What Apple did was moving all the cities together into a mega city without any highways in between, it's just one big city and you have access to everything basically instantly.

 

While one can argue Apple just reinvented a wheel and there is no innovation, the thing is, only Apple can pull such a thing and do it in a span of under 3 years with stellar results, because they control the software (OS) and also hardware (A14/M1) and make sure both operate at its peak. And send it out and make it work this moment. Plus, they did the translator tools/features in a different way speeding that up too for older apps. Where on PC's, just look at Ryzen launch. AMD did something unique and then we had to wait few years for Microsoft to even catch up updating the OS scheduler and give Ryzen a full potential. Apple will have that the moment they launch M1 and MacOS Big Sur. There won't be any years long delay in between. They'll also fine tune the scheduler to properly and fully utilize those CPU cores. Big cores for fast tasks exclusively and small cores for background tasks and light loads. And it'll just work perfectly. Just wait how long it'll take Windows to fully and properly support Intel's new CPU's that have combination of big and small x86 cores...

 

There is no doubt Apple will crush all mobile/portable devices with this M1 chip. What I'm mostly interesting to see now is how Apple will scale this to Mac Pro's, the desktop systems in big format. Also, I just had a future flashback of Apple entering console market and with that, gaming. Just think about it. They already have Apple Arcade service. They now have a very powerful SoC. I'm predicting Apple will compete with Xbox and PlayStation, while in a way not compete with them entirely directly as they'll create their own niche just like with basically all their devices. And again, have 100% control over the hardware and software, unlike Sony or Microsoft who had to order all the guts from someone else (AMD). This can actually be quite a big thing...

[LAwLz]

29 minutes ago, leadeater said:

But it's a really good benchmark if it wanted to know how ~10 year old software runs, which might be useful to at least a few people.

Please explain why you think SPEC2006 tests "how ~10 year old software runs". Please remember that "it was written 14 years ago" makes as much sense as saying we shouldn't teach addition or subtraction in school because "it was invented 4000 years ago". 

 

The benchmarks in it were compiled with modern compilers such as LLVM. And yes, they were compiled with AVX2 support. So the benchmarks do use the latest instructions in use today. Even though the benchmarks were written 14 years ago, they are just as relevant and optimized as the latest and greatest game or application released today are.

 

You have to remember what benchmarks are and what we are actually testing. Benchmarking is just feeding a CPU a certain workload and seeing how quickly it does it. When comparing two processors you feed them the same workload and see which one finishes the fastest. It doesn't actually matter when the workload was written. All that matters is optimizations such as which instructions are used (which Anandtech takes care of by using modern compilers with flags for things such as AVX) and that the workloads are good for what you want to test (in this case, SPEC includes a very wide variety of workloads).

That's why SPEC2006 is a perfectly valid CPU benchmark even today, and I strongly believe (and have seen 0 evidence that contradicts it) that the differences between two processors depicted in SPEC2006 can accurately describe the difference in performance you will see even when running modern code.

 

 

Remember, we are comparing processors here, not applications. Here is an example of what I mean.

CineBench R20:

5600X - 600

3600XT - 521

Difference: ~15%

 

CineBench  R10:

5600X - 12479

3600XT - 10644

Difference: ~17%

 

Sure the difference in score are completely different between R20 and R10, but that's not what we are comparing. We are comparing how quickly each processor were able to finish the given task. Cinebench R10 was released in 2007 and somehow it is still able to show very similar results to R20 when using it to compare two different CPUs.

We are not interested in exactly what tests are being done. What we are interested in is "how much faster is processor X vs processor Y".

[leadeater]

1 minute ago, LAwLz said:

Please explain why you think SPEC2006 tests "how ~10 year old software runs". Please remember that "it was written 14 years ago" makes as much sense as saying we shouldn't teach addition or subtraction in school because "it was invented 4000 years ago". 

By the simple fact of it's age and it's deprecated status by SPEC and that the performance margins between the same CPUs on the current equivalent tests are different for SPEC2017. Not all the margins are vastly different but there are some that are.

 

SPEC2006 better represents 10 years ago and SPEC2017 better represents today, I don't think that is too hard to understand. Anandtech is using SPEC2006 because of their historical data, they have also used SPEC2017 on current Intel and Ryzen CPUs so that data exists if you are interested.

 

5 minutes ago, LAwLz said:

Remember, we are comparing processors here, not applications. Here is an example of what I mean.

CineBench R20:

5600X - 600

3600XT - 521

Difference: ~15%

 

CineBench  R10:

5600X - 12479

3600XT - 10644

Difference: ~17%

Cinebench is equivalent to comparing a single test case in SPECPerf, you might have 2% difference there but not so in the other slew of test cases.

[LAwLz]

9 minutes ago, RejZoR said:

Seeing how tightly they integrated all the subsystems, I find these claims entirely plausible. What Apple did is essentially stuffing not only GPU into the CPU but also Neural processor and RAM. This last one in particular skews my perception of how big the M1 actually is since you can't make it as tiny as it looks on the slideshows and have 16GB of RAM built in. That memory takes some big surface estate. Unless it's HBM style approach and you could technically build that right into the chip itself instead of placing it next to the CPU chip like AMD did with GPU's. This also means that if Apple has 2 SKU's of M1, one with 8GB RAM and another with 16GB, we're talking about two physically different M1 chips.

The A14 in the iPhone has the memory on the same chip as all other components (CPU/GPU/NPU/etc) so my guess is that it is the same for the M1.

That would explain why you can only pick between 8GB or 16GB as well.

 

Apple also showed a picture of a HBM-style packaging with the memory on the same chip, but to the side when they talked about the unified memory architecture. 

The same slide even included a picture of the M1 die (not including the DRAM).

 

[LAwLz]

10 minutes ago, leadeater said:

By the simple fact of it's age and it's deprecated status by SPEC and that the performance margins between the same CPUs on the current equivalent tests are different for SPEC2017. Not all the margins are vastly different but there are some that are.

 

SPEC2006 better represents 10 years ago and SPEC2017 better represents today, I don't think that is too hard to understand. Anandtech is using SPEC2006 because of their historical data, they have also used SPEC2017 on current Intel and Ryzen CPUs so that data exists if you are interested.

 

Cinebench is equivalent to comparing a single test case in SPECPerf, you might have 2% difference there but not so in the other slew of test cases.

[Citation Needed]

 

I would like some evidence that:

1) SPEC2006 is outdated to the point where it no longer gives accurate depictions of the performance difference between two processors in modern workloads.

2) That the tests in SPEC2006 and SPEC2017 are different enough that they provide a significant difference in overall performance scores, to the point where they might be used to come to different conclusions regarding performance difference between two chips. 

[RejZoR]

@LAwLz

Nice spotting. So it's not quite L3 design, but more like HBM, stuck on same interposer as the CPU chip itself. Still, having memory right there next to it using more direct interconnect will give way bigger bandwidth for sure while keeping latencies low. Remember, physical distance between these things actually has an affect on latency when we're talking nanoseconds.

[leadeater]

38 minutes ago, LAwLz said:

[Citation Needed]

Like did you not just check Anandtech like I said to, it's not exceptionally hard to find.....

 

 

https://www.anandtech.com/show/16214/amd-zen-3-ryzen-deep-dive-review-5950x-5900x-5800x-and-5700x-tested/9

 

See link for the more detailed test results but the above illustrates well enough when doing performance analysis using a more current tool shows a little bit different story than an older one. It's just a shame SPEC2017 results are more limited. Nothing earth shattering but outside margin of error and when comparing products it could be the difference between for example the 10900k looking better or worse compared to it's competition so it does in fact matter.

 

Now the specific tests to go look for would be FP ibm and namd if you want to see an then and now difference of the same test, rather than noticing that SPEC2017 has replaced and added modern equivilents/alternatives. There are also two INT tests that are common between both and show a delta difference.

 

Call me crazy but a newer benchmark suite seems like a better idea to me, SPEC didn't create it for no reason.

[LAwLz]

37 minutes ago, leadeater said:

Like did you not just check Anandtech like I said to, it's not exceptionally hard to find.....

 

https://www.anandtech.com/show/16214/amd-zen-3-ryzen-deep-dive-review-5950x-5900x-5800x-and-5700x-tested/9

 

See link for the more detailed test results but the above illustrates well enough when doing performance analysis using a more current tool shows a little bit different story than an older one. It's just a shame SPEC2017 results are more limited. Nothing earth shattering but outside margin of error and when comparing products it could be the difference between for example the 10900k looking better or worse compared to it's competition so it does in fact matter.

 

Now the specific tests to go look for would be FP ibm and namd if you want to see an then and now difference of the same test, rather than noticing that SPEC2017 has replaced and added modern equivilents/alternatives. There are also two INT tests that are common between both and show a delta difference.

 

Call me crazy but a newer benchmark suite seems like a better idea to me, SPEC didn't create it for no reason.

Oh you gotta be kidding me.

 

The argument that was made in the thread was "SPEC2006 is old so therefore it does not accurately depict performance difference between two products using modern programs". The best you can come up with is that SPEC2006 and SPEC2017 shows, in your own words "a little bit different" results? Come on. Saying that SPEC2006 doesn't accurately depict performance difference between two processors in modern applications is laughable. It absolutely does.

SPEC2017 shows SLIGHTLY different results, sure, but it's not like they are so widely different SPEC2006 can be discarded or are irrelevant. They are still accurate depictions of the performance differences between two processors.

 

Here is what you wrote:

Quote

But it's a really good benchmark if it wanted to know how ~10 year old software runs, which might be useful to at least a few people.

 

You were essentially trying to dismiss the benchmarks because "lol it's old software so they give inaccurate results". You even said it is only useful for "a few people". That is bullshit. The results from SPEC2006 will accurately depict performance difference in modern software too. 

SPEC2006 DOES NOT just represent old software. For crying out loud the tests even use instructions that were not invented when the tests were first created. Newer tests might show a couple of percentages difference in performance but the same can be said about two brand new programs too.

The age of the benchmark has nothing to do with what it is testing. Any attempt to dismiss SPEC2006 as "old and outdated benchmark that isn't relevant today" is completely and utterly unfounded and reeks of ignorance.

[leadeater]

11 minutes ago, LAwLz said:

You were essentially trying to dismiss the benchmarks because "lol it's old software so they give inaccurate results".

Um no... literally SPEC2006 contains tests that have been superseded by more modern options that are included in SPEC2017. Unless you're saying h264ref is more relevant than what it actually is today? As an example.

 

You're confusing relevancy with accuracy. Something can be accurately less relevant or irrelevant.

 

11 minutes ago, LAwLz said:

Any attempt to dismiss SPEC2006 as "old and outdated benchmark that isn't relevant today" is completely and utterly unfounded and reeks of ignorance.

So what you are essentially saying is it better to run SPEC2006 than it is SPEC2017 then?

 

11 minutes ago, LAwLz said:

The best you can come up with is that SPEC2006 and SPEC2017 shows, in your own words "a little bit different" results?

No I said the same tests in SPEC2006 and SPEC2017 have different performance deltas for the same products. I did point you to the names of two of them.

[LAwLz]

5 minutes ago, leadeater said:

So what you are essentially saying is it better to run SPEC2006 than it is SPEC2017 then?

No, what I am saying is that the results from SPEC2006 accurately depicts the performance the A14 is capable of compared to Intel and AMD in modern workloads.

 

If SPEC2006 says chip A performs 10% better than chip B, then I think it is accurate to say chip A has a 10% higher performing CPU than chip B even in modern applications.

Sure, SPEC2017 might say chip A performs 11% or 12% better than chip B, but those are small variances that are not big enough to completely change the conclusions drawn from the results.

 

That is what I am saying, and some people like you and Curufinwe_wins seem to be arguing against that.

 

Just because SPEC2017 was released doesn't mean results from SPEC2006 are inaccurate.

Just because SPEC2006 was created in 2006 doesn't mean the results are inaccurate for workloads created in 2020. Especially not since SPEC2006 has been recompiled to take advantage of the newer instructions.

 

 

Do you agree or disagree with those statements?

[leadeater]

3 minutes ago, LAwLz said:

If SPEC2006 says chip A performs 10% better than chip B, then I think it is accurate to say chip A has a 10% higher performing CPU than chip B even in modern applications.

Sure, SPEC2017 might say chip A performs 11% or 12% better than chip B, but those are small variances that are not big enough to completely change the conclusions drawn from the results.

No you're still missing a critical point, again go look at SPECfp2006 namd/ibm and SPECfp2017 namd/ibm.

 

 

 

 

 

May I ask you if there is a difference between the indicated performance of the very same CPUs between SPEC2006 and SPEC2017? The answer is yes there is. If either of these workloads were important to me I'd want the SPEC2017 data not SPEC2006

[Taf the Ghost]

I'm looking forward to the big "catch" when these parts arrive. I don't doubt it really is the "best" core, but ask AMD how well that was working on Zen1. Apple has an incredibly "thicc" core, which also means it's probably going to be deeply problematic to keep them feed. At low clocks, that means likely a lot of latency & stall issues.

 

As for the SPEC2006 stuff, I feel like Bulldozer has some thoughts on the value of raw Int Perf.

 

On the marketing side of things, Apple basically sandbagging with the low powered Skylake parts has let them basically grab 2 full node jumps (14nm Intel = 10 nm TSMC, then 7nm, then 5nm) worth of Perf/Watt. That helps matters in general.

[LAwLz]

11 minutes ago, leadeater said:

No you're still missing a critical point, again go look at SPECfp2006 namd/ibm and SPECfp2017 namd/ibm.

 

-snip-

 

May I ask you if there is a difference between the indicated performance of the very same CPUs between SPEC2006 and SPEC2017? The answer is yes there is. If either of these workloads were important to me I'd want the SPEC2017 data not SPEC2006

 

https://en.wikipedia.org/wiki/Cherry_picking

 

 

Someone: "These benchmarks are old so we shouldn't look at them!"

Me: "Just because they are old doesn't mean they don't give a good indication of performance".

You: "They should use the newer benchmarks instead! The old ones aren't relevant to today's programs!"

Me: "What? Of course the old benchmark is relevant to new programs"

You: "No, in these two particular tests out of like 40 different tests the results are slightly different!"

 

We are talking about general performance. If you specifically care about namd and ibm then sure, SPEC2017 is a better benchmark that gives more accurate representation of performance in modern programs. But we are talking about general performance here, not a very specific benchmark.

 

If the summarized SPEC2017 results showed Intel being on top and SPEC2006 showed AMD being on top then I could understand your argument, but that's not what is happening. What is happening is that there are slight differences (sometimes even fairly big differences) in small sections of the test but what we should be focusing on is the overall score.

 

If you want an analogy then I am saying:
 

Quote

This new graphics card performs on average 30% better than the old one. That's really good.

and you responding with:

Quote

No it's not 30% better than the old one because in this particular game and in this particular segment the difference is 37% so therefore we can't look at the overall results because I really care about this specific segment in this specific game!

 

[leadeater]

1 minute ago, LAwLz said:

But we are talking about general performance here, not a very specific benchmark.

And yet like I said SPEC2017 exists and is more relevant to day than SPEC2006 is, like I said. Would you not rather SPEC2017 was run which has other things in it like neural network workloads so you can see how the hardware accelerated paths in CPUs perform with that?

 

There's so much more in SPEC2017, it simply is more relevant than SPEC2006 is today.