"Exclusive: Why Apple M1 Single "Core" Comparisons Are Fundamentally Flawed (With Benchmarks)"

[Jailbreaker]

"Exclusive: Why Apple M1 Single "Core" Comparisons Are Fundamentally Flawed (With Benchmarks)" https://wccftech.com/why-apple-m1-single-core-comparisons-are-fundamentally-flawed-with-benchmarks/amp/

[LAwLz]

Ah yes, Usman Pirzada. The person who is really salty his favorite company is now being beat by Apple so he post article after article, tweet after tweet, grasping at straws for why anything showing the M1 in good light is invalid.

 

For those who do not want to give WCCFTech any clicks (since they don't deserve it), the claims the article makes is that since Intel and AMD have SMT, it is unfair to do single core tests against the M1 which do not have SMT.

"It's unfair to run a benchmark which only executes on one thread on a CPU that could run two threads on a single core" is the argument. There, saved you a click.

 

The problem with that argument is that if a program only has one thread, then it won't care that your CPU could execute two threads on a single core. All it cares about is per thread performance. If you change the wording from "per core performance" to "per thread performance" all of Usman's arguments fall flat on their face.

 

But like I said, since his favorite company is threatened he has to grasp at straws and go

Quote

noooo! Apple doesn't have the best single core performance! (they have the best single threaded score)

Except you know... when people say single core score they generally mean single threaded score. Hell, Windows and other schedulers even sees each thread as an individual core. So single thread and single core performance are interchangeable words even though WCCFTech tries to make the argument that they are completely different and therefore people should stop saying Apple has the best single core performance.

 

They "prove" this then by running a benchmark at two threads for the x86 chips, and one thread for the M1 tests. Yeah... Totally fair comparison... In some tests he didn't even care to run them, he just straight up took AMD and Intel scores, added 30% to the score and went "herp derp this is their true single core score, not what the benchmark actually shows".

 

 

This comment summed it up really well:

Quote

Rohit Jain • an hour ago • 

Little doubt here: If a application coded for single core work only uses singlet thread not whole core on x86 and is unable to use the core to it's full potential. Then sure the numbers given by benchmarks are not exactly for a core, but to the end user does that matter. The single core score is representative of the performance of application that is not coded to use multiple cores(/threads) which is true in this case. Irrespective of what the actual performance is the whole core is.

 

 

Stop reading WCCFTech. 

It really is a garbage website. The people over there have very little technical understanding of the things they cover, and most of their content is either rumors they saw on Twitter, or stuff like this where they deliberately try to stir up a controversy to get clicks both from people desperate to find some reason to say "Apple bad" (or whichever company they are covering) and from people who feel like they have to jump in and defend Apple (or whichever company they are covering).

[Spindel]

The author of the article seem to miss that's the reason most benchmarks present the result as single threaded and not singe core results.

 

Also let's ignore that the M1 still beats most (all?) 4 core 8 threaded x86 CPUs in the market.

 

[Pixel5]

2 minutes ago, Spindel said:

Also let's ignore that the M1 still beats most (all?) 4 core 8 threaded x86 MOBILE CPUs in the market.

clarified your statement here a bit.

[Lord Vile]

30 minutes ago, Jailbreaker said:

"Exclusive: Why Apple M1 Single "Core" Comparisons Are Fundamentally Flawed (With Benchmarks)" https://wccftech.com/why-apple-m1-single-core-comparisons-are-fundamentally-flawed-with-benchmarks/amp/

On the flip side the M1 has ML capabilities that traditional chips don't have so if an application can take advantage of that surely benchmarks don't matter because the M1 will be miles faster.

[Spindel]

35 minutes ago, Pixel5 said:

clarified your statement here a bit.

I actually had to look this up.

 

There is not a single 4 core 8 thread CPU (desktop or laptop) that beats M1 in benchmarks at least. I actually can't find anything under 8 cores that beats the M1. 

 

Of course benchmarks are benchmarks YMMV in real world workloads. In those usually the M1 seem to perform better against CPUs up to 6 cores but there are some loads where it gets beaten.  

[RejZoR]

Single core is single core. If CPU can execute 4 threads on 1 core, then that's that. That's like saying M1 is unfair because it's ARM and comparing it to x86...

[Jailbreaker]

1 hour ago, LAwLz said:

Ah yes, Usman Pirzada. The person who is really salty his favorite company is now being beat by Apple so he post article after article, tweet after tweet, grasping at straws for why anything showing the M1 in good light is invalid.

 

For those who do not want to give WCCFTech any clicks (since they don't deserve it), the claims the article makes is that since Intel and AMD have SMT, it is unfair to do single core tests against the M1 which do not have SMT.

"It's unfair to run a benchmark which only executes on one thread on a CPU that could run two threads on a single core" is the argument. There, saved you a click.

 

The problem with that argument is that if a program only has one thread, then it won't care that your CPU could execute two threads on a single core. All it cares about is per thread performance. If you change the wording from "per core performance" to "per thread performance" all of Usman's arguments fall flat on their face.

 

But like I said, since his favorite company is threatened he has to grasp at straws and go

Except you know... when people say single core score they generally mean single threaded score. Hell, Windows and other schedulers even sees each thread as an individual core. So single thread and single core performance are interchangeable words even though WCCFTech tries to make the argument that they are completely different and therefore people should stop saying Apple has the best single core performance.

 

They "prove" this then by running a benchmark at two threads for the x86 chips, and one thread for the M1 tests. Yeah... Totally fair comparison... In some tests he didn't even care to run them, he just straight up took AMD and Intel scores, added 30% to the score and went "herp derp this is their true single core score, not what the benchmark actually shows".

 

 

This comment summed it up really well:

 

 

Stop reading WCCFTech. 

It really is a garbage website. The people over there have very little technical understanding of the things they cover, and most of their content is either rumors they saw on Twitter, or stuff like this where they deliberately try to stir up a controversy to get clicks both from people desperate to find some reason to say "Apple bad" (or whichever company they are covering) and from people who feel like they have to jump in and defend Apple (or whichever company they are covering).

oh damn, i didnt know all that about that website, and the article seemed convincing...

[exhaustedbrit]

Honestly the M1 chip being extremely competitive at little over 3GHz is a really convincing start and good competition which can only be a positive thing in the long run. I really can't understand the bitterness of some people, for the first time in maybe 30 years we have genuine competition from three giants, granted the M1 is a SoC/PoP exclusively for Apple hardware - but they're enough of a beast to be looked at as genuine competition, especially given Japan's phenomenal uptick in Mac minis the last few weeks.

The only chips I see that actually trade blows with the M1 in single threaded workloads are the tiger lake CPUs running at 5GHz in meltdown mode and the 5000 series chips from AMD, which at least for now I'd argue is broadly a difficult comparison given how few people can actually buy the 5000 series chip they want. All this performance is irrelevant if the user isn't willing to use a Mac/macOS as their platform or the software is not compiled for Apple ARM - but that much should be obvious.

[LAwLz]

Just now, Jailbreaker said:

oh damn, i didnt know all that about that website, and the article seemed convincing...

Did it really though?

 

Like I said, the argument is "we should run two threads on the x86 cores because they can execute two threads, and only one on the M1 because it can only do one thread" completely misses the point of why "single core" benchmarks are run.

We run single threaded benchmarks like Cinebench and Geekbench to measure how a single threaded program will perform. If the program you want to run can use two threads then it will be able to run on two threads with both the M1 and x86 processors.

 

The entire argument is just him arguing semantics that "single core score" should be called "single threaded score" instead. He kind of has a point, but the message he is trying to push is completely wrong.

The "single core scores" that gets outputted by Cinebench and Geekbench are measurements of how a single threaded program will perform on the given CPU, but that's what people who look at those scores are interested in.

 

He is trying to muddle the water. He might be technically correct when he says one Intel core can do more computations on a single core in a given timeframe than the M1 and thus has "higher single core performance", but when people say single core performance they typically mean single threaded performance, and for single threaded performance the M1 wins.

 

 

What people say: The M1 has higher single core performance than Intel.

What people mean: The M1 has higher single threaded performance than Intel.

What Usman says: Intel has higher single core performance than the M1.

What Usman wants people to think: Intel has higher single threaded performance than Intel (because in their minds the words are interchangeable but Usman has made a very distinct differentiation).

 

He is technically right, but he is using that info to make people jump to the wrong conclusions.

He also doesn't seem to understand why single threaded benchmarks are a thing. If your program can take advantage of two threads (and thus being able to take advantage of SMT), then it will simply run on two cores on the M1. The benchmark results he presents do not show any kind of real world performance. It is just a futile attempt to score some win against the M1 by claiming something along the lines of "if we don't look at performance benchmarks but rather talk about some theoretical performance metric not relatable to any real world program then the Intel core is slightly better than the M1".

[Arika]

Lol, WCCF.

[captain_to_fire]

I thought wccftech is the daily mail of tech news

[RejZoR]

48 minutes ago, exhaustedbrit said:

Honestly the M1 chip being extremely competitive at little over 3GHz is a really convincing start and good competition which can only be a positive thing in the long run. I really can't understand the bitterness of some people, for the first time in maybe 30 years we have genuine competition from three giants, granted the M1 is a SoC/PoP exclusively for Apple hardware - but they're enough of a beast to be looked at as genuine competition, especially given Japan's phenomenal uptick in Mac minis the last few weeks.

The only chips I see that actually trade blows with the M1 in single threaded workloads are the tiger lake CPUs running at 5GHz in meltdown mode and the 5000 series chips from AMD, which at least for now I'd argue is broadly a difficult comparison given how few people can actually buy the 5000 series chip they want. All this performance is irrelevant if the user isn't willing to use a Mac/macOS as their platform or the software is not compiled for Apple ARM - but that much should be obvious.

Though in all fairness, those Intel (or AMD ) chips in "meltdown" mode are that hot because they run all cores at 5GHz. If you'd lock it down to only 1 core that reaches such clocks it would be way cooler.

[Video Beagle]

54 minutes ago, exhaustedbrit said:

we have genuine competition from three giants, granted the M1 is a SoC/PoP exclusively for Apple hardware

I would think that because Apple has shown it can be done, we'll start to see others working toward that as well, it being easier to make a second wheel than a first....more funding and research will be thrown that way in the ARM space because Apple has shown there's something there.  Seems win/win/win to me, whether you like Apple or not.

[exhaustedbrit]

1 hour ago, RejZoR said:

Though in all fairness, those Intel (or AMD ) chips in "meltdown" mode are that hot because they run all cores at 5GHz. If you'd lock it down to only 1 core that reaches such clocks it would be way cooler.

I could be wrong because tiger lake is a newer process & architecture, but I highly doubt it's running that all-core. The i9-9880H I had would pull about 50 watts at 4.8GHz on a single core, this wasn't all core turbo. When I stressed all cores it barely met 3 GHz with the multi core tests, generally because of the voltages required to reach that 5GHz and keep stable, performance per watt drops off a cliff the closer you get, which shows in my anecdotal example that it took the same power to reach 4.8GHz with a single core as it did to swamp 8-cores at 3GHz.

[saltycaramel]

We’re 120 days at most from the new iMac with the desktop version of the M1, for all we know it may have 4-5GHz freq, 65-95W tdp, with 8-12 performance cores instead of just 4.

 

That’s gonna be fun to see the damage control for that.

[Pixel5]

3 hours ago, Spindel said:

I actually had to look this up.

 

There is not a single 4 core 8 thread CPU (desktop or laptop) that beats M1 in benchmarks at least. I actually can't find anything under 8 cores that beats the M1. 

 

Of course benchmarks are benchmarks YMMV in real world workloads. In those usually the M1 seem to perform better against CPUs up to 6 cores but there are some loads where it gets beaten.  

which benchmarks are you looking at here?

also looking for 4 core benchmarks to compare against doesnt really make sense, after all the M1 has 8 cores even if 4 of them are more optimized for power efficiency it can still use all 8 at the same time.

[LAwLz]

37 minutes ago, Pixel5 said:

which benchmarks are you looking at here?

also looking for 4 core benchmarks to compare against doesnt really make sense, after all the M1 has 8 cores even if 4 of them are more optimized for power efficiency it can still use all 8 at the same time.

The efficiency cores sucks ass for performance. They really don't matter at all for performance. All 4 of the small cores doesn't even add up to the performance of a single big core.

You get pretty much the same performance if you run a benchmark on just the 4 big cores as you do if you run them on all the cores. I wouldn't even be surprised if including the small cores will REDUCE performance since they will be competing with the big cores for memory bandwidth.

 

For all intents and purposes, the M1 is a quad core.

[exhaustedbrit]

46 minutes ago, LAwLz said:

For all intents and purposes, the M1 is a quad core.

When running things under Rosetta 2 it appears it's exposed as a quad core part which lands some credence for this, this would make sense as perhaps some of the additional instructions for accelerating memory address translation are not present in the efficiency cores. In CB23 compiled natively all 8 cores are utilised and this results in a MP core ratio of just above 5.

As an experiment, I limited CB23 to four threads and watched utilisation and power usage on my M1 Air. I then did the same with 8 threads.

CB23 4T
Total Package Power: 14593 mW
Score: 5169pts

CB23 8T:
Total Package Power: 15394 mW
Score: 7120pts

Some extra info for anyone who cares:

E-Cluster Power: 1256 mW
E-Cluster HW active frequency: 2064 MHz
E-Cluster HW active residency: 100.00% (600 MHz:   0% 972 MHz:   0% 1332 MHz:   0% 1704 MHz:   0% 2064 MHz: 100%)
E-Cluster idle residency:   0.00%
cpu 0 frequency: 2064 MHz
cpu 0 idle residency:   0.00%
cpu 0 active residency: 100.00% (600 MHz:   0% 972 MHz:   0% 1332 MHz:   0% 1704 MHz:   0% 2064 MHz: 100%)
cpu 1 frequency: 2064 MHz
cpu 1 idle residency:   0.00%
cpu 1 active residency: 100.00% (600 MHz:   0% 972 MHz:   0% 1332 MHz:   0% 1704 MHz:   0% 2064 MHz: 100%)
cpu 2 frequency: 2064 MHz
cpu 2 idle residency:   0.00%
cpu 2 active residency: 100.00% (600 MHz:   0% 972 MHz:   0% 1332 MHz:   0% 1704 MHz:   0% 2064 MHz: 100%)
cpu 3 frequency: 2064 MHz
cpu 3 idle residency:   0.00%
cpu 3 active residency: 100.00% (600 MHz:   0% 972 MHz:   0% 1332 MHz:   0% 1704 MHz:   0% 2064 MHz: 100%)

P-Cluster Power: 14138 mW
P-Cluster HW active frequency: 2983 MHz
P-Cluster HW active residency: 100.00% (600 MHz:   0% 828 MHz:   0% 1056 MHz:   0% 1284 MHz:   0% 1500 MHz:   0% 1728 MHz:   0% 1956 MHz:   0% 2184 MHz:   0% 2388 MHz:   0% 2592 MHz: 1.2% 2772 MHz:   0% 2988 MHz:  99% 3096 MHz:   0% 3144 MHz:   0% 3204 MHz:   0%)
P-Cluster idle residency:   0.00%
cpu 4 frequency: 3204 MHz
cpu 4 idle residency:   0.00%
cpu 4 active residency: 100.00% (600 MHz:   0% 828 MHz:   0% 1056 MHz:   0% 1284 MHz:   0% 1500 MHz:   0% 1728 MHz:   0% 1956 MHz:   0% 2184 MHz:   0% 2388 MHz:   0% 2592 MHz:   0% 2772 MHz:   0% 2988 MHz:   0% 3096 MHz:   0% 3144 MHz:   0% 3204 MHz: 100%)
cpu 5 frequency: 3204 MHz
cpu 5 idle residency:   0.00%
cpu 5 active residency: 100.00% (600 MHz:   0% 828 MHz:   0% 1056 MHz:   0% 1284 MHz:   0% 1500 MHz:   0% 1728 MHz:   0% 1956 MHz:   0% 2184 MHz:   0% 2388 MHz:   0% 2592 MHz:   0% 2772 MHz:   0% 2988 MHz:   0% 3096 MHz:   0% 3144 MHz:   0% 3204 MHz: 100%)
cpu 6 frequency: 3204 MHz
cpu 6 idle residency:   0.00%
cpu 6 active residency: 100.00% (600 MHz:   0% 828 MHz:   0% 1056 MHz:   0% 1284 MHz:   0% 1500 MHz:   0% 1728 MHz:   0% 1956 MHz:   0% 2184 MHz:   0% 2388 MHz:   0% 2592 MHz:   0% 2772 MHz:   0% 2988 MHz:   0% 3096 MHz:   0% 3144 MHz:   0% 3204 MHz: 100%)
cpu 7 frequency: 3204 MHz
cpu 7 idle residency:   0.00%
cpu 7 active residency: 100.00% (600 MHz:   0% 828 MHz:   0% 1056 MHz:   0% 1284 MHz:   0% 1500 MHz:   0% 1728 MHz:   0% 1956 MHz:   0% 2184 MHz:   0% 2388 MHz:   0% 2592 MHz:   0% 2772 MHz:   0% 2988 MHz:   0% 3096 MHz:   0% 3144 MHz:   0% 3204 MHz: 100%)

ANE Power: 0 mW
DRAM Power: 221 mW
Clusters Total Power: 15394 mW
GPU Power: 17 mW
Package Power: 15807 mW

 

[Jeppes]

2 hours ago, saltycaramel said:

We’re 120 days at most from the new iMac with the desktop version of the M1, for all we know it may have 4-5GHz freq, 65-95W tdp, with 8-12 performance cores instead of just 4.

 

That’s gonna be fun to see the damage control for that.

But why would have they ruined the mac mini line with the current m1 then?

[igormp]

6 hours ago, Lord Vile said:

On the flip side the M1 has ML capabilities that traditional chips don't have so if an application can take advantage of that surely benchmarks don't matter because the M1 will be miles faster.

Intel's 10th gen has VNNI, which is basically the same. The difference is that apple can force app devs to use their NPU while intel can't.

[Lord Vile]

6 minutes ago, igormp said:

Intel's 10th gen has VNNI, which is basically the same. The difference is that apple can force app devs to use their NPU while intel can't.

Intel chips don't have a hardware solution for machine learning. VNNI is just an x86 extension using traditional CPU and GPU tech. The M1 uses the neural engine made specifically for ML.

[igormp]

3 hours ago, Lord Vile said:

Intel chips don't have a hardware solution for machine learning. VNNI is just an x86 extension using traditional CPU and GPU tech. The M1 uses the neural engine made specifically for ML.

You do know that usually those x86 extensions require extra hardware, right? CPUs with VNNI have hardware-accelerated FMA for low precision matrices, exactly the same thing that the NPU on the M1 does.

[saltycaramel]

4 hours ago, Jeppes said:

But why would have they ruined the mac mini line with the current m1 then?

Because this is the silver, low-end, mobile-hw-based Mac Mini. And because it was paramount to put out a desktop immediately for developing, especially for developers who weren’t in the DTK program. 

 

If we look at the history of MacMini hardware, the actual exception is the dark gray 2018 one, with a 65W desktop CPU. Not the other way around.

 

Let’s hope they’ll bring this beefier Mini back next summer. Maybe call it “Mac Mini Pro”. In a dark gray chassis.

[Lord Vile]

39 minutes ago, igormp said:

You do know that usually those x86 extensions require extra hardware, right? CPUs with VNNI have hardware-accelerated FMA for low precision matrices, exactly the same thing that the NPU on the M1 does.

It's not exactly the same, like at all. FMA isn't even related to ML.