Does Linus ever look at the benchmarks of the M1?

[whm1974]

7 minutes ago, Lord Vile said:

Maybe you should look at performance not the brand 

Well MacOS X back when Apple wanted it to meet UNIX Standards and gotten the OS Tested for that, was really good. Even still meet the Standards after switching to Intel.

 

The Current AmigaOS which is something like Ver 4.1 is only a few PowerPC boards, one of which is in the AmigaOne 500 priced around ~$800 or so. Which can run AmigaOS 4.1, MorphOS, or the few Linux Distros still supporting PPC.

 

Better off off getting the Raspberry Pi 400...

[LAwLz]

1 hour ago, dalekphalm said:

People consistently keep dismissing Geekbench - but, do you have a source that actually verifies that it's "not a real benchmark"?

 

I've heard the claim many times. I've also heard the opposite claim - the justification behind the defense of Geekbench is that it's a pure CPU benchmark with little to no RAM usage, whereas other benchmarks often are more CPU+RAM dependent. Essentially Geekbench does realworld tasks like running "C compiler Clang, the PDF reader from Chrome and compression engine from 7-Zip" (snippet taken from a quote by @LAwLz in the M1 review thread).

 

Now, I personally don't quite believe the hype around the M1 - it definitely looks good, and I think compared to U series Intel CPU's and comparable low power AMD CPU's, I bet it probably is just as good as they've said.

 

With that in mind, I do not believe that the M1 is faster and more powerful than an 8-core/16-thread Zen 3 CPU or even the higher end Epyc/Threadripper ones, etc. It might be true, but I'm not convinced about that yet.

For those interested, Geekbench has a fairly detailed breakdown of each of the individual tests. Here is the link.

Like you said, the "problem" with Geekbench is that the data sets are very small so there is not a whole lot of memory usage going on in the tests.

For example the compile test I mentioned is only 1094 lines of C code. The HDR test uses images that are 3 megapixels in size. 

 

That doesn't mean it isn't a good test though. It just means it might not give the full picture of how a system performs.

[dalekphalm]

12 minutes ago, LAwLz said:

For those interested, Geekbench has a fairly detailed breakdown of each of the individual tests. Here is the link.

Like you said, the "problem" with Geekbench is that the data sets are very small so there is not a whole lot of memory usage going on in the tests.

For example the compile test I mentioned is only 1094 lines of C code. The HDR test uses images that are 3 megapixels in size. 

 

That doesn't mean it isn't a good test though. It just means it might not give the full picture of how a system performs.

My interpretation of that, is that Geekbench is specifically benchmarking the CPU, while isolating the rest of system performance as much as possible. Would you say that's an accurate understanding of what it does?

 

Overall system performance is very important - particularly when reviewing an entire system as one product (Example: M1 Macbook Air or whatever). But if you're trying to isolate the specific performance of just the CPU, I would think these tests can be very useful, along with other tests.

[SpaceGhostC2C]

On 11/24/2020 at 12:33 PM, dalekphalm said:

People consistently keep dismissing Geekbench - but, do you have a source that actually verifies that it's "not a real benchmark"?

I don't think there is a discussion to be had about "being a real benchmark", since anything can be: if i want to open and close the calculator 64 times in a row and measure the total time it took in each system, I can do that. I guess the only requirement for a benchmark is repeatability, that is, that the results for the same system are always the same or at least don't vary too much. Hence, I don't think there is room to say Geekbench is "not a real benchmark".

 

What most people take issue with are benchmarks that, for whatever reason, are not typically used suddenly being the center of attention when they favor a particular product. And for all the cultists' cries about Apple hate, M1 isn't even the most salient case: look at the reactions to Nvidia's "gigarays" results, or to Intel's "real world" benchmarketing. it is always possible to innovate in benchmarks, and one could always argue that the industry has been wrong all along and actually some new metric is the relevant one, but then one would have to argue how exactly these other metrics are worse and how this new or less used information is more useful. Indeed, Intel has been trying, and failing, at doing that. How many reviewers told us the geekbench scores for other CPUs in their reviews? And know some did. How much attention did anyone pay to those results, however? Does anyone remember, or based their purchase decisions or recommendations in the past, on the relative Geekbench results of a Ryzen 7 vs an i7? Did we measure Intel's inter-generational improvements by looking at Skylake geekbench vs Kaby Lake geekbench?

 

Far from being held to harsher standards, I think Apple actually gets kind of a pass. Technically, if you want to run a software benchmark to test hardware, you should run the exact same software. As far as I know, M1 can't run Windows, so for most reviewers the only result would be "it didn't even run" (note: this literally appears in other product reviews), at least for reviewers who only test Intel or AMD CPUs on windows software. I guess Level1techs could do a Linux showdown, given that they test under Linux too, assuming you can get Linux to run on the M1. Otherwise, you are already not doing a "benchmark" because you are changing the software used from product to product.

Now, for the subset of programs that have versions for different OSs, you could in principle do a "hardware+OS" joint benchmark, in which you are testing two things at the same time (say, Photoshop in MacOS with CPU1 and Photoshop in Windows with CPU2), but you will not be able to draw conclusions on the silicon by doing that. Maybe the only way around it is to test Windows on Mac vs Windows elsewhere, or Linux everywhere, or Mac with MacOS vs. Hackintosh. But Photoshop in my Mac vs Photoshop in this other laptop is not a CPU test, it's a full product test (cue intel marketing comparing laptops with different dGPUs ).

 

Yet instead of seeing an avalanche of reviews saying "it can't run Windows, so it failed 100% of our standard benchmarks, move along", people actually go out of their ways to find something that resembles a comparable benchmark across architectures, which for the reasons stated above it will never 100% be, and end up presenting this mixture of test that aren't quite standard in CPU reviews, and confound the CPU with other characteristics of the end products, to accommodate Apple and say something useful to the viewer. And then, because among all of these imperfect tests that already deviate from a standard CPU review, they choose not to overplay one particular test, it turns out they all just hate Apple....

 

 

[LAwLz]

45 minutes ago, dalekphalm said:

My interpretation of that, is that Geekbench is specifically benchmarking the CPU, while isolating the rest of system performance as much as possible. Would you say that's an accurate understanding of what it does?

Yes I'd say that an accurate understanding. I don't think they intentionally designed it for the purpose of only testing the CPU in isolation though, even if that's what it does. That's just a consequence of designing a benchmark that can run on really low end mobile devices. You can't really design a test that loads 2GB worth of data if the test needs to run on a device that only has 512MB of RAM.

 

It's also worth noting that the M1 has a freakishly fast memory interface. It is absolutely insane how good the memory is, probably thanks to being on the same chip.

So the thing that Geekbench doesn't measure, is one of the M1's big strengths. If Geekbench had been more reliant on memory then we might have seen the M1 perform even better relative to other processors.

 

Edit: Found some stats for the memory on the M1. Please bear in mind that the Zen numbers are very rough:

  

On 11/17/2020 at 4:48 PM, LAwLz said:

The "problem" is that the workloads are very small in terms of memory size, so it doesn't stress the memory very much. This means that a really good CPU but with weak memory interfaces will look just as good as a really good CPU with a really good memory interface.

The thing with the M1 is that it has a REALLY fucking good memory interface. 

 

The 3900X has something like 20GB/s bandwidth for copies, and around 15GB/s for writes. I think the 5950X has similar results.

The M1 gets 61GB/s for copies and around 34GB/s for writes. And it does that with similar memory latency to Zen3 as well. It is fucking insane how good these results are.

 

[LAwLz]

24 minutes ago, zhnu said:

What does it matter what it scores on geekbench? Meanwhile I'm here wait for them to have a proper software support so it can be compared to other devices.

It has "proper software support".

[LAwLz]

1 hour ago, zhnu said:

Really so Adobe Suite not running properly is just a little annoyance not like it's a productivity software, or an hypervisor?
Proper software is not running their own stuff and calling it a day.

The Adobe suit runs properly. 

It runs as good of not better than on the previous gen Intel Macs. 

There are also betas out if you want it compiled for ARM which makes the performance even higher. But that's more like icing on the cake. 

[LAwLz]

41 minutes ago, zhnu said:

See this is what I don't understand have you seen it run? It lags, Adobe said it will release next year proper versions of their software people that need to work don't run on betas.

Well if you are in the minority of people who run Photoshop, and you aren't satisfied with the performance of it running in Rosetta (even though it seems to perform about the same as on the Intel Macs before it), and don't want to run the beta then sure, you should probably wait a couple of months for the full release. 

 

That's a rather niche group of people though, and your statement was way more generalized than that. Just because one specific program hasn't been fully optimized in the stable branch doesn't mean it doesn't have "proper software support". It's such a subjective term it's a fairly meaningless statement. 

[Rlmva]

On 11/23/2020 at 1:07 PM, dalekphalm said:

It's rather odd because it seems to be one or the other.

 

The "more traditional" forum members seem to frequently be outright apple haters, in which they will very easily call people sheep or isheep or apple fanboys for the smallest things.

 

Then you've got the folks who are claiming that Apple can make a DIY CPU that would "destroy" AMD/Intel.

 

And there's me in the middle - I don't particularly like macOS, but I don't hate it. I do quite like iOS and iPhones, but I'm sure no Apple fanboy. I'm also not an Apple hater either.

I agree with this a lot, mostly because I don’t really like MacOS (mostly because I didn’t grow up using it and figuring it out) but I also don’t think MacOS is bad, I just don’t really know how to use it. I also use an iPhone (the only cell phone I’ve ever had, besides a windows phone for a week) I guess you could also say I am split down the middle.

[Rlmva]

On 11/22/2020 at 2:57 PM, Lord Vile said:

He’s been very dismissive of it.

I agree with this because it’s kind of hard to point to a specific video and time because of when he talks about the M1 (a lot) it just seems like he doesn’t consider it to be up to spec with intel and amd cpus, which is fair, considering I did the same thing. But after I looked at benchmarks and tests I was like “wow, this thing doesn’t totally suck!” But yeah, he just seems kinda dismissive of it. 

[LAwLz]

8 hours ago, zhnu said:

A "niche group" of people who use Photoshop, Unity, Docker, After Effects, Illustrator, etc there's a lot of tools that don't work properly.
Sure if you only want to use apple software it all works well, the majority of  people want to run more than that. Support will come eventually I know that but by then you'll have better options.

The only thing you have mentioned that doesn't work on the M1 is Docker (since it's a VM). 

Everything else works. Just because it doesn't perform as well as it potentially could doesn't mean it performs poorly or doesn't work at all. Far from it. 

Non-native apps works so well on the M1 that they often outperform the native x86 apps. 

[Arika]

15 minutes ago, LAwLz said:

The only thing you have mentioned that doesn't work on the M1 is Docker (since it's a VM). 

Everything else works. Just because it doesn't perform as well as it potentially could doesn't mean it performs poorly or doesn't work at all. Far from it. 

Non-native apps works so well on the M1 that they often outperform the native x86 apps. 

Given that it's a first generation product, the fact that it runs as good as it does right now, is actually pretty incredible.

 

Remember people, just because Apple made it, doesn't mean it's a bad product. you can hate Apple as much as you want (and i will continue to do so as well), but they have created something pretty amazing.

 

*aggressively points down at signature*

[Lord Vile]

56 minutes ago, Arika S said:

Given that it's a first generation product, the fact that it runs as good as it does right now, is actually pretty incredible.

 

Remember people, just because Apple made it, doesn't mean it's a bad product. you can hate Apple as much as you want (and i will continue to do so as well), but they have created something pretty amazing.

 

*aggressively points down at signature*

You can point at the signature all you want people won’t listen, bring it up in most threads regarding Apple but that hate is just too strong 

[LAwLz]

1 hour ago, LAwLz said:

The only thing you have mentioned that doesn't work on the M1 is Docker (since it's a VM). 

Everything else works. Just because it doesn't perform as well as it potentially could doesn't mean it performs poorly or doesn't work at all. Far from it. 

Non-native apps works so well on the M1 that they often outperform the native x86 apps. 

I found a benchmark from Puget where they tested the M1 vs desktops.

 

 

Please note that this is Photoshop running on Rosetta 2, not natively ARM.

 

The idea that you can't run Photoshop on the M1 Macs is completely and utterly false. Just because it isn't optimized yet doesn't mean you can't run it or that it performs poorly. For crying out loud a system with a Ryzen 5950X, 64GB of RAM and an RTX 3080 "only" gets ~85% higher score.

If the argument is that "Photoshop doesn't perform well enough on the M1" then the same can be said for the vast majority of Windows PCs. I don't think "only" half the performance of a 5950X, 64GB of RAM and an RTX 3080 in Photoshop is a strong argument for "it doesn't perform well enough".

 

It's a shame that Puget didn't test the old Intel Macs so that we could get more of an apples to apples comparison. I wouldn't be surprised if the M1 outperforms the old Intel Macs though, and I never heard anyone say "a Macbook doesn't have proper software support because Photoshop doesn't run properly".

[LAwLz]

20 minutes ago, zhnu said:

Really you got to read no one said it didn't run

This is our conversation:

You: The M1 Macs does not have "proper software support".

Me: Yes it does.

You: The Adobe suite does not run properly on the M1.

Me: It runs properly. It works, and performance is about the same as the previous Intel Macs.

You: Have you seen it running? It lags.

Me: Well if you aren't satisfied with the performance in Rosetta 2 then you should probably wait. Just because it doesn't perform as well as it could do doesn't mean it doesn't have "proper software support" though. 

You: Photoshop does not work properly.

Me: But it does work. Just because it doesn't perform as well as it potentially could doesn't mean it performs poorly not doesn't work. Here is a benchmark of how well Photoshop performs on the M1 even when it is not running natively. 

 

 

What would you classify as "proper software support"? 

Is it not enough that programs work and run about as well as the previous generation for it to "work properly" in your eyes?

I think the "problem" here is that you don't really care about performance or compatibility, because both of those are good on the M1, even in Photoshop. 

 

Here is a video from photoshopCAFE, where he runs different tests on the Macbook Pro 16", the M1 natively and the M1 through Rosetta.

Even when you run Photoshop in Rosetta, the performance is the same or better than the Intel Macs. In some specific tests the Intel one performs better, and in some tests the M1 (even when running through Rosetta) performs better.

 

So let me ask you this, what exactly do you mean when you say "proper software support"?

Do you mean performance, compatibility or something else? Because the M1 Macs has both performance and compatibility comparable to the older Intel Macs, even in non-ARM-native programs.

 

 

20 minutes ago, zhnu said:

it lags it's not usable for regular work

What would you define as "regular work"? I'd say the M1 is more than enough for most jobs. From the list of things you have posted, Docker is the only one that doesn't work at all so yeah, if your work relies on docker then the M1 is not ready for your job yet.

Everything else you have mentioned though, including Photoshop, both runs without problems and performs as expected so I really don't see why someone couldn't use them for work.

Also, the fact is that "regular work" and "Photoshop" are not really the same category. Most people basically only need some email app, a browser and maybe some spreadsheet to do their jobs. That is regular work. Processing pictures in Photoshop (which does work really well on the M1) is not "regular work", it's "niche work".

 

23 minutes ago, zhnu said:

people calling out defects it's not hate it's a statement of facts.

1) I don't think what you are describing is a defect.

2) I never said what you did was make "hate statements".

3) Your post are very, VERY lacking when it comes to facts. They are 99% opinions and vague stuff. I am trying to have an objective conversation with you by posting things like benchmarks from multiple sources but you responses are basically putting fingers in your ear and going "lalala you're wrong and I am right and I won't explain why".

[Blademaster91]

3 hours ago, Lord Vile said:

You can point at the signature all you want people won’t listen, bring it up in most threads regarding Apple but that hate is just too strong 

But there wasn't anything to judge the product with except some vague pointless graphs and a claim of being "2X faster" with no indication of what its being compared to. But that and any valid criticisms or facts can't be brought up in apple threads or people call names or accuse anyone bringing up facts a hater.

Also going to leave this here, people need to understand the difference between an unboxing and a review.

 

[Lord Vile]

12 minutes ago, Blademaster91 said:

But there wasn't anything to judge the product with except some vague pointless graphs and a claim of being "2X faster" with no indication of what its being compared to. But that and any valid criticisms or facts can't be brought up in apple threads or people call names or accuse anyone bringing up facts a hater.

Also going to leave this here, people need to understand the difference between an unboxing and a review.

 

If you know how to graph you can tell it’s being compared to the something around the Ryzen chips like the 4300U

[BlakSpectre]

On 11/24/2020 at 8:12 AM, whm1974 said:

Yawn!!! I swear the only Users worse then those of The Cult of Apple are the Members of The Cult of AmigaOS. Those Guys Make Apple Worshipers look perfectly Sane...

 

Yes AmigaOS is still around. It is at Ver 4.1 for PowerPC CPUs/SoCs.

I used to think that the apple cult was much better than the anti apple cult because they did not actively insult, degrade and demean people on "the INTERNET" (except blog writers) like the anti apple cultists do. This whole thing proved that I was wrong.... the gap between them is relatively minor.

 

On an unrelated note: It is a bloody RISC chip, based on TSMC 5nm. It better be more power optimized for things it is designed for, there is a reason every Intel attempt at mobile x86 failed and every phone uses arm chips. Does not take away from the engineering apple did, but I think keeping in mind the benefits the architecture affords to M1 as evaluate the jump that is this chip would be beneficial.

Edit: Clarification, last paragraph was regarding the power efficiency using ARM affords. I dont think the text was eloquent enough.

[whm1974]

8 minutes ago, BlakSpectre said:

I used to think that the apple cult was much better than the anti apple cult because they did not actively insult, degrade and demean people on "the INTERNET" (except blog writers) like the anti apple cultists do. This whole thing proved that I was wrong.... the gap between them is relatively minor.

 

On an unrelated note: It is a bloody RISC chip, based on TSMC 5nm. It better be more power optimized for things it is designed for, there is a reason every Intel attempt at mobile x86 failed and every phone uses arm chips. Does not take away from the engineering apple did, but I think keeping in mind the benefits the architecture affords to M1 as evaluate the jump that is this chip would be beneficial.

Edit: Clarification, last paragraph was regarding the power efficiency using ARM affords. I dont think the text was eloquent enough.

Well I never demeaned people who using whatever Platform they preferred. But I do point out any Issues that Platform may have that I'm aware of. In case of Apple Macs, well they are expensive and "Walled Gardens".

https://en.wikipedia.org/wiki/Closed_platform

 

The AmigaOS 4.1? Sorry but I am not going to buy a well used PowerPC Mac or a SAMcr PPC board to run that.

[BlakSpectre]

1 minute ago, whm1974 said:

Well I never demeaned people who using whatever Platform they preferred. But I do point out any Issues that Platform may have that I'm aware of. In case of Apple Macs, well they are expensive and "Walled Gardens".

https://en.wikipedia.org/wiki/Closed_platform

 

The AmigaOS 4.1? Sorry but I am not going to buy a well used PowerPC Mac or a SAMcr PPC board to run that.

Mate I never said you demeaned people for buying what they wanted.

It gets annoying when people keep telling me how I overpaid for a mac like I am stupid but as long as they are not insulting I don't think they belong to the anti apple cult.

 

I did the math, I have like 11 computers collected over time and 4 of them are macs and unless I am gaming they get used the most. I know what I am buying into. The $800 a year for my Mac primary is well worth having a managed unix env for me. It is similar to $700 year I end up paying for the XPS (my Linux primary) and have to deal with issues installing linux on it brings.. (pop OS has made things simpler tho) and the worse battery life. And the performance is the same for both machines for my tasks, (Mac actually is a bit better for short tasks due to optimizations and XPS on long running tasks). And do not talk of cheaper/heavier devices.. i have burned through enough of those because of bad linux support or just literally dying just out of warranty.

(with that said, i have set up servers now and will be downgrading to $350/year on mac when I buy the next one and use the server for power hungry tasks as i am gradually introducing that to my workflow).

 

What I am trying to say is that most anti mac people just don't understand the benefits it may bring to a user and just assume other people are stupid and have not considered the pros and cons of the decisions they have made. Some people prefer the convenience it brings, some people prefer the feature set. People non stop "dunking" on the supposed expensiveness of those machines as they sometimes do gets annoying.

These devices are tools, like a screwdriver... some people need a ratcheting one some a longer one.. some need one with replaceable head and some need an electric one... they all will have to live with the trade-offs of decisions they have made.

 

PS: I just vented on you... sorry about that.

[whm1974]

1 minute ago, BlakSpectre said:

Mate I never said you demeaned people for buying what they wanted.

It gets annoying when people keep telling me how I overpaid for a mac like I am stupid but as long as they are not insulting I don't think they belong to the anti apple cult.

 

I did the math, I have like 11 computers collected over time and 4 of them are macs and unless I am gaming they get used the most. I know what I am buying into. The $800 a year for my Mac primary is well worth having a managed unix env for me. It is similar to $700 year I end up paying for the XPS (my Linux primary) and have to deal with issues installing linux on it brings.. (pop OS has made things simpler tho) and the worse battery life. And the performance is the same for both machines for my tasks, (Mac actually is a bit better for short tasks due to optimizations and XPS on long running tasks). And do not talk of cheaper/heavier devices.. i have burned through enough of those because of bad linux support or just literally dying just out of warranty.

(with that said, i have set up servers now and will be downgrading to $350/year on mac when I buy the next one and use the server for power hungry tasks as i am gradually introducing that to my workflow).

 

What I am trying to say is that most anti mac people just don't understand the benefits it may bring to a user and just assume other people are stupid and have not considered the pros and cons of the decisions they have made. Some people prefer the convenience it brings, some people prefer the feature set. People non stop "dunking" on the supposed expensiveness of those machines as they sometimes do gets annoying.

These devices are tools, like a screwdriver... some people need a ratcheting one some a longer one.. some need one with replaceable head and some need an electric one... they all will have to live with the trade-offs of decisions they have made.

 

PS: I just vented on you... sorry about that.

That is OK. I am well aware of Issues Linux can when dealing with them. I went to Linux back in 2001/2002 from BeOS.

[LAwLz]

10 minutes ago, zhnu said:

 

What is your point exactly?

 

Can you give me a timestamp or at least explain what I should be listening for? Just linking an entire video as a response to multiple question is yet another very vague and ill-defined response from you.

I don't want to guess which question you are responding to, guess what you mean when you use vague words like "proper support" and guess which part of the video refers to which part of my post.

[LAwLz]

37 minutes ago, zhnu said:

Someone made a video with issues they had after the first week of use starts at 58s. The video isn't very long.

-video-

Yes, and?

You are not responding to anything I said. I asked you several questions and your response is "someone on the Internet says they have some issues sometimes with the M1". So what? That doesn't answer the questions I asked you.

 

Also, the comments you made predates this video, so it's not like you saw these issues and then made your comment. My guess is that you made your comment, got called out and then started searching for evidence to validate your point, rather than base your point on evidence.

[Blademaster91]

2 hours ago, zhnu said:

Someone made a video with issues they had after the first week of use starts at 58s. The video isn't very long.

-snip-

I guess you can't make someone look at proof, with the M1 being new it doesn't surprise me that people may be having issues.

And not everyone will want to trust their work to an emulation layer or want to figure out what works and what doesn't. It's not the same as running the application natively, not sure why the comparison is being made in benchmarks either lol.

[LAwLz]

2 hours ago, Blademaster91 said:

I guess you can't make someone look at proof, with the M1 being new it doesn't surprise me that people may be having issues.

And not everyone will want to trust their work to an emulation layer or want to figure out what works and what doesn't. It's not the same as running the application natively, not sure why the comparison is being made in benchmarks either lol.

There is no proof in that video. Please don't barge into a conversation you obviously haven't read.

The person I am talking to are throwing around wild and unsubstantiated claims and very vague wording. You can't just post a video without saying anything else to someone who just asked you multiple questions.

Also, I am very open to proof. I just want him to accurately define what his arguments are first. That's what I keep asking for and he ignores.