Neural Engine on M1 is amazing...

[TheReal1980]

We have read all about how good the M1 is in all sorts of ways but something not spoken about as much is the M1's Neural Engine, with 16 Cores and the capability of doing 11 trillion operations per second.

We are now seeing what can be done in the picture software Pixelmator as an example.

 

 

 

My thoughts

To me it seems the M1 is an even bigger deal than most people understand. To be able to pack that kind of punch in a fanless laptop is insane and this Neural Engine is just the icing on the cake.

I wonder how Intel, AMD and different mobile chip developers are thinking right now.

 

 

Sources

https://www.pixelmator.com/blog/2019/12/17/all-about-the-new-ml-super-resolution-feature-in-pixelmator-pro/

[Sousuke]

I've not really looked at the ML Cores on my M1 MacBook Air but this is easily an amazing feature if supported by the software manufacturers! 

[A51UK]

Cool, but RTX GeForce had Tensor Cores for year now and AMD have them own version on them up coming GPU. Also this type of system have been in android phone for some time like in the Huawei Mate 40 Pro.  It other words Android and PC been having this type of system for years and apple just catching up.

 

There nothing new. 

[Vishera]

14 minutes ago, TheReal1980 said:

11 trillion operations per second

Boasting about numbers yet doesn't compare it to anything.

[exhaustedbrit]

3 minutes ago, A51UK said:

Cool, but RTX GeForce had Tensor Cores for year now and AMD have them own version on them up coming GPU. Also this type of system have been in android phone for some time like in the Huawei Mate 40 Pro.  It other words Android and PC been having this type of system for years and apple just catching up.

 

There nothing new. 

The A11 chip released for iOS devices in 2017 had fixed function hardware for accelerating machine learning workloads (INT8/FP16 matrices), at this time nvidia hadn't even released the Volta architecture. This is simply misinformation.

[saltycaramel]

3 minutes ago, A51UK said:

Cool, but RTX GeForce had Tensor Cores for year now and AMD have them own version on them up coming GPU. Also this type of system have been in android phone for some time like in the Huawei Mate 40 Pro.  It other words Android and PC been having this type of system for years and apple just catching up.

 

There nothing new. 

Apple started shipping the NE in every new iphone/ipad starting with the A11 Bionic in 2017.

Now in every new Mac.

Completely different chances of app developers leveraging it in everyday usage for regular people.

But go ahead with your youtube-comment style superficial responses. “Catching up”..

[saltycaramel]

4 minutes ago, saltycaramel said:

Now in every new Mac.

Completely different chances of app developers leveraging it in everyday usage for regular people.

 

That’s what some people don’t get about these AS Macs.

 

Imagine a software ecosystem where that CPU power is the baseline, a given.

Imagine a software ecosystem where that iGPU power is the baseline, a given.

Imagine a software ecosystem where the presence and the speed of that Neural Engine is the baseline, a given.

Imagine a software ecosystem where the speed of that Unified Memory is the baseline, a given.

Imagine a software ecosystem that can assume that the slowest internal storage to EVER ship was 2500MB/s. 

 

What will software leveraging all of this look like in a few years?

What this being the baseline, the bottom of the barrel, the lowest end will imply for developers designing an app?

[LAwLz]

14 minutes ago, A51UK said:

Cool, but RTX GeForce had Tensor Cores for year now and AMD have them own version on them up coming GPU. Also this type of system have been in android phone for some time like in the Huawei Mate 40 Pro.  It other words Android and PC been having this type of system for years and apple just catching up.

 

There nothing new. 

You're being very disingenuous right here.

• On PCs, we have only had NPUs for 1 generation of GPUs, and only from Nvidia. AMD and Intel do not have NPUs in any currently available consumer products. Much less in actual products like laptops.
• If we are going to bring mobile phones into the discussion like you did then this is not the first time Apple uses NPUs either.
• We have "had this type of system for years" (from one manufacturer) and we barely see it used in any applications. Meanwhile, Apple launched their computer with it yesterday and multiple apps are already using it.
• Just because something isn't new doesn't mean it is bad. The 3070 and RX 6800 for example do not really offer anything "new" either. It's performance that was possible to get several years ago, and mostly features that were available several years ago too. But despite that, it is news when it gets announced and released. Right?

[Spindel]

As I wrote in some other thread surrounding the M1 announcement.

It will be interesting to see how developers will leverage the power of the NE that is built in to the base system in other applications. It should be able to provide a compute boost even in non ML applications as long as you write code to use it. 

[Kisai]

32 minutes ago, A51UK said:

Cool, but RTX GeForce had Tensor Cores for year now and AMD have them own version on them up coming GPU. Also this type of system have been in android phone for some time like in the Huawei Mate 40 Pro.  It other words Android and PC been having this type of system for years and apple just catching up.

 

There nothing new. 

Think ASR and TTS. Just to get to "Siri" level of functionality still requires a high end GPU with a lot of RAM. The stuff that runs on laptops and phones is entirely in the cloud. Just to run a neural TTS system requires 4GB of video memory with a single voice, alone. You don't necessarily need all the cores if you're not training a system, but you still need some to use it, and the CPU is phenomenally bad at it, even with all the AVX instructions.

 

Likewise for OpenCV/ARKit, requires some level of functionality here because running it entirely on the CPU is going to have poor performance.

 

To that end, I think this is something that needs to be looked at vs comparable nvidia, amd and intel solutions and since there is no common middleware to test that, it might not be be doable for some time.

[RejZoR]

Benvista PhotoZoom didn't use ML and it had spectacular results years ago before ML was even a thing anywhere. It made some insane results with upscaling. If Bicubic or Lanczos looked blocky and blurry, PhotoZoom's S-Spline Max delivered massive detailed and sharp images that looked light years better than anything else. Worth chekcing out.

Where ML might shine is computational time though...

[Video Beagle]

Interesting.. Pixelmator specifically was called out in MKBHD's video for not working on his M1 mbp. Guess an update happened between him trying it and now.

[TheReal1980]

7 minutes ago, Video Beagle said:

Interesting.. Pixelmator specifically was called out in MKBHD's video for not working on his M1 mbp. Guess an update happened between him trying it and now.

Yeah they came out with an update afterwards.

[Kilrah]

56 minutes ago, saltycaramel said:

What this being the baseline, the bottom of the barrel, the lowest end will imply for developers designing an app?

Probably the same as 2 decades ago, a bunch of quickly thrown together and unoptimized software that runs like shit, worse than good software running on hardware that doesn't have all those bells and whistles...

That's what tends to happen when "we've got more power than we need, let's use it to save us dev effort"

[saltycaramel]

16 minutes ago, Kilrah said:

Probably the same as 2 decades ago, a bunch of quickly thrown together and unoptimized software that runs like shit, worse than good software running on hardware that doesn't have all those bells and whistles...

That's what tends to happen when "we've got more power than we need, let's use it to save us dev effort"

Yeah, just like iOS/iPadOS apps compared to Android apps, am I rite?

[saltycaramel]

Also: if everything moved to the web or cross-platform desktop webapps, Apple CPUs would still be preferable being the absolute champions at crushing every web/html/js benchmark..

[RedRound2]

17 minutes ago, saltycaramel said:

Also: if everything moved to the web or cross-platform desktop webapps, Apple CPUs would still be preferable being the absolute champions at crushing every web/html/js benchmark..

And battery life

Exciting times

[igormp]

2 hours ago, LAwLz said:

Intel do not have NPUs in any currently available consumer products.

Playing devil's advocate, Intel has had the movidius for quite a while now, and Google has the Coral as a competitor. 

 

However, those are pretty useless for the general consumer since most software won't make use of it and it's more of a dev/embedded thing 

[JoseGuya]

4 hours ago, saltycaramel said:

 

That’s what some people don’t get about these AS Macs.

 

Imagine a software ecosystem where that CPU power is the baseline, a given.

Imagine a software ecosystem where that iGPU power is the baseline, a given.

Imagine a software ecosystem where the presence and the speed of that Neural Engine is the baseline, a given.

Imagine a software ecosystem where the speed of that Unified Memory is the baseline, a given.

Imagine a software ecosystem that can assume that the slowest internal storage to EVER ship was 2500MB/s. 

 

What will software leveraging all of this look like in a few years?

What this being the baseline, the bottom of the barrel, the lowest end will imply for developers designing an app?

IB4 more Javascript shenanigans and unoptimization... 

[Blade of Grass]

The real challenge is going to be driving adoption of the CoreML library to benefit from this inclusion. Lots of data science/ML work is done with Python, but as far as I know there are no python libraries that have been updated to support the hardware acceleration yet. Even some other very crucial libraries like TensorFlow do not currently have support (in fact, TensorFlow on macOS doesn't support GPU acceleration because Metal doesn't support templetized kernels). 

 

Bit of a road ahead, but still promising. 

[thechinchinsong]

Apple's custom hardware/software integration in terms is so far ahead of everything else in the market. Some might be able to leverage custom SoCs like Samsung's exynos line, some might be able to hyper-focus on software, like Google, but nobody can match Apple's silicon-software fusion and this is another example of that. A fantastic neural engine on the hardware side coupled with what is looking like a good software base that can only get better.

[StDragon]

A "Deepfake" accelerator.

 

2021 is going to be an interesting year.

[RejZoR]

5 hours ago, StDragon said:

A "Deepfake" accelerator.

 

2021 is going to be an interesting year.

Instagram fake influencers are already going wild with all the sharp jaw lines they could have and big eyes and it'll all be perfect lol

[Fnige]

1 hour ago, RejZoR said:

Instagram fake influencers are already going wild with all the sharp jaw lines they could have and big eyes and it'll all be perfect lol

how big?

[Letgomyleghoe]

14 hours ago, saltycaramel said:

magine a software ecosystem where that CPU power is the baseline, a given.

Imagine a software ecosystem where that iGPU power is the baseline, a given.

Imagine a software ecosystem where the presence and the speed of that Neural Engine is the baseline, a given.

Imagine a software ecosystem where the speed of that Unified Memory is the baseline, a given.

Imagine a software ecosystem that can assume that the slowest internal storage to EVER ship was 2500MB/s. 

this is all hardware.

 

Imagine a hardware ecosystem where 16gb memory is the max.