ARM designed an image signal processor.. and yes, it's important (well, I guess not for this forum)

EunSoo · April 27, 2017

ARM has been quietly residing in the heart of the mobile revolution, designing the processors and graphics systems that go into our phones. This week ARM has expanded its portfolio to also include image signal processors (ISPs), which is the first major product from its year-old Imaging and Vision Group and a major foray into pushing mobile camera technology forward. The product they announced is the Mali-C71 Image Signal Processor, the first product in the Mali family. It was built from the ground up for self-driving cars and to manage and process the data from the sensors and cameras throughout the car.

Quote

As the number of cameras per vehicle increases and sensor fusion technologies get smarter, the ISP function – processing raw pixels into high quality images for displays – encoding and further processing by computer vision algorithms is being centralised within the ADAS SoC. Therefore the SoC is capable of managing multiple cameras and delivering reliability and the highest levels of safety in the automotive market. This requires a new approach with built-in functional safety features and parallel virtual processing output for computer vision applications. To date, implementing this in a single SoC has been an elusive goal. Until now.

- ARM

Quote

ARM’s ISP exists specifically to address that weakness of the mobile market. Its Mali-C71 iteration targets autonomous cars and all the situational awareness they need to achieve through imaging, so it’s a little overpowered for smartphones, but the company tells me that its mobile solution is already released to silicon partners and we can expect it to become official some time over the next few months.

In terms of integration into a mobile chip, the ISP is little different to ARM’s GPU designs. A silicon vendor licenses the intellectual property from ARM and then works to integrate the ISP into its system-on-a-chip design, which would also include the various CPU and GPU cores as well as power management and other integrated parts. Then a smartphone maker can just order an all-in-one solution to build a new smartphone or tablet device around: CPU, GPU, and ISP all nicely tucked into the same chip.

- The Verge

ARM’s ISP puts the image sensor’s data through 15 stages of refinement and correction: de-noising, dead pixel correction, de-mosaicing, tone mapping, white balance, color space conversion, gamma correction, sharpening, and then final adjustments to account for whether you want to show the image on the device’s own display or export it elsewhere.

Quote

while the focus of ARM’s announcement is on the ISP IP itself, the company is offering more than just hardware. Mali-C71 is intended to be a combined hardware + software product, with ARM also including a full reference software suite to go with the ISP. Customers will still want to tune the system to their specific needs – and ARM provides those tools as well – but it means that customers can hit the ground running with ASIL-compliant software as well.

- AnandTech

If you want to learn more about this, you can read more about it on the Anandtech and ARM article:

https://community.arm.com/graphics/b/blog/posts/driving-next-generation-image-processing-for-automotive

http://www.anandtech.com/show/11293/arm-announces-mali-c71-automotive-isp

SansVarnic · April 27, 2017

Very cool

EunSoo · April 27, 2017

3 hours ago, SansVarnic said:

Very cool

I'm glad at least one other person here appreciates it. I love reading about this kind of stuff in the news.

rattacko123 · April 28, 2017

So ARM is an ISP?

EunSoo · April 28, 2017

Just now, rattacko123 said:

So ARM is an ISP?

haha no. but i bet they would do a better job than most ISPs out there.

HelplmChoking · April 28, 2017

Awesome, read about it this morning, really exciting stuff.

Godlygamer23 · April 28, 2017

Bear in mind the diagram says Defect Pixel Correction, not Dead Pixel Correction - while a Defect Pixel Correction could include a dead pixel, it's not exclusive to that.