Apple Abandons Controversial Plans to Detect Known CSAM in iCloud Photos

[DrMacintosh]

In addition to Advanced Data Protection in iCloud, Apple has announced that it has abandoned its plans to screen your photos for CSAM. Apple Provided this statement:

 

Quote

After extensive consultation with experts to gather feedback on child protection initiatives we proposed last year, we are deepening our investment in the Communication Safety feature that we first made available in December 2021. We have further decided to not move forward with our previously proposed CSAM detection tool for iCloud Photos. Children can be protected without companies combing through personal data, and we will continue working with governments, child advocates, and other companies to help protect young people, preserve their right to privacy, and make the internet a safer place for children and for us all.

 

My thoughts

Back in 2021 when Apple announced its intentions to use this hashing tool to check against known CSAM, people were furious. Despite Apples attempts to show that the service was "secure enough" and had privacy in mind (despite its end goals to scan every photo and document you have for potentially incriminating photos), there was no good way to spin it. The feature was lambasted by security professionals, politicians, and users around the globe. I'm happy that Apple has decided to not push this feature forward and has instead decided to double down on privacy, even though adding full encryption puts more responsibility on the user and abandoning this feature means potential criminals will have to be caught outside of iCloud. 

 

Sources

https://www.macrumors.com/2022/12/07/apple-abandons-icloud-csam-detection/

[AlTech]

Good but the damage of proposing it has already been done. Apple will forever be known as having wanted to implement it and only backing down when faced with incredible opposition.

 

Also, we can only take Apple at their word that they have backed down because IOS isn't open source. So I'll believe it when an independent auditor is able to view IOS' source code to verify that it's gone.

[WolframaticAlpha]

6 hours ago, DrMacintosh said:

In addition to Advanced Data Protection in iCloud, Apple has announced that it has abandoned its plans to screen your photos for CSAM. Apple Provided this statement:

 

 

My thoughts

Back in 2021 when Apple announced its intentions to use this hashing tool to check against known CSAM, people were furious. Despite Apples attempts to show that the service was "secure enough" and had privacy in mind (despite its end goals to scan every photo and document you have for potentially incriminating photos), there was no good way to spin it. The feature was lambasted by security professionals, politicians, and users around the globe. I'm happy that Apple has decided to not push this feature forward and has instead decided to double down on privacy, even though adding full encryption puts more responsibility on the user and abandoning this feature means potential criminals will have to be caught outside of iCloud. 

 

Sources

https://www.macrumors.com/2022/12/07/apple-abandons-icloud-csam-detection/

What will be very funny is how to people who supported apple for this earlier are going to spin it now

[leadeater]

6 hours ago, AluminiumTech said:

Good but the damage of proposing it has already been done. Apple will forever be known as having wanted to implement it and only backing down when faced with incredible opposition.

Thing is I respect a company that listens to feedback and makes changes based on that. 

[LAwLz]

Good. It was a terrible idea to begin with and I am happy they scrapped it.

Even if their intentions were good (which I believe they were), the whole "let's treat everyone has a potential criminal and maybe we'll catch a few" mentality that often gets used to push for various things always rubs me the wrong way.

Not to mention this had the potential to be abused if it ended up in the wrong hands. 

[Commodus]

8 hours ago, AluminiumTech said:

Also, we can only take Apple at their word that they have backed down because IOS isn't open source. So I'll believe it when an independent auditor is able to view IOS' source code to verify that it's gone.

We do have to take Apple's word on it, but there's also no good reason to think the company is lying.

 

And the simple reality is that most major consumer platforms are unlikely to ever be truly open source; even Android has proprietary code if your phone has Google apps. We can't all be Richard Stallman, waiting for the FOSS paradise that will never come. So we either put a reasonable amount of trust in companies like Apple, Google and Microsoft, or become a Stallman-like hermit who has less real-world freedom than a Chinese iPhone user.

[hishnash]

5 hours ago, LAwLz said:

Good. It was a terrible idea to begin with and I am happy they scrapped it.

Even if their intentions were good (which I believe they were), the whole "let's treat everyone has a potential criminal and maybe we'll catch a few" mentality that often gets used to push for various things always rubs me the wrong way.

Not to mention this had the potential to be abused if it ended up in the wrong hands. 

If you're going to scan for that content who apple did it was the best way to scan.   All the other photo services currently scan for this server side. Scanning client side is much better for a few key reasons:

* It is possible for third parties to audit what the hashes that are being used are (yes this was done by sec firms when this feature was in beta)
* Worth remembering this local scanning was only scanning content as you were about to upload it. It was not scanning local content that was set to not be synced. 
* Much harder for potential abuse (govments etc) to be able to manipulate this than a server side scanning solutaiton
* even through at the time iCloud Photos was not end to end this system was designed to deal with that possibility such that it only resulted in the full decryption key being exposed if enough photos passed the threshold (each time the phone flagged a photo it put a small part of your encryption key with it in the metadata.. this meant by design a single image being flagged would not expose your lib only if you had 100s of images flagged were there enough parts of your crypto key to open up your lib to be inspected). 


When you compare this to what others are doing (already) in the form of server side scanning (google, etc) apples approach was much better.  

[hishnash]

12 hours ago, AluminiumTech said:

Good but the damage of proposing it has already been done. Apple will forever be known as having wanted to implement it and only backing down when faced with incredible opposition.

 

They backed down very fast, within in being in beta (optionaly that you turned on) for less than 3 weeks.  And it is worth nothing every other cloud photo provider does this currently without any of the protections that apples solution had. If you're upset at the idea of what apple did then you would be extremely upset at what google are doing and have been doing for the last 5+ years server side. Scanning client side just before you upload the image is much better than scanning server side after the image is uploaded since client side the list of hashes can be audited (and this was done by sec ops companies) and you cant apply new hashes retrospectively to an image after upload. Also how it was done meant only once enough images were flagged by your device (on upload) was enough of your crypto key exposed to let anyone look at these images so just having a few random images be flagged did not result in the police looking at them as they were still encrypted. 

[Kisai]

7 hours ago, WolframaticAlpha said:

What will be very funny is how to people who supported apple for this earlier are going to spin it now

I don't think anyone did.

 

It's generally a bad idea to let the device "decide" to do something against it's users wishes, even if it's deplorable. That's a slippery slope of "well if apple can block CEI, then apple can block all porn, be it real, cartoon, snapchat filters, ai-generated, etc"

 

If there is a concern about CEI, then that belongs to the public access point. If one adult receives images from someone they had consent from, and then the machine goes "uh oh, that looks like CEI, better block it", you see how it starts mistaking images and work around it. There was no reason for CSAM to exist, even if good-intentioned.

[mr moose]

17 hours ago, Kisai said:

I don't think anyone did.

 

 

I remember quite a large thread on the topic back when it was news that contained a few who supported it vehemently.  But for whatever reason I cannot find it using the search function.

[LAwLz]

3 hours ago, mr moose said:

I remember quite a large thread on the topic back when it was news that contained a few who supported it vehemently.  But for whatever reason I cannot find it using the search function.

I found the thread but I didn't really see anyone defending it in the 5 or so pages I went through. 

 

 

There were some people talking about the technical details and saying things like "I want more info before deciding if this is good or bad". That's as close to someone defending it that I could find. Everyone else was against it for varying degrees of legitimacy. 

[Levent]

I doubt anything is killed. I wouldnt be surprised if they pulled a Google and kept on developing this thing further under a different name and covers.

[mr moose]

18 hours ago, LAwLz said:

I found the thread but I didn't really see anyone defending it in the 5 or so pages I went through. 

 

 

There were some people talking about the technical details and saying things like "I want more info before deciding if this is good or bad". That's as close to someone defending it that I could find. Everyone else was against it for varying degrees of legitimacy. 

The two most usual suspects for defending apple were more to the end (page 9 or so).  But it wasn't as bad as I remember (thank fuck for failing old man memory, the world may not be as shit as I think it is).

[divito]

Chalk one up for Epstein.

[BlueChinchillaEatingDorito]

On 12/8/2022 at 9:38 PM, AluminiumTech said:

Apple will forever be known as having wanted to implement it and only backing down when faced with incredible opposition.

You mean like how all companies are viewed? LMG, Microsoft, Apple, Google, ... Air Canada, the list goes on. I don't think there is any company that will immediately back down once faced with any slight bit of opposition. 

[WolframaticAlpha]

On 12/10/2022 at 9:13 PM, LAwLz said:

I found the thread but I didn't really see anyone defending it in the 5 or so pages I went through. 

 

 

There were some people talking about the technical details and saying things like "I want more info before deciding if this is good or bad". That's as close to someone defending it that I could find. Everyone else was against it for varying degrees of legitimacy. 

Oh well, then I must be misremembering it. Apologies.

[Brooksie359]

On 12/9/2022 at 12:40 PM, Kisai said:

I don't think anyone did.

 

It's generally a bad idea to let the device "decide" to do something against it's users wishes, even if it's deplorable. That's a slippery slope of "well if apple can block CEI, then apple can block all porn, be it real, cartoon, snapchat filters, ai-generated, etc"

 

If there is a concern about CEI, then that belongs to the public access point. If one adult receives images from someone they had consent from, and then the machine goes "uh oh, that looks like CEI, better block it", you see how it starts mistaking images and work around it. There was no reason for CSAM to exist, even if good-intentioned.

I think the biggest issue would he teenagers and nudes. It's an unfortunate truth that most of cp out their is probably just kids sending each other nudes and if Apple started to scan for that I can imagine it would be a nightmare. I mean do they now send all the teenagers who are sending nudes to each other to jail for cp? 

[Obioban]

On 12/15/2022 at 2:44 AM, Brooksie359 said:

I think the biggest issue would he teenagers and nudes. It's an unfortunate truth that most of cp out their is probably just kids sending each other nudes and if Apple started to scan for that I can imagine it would be a nightmare. I mean do they now send all the teenagers who are sending nudes to each other to jail for cp? 

Apple's system only detected photos from the known CSAM database, so it would not have flagged those photos.