"I want to ban Maths" - EU wants Big Tech to Break Encryption and scan private messages

[Lightwreather]

Summary

A European Commission proposal could force tech companies to scan private messages for child sexual abuse material (CSAM) and evidence of grooming, even when those messages are supposed to be protected by end-to-end encryption.

Online services that receive "detection orders" under the pending European Union legislation would have "obligations concerning the detection, reporting, removal and blocking of known and new child sexual abuse material, as well as solicitation of children, regardless of the technology used in the online exchanges," the proposal says. The plan calls end-to-end encryption an important security tool but essentially orders companies to break that end-to-end encryption by whatever technological means necessary

 

Quotes

Quote

    In order to ensure the effectiveness of those measures, allow for tailored solutions, remain technologically neutral, and avoid circumvention of the detection obligations, those measures should be taken regardless of the technologies used by the providers concerned in connection to the provision of their services. Therefore, this Regulation leaves to the provider concerned the choice of the technologies to be operated to comply effectively with detection orders and should not be understood as incentivising or disincentivising the use of any given technology, provided that the technologies and accompanying measures meet the requirements of this Regulation.

    That includes the use of end-to-end encryption technology, which is an important tool to guarantee the security and confidentiality of the communications of users, including those of children. When executing the detection order, providers should take all available safeguard measures to ensure that the technologies employed by them cannot be used by them or their employees for purposes other than compliance with this Regulation, nor by third parties, and thus to avoid undermining the security and confidentiality of the communications of users.

 

Quote

An EC announcement said the problem of CSAM has gotten out of hand and that the current "voluntary" system isn't enough. "With 85 million pictures and videos depicting child sexual abuse reported worldwide in 2021 alone, and many more going unreported, child sexual abuse is pervasive," the announcement said. "The COVID-19 pandemic has exacerbated the issue, with the Internet Watch foundation noting a 64 percent increase in reports of confirmed child sexual abuse in 2021 compared to the previous year. The current system based on voluntary detection and reporting by companies has proven to be insufficient to adequately protect children."

The proposal's detection orders would be "issued by courts or independent national authorities," the announcement said. A detection order would be "limited in time, targeting a specific type of content on a specific service," and instruct the company receiving the order to scan "for known or new child sexual abuse material or grooming." Grooming means "solicitation of children," the announcement said.

Other parts of the proposal "require app stores to ensure that children cannot download apps that may expose them to a high risk of solicitation of children." Additionally, "providers that have detected online child sexual abuse will have to report it to the EU Centre," and "national authorities can issue removal orders if the child sexual abuse material is not swiftly taken down. Internet access providers will also be required to disable access to images and videos that cannot be taken down, e.g., because they are hosted outside the EU in non-cooperative jurisdictions."

 

Quote

Scanning the content of private messages shouldn't be possible with encryption that is truly end-to-end. As Proton Mail explains, "E2EE [end-to-end encryption] eliminates this possibility because the service provider does not actually possess the decryption key. Because of this, E2EE is much stronger than standard encryption."

The European proposal was criticized by security experts including Alec Muffett, a network security researcher who—among other things—led the team that added end-to-end encryption to Facebook Messenger. "In case you missed it, today is the day that the European Union declares war upon end-to-end encryption, and demands access to every person's private messages on any platform in the name of protecting children," Muffett wrote.

 

The EC's announcement said that companies will be instructed to implement the proposed detection orders in the "least privacy-intrusive" way. "Companies having received a detection order will only be able to detect content using indicators of child sexual abuse verified and provided by the EU Centre," it said. "Detection technologies must only be used for the purpose of detecting child sexual abuse. Providers will have to deploy technologies that are the least privacy-intrusive in accordance with the state of the art in the industry, and that limit the error rate of false positives to the maximum extent possible."

However, Bits of Freedom wrote that it is "simply impossible to filter someone's Internet connection the way the European Commission wants." The group explained further with an example involving WhatsApp:

    To give an example: based on this proposal an instant messaging platform can be given the task to detect material of the sexual exploitation of children. That could be known material, or "new" material, or grooming, so text. Let's assume, for the sake of the argument, that the order is given to Meta with regards to WhatsApp. A platform that, as you know, is protected with end-to-end encryption. This type of encryption means Meta can see who is communicating with whom, but is unable to read the content of that communication. But how is Meta supposed to detect something in a conversation it's not supposed to be able to access? For the sake of convenience, the Commission leaves that decision (the "how to do it") to the platform. Our guess is that the only way to do it, is by installing some sort of (now, government-mandated!) spyware on the phones of the people using a particular service. After all, that is the only place where the content of the chats is readable.

My thoughts

 "You were the Chosen One. EU, you were supposed to destroy the Privacy and Security Jeopardists not join them." Oh dear EU, why? Now all will be ruined. Putting the melodrama aside, I'm not really sure what was going through the head of whoever proposed this, because not only is it asking regular citizens to give up a part of their privacy (and security) whilst playing the "Oh won't someone think of the children" card, it's asking Tech companies to do a freaking impossible task!  Now look at me, I'm already barking, I didn't expect to be this fired up today. A couple things: One, this doesn't affect servers outside the EU, so it's likely going to be ineffective. Two, There isn't going to be a way to read through the encryption without also opening it up to hackers all over the world "For on the Internet, there is no such thing as distance." Well, it's a good thing to note that so far this is just a proposal and hasn't been put into legislature yet.

I'll close with a few of CGP Grey's word, "No matter how much we wish it, there is no way to build a digital lock that only angels can open but demons cannot. Anyone saying otherwise is either ignorant of the mathematics or less of an angel than they appear"

ps, I'll link said CGP grey video down below.

Sources

ArsTechnica (Quoted)

EU - Proposal, Q&A, Press Release

BitsofFreedom

Twitter - AlecMuffet, Matthew Green

Euractiv

 

[Forbidden Wafer]

It's about power, not privacy nor security. 

[Senzelian]

Shouldn't the postal office not also start opening letters then?

[Paul Thexton]

Governments (not just the EU): "Put a good-actor back door in to your encryption please"

Everyone: "But that contravenes the very essence of security and privacy. A backdoor will be exploited by bad actors."

Governments: "But only we'll be able to use it, we'll create legislation that says so!"

Everyone: "....."

 

The UK gov have been trying to do this for bloody years, even as far back as the BBM days they were threatening to put legislation in place to allow for legal intercept of messages.  It's almost as if politicians don't understand what peer to peer encryption is or how it works.

[Sauron]

I don't think this is going to solve the problem so hopefully it remains just a proposal.

[Senzelian]

17 minutes ago, J-from-Nucleon said:

it's asking Tech companies to do a freaking impossible task!

Apple is already doing something similar:
https://support.apple.com/en-us/HT212850

 

Afaik this happens all on the device itself, so it doesn't break any encryption. It also doesn't send any data to anyone. It's safe, secure and private, within certain boundries of course and also "according to Apple".

 

[Paul Thexton]

Just now, Senzelian said:

Apple is already doing something similar:
https://support.apple.com/en-us/HT212850

 

 

That's on the device once the actual data pops out of the encrypted transport stream.  Asking to do it in-transit (which I think is what this EU proposal is about?) is a whole different matter.

[Senzelian]

1 minute ago, Paul Thexton said:

That's on the device once the actual data pops out of the encrypted transport stream.  Asking to do it in-transit (which I think is what this EU proposal is about?) is a whole different matter.

Yes indeed that is a completly different matter, but on-device should be the way to go if something like this makes it to market.

[Lightwreather]

1 minute ago, Paul Thexton said:

 Asking to do it in-transit (which I think is what this EU proposal is about?) is a whole different matter.

Well it leaves the How entirely up to the platform, be that in transit or on device.

Neither are particularly nice.

[Paul Thexton]

2 minutes ago, J-from-Nucleon said:

Well it leaves the How entirely up to the platform, be that in transit or on device.

Neither are particularly nice.

Agreed. I'm not going to pretend to know what the correct solution is to detecting and policing online child abuse/grooming, but I'm pretty sure infringing on *everyone's* privacy isn't it.

[porina]

4 minutes ago, Senzelian said:

Apple is already doing something similar:

I was wondering if it was possible to comply with the requirements by scanning on the endpoints only. The workaround to those wishing to circumvent that would be to modify the app to not operate as expected which while far from impossible will make it much more inconvenient. 

[HarryNyquist]

10 minutes ago, porina said:

I was wondering if it was possible to comply with the requirements by scanning on the endpoints only. The workaround to those wishing to circumvent that would be to modify the app to not operate as expected which while far from impossible will make it much more inconvenient. 

That has the possible chilling effect of making open-source software illegal. If the source is open, then anyone can modify the program to remove the scanning methods, and that would be against the laws the governments would write.

[PDifolco]

Those EU fools have no limits to their stupidity... from the same guys that created GDPR, it's hilarious

[TVwazhere]

49 minutes ago, Senzelian said:

Shouldn't the postal office not also start opening letters then?

Reminded me of:

[wanderingfool2]

1 hour ago, Senzelian said:

Shouldn't the postal office not also start opening letters then?

Well I mean if there is reasonable suspicion that a crime is being committed by mail then yea...they are actually authorized to open the mail provided they get a warrant first.

 

 

I don't agree with adding any types of backdoors into devices or encryption.  It is my belief though that there needs to be some mechanism for the authorities to search a device when a warrant is presented...which the only way I can reasonably think of is weakening the self incrimination portion regarding passwords/access.  I've said it before, and I'll say it again...I can understand the reasoning why these types of bills are brought up.  In the modern age of e2e encryption and fully encrypted devices it's at the point where literal serious crimes could be committed without ever any "evidence" available to the police (because it's all locked up on a phone or computer that they cannot access).  I do disagree with backdoors though, but as a society we must admit that there will be a point where it becomes common enough that something somewhere has to give (just I don't think backdoors are the solution)

[darknessblade]

Didn't companies like Facebook, apple, instagram, twitter and so on made a AI for this as well.

 

But it was overly sensitive False-flagging something like family pictures with parents and their children at a waterpark.

 

I remember reading a article about this around a year or so ago.

----

On paper it is a good thing, but in Practice/the real world it could pose nothing but issues, especially if the AI used is overly sensitive, false-flagging even the smallest bit of visible skin on a picture of a "child"

 

 

[porina]

1 hour ago, HarryNyquist said:

That has the possible chilling effect of making open-source software illegal. If the source is open, then anyone can modify the program to remove the scanning methods, and that would be against the laws the governments would write.

It requires those running "online services" to only take those actions when specifically required to do so by an appropriate authority, like a traditional wiretap. Client software in itself does not directly fall under that. Someone attempting to use the client software does not fall under that. Someone outside the jurisdiction of the EU does not fall under that. As such I don't think it would have a significant impact to open source beyond non-compliant solutions being worked on less in EU. Someone sufficiently determined to do so may find alternate communications methods not covered by this. I see the requirement as removing the low hanging fruit and make it more difficulty for those engaged in illegal activities to continue doing so as easily. It doesn't have to be 100% effective to make some impact.

[Rauten]

For a while, it looked like the EU was really knocking it out of the park, and now... this...

*facepalm*

As a European citizen, this is stupid as all hell.

I'm actually trying to find out if there are any proper channels to contact the commission, as a citizen, to tell them how utterly stupid and pointless this is. Ugh.

[poochyena]

3 hours ago, Senzelian said:

Shouldn't the postal office not also start opening letters then?

They already do, especially international packages.

[Quackers101]

just let facebook handle that, so they can give you their password for free *stored in regular text*

While I do agree that more should come to digital communication and court, so important information is not destroyed and as with stated 30 day of stored information on servers for various apps. But also ways to get that information when you get your hands on the device. Like an text chip on communication devices that only delivers but not altered, from text logs. If needing their log quickly in critical cases connected with your phone through USB or other, and access this chip for those details. (but then gets abused by others). With an APP that has the region code and access, and the needed government ID (police etc) and code to access this USB to chip for text information? but dunno, sounds a bit like those "security serialisation chips" that AMD and lenovo, apple have done?

 

messing around with the topic.

Spoiler

But I thought messages was scanned to a degree? maybe not stored, but like keywords, ads etc?

 

Do disagree with the EU, I do wonder if they can have red flag system built into the encryption. Like bomb, school shoot* that gets pinged.

*starts color coding every message and arrested on false positives* and other nations adding their own keywords, oh no. AI detecting behaviors and if you are a western spy!

anyways, leave my keyhole alone!

Edited May 12, 2022 by Quackers101

[HarryNyquist]

1 hour ago, porina said:

It requires those running "online services" to only take those actions when specifically required to do so by an appropriate authority, like a traditional wiretap. 

Regulations/requirements as to when this is needed to be done can be (will be) abused.

[RejZoR]

I remember how ProtonMail once told them to kindly f**k off when they were approached by this idea. Possibly during draft design of this idiotic law. They told them it's impossible without breaking entire cryptographic chain. And I think Signal did the same. EU clowns don't seem to even understand the whol epoint of encryption and why we have it in the first place. Encryption is always absolute and unconditional. The moment you start fiddling with it and placing backdoors in it, you're defeating the whole point of it which also means you're defeating safety of all internet communication. Because at the moment we all presume it's always secure. The moment they start dicking around like this you assume by default that it's insecure by design.

[Ydfhlx]

38 minutes ago, RejZoR said:

EU clowns don't seem to even understand the whol epoint of encryption and why we have it in the first place.

They do understand. And that's precisely why they want to introduce law to break it, because they know they can't spy on citizens without breaking encryption.

 

And before you say "they would never do that!", they will. Literally now there is a spy afera in Poland (member of EU), because they put pegasus spying software on politicians of opposition. If they get legal power to read private messages... Same in the US btw, go read CIA controversies page on Wikipedia.

[Quackers101]

19 minutes ago, Ydfhlx said:

Same in the US btw, go read CIA controversies page on Wikipedia.

CIA, FBI, microphone always on in TV's sold. and much more.

kinda sus ngl.

[wanderingfool2]

3 hours ago, Zodiark1593 said:

. 

It gets back to what I was talking about though, eventually it needs to be talked about in regards to what is and isn't acceptable though.

 

It's easier and easier in this day and age to have crimes where the only evidence is encrypted.  Prior we use to have things like wire-taps, phone record searches, text message searches and other surveillance means.  In the day of e2e encryption and device encryption though there is a whole lot of evidence that is now hidden way.  It's why I think it's reasonable to allow a search based on warrants that require the list of exact information being searched for.  Could it be abused, yea maybe, but realistically not as much as people make it out to be.  I also like this method over the method of having devices scan for illegal content or having backdoors installed on e2e protocols (as either of those could be truly abused)

Edited May 12, 2022 by SansVarnic