Apple is (going to) scan your Apple devices

[MageTank]

11 hours ago, Jtalk4456 said:

OR, and stay with me here, parents TEACH their children about online safety and not to send nudes to strangers
Also IMHO, kids don't need a cell phone unless they have a job in high school. 

Ah yes, children that listen to the teachings of their parents without question. A common sight in the wild, lol.

 

That said, I don't disagree with your second statement, but it isn't just cell phones. Parents buy kids tablets with cameras integrated in them, not to mention they have webcams for online schooling (now a requirement during the pandemic). Drawing the line at not buying kids phones doesn't really solve this specific problem all that much, but I get where you are coming from here.

 

If I ever have kids, they'll have a two-way communication watch, out here looking like a budget James Bond character and only be programmed with me and my most trusted allies. You want to sext in my household? You better master the art of carrier pigeons.

[freeagent]

You are being monitored.. make good choices.

 

That is all.

[Commodus]

19 minutes ago, HSF3232 said:

Late to this discussion, but I would like to share my thoughts on this. First, it appears they are using the same system that they use for FaceID, only they revamped it for this purpose. IIRC, that system requires training data. Wonder where they got it from?

Secondly, Apple claims that the system will have a "1-in-1 trillion chance of false positives". Somehow I don't believe that, if the system I used classified star wars soundtrack album art as "drugs". Granted, these systems that Apple is using has way more training system than a consumer forensics toolkit, but mistakes still happen.

 

All in all, if Apple rolls this out, there a going to be a lot of arrests of innocent people. Either because of innocent images that get misclassified or because of consensual "love-bird" images between two adults. In a nutshell - Royal Mail Horizon 2: Electric Bogoloo.

It's not the same system as for Face ID. Image training is a fairly common thing... sites like Facebook already use methods like this to screen for inappropriate material.

 

I also don't think we'll see significant numbers of false arrests. For one thing, the police can't just arrest someone because a hashing system showed a match... you have to prove the content is illegal.

 

Honestly, a lot of this thread can be characterized as people misunderstanding the technology and/or using it as an excuse to justify their existing Anything But Apple mindsets. There's a tremendous irony to balking at this while running into Google's arms.

[RejZoR]

3 hours ago, Roswell said:

From the quote it only mentions the photos app when it backs up to iCloud. It also mentions iMessage, which uploads photos to Apple for delivery/hosting. 
 

So in both instances they are only analyzing photos you choose to upload to their services, not everything on the phone.

Yeah, I've later spotted some articles where Apple allegedly explained it's iCloud only. Which would make sense as Apple hosting such content is what's usually the whole issue for it. So, that's fine for now I guess. I don't use iMessage at all and I don't use iCloud (I use some other service from Switzerland which has servers located in EU). It's still valid to always question such things because they often quickly cross the line and pretend it was something else or downplay it in the name of saving the children.

[Sauron]

Obviously the stated goal is unobjectionable but... that doesn't mean Apple just gets to scan and collect information about every image you store or see on your phone. This stuff is best left to law enforcement rather than AI vigilantism. What's next, automatically snapping a picture at fixed intervals to check if you're currently abusing a minor?

 

I wouldn't want to be the poor Apple employee who gets to check all suspect images... and I don't get how this can even pretend to respect your privacy when you could literally have your nudes sent to an Apple employee for review. I mean, what happens if a minor has explicit photos of themselves on their phone for whatever reason (which I would strongly discourage, by the way)? Do those get automatically collected and looked at by Apple for their ""safety""?

 

I would be ok with just the warnings being locally displayed, after all if it's all local then no harm can be done. anything more is an unacceptable invasion of privacy in my opinion.

[Bombastinator]

52 minutes ago, The Unknown Voice said:

Old News: The Govt has been watching for many years, now.

So, when the Govt doesn't want to fund an operation,, they get the "Private Business" to do it for them.

Just give us the evidence, and we will prosecute.

 

"Junior, Put the evidence in the car"....

 

Implies businesses do stuff spontaneously for free.  To my knowledge this rarely happens, if for no other reason that companies have to justify expenditures during earnings calls.

 

There is an employee in this and probably a lot more than one so this is not a wildly cheap endeavor.  Makes big “why?” noises.  Either there is money to be made here or there’s a regulation requirement, or some kind of voluntary thing to avoid a future regulation, or something.  More to uncover.  I saw a link about it on basically every major news page I read though, so it is being noticed.

[Bombastinator]

5 hours ago, Stahlmann said:

What's a personal iPhone? iPhones are not meant to be owned by the user. This whole topic confirms it yet again.

 

It's kinda sad that the choice nowadays is between bad and worse isn't it?

 

No it's not.

 

While it would be the best thing to do for privacy reasons, it would be a pretty big tradeoff. Nowadays people want to be connected to each other all the time. While i'd probably do fine as i like to call people rather than send a message via WhatsApp etc, most people would just lose contact with a person who decides not to use a smartphone anymore.

That’s sort of always been the choice in most cases. Not saying it isn’t sad though.

[Quackers101]

I thought others did this too to a certain degree?

From google libraries, samsung and others, with their online functionality that I hate but still use since it's default app/storage. (not removeable)

If there was an AI that did check your phone and not for other purposes, then that could be nice for abuse etc, but it's then easy to blurr the lines and maybe make vulnerable people less safe. *gets notified about being gay to apple china* oh boi, but I'm not gay! I swear on the furry above!

 

Kinda sad on how invasive they got to be and force it upon us.

Also when there is talk about porn and hentai, that is a rabbit hole and how spreading such stuff today, what is legal and what is not even if you got it from third parties, what it shows or doesn't show and can be taken out of context and how healthy our sexual habits are and if they need help. Hentai is a massive hole, as you can portray things very differently and be taken quite differently too, from size, age, creature, what it actually shows and what to classify it as. Kind of annoyed at japan though, as they do push some horrid and unhealthy stuff through their business in hentai. While some is just for fun, others are a bit ill and unhealthy content to consume.

[Bombastinator]

10 minutes ago, Betterthanever said:

To me personally this looks like an introduction to "Big Brother made by Apple". Maybe they just plan to release a movie about "Apple Users" or a new commercial and they need to understand how we users use their technology... I don't know how to call it but it looks like a conspiracy between companies. When a company being a giant starts to rule the market it just manifests that it is entitled to do whatever it feels like doing at the time and shows it by sending such notifications... it's just a warning for those who went tooooo far while browsing

Conspiracy between companies doesn’t happen spontaneously, or possibly at all. There are too many people and interests for that to possibly be effective.   There has to be a driver.  Various suppositions on what that driver might be have been proposed, but they don’t seem strong enough to me to justify the kind of expenditure this represents.  I don’t know how expensive it is but it’s not zero.  Something may be being missed.

[Bombastinator]

3 minutes ago, Betterthanever said:

There is one obvious "driver". It is called "how to earn more money in a shorter period of time". This is a challenge that has been accepted long time ago by the biggest companies...

But how are they doing so with this?  On can surmise obfuscated data mining, but data mining sales would show up in a balance sheet and be caught. The driver is not clear.

[Guest willies leg]

I'm thinking they already do with their cloud, but now it's gonna take your power from your battery on your phone to do it.

 

[Sauron]

16 minutes ago, Quackers101 said:

I thought others did this too to a certain degree?

If they do it then they're very good at hiding it. Obviously any image you intentionally upload to a google service is liable to be scanned, however as far as I know and as far as anyone can tell they do not scan images that are just sitting on your phone. They also don't show those potentially sensitive images to employees to determine whether they need to call the police on you or something.

34 minutes ago, Bombastinator said:

There is an employee in this and probably a lot more than one so this is not a wildly cheap endeavor.  Makes big “why?” noises.  Either there is money to be made here or there’s a regulation requirement, or some kind of voluntary thing to avoid a future regulation, or something.  More to uncover.  I saw a link about it on basically every major news page I read though, so it is being noticed.

I would assume they just want to scan your pictures to train their AI and use the concern trolling as an excuse - and if you claim you're looking for minor abuse you'd better at least try to do something about it if you find it.

[Bombastinator]

Just now, willies leg said:

I'm thinking they already do with their cloud, but now it's gonna take your power from your battery on your phone to do it.

 

This seems to be being proposed to be done company side and adds a step to uploads.  So it wouldn’t use consumer side resources.  Resources are being used though.

[Bombastinator]

7 minutes ago, Sauron said:

If they do it then they're very good at hiding it. Obviously any image you intentionally upload to a google service is liable to be scanned, however as far as I know and as far as anyone can tell they do not scan images that are just sitting on your phone. They also don't show those potentially sensitive images to employees to determine whether they need to call the police on you or something.

I would assume they just want to scan your pictures to train their AI and use the concern trolling as an excuse - and if you claim you're looking for minor abuse you'd better at least try to do something about it if you find it.

AI need to be trained for very specific things though.  They’re not actually smart, they learn slow, and they don’t always learn what is expected regardless of if they even ARE being trained for a very specific thing. If this is the case they could use other things that are cheaper and less controversial. This appears to be QUITE controversial judging from the number of column inches it is generating in various major news media. Apple could do that then without creating nearly so much Faff. So why are they? 

Edited August 6, 2021 by Bombastinator

[BuckGup]

I'm confused, so is this using perceptual hashing, which is shit and isn't good enough for this, or does apple have petabytes of child porn they used to train their models on? Considering how anything related to child abuse and porn is guilty until proven innocent and allegations ruin all credibility regardless if they are found to be true or not this is a massive issue. You can now weaponize messages to frame people or at least destroy their credibility. Also this doesn't help solve the issue of child abuse at all. Apple is going to get sued out their ass if they actually implement this

[Bombastinator]

9 minutes ago, BuckGup said:

I'm confused, so is this using perceptual hashing, which is shit and isn't good enough for this, or does apple have petabytes of child porn they used to train their models on? Considering how anything related to child abuse and porn is guilty until proven innocent and allegations ruin all credibility regardless if they are found to be true or not this is a massive issue. You can now weaponize messages to frame people or at least destroy their credibility. Also this doesn't help solve the issue of child abuse at all. Apple is going to get sued out their ass if they actually implement this

The anti-pedo organization groups do which is where the file would be coming from most likely.  Apple is a corporation.  What would it do with porn of any kind? It’s not even alive.

[Sauron]

39 minutes ago, Bombastinator said:

AI need to be trained for very specific things though.  They’re not actually smart, they learn slow, and they don’t always learn what is expected regardless of if they even ARE being trained for a very specific thing. If this is the case they could use other things that are cheaper and less controversial.

There are many different kinds of machine learning algorithms, not all of them require pre-labeled datasets. As for the controversy, "preventing child exploitation" is perhaps the least controversial excuse they could come up with...

46 minutes ago, Bombastinator said:

This appears to be QUITE controversial judging from the number of column inches it is generating in various major news media.

The part that is controversial is the scanning itself. Regardless of why they're doing it, it's still a significant invasion of privacy.

[BuckGup]

8 minutes ago, Bombastinator said:

The anti-pedo organization groups do which is where the file would be coming from most likely.  Apple is a corporation.  What would it do with porn of any kind? It’s not even alive.

Wait what I don't understand the last part about being alive. I was talking more like the death penalty perspective. Killing is wrong and you aren't suppose to do it so we kill you for it. CP is bad and wrong and you shouldn't have it but we need petabytes of it. If anything this leads to new methods to fool AI and the cycle continues. I would have much rather heard Apple is spending billions on mental health programs/facilities in every state to help rehabilitate sexual offenders. But nah some janky AI solution that'll ruin innocent lives because tech broooo AI solves everything.

[Bombastinator]

10 minutes ago, BuckGup said:

Wait what I don't understand the last part about being alive. I was talking more like the death penalty perspective. Killing is wrong and you aren't suppose to do it so we kill you for it. CP is bad and wrong and you shouldn't have it but we need petabytes of it. If anything this leads to new methods to fool AI and the cycle continues. I would have much rather heard Apple is spending billions on mental health programs/facilities in every state to help rehabilitate sexual offenders. But nah some janky AI solution that'll ruin innocent lives because tech broooo AI solves everything.

They need hashes anyway.  Hashes of copies of stuff already being distributed. It won’t stop the perv/non perv arms race dead.  I don’t think anything can. Not seeing your connection with ruining innocent lives.  Seems like a jump. The unknowns here seem to be how actually effective it will be in net reduction, what the costs are, and who is paying them.  This seems to imply the costs is children and this is effectively distribution of such things.  Hashes aren’t pics. On this case it seems “who is paying them” does not include the victims.  The issue seems to be in the conclusion.  A does-not-follow error.

[BuckGup]

I read a break down on arstechnica on how it works and it is using perceptual hashing against the CSAM database. So one could just acquire the CSAM database and change the images by a few pixels and they won't match. If the CSAM is not public then they could test images to see how different they need to be in order to bypass it. Now Apple would have to rehash all the new variants of the image they find and add them to the database. So basically Apple made child abuse images harder to find as people will just implement security measures to bypass perceptual hashing checks. Sure one in a trillion if the images are identical pixel for pixel. For as big as Apple is they have some dogshit management because I guarantee engineers expressed how terrible of a solution this is but here we go again with a Challenger incident. 

[BuckGup]

3 minutes ago, Bombastinator said:

I don’t think anything can. 

I think a total overhaul of mental health and how we deal with it could easily reduce the numbers significantly and over a long period of time potentially zero. 

[wamred]

So, I am glad that they are going to do something about sexual abuse but that does mean that any end to end encryption being used in iMessage is garbage. 

[Bombastinator]

5 minutes ago, BuckGup said:

I think a total overhaul of mental health and how we deal with it could easily reduce the numbers significantly and over a long period of time potentially zero. 

Reduction is not stopping though. I’m sure the mental health angle is being addressed too. Seems to sound like such in the news.  One doesn’t have to give one up to get the other. Which is a more effective use of what resources and with what cost? I don’t know.  That one is still murky. 

[BuckGup]

2 minutes ago, Bombastinator said:

Reduction is not stopping though. 

Correct as the issue overlaps from generation to generation so it could take 3-4 generations to eradicate completely. It's just a much more effective method than something like this. The logistics, cost, and work involved will be monumental that's why no one has done it yet. Also money, this will make some Apple investors happy opposed to a massive money pit were the only return is less pedos but a better society. 

[DildorTheDecent]

Inb4 weebs have their basement raided by the feds because it picked up their loli's as CP.