Adam’s absolutely right about the Fourth Amendment applying to government entities not private ones. That said, that doesn’t mean that Apple searching your phone without your permission (or car or house) wouldn’t be illegal. What constitutes “your permission” is an interesting question.
1 Like
You may be right there…I’m sure folks like you are much better informed about all things Apple than us mere mortals are. Now whether than actually change the way that this is supposed to work because of what they see here or elsewhere…or whether they just send our FAQs to try and silence the rumbling…doesn’t really change what they’re doing and why. Just like all the complaints I see on MacStories and other places about the horrible things being done to iOS and iPadOS Safari…yes, the thing keeps changing in beta; but will/does Apple really go back to the old way of how Safari works on iOS/iPadOS or will they just do what they’re going to do anyway? There might be a little of the former…but by and large Apple’s going to do what Apple is going to do and mostly isn’t going to change course because a few users complain. They’ve always been about a consistent UI and not allowing things to change…even though many users would prefer a different thing than Home Screens and default apps…but Apple in it’s wisdom ignores that (mostly) and does what they feel like doing. Given that history…are they really going to change how this works because a few users complain about it? I’m thinking the odds on that are pretty darned low.
Protecting children is good…but as I said in another message on the thread this will only affect a small percentage of the relevant material around. Anything that was originated by an individual user…or that was shared by another user to him/her and hasn’t been through the official declaration as illegal and got the classifying/hashing voodoo done to it…will not be discovered by this system…and while I’m not an expert on the subject I would guess the home grown variety quantity of material far exceeds the known/classified/hashed quantity…so this seems much more like a feel good PR move so that Apple can say “we’re doing something about it”…and it may even be more secure and less privacy invasive than what FB/Twitter/google/whatever is doing…but it’s still a big step back from user privacy first which has heretofore always been the top priority. I just can’t see how something that (a) won’t affect most of the material in question and (b) is thus mostly a feel good PR thing is worth it for taking that big step back.
1 Like
Maybe your filter is different from mine…but I haven’t noticed any incivility or personal attacks…just passionately expressed ideas and explanations of their opinions. It ain’t my circus here though…it’s Adam’s so he and whoever he allows get to make the rules and enforce them…and the rest of us get to live by those rules even if we don’t particularly like some of his decisions regarding what gets posted…but then his filter might be as different from mine as Glenn’s is.
Just sayin’
Technically…you’re correct. However…if Apple is turning this over to the cops…or to an organization that will turn them over to the cops…then IMO they’re almost acting as a government agent and are at least violating the spirit of the constitutional right if not the actual letter of the law and we would have to see what the courts said about that.
If Apple was officially acting as a government agent for this…then I believe that the courts have held that the constitutional right still applies. If Apple is not acting as an agent…and happens to come across it accidentally and turns it into the cops…then the evidence is admissible in court.
However…if Apple is doing this as a result sort of a wink wink nudge nudge conversation in the mens room that was along the lines of “it would be nice if you could do t his but we’re not officially asking”…then IMO they are acting as a de facto government agent and the right applies.
1 Like
Useful TechCrunch interview with Apple’s head of privacy.
Since 9/11 we’ve all been asked, “if you see something, say something.” This can be thought of as Apple saying something. If the authorities take action after being informed by Apple, they will be acting only after obtaining a warrant.
Another way to think of this is that Apple is taking action to prevent collections of CSAM from being uploaded to their iCloud servers. If they scan after the upload, that means that they were hosting these collections prior to the scan.
Only dumb or careless people will be caught. I don’t think that this is really about catching child pornographers, etc. - I think it is about Apple not wanting collections of this material on their property.
IMHO that’s a false equivalency.
Apple isn’t just saying something after they happen to “see” something. They’re building the Hubble Space Telescope and then plan to report on observing alleged occurrences in the outer galaxy.
In essence, they are now trying to sell to us that their telescope can only be used to detect criminal behavior at the far outer edges of the galaxy and that if we’re all good citizens of the universe, we have nothing to fear from their peaking.
1 Like
Yep! Which has led to countless people of color getting the cops called on them by overaggressive observers, so I’m not sure we should invoke that as a model for Apple.
1 Like
The app requires parents to opt in to run the app on the iPhones of their kids; they need to make the choice to activate it. It works by scanning Messages and iCloud images, not everything that lives on anyone’s iPhone. The app does not directly scan the photos on the phones. It uses hash case matching instead, and Apple’s live monitors will check the images to verify them. They won’t look at anything that isn’t isn’t a match with the database. If the match is verified, children and parents will be warned. Children will also be prevented from sending images that match the database. They are not randomly scanning photos on anyone or everyone’s iPhones, as the CEO of Facebook’s WhatsApp and many others are claiming:
“By the way, do you know which messaging service isn’t encrypted? Facebook’s. That’s why Facebook is able to detect and report more than 20 million CSAM images every year sent on its services. Obviously, it doesn’t detect any messages sent with WhatsApp because those messages are encrypted, unless users report them. Apple doesn’t detect CSAM within Messages either.”
1 Like
You’re still conflating two separate parts of the program, MM. You need to stop doing that.
1 Like
IMHO, two halves = one whole.
OK, first off, @silbey is right. The Communications Safety in Messages and the CSAM Detection for iCloud Photos uploads are completely different systems that work in completely different ways. The only sense in which they’re related is that they’re both designed to protect children. So let’s not continue that branch of the discussion.
Second, Alex Stamos of Stanford has an extremely good thread on Twitter about this.
1 Like
As I said before…I’m happy for Apple to see something say something…but they should scan on their end and not my end. That’s better…much better…for my privacy on my iPhone and also according to a link Adam (I think) posted earlier today Apple might be in the wrong legally if they scan on the device.
Transmission of an illegal image to anybody but the NCEC (or whatever it’s name is) is expressly prohibited by federal law…the article referenced above has the cite…so if Apple is scanning on the device and then transmitting the ‘highly suspected as illegal’ image to themselves instead then technically they’re in violation of federal law. Checking on their end…they did no transmission to themselves…but saw something on an image that was uploaded (exactly as google, FB, et al do) and then tell the appropriate authorities.
For Apple to scan on our devices…and you know that most likely eventually they’ll change this to all photos on the device period…is essentially saying (as another reply suggested) that all iPhone users are subject to a warrantless search because of a few bad apples. And while Apple doesn’t need a warrant…they’re getting quite close to acting at the behest of the government here and in that case the warrant requirement applies as well as privacy and unreasonable search rights.
I applaud them for doing…something…even through I personally think it’s a feel good PR thing and not an actual help with the problem…but they’re giving up a long held “user privacy is our primary goal, it’s in our DNA” position for essentially little or no gain that I can see.
My personal guess is that either Tim Cook…or somebody that has his ear…has a mission in life to help eliminate this illegal material, perhaps because of a family member it happened to or whatever…and this whole thing started out from that mission. I’m against illegal material myself…but an effective solution that doesn’t treat all users as guilty until proven innocent seems like a much better “privacy first” company approach. I don’t know what exactly that approach should be…but this doesn’t seem like the right one…especially as we absolutely know that the Chinese will require Apple to scan for their ‘we don’t like these images’ hashes as well…China will pass a law saying they have to do so and Apple obeys local country laws so they’ll give in just like they did with other privacy related stuff in China.
1 Like
Another article that may be useful here. The writer here points out why what Apple is doing is legal, but what we’re afraid of is not.
Of course, if you believe that Apple will violate the law and their own policies despite their statements to the contrary, this won’t do a thing to change your mind.
1 Like
A good analysis, but he seems to miss one key point (that everybody else seems to have missed as well).
The CSAM database is a set of hashes of specific, already identified photos.
If someone takes a different picture of the same subject, and that photo’s hash isn’t in the CSAM database (e.g. because it hasn’t been circulated), then Apple’s scanner won’t detect anything.
This is what makes it less than useful for governments trying to use it to crack down on subversive people. Sure, they can use the tech to detect, block and report specific images, but they can’t use it to detect all similar images.
In other words, while China may be able to use this tech to block well-known photos and videos of Tank Man, they wouldn’t be able to block the generic category of all content referring to the Tiananmen Square massacre, unless they somehow managed to collect a copy of every picture, video clip and audio recording taken. And even then, they still wouldn’t be able to block new and original content (e.g. artwork, re-enactments, discussion and analysis, etc.)
It’s the difference between (for example) looking for a pirated copy of a music song and looking for any person or band’s cover of that song or for every audio recording of people taking about the song. One is pretty easy (once you have the database of hashes) and the other requires a massive amount of computing horsepower.
And this is why on-device scanning is actually better than in-cloud scanning.
The amount of processing needed to accurately identify the subject of a piece of media (photo, video, audio clip, .etc.) is more than can be done practically on a phone. A phone might be able to do the scan if it has enough memory and a big enough neural processor, but it would consume all of its CPU power while scanning (making it get very hot and drain the battery) and the AI model necessary to accurately perform the identification would be huge - enough that people would easily notice the amount of storage consumed by it. Especially when you consider that, to be useful, the model would have to be updated rather frequently (to accommodate new subjects to detect), meaning lots of very large downloads and re-scans on a regular basis.
In other words, although it might be possible, it would have a massive impact on user experience, and there is no way they could do it in secret. People would immediately notice the massive and constant drain on all system resources, investigations would happen, and we’d all find out very soon afterward.
On the other hand, that kind of surveillance/analysis is nothing unusual for a cloud-based service, and it happens all the time. While we certainly know about much of it (e.g. YouTube content matching), I’m certain there’s a lot more analysis going on that we never find out about until some whistleblower creates a scandal.
In short, Apple’s decision to do all this scanning on your phone and not in the cloud probably goes a long way toward preventing the kinds of abuses many of us are afraid of. Of course, the tech can still be abused, but not nearly to the same extent that would be possible with in-cloud scanning (which some have said would be better).
2 Likes
That would mean that someone owning CSAM could change something small about a known image and have it escape detection. I’m pretty sure Apple is doing something to guard against that, which suggests that no, it’s not quite the same as only the exact precise image will get picked up.
Yep! From Apple’s tech summary:
“The main purpose of the hash is to ensure that identical and visually similar images result in the same hash.”
Emphasis mine. What “visually similar” means is an interesting question.
When I say “similar”, I’m not describing the same thing Apple is.
According to Apple’s CSAM technical summary document, they are using a “Neural Hash” algorithm. This document doesn’t explain it in detail, but they say it generates the hash based on “perceptual features” of an image instead of the specific bit-values.
People familiar with lossy audio encoding (e.g. MP3, AAC) will recognize this term. When used for data compression, you delete data that the algorithm believes will not be perceived by the listener, in order to get better compression ratios. I think something similar is going on here - an algorithm is identifying the data that is most significant to recognizing the subject of the image and is only hashing that.
Their example shows an image converted from color to B&W without changing the hash value. I would assume that other basic transformations (different color depths, different DPI resolutions, cropping out backgrounds, rotations, etc.) would also not impact the hash unless they are so extreme that the subject is no longer recognizable (e.g. reduce the resolution to the point where the subject is significantly pixelated).
Assuming this is correct, it would be hard (but definitely not impossible) to modify an image enough to produce a different hash. The intent of the algorithm (as I understand it) is that the kinds of changes needed to produce a different hash will be big enough so that the result is no longer “visually similar” to the original.
But this is talking about detecting well-known images after various photographic transformations have been applied to them. They are not talking about trying to (for example) detect a different photo of the same person, which would generate a different hash.
Again, this algorithm, if used for a hypothetical situation of the Chinese government looking to squash dissident material, would be able to detect well-known photos of Uyghur concentration camps, but it wouldn’t be able to detect photos of these camps not previously known to the Chinese government (e.g. ones that were recently taken, or taken from a different angle).
1 Like
Given the lack of detail in Apple’s explanations and the assumptions you’re making in your analysis, I don’t think we can be particularly confident about things.
1 Like
Apple actually supplies quite a bit of detail, but do you want a lesson in neural networks?
A convolutional neural network is a common model that is used for image processing. It is used for many different purposes including image identification (e.g. identify what a photo is a picture of), object detection (e.g. identify every instance of different kinds of known objects), pose detection (e.g. determine the position of a person’s body) and facial recognition.
The NeuralHash algorithm is an instance of this that is trained to produce a large number (a “hash”) that uniquely identifies an image.
They train the neural network using the original CSAM images (from the NCMEC database). But they don’t just use the original image for training. They apply a variety of transformations to the image (e.g. color space changes, rotation, scaling, etc.) and train the network on those images as well, so that they will all produce the same hash. And they use transformations that produce dissimilar images for negative training (ensure that they don’t produce the same hash).
Assuming they didn’t screw it up in the implementation, the result will be a neural network that is trained to recognize images from the set in the NCMEC database, that is resilient to image alteration but with a low probability of false positives.
Unfortunately, I don’t know enough about the guts of neural network training to be able to explain this in greater detail. I hope it helps you understand what’s actually going on in here.
1 Like