New CSAM Detection Details Emerge Following Craig Federighi Interview

[ace (Adam Engst)]

Bringing the state actor discussion over from another thread:

@Simon

I’ve pointed this out before, but in principle, the damage is already done. They have demonstrated that they have created this tech and that they can deploy and use it. That is now public knowledge. They can swear they have good intentions all day long (and I’m inclined to actually believe that), but that won’t shield them from the pressure they can now be exposed to by authorities in countries where they can’t just walk away (eg. certainly China and the US).

@silbey

I’m worried about state actors forcing Apple to do this. In the latter case, I don’t see how people notice easily.

@silbey

It’s not that Apple can’t be trusted, it’s that state level actors have a whole different level of power and you don’t create tools for them to use easily, ringed around with promises like “we will refuse” which are not possible to keep.

So let’s think this through.

1. We now know that Apple’s CSAM hash database is the intersection of several other CSAM hash databases, NCMEC’s and at least one from another non-US organization. So a state actor would have to subvert not one, but two or more organizations like NCMEC. Not unthinkable, but significantly harder and more likely to result in exposure.

2. Let’s say Apple’s CSAM hash database is subverted. The only thing that could be put in there would be CSAM itself. Remember, Apple’s human reviewers only see the “visual derivative” of the matched CSAM (and only after there are 30 matches). So if Apple’s reviewers see an image of a dissident or whatever, they’ll chalk it up to a false positive and will send it to Apple engineering for analysis. Which might lead to the exposure of the subversion of the CSAM hash database, since the engineers wouldn’t rest until they figured out how NeuralHash failed (which it didn’t). I have trouble seeing what kinds of images would be useful to gather this way as well, since they have to be known in advance to be matched.

3. So now let’s say that the state actor subverts Apple’s CSAM hash database with more CSAM, with the idea of planting it on the devices of dissidents to discredit them. This is apparently common in Russia. Apple will get the matches, confirm the CSAM, and report the person to NCMEC, and thus US law enforcement. If the person in question is a Russian citizen not on US soil, it seems unlikely that anything of interest happens, unless US law enforcement regularly cooperates with places like Russia on such investigations. This seems like a possible attack vector, but given that Russian dissidents also suffer from poisoning, it seems like way more work than is worth the effort. See xkcd.

4. What about a state actor that completely subverts Apple? There’s a true backdoor in the code (one that no one knows about—it’s not a backdoor if it has been publicized), Apple’s human reviewers have been forced to identify dissidents when their images show up (but how is this useful if the images are already known?), and Apple reports to the state actor instead of just NCMEC. At this point, we’re so deep in the conspiracy theory that you may as well assume every photo you take on your iPhone is being sent directly to China or whatever. The only answer to this level of conspiracy theory is that there are numerous security researchers looking at iOS at all times, and something like this would likely be discovered. And if Apple was so completely subverted, why would they make a public announcement of all this tech?

Am I missing any attack vectors?