New CSAM Detection Details Emerge Following Craig Federighi Interview

At the risk of speaking for @ace:

  • There’s a difference between scanning for a particular kind of content (e.g. offensive images) and scanning for a specific set of well-known images without any kind of content identicfication.

  • Apple doesn’t have the NCMEC files. But they have the NeuralHash values for these files, which NCMEC computes and sends to Apple. The algorithms (both device-side and server-side) use this database to determine if an image is or is not one of the NCMEC files.

Apple wrote a technical description (which I attempted to summarize) that explains exactly what is being done.

A superbrief summary is:

  • Apple generates a “blinded hash” database from all of the NeuralHashes provided by NCMEC. This database is stored on iCloud servers and on all iPhones equipped with the software (distributed via iOS updates).
  • Your phone, as a part of the iCloud upload process, computes the NeuralHash of each image and generates a derivative image (a low-res version of the original) and encrypts them using two different algorithms (PSI and TSS). The contents of the blinded hash is used to generate the key used by the PSI algorithm. The encrypted data is called a security voucher and is uploaded with the image.
  • The nature of PSI is that the security voucher can not be decrypted unless both the image’s NeuralHash belongs to the blinded hash (meaning it’s in the CSAM database) and Apple’s secret key is known. Since your phone doesn’t have the secret key, it can not know if the image matches the database or not.
  • The nature of the TSS algorithm is that, after decrypting the PSI layer, the actual content can’t be viewed unless a threshold number of PSI-matching images have also been uploaded. Craig has said that the threshold is 30.
  • In order to prevent Apple from knowing how many matches have been uploaded before the threshold has been crossed, your phone also uploads synthetic vouchers, which match the PSI algorithm but always fail the TSS algorithm.

The upshot of all this is:

  • The system can only detect NCMEC’s images (or basic transformations of them, like color-space changes, cropping, rotation, resizing), not other images, even if they are of similar subject matter.
  • Your phone doesn’t know if any images match the database
  • Apple doesn’t know if any images match the database until 30 matching images have been uploaded. Once there are 30, Apple can view the derivative images of the matches, but not of any other image.

And, has also been mentioned several times, Apple will have humans review these derivative images, to make sure they really are CSAM and not false-positive matches, before law enforcement is notified.

2 Likes