Originally published at: When Are Summaries Valuable? - TidBITS
Apple Intelligence has introduced uncertainty into formerly verbatim news article notifications, sometimes producing blatantly erroneous summaries. The company’s response to a formal complaint from the BBC and widespread negative media coverage? It will update the feature to perform better. Jason Snell of Six Colors thinks that’s insufficient. As it stands, apps can’t opt out of having their notifications summarized by Apple Intelligence; Jason is calling on Apple to allow individual apps or similar classes of apps to opt out of notification summarizations. I’m with him on this topic—it’s problematic for Apple to put words in the mouths of others. The Verge’s collection of notification summarization mistakes is reminiscent of auto-correct fails, but at least with those, the user can revert to their original text. With news notifications, Apple Intelligence summarizes a collection of unrelated content, often providing actively unhelpful results.
These AI summarization mishaps prompted me to think about summaries in general. I’ll admit to a knee-jerk negative reaction whenever I have been offered an option to summarize, whether AI-generated or not. As a fast reader, I was never intimidated by long books in school, and I picked up on my teachers’ disdain for CliffsNotes summaries of classic works of literature.
Upon reflection, though, my reaction is unfair. While summarization certainly has its problems, dismissing it overlooks something fundamental: summarization isn’t just an overhyped AI feature—it’s core to the human experience.
Think of summarization as a form of lossy compression, similar to how digital photos are compressed to save space. Both attempt to reduce the amount of data required from the original to convey its meaning. Damage is always done in the process—a JPEG-compressed image loses fine details from the original, and text summaries lose detail and nuance. Romeo and Juliet is more than a tragedy about two young lovers whose devotion to each other defies their families’ bitter feud and ultimately ends in their untimely deaths. Thanks, ChatGPT, for getting it right.
We accept the loss of detail because one or more constraints often make summaries more practical or useful for specific purposes. The most common constraint is time—you can read that one-sentence summary of Romeo and Juliet in a few seconds, but watching the play or reading the text would take several hours.
Another constraint is background. Without a solid grounding in physics, you may not get much from reading “On the Electrodynamics of Moving Bodies,” the paper in which Albert Einstein introduced his special theory of relativity. Those of us who lack that background or a desire to achieve such a state—life is finite, and we can only learn so much—are better off knowing that the paper demonstrates that the laws of physics are the same for all non-accelerating observers and establishes the relationship between space and time, fundamentally altering concepts of simultaneity and motion. I hope that’s a reasonable one-sentence summary.
Physical display space is a third type of constraint. When you look at the list of messages in Mail, that’s a form of summary—reading your email as a single scrolling document would be insane. One of Apple Intelligence’s features enhances the message list to replace snippets from the first few lines of an email message with a summary. These summaries will be more descriptive than the snippet preview, as the preview is just the first part of a message instead of its meat. However, they can suffer from the same sort of errors as the news notifications.
The value of a summary is, within limits, inversely proportional to the difference in length between the source and the summary. The more compressed the summary, the better—again, within limits.
Those limits vary by situation—I needed a single sentence for the examples above, but such short summaries lose so much of the originals that they aren’t otherwise all that useful. Asking ChatGPT for longer summaries provides significantly richer results. In other words, there’s always a sweet spot between how tightly the summary compresses the original and how much of the original’s information is retained.
That value explains my discomfort with Apple Intelligence’s summarization options. Because I read quickly, I see no reason to ask Apple Intelligence to generate a summary of a Web page or a conversation in Mail. The downside of losing detail and nuance—and of possible errors—outweighs the upside of saving a few minutes of reading time. Notification summaries are even worse; for me, they save seconds at most and often introduce confusion by summarizing unrelated news articles or information that has changed multiple times within the summary period. The main utility I see for notification summaries is to reduce the irritation of too many notifications from chatty conversations or overactive apps, but Apple has already addressed that by grouping notifications.
While AI-generated summaries raise valid concerns, it’s essential to recognize that human-created summaries permeate nearly everything we read. For instance, every email message and discussion forum post has a subject line that’s supposed to summarize the message’s intent. People often write poor subject lines, but they remain an essential form of summary—one that AI could actually help improve.
That’s just the start. Nearly every article or non-fiction book has a title that is, most of the time, the shortest possible summary the author or editor can think of that is both attractive to a potential reader and accurate to its contents. Many articles, including ours, have short summaries that serve as teasers in a list. All academic papers have built-in summaries in the form of abstracts—I rely heavily on those when researching topics outside my sphere of expertise.
The need to summarize goes even deeper. Most news articles are themselves summaries of the events they cover. Wikipedia may contain 6.9 million articles, but the average length of an article is a mere 690 words—it’s a collection of summaries. While few people would consider a book to be a summary, most non-fiction titles are distillations of the author’s more extensive research.
I would even argue that human language is itself a form of summary. There’s a reason we say that we “choose our words”—we’re summarizing the rich, complex, and chaotic thoughts and feelings in our minds into a limited but hopefully understandable collection of words. Just as summaries lose nuance and detail, language often falls short of conveying precisely what we’re thinking. Without full-bandwidth telepathy, it’s the best we have for sharing ideas. Summaries are intrinsic to human expression.
To summarize—I had to!—summaries offer a different value proposition for everyone. Reading speed, language fluency, topical understanding, display space, and other factors play into how valuable a summary of a particular length will be in any given situation. You should ask for AI-generated summaries only when they will provide actual value and you can verify their accuracy when it matters. Finally, remember that just because something can be summarized doesn’t mean it should be.
) but - Apple Intelligence remains in what Apple calls beta, so I’m fine giving any errors like that a pass for now as long as Apple commits to making it better. And in the case of these erroneous summaries - to me they are a really small matter. And, summaries are under user control - they can be turned off on an app-by-app basis, or even altogether, if users don’t like them.
)
