Originally published at: Apple Intelligence Arrives in macOS 15.1 Sequoia, iOS 18.1, and iPadOS 18.1 - TidBITS
In keeping with its WWDC promise of a slow rollout (see “Examining Apple Intelligence,” 17 June 2024), Apple has released macOS 15.1 Sequoia, iOS 18.1, and iPadOS 18.1 with support for the first set of Apple Intelligence features, with more to come. Accompanying those releases are watchOS 11.1, tvOS 18.1, and visionOS 2.1.
Changes Unrelated to Apple Intelligence
Before I look at the Apple Intelligence features, I want to call out a few unrelated changes in macOS 15.1 Sequoia, the only update for which Apple has published release notes:
- iPhone Mirroring now allows you to drag and drop files, photos, videos, and more between your Mac and iPhone.
- A new option in the App Store app’s settings lets you automatically download and install games and apps to an external drive.
- You can send Game Center friend invites directly from the Contacts app, and Friend Suggestions and receivers can see the invites in the inbox in Settings. These features also appear in iOS 18.1 and iPadOS 18.1.
Despite holding back on the published release notes so far, Apple did call out a few other changes in iOS 18.1 and iPadOS 18.1 when installing the updates:
- Control Center has new controls to add connectivity controls individually and reset your configuration.
- The App Store now lets you use natural language searches.
- Podcasts no longer incorrectly marks unplayed episodes as played.
- 4K videos recorded while the device was warm no longer stutter in playback scrubbing in Photos.
- Digital car keys should now work with vehicles with passive entry after restoring from a backup or transferring directly from another iPhone.
- Apple fixed a bug that could cause iPhone 16 models to restart unexpectedly.
Also, when the AirPods Pro 2 with firmware update 7B19 are used with an iPhone or iPad running iOS 18.1 or iPadOS 18.1 or later, they gain the promised Hearing Test, Hearing Aid, and Hearing Protection features (see “Apple Releases AirPods 4, Adds Hearing Aid Mode to AirPods Pro 2,” 12 September 2024).
watchOS 11.1 also includes a fix for an issue with writing breathing disturbances data to HealthKit. tvOS 18 and visionOS 2 only have bug fixes.
Finally, there are numerous security fixes in all the new operating systems, along with security updates to the older iOS 17.7.1, iPadOS 17.7.1, macOS 14.7.1, and macOS 13.7.1. None of the vulnerabilities are being exploited in the wild, but with 50 fixes in Sequoia and 27 in iOS 18.1 and iPadOS 18.1, you’ll want to update to take advantage of them.
Initial Apple Intelligence Features
The initial Apple Intelligence features are the real news surrounding these releases and the main reason you might want to update. However, you may not be able to take advantage of them unless you have new enough devices. Apple Intelligence is available only on M-series Macs, the iPhone 15 Pro and iPhone 16 lineups, and iPads with an A17 Pro or any M-series chip. Otherwise-compatible Intel-based Macs and older iPhones and iPads can still upgrade but lack the processing power for Apple Intelligence (see “The Real System Requirements for Apple’s 2024 Operating Systems,” 12 July 2024).
Writing Tools
For some, the most compelling Apple Intelligence features will be Writing Tools, which are available in nearly any app that accepts text. A new Edit > Writing Tools menu appears in most Apple apps and will undoubtedly start to appear in third-party apps as well. Until then, you can usually access Writing Tools by selecting some text, Control-clicking the selection, and looking in the Writing Tools submenu. Capabilities include:
- Proofread: The Proofread tool attempts to fix spelling, grammar, and punctuation mistakes. Although Proofread appears to do a decent job (if nowhere near what Grammarly can do), it walks you through changes only in a few Apple apps like Mail and Messages, making it difficult to use in other apps. In those apps, you have to pore over the original and the rewrite to determine what would change. Nor does it tell you if it has no suggestions. You can replace or copy the suggested revision.
- Rewrite: The Rewrite tool suggests different versions of your text, with options to make it friendlier, more professional, or more concise.
- Summarize: Along with a straight summary, Writing Tools can also create a list of the key points in your text, make a list based on the selection, and create a table.
Although I thought all Writing Tools relied on local processing rather than Apple’s Private Cloud Compute data centers, when testing the summarization tools, I was informed that “certain capabilities” were unavailable at that time.
Siri Improvements
Apple has redesigned the Siri interface, replacing the animated circle with a glowing light on iPhone and iPad, while on the Mac, it now features a floating lozenge that displays voice input or lets you type Siri commands. Typing to Siri is also available on the iPhone and iPad—double-tap at the bottom of the screen to reveal the text field. I don’t love the new Siri interface because of how it makes everything on the iPhone screen go all wobbly; you can eliminate that animation by turning on Settings > Accessibility > Motion > Reduce Motion.
In theory, Siri has a richer language understanding that will let it follow along even if you stumble over your words or change them mid-sentence. Plus, Apple claims Siri maintains conversational context throughout a session. I’m unsure what sort of conversations you could have with Siri that would benefit from these improvements. As far as I can tell, Siri’s only new trick is that it knows about the contents of the Mac, iPhone, and iPad user guides, so you can ask it how to accomplish tasks explained in those guides.
So far, I’m underwhelmed. I haven’t found any of the purported improvements to make a difference in my Siri usage. Plus, Apple has said nothing about Siri on the HomePod, which is by far the most common destination for my Siri commands.
Photos Enhancements
Thanks to Apple Intelligence, Photos gains three new features:
- Clean Up: The marquee addition is Clean Up, which can identify and remove distracting people and objects in your photos. It’s not perfect, but in the right conditions, it can work well. I’ll be looking more at Clean Up shortly.
- Descriptive search: Although Photos has long had a machine-learning-powered search that lets you find photos based on their contents, you can now find photos based on more extensive and natural descriptions. It also seems to perform pretty well.
- Descriptive memory movies: The Memories feature in Photos has always left me cold—it’s not how I want to interact with my photos—but if you like it, perhaps you’ll enjoy creating movies by describing what you want in them. Some of my tests were unsuccessful, presumably due to too few photos.
Priority Messages and Summaries in Mail
In Mail, Apple Intelligence powers a handful of features that might enhance the email experience:
- Priority: Mail attempts to identify conversations that are more important to you and display them at the top of the message list. The conversation it chose for me was reasonably important, but oddly, I couldn’t display or open the messages.
- Message summaries: In the message list, Mail can now summarize the message or conversation rather than showing the first few lines. That’s a win and seems to work well.
- Conversation summaries: For longer messages and conversations, a Summarize button appears at the top of the message pane. Every time I click it, Apple Intelligence processes briefly and then reports, “Unable to summarize these messages.”
- Smart Replies: A Smart Reply feature suggests replies in both Mail and Messages. The replies are reasonable but obvious and may be worth using if you were going to write the same thing anyway.
Even though I have Mail configured to access my Gmail and iCloud accounts, I don’t use it regularly, so something may be messed up with my installation.
Notification Summaries
As with messages in Mail, Apple Intelligence can summarize notifications, either individual notifications that contain more content than can fit in the notification or stacks of notifications that encapsulate multiple messages or actions. If you don’t like it for certain apps, it’s easily turned off in Settings > Notifications > Summarize Previews.
Audio Recording, Transcription, and Summarization
Finally, both the Notes and Phone apps can now record, transcribe, and summarize audio. When you turn on recording in the Phone app, it notifies participants. I’ve tested this feature in Phone only once, on a customer support call, but the tech’s accent was too strong for Apple Intelligence to cope.
I had better transcription luck in Notes. Recording Apple’s Glowtime event produced a decent transcript of the talk, partly because the audio was very high quality (I had positioned my iPhone next to the speaker through which the sound was playing). There were still many mistakes, but I found it helpful to be able to search the exported text. A second test, at a public talk about a book to which my mother had contributed, proved less successful because the iPhone had trouble picking up the audio—it was sitting in my lap in the fourth row of a small lecture room. It also failed miserably at nearly all proper names.
The combination of audio and transcript was quite valuable. I appreciated being able to jump around in the audio by tapping within the transcript, and I’ve returned to the Glowtime transcript several times to confirm what Apple said.
Interestingly, the Writing Tools summaries of the transcripts eliminated many of the more egregious transcription errors, though they introduced other problems. In neither case did I have a use for a summary, so I can’t say if it would have been beneficial.
Update or Upgrade?
Should you install macOS 15.1, iOS 18.1, and iPadOS 18.1? If you have already upgraded to macOS 15.0, iOS 18.0, and iPadOS 18.0, yes, update in the next week or so. Apple is still working toward the full feature set it promised for these operating systems at WWDC, and there’s no reason to put off playing with Apple Intelligence any longer.
However, if you haven’t yet made the jump to macOS 15, iOS 18, and iPadOS 18, I suggest waiting a bit longer. Apple is now promising another release in December with more Apple Intelligence features like ChatGPT integration, Genmoji, and Image Playground. That won’t be the final feature release of this generation of operating systems, but it seems like a reasonable compromise between upgrading immediately and waiting for everything to settle down later next year.
