Siri AI Hits and Misses

Article Comments
1

Originally published at: Siri AI Hits and Misses - TidBITS

In 125 Tests of the New AI Siri, David Pogue has done the yeoman’s work of testing not just a few Siri AI requests but a whopping 125 across the categories of personal information, phone and app control, Internet queries, visual intelligence, and screen intelligence. He shares each request, briefly explains how Siri responded, and notes whether or not it succeeded. Many of his examples won’t apply directly to your situation, of course, but they give you a sense of what you can accomplish with Siri AI.

For those who are using OS 27 and have gotten into the Siri AI beta, let’s crowdsource our own list of Siri AI hits and misses in the comments below. Follow Pogue’s example of starting successful prompts with ✅ and unsuccessful ones with ❌. You decide whether partial success gets a ✅ or an ❌, perhaps based on how close it came or whether a follow-up made it work. Let’s also use ⏳ to indicate prompts that worked but took so long that it wasn’t worth having Siri do it for you. Please remove any sensitive information before posting.

3 Likes
2

I’ll kick things off. Here are some recent ones culled from my Siri app.

:white_check_mark: Text Gary’s mobile number to Zsophia. (I’m giving this one to Siri even though it couldn’t figure out the right person from Zsophia’s name and I had to enter it manually when it ask for more help.)

:cross_mark: Has anyone said they’re creating a carpool for the annual picnic tonight? (Only searched Messages, so it missed several that were in email. Perhaps that’s because I don’t use the Mail app?)

:white_check_mark: How much of a percent increase is 1715 up from 1510? (I never know how to phrase this sort of thing.)

:white_check_mark: Is it possible to use Siri AI in Golden Gate while voice control is turned on? (It turns out that when Voice Control is turned on, you have to use the phrases Open Siri or Show Siri rather than Hey Siri.)

:white_check_mark: Is New Mexico a two-party state for audio recording consent? (No.)

:white_check_mark: Find photos of the Robert H. Treman North Shelter where we had the FRC picnic. (It succeeded even though it didn’t get FLRC right in the dictation recognition.)

:white_check_mark: Find the video that Sarah texted me about BPPB. (Again, it succeeded even though dictation confused BPPV.)

:cross_mark: Is there music and dinner at Honscher tonight? (This was a comedy of errors. The brewery’s name is Hopshire, and Siri recognized it wrong repeatedly. Once it figured out what I was talking about, it failed to check the actual events page, so it gave me the wrong answer.)

:white_check_mark: Can I import a GPX track into Strava as a route? (But it was not a successful ride. :face_with_head_bandage:)

:cross_mark: What’s the cultural reference of puffins and pancakes? (Siri made up something completely insane about a hybrid food of pancakes and muffins and some phrase from an Austin Powers movie. The real answer was that it came from “There once was a puffin,” which it only found once I told it to look for the children’s poem.)

:white_check_mark: How do I set a keyboard shortcut for Spotlight clipboard history? (But I’m still not happy, because the answer is that you can’t. I suspect there still is a way, but that it’s not obvious.)

:white_check_mark: How can I toggle start and stop listening for voice control in macOS 27 from the keyboard? (Same problem–I got the correct answer, just not the answer I wanted.)

:white_check_mark: Is there a shortcut for taking a video from the Camera Control? (Press and hold.)

:white_check_mark: When was my last eye doctor appointment? (Interestingly, Siri found Tonya’s more recent eye doctor appointment first because it starts with T:. When I told it that was Tonya’s and not mine based on the T:, it successfully found the one a year ago with that started with A:.)

3

:white_check_mark: How long would the trip to pick up XXX near Grand Rapids, MI take in hours and miles? How much would the round trip cost a) in my EV6 and b) in my gas car at 25 mpg? How do those compare with a $600 round trip flight cost? (It gave me a detailed cost comparison and noted that driving the EV6 is by far the cheapest, but would be 23 hours roundtrip. When prompted, it added in toll costs for two plausible routes as well)

:white_check_mark: How many days have I stayed in [Adirondack vacation spot] since 2020? (Counted multiple calendar events to come up with the total)

:white_check_mark: How do you pronounce “Saoirse”? (Gave variants and indicated the most common one, used by a specific well-known actress)

:white_check_mark: Did I email or text anyone at work regarding where I left their new credit cards for them? (Found and accurately summarized texts and email to several peopleindicating my plans, as well as where I eventually left them)

:cross_mark: Find me local event listings, concerts, outdoor activities and plays this coming Sunday in Hudson Valley. (Came up dry, and asked me about specific counties or cities)

:white_check_mark: Hey Siri what are the rules for paying for a student’s off-campus food via a 529? [Followed by] Find me the COA limit for State University of New York at New Paltz for this year. Accurately summarized the IRS rules, with links, and found and quoted the most recent published amount on the university’s website

:white_check_mark: [Returning from a Weird Al concert] Hey Siri who are the backup singers for Weird Al Yankovic on summer tour? Gave a detailed answer of his regular band members + this summer’s expanded tour band with their instruments played, which answered our question about who the amazing young guitar player had been.

4

Holy smokes! Reading Pogue’s column and your entries here–Siri AI is unbelievable! I would never have dreamed of trying some of the questions that it successfully answered.

Dave

5

I have been quite impressed so far. I asked it about the new Telo EV truck and it gave me deets. Then I told it to “save” all that info and it created a new Note in Notes with everything. Then I pointed out that it still had the old 6,600-lb towing amount and the revised one is now 8,000lbs and it agreed, brought up the new details about that, and then offered to update my Note with the new details without me even suggesting that!

I asked it about weather projections for areas where I’m planning a camping trip next week and it gave me a nice overview (I did not check to see if the projections were accurate).

I asked it about new windshield wipers for my campervan and it told me what I needed (which confirmed what I had found on Amazon). It then offered to find them for me online, but I told it I already had them on order from Amazon (a slight lie as I hadn’t clicked the buy button yet). It then said it couldn’t find my order in my mail. I went to Amazon and ordered the wipers, but Siri couldn’t see the confirmation email until I refreshed the view on my phone (I could see the email order on my laptop, which is not upgrade to Golden Gate yet). As soon as I refresh Mail on my phone, I told Siri to look again and it found it order.

I have found a few small glitches and some I need to practice with. Like it tried to open Settings for a few things and failed. A few times it said it couldn’t do change a setting for me. Minor stuff.

Yesterday I got a text message from K and told Siri the way I have done thousands of times before, “Reply to K” and Siri said I had no texts from K! It then asked if I wanted to email K and I was “No, reply to K’s text!” I could literally see the text on my lock screen but Siri was claiming I had no texts from K. I don’t remember exactly what I did, but somehow I got it to see it finally and it did reply. Very weird as that was one thing that used to always work!

(Maybe the phone needed to be unlocked for Siri to see the new text? I was out walking the dog and listening to the text on my AirPods.)

Overall a solid B+ or A- so far.

6

Almost all of Siri’s answers were “I found this on the Web,” followed by a notification-style on-screen list of web pages. I think Siri in Golden Gate is no different from Siri in Tahoe.
The question “Show a photo of me playing piano” produced “Here are some images of me playing piano from the Web” (sic) with a few thumbnail pictures, like from https://www.istockphoto.com/jp/写真/cats-playing-piano.

7

Are you sure you’ve given it enough time to build the indices? In particular, if you have a large photo collection, in the multiple thousands, it could take days for it to process them.

Dave

8

Sounds like regular Siri, not A.I. Siri. What hardware are you using?

9

Sounds like you don’t have the new AI Siri turned on. You have to activate it in Settings and join a waitlist and it can take a day or two to turn on, and more time (days) to index all your content. At least that’s how it works on the phone – I haven’t installed it on the Mac yet.

10

Yes, on the Mac it can take days after activating it for it to finish its indexes and surveys.

Dave

11

M4 Mac mini (Mac16,10, 2024), macOS Golden Gate 27.0 (26A428)
Settings > Siri shows no way to “Activate AI Siri” at all. It only shows the options:
Request: “Hey Siri”
Language: English (United States)
Voice: English (U.S.) (Voice 1)

Above those options, there is some text (in Japanese) that says something like:

To use Apple Intelligence, your Mac and Siri must be set to the same language. Currently, your Mac is set to Japanese (Japan), while Siri is set to English (United States).

But I definitely do not want to set Siri to Japanese (I do want English voices.)

I don’t have any kind of smartphone whatsoever (never in my life - my mobile phone is phone/SMS only, not connected to the Internet.) I don’t take pictures, I don’t use Messages. I do Zoom/WebEx meetings, but I don’t make phone calls often. E-mails on the Mac is enough for me. My Photos collection contains 104 (one hundred and four) photos, and I don’t appear in any of them. – So, I guess Siri AI wouldn’t be of much use anyway.

Oh, my Contacts.app contains only 2 contacts - one is Apple Japan (somehow pre-set from the beginning) and the other is myself. Gmail handles all other contacts…

12

I’m on the Siri “waitlist.” Running 202 4 MBP with Golden Gate. I just typed in Siri AI and “Extensions Siri” and came up. Then click on that then work through and Activate Sir AI (Beta) and it will put you on waitlist.
David

w/r,
David

13

:cross_mark: Set a timer on the watch. This absolutely flummoxes me.

Previously, I didn’t use Siri a lot, except to set timers on my watch. Especially in the kitchen. What I’ve discovered since the update is that I cannot set timers using Siri on my watch. I can set timers using Siri on my phone, and I can set timers by hand (like a savage) on my watch, but telling Siri to “set a timer for 5 minutes” on my watch results in the animation spinning for a few seconds, and then refusing. The exact refusal isn’t always the same, but sometimes is “I can’t set a timer because the clock is unavailable.”

This is just about the simplest thing that Siri could do, and it can no longer do it, apparently. There may be some obscure setting that needs to be changed (I’ve looked and haven’t found anything).

14

In an early beta of watchOS 27, setting timers via Siri didn’t work for me either. But a later beta fixed that, and I’m not having any troubles with commands like:

  • Set a timer for five minutes
  • Five minute timer
  • Five minutes
1 Like
15

I just asked Siri about this and here is its response:

Siri AI Language Limitations

To use Siri AI, both your device language and Siri language must be set to English.

Dave

16

Here’s a sentence from an article I’ve been reading:

“the U.S. military’s important staging points: the Fifth Fleet’s headquarters in Bahrain as well as the Ali Al Salem Air Base, Camp Arifjan, and Camp Buehring in Kuwait.”

Selecting a place and asking Siri to “Show on map” worked only on “Bahrain”; for Kuwait and the bases, it directed me to a nearby soul food restaurant.

1 Like
17

I presume most of these Siri AI answers have to be checked for verification at which point the time saving is undone.

I guess eventually everyone will trust the results enough to not bother checking but we all know that the way “AI” works makes it prone to confidently spouting BS as facts. So I expect there will be a lot of lawsuits incoming from people who missed important appointments and tax bills etc :)

For important stuff I’d always choose to spend the time to manually search for the correct information. That seems sensible.

I’m in the EU so I don’t get the option of using Siri AI anyway (no I’m not happy about bureaucrats deciding what technology is available to me.) and for trivial stuff I could possible use it.

[Yeah I have a Mac but too old for the latest OS]

18

You need to call Apple and report that bug, because I absolutely can make timers with Siri on my Apple Watch. Remember, Apple doesn’t read third- party websites to find out what’s wrong with their hardware or software. They rely on people calling AppleCare and reporting them if you aren’t participating in the beta and using the Feedback app to report them.

19

:white_check_mark: Siri, transcribe this sign-in sheet from a meeting; “/…” is the path to a photo of the sheet taken handheld with my phone; each line has a handwritten name, phone, and email. Put the results in a spreadsheet.

Siri did this as well as Claude Code - with an appropriate warning that transcribing handwriting is likely to have errors. And it did have errors, a few on names that should have been clear, others on genuinely unclear and overlapped writing. Claude Code does a better job of flagging ambiguous lines, adding a column of comments to the spreadsheet to guide my checking. I imagine Siri might do that with the right instructions.