Testing Jetpack Search’s AI Answers Feature on TidBITS

[ace (Adam Engst)]

Originally published at: Testing Jetpack Search’s AI Answers Feature on TidBITS - TidBITS

A renewal notice for the Jetpack Search feature I use to search to power the TidBITS search engine alerted me that Jetpack Search has launched a beta AI Answers feature. I’m always game to try new things, so I’ve turned it on.

When you search the TidBITS website using the search field (at the top left on Mac and iPad, or in the ••• menu on iPhone), along with the traditional results, you’ll get a short AI-generated summary of the articles that match the search results. I haven’t found these short summaries useful yet.

However, if you click Show More below the summary, Jetpack Search expands its answer significantly. When expanding the “Apple 2026 earnings” search, it provides a rather informative bullet list, with each section linked to the appropriate article.

With this addition of AI, you might be tempted to ask real questions. For instance, let’s say you wanted to know when the M4 MacBook Pro was released. If you search for “When was the M4 MacBook Pro released?” neither the AI answer nor the traditional search results provide what you’re looking for. Presumably, Jetpack Search is doing a standard search, and one or more of the words in that search phrase don’t appear in the article about the release of the M4 MacBook Pro.

However, if you perform a simple search for “M4 MacBook Pro,” Jetpack Search finds the correct article and the issue it was in. Again, the AI Answers expansion is noticeably more informative, though neither it nor the summary shows the release date. You would have to extract that fact from the article itself.

Although Jetpack Search has a Personality field that theoretically lets me adjust how the summaries read, it doesn’t seem to make any difference, no matter how I prompt it. Perhaps that’s what Jetpack means by “beta.”

I’m uncertain if I’m going to leave the Jetpack Search AI Answers enabled or not, but I’m curious what those of you who regularly search the TidBITS site think. Are the AI Answers helpful enough that it’s worth devoting a little screen real estate to them above the traditional list of articles and issues?

[matt3 (Matt Neuburg)]

I tried “Neuburg HyperCard” and found the results both excellent and silly. My full review of HyperCard was first, and was rightly described as “in depth”. On the other hand, one result was included because that issue contained a HyperCard update notice and an unrelated review by me. On the other other hand, the AI correctly summarized that fact. So I’m tempted to vote for keeping the AI search.

[mpainesyd (Michael Paine)]

I think it could be useful, provided some scepticism about AI results is applied by the user.

It might also be handy for searching the discussions.

[Simon]

I must be doing it wrong. Using the search field at the top right I only get posts/threads as in the past. No AI summary or AI otherwise in sight.

[ace (Adam Engst)]

This is on the TidBITS site, not TidBITS Talk.

[Simon]

I would something like that most helpful here on Talk. Basically to aggregate latest advice on a topic and point to relevant posts or to see if something similar has been posted before. If that’s at all possible with Jetpack.

[ace (Adam Engst)]

Jetpack is a WordPress thing, but there is something similar for Discourse. I haven’t looked into it because it requires API keys and thus generates a cost, but from what Claude is telling me, our traffic would likely only generate pocket-change expenses. Let me look into it…

OMFG, I should have known that the Discourse team would have gone all out. I’ve set up the AI plug-in and enabled a bunch of stuff. I’ll start a new Site Feedback topic to discuss.

[mpainesyd (Michael Paine)]

Apparently there is another Discourse plug-in that can generate an index of topics. This would be useful for @simons’ request.

Discourse Doc Categories