AI Answer Engines Are Worth Trying

Article Comments
21

Thanks for an excellent article, @ace . I’ll recommend it even to non-Apple folks.

A few comments:

Perplexity is my primary generative AI engine, but I’ve found Grok to be quite easy to use and often to yield very good results. It’s my secondary engine.

I did have a sobering experience with Perplexity in recent weeks that highlights the need for caution when using generative AI. I’ve been working on a project where I’ve needed to generate around a dozen one-page information sheets for a subject about which I have significant first-hand knowledge.

The info sheets needed to be in a very specific format, and they each needed the equivalent of 10-15 bullet points each. Since I was under a lot of time pressure, I asked Perplexity to create a first draft for each topic.

On the positive side, Perplexity generated each document in seconds. Each was perfectly formatted and written to a very high, professional standard of vocabulary and grammar. Further, all were “directionally correct” in the sense that a non-expert of reasonable intelligence could be expected to understand them and, if asked about them a day or two later, could say a few things about each that, at a high enough level, matched reality.

On the negative side:

  • Perhaps 10% of the bullet points were fully incorrect.
  • Around a quarter of them overstated a fact or missed a subtlety in ways that that would have seemed fine to a casual reader but would have damaged the work’s credibility if read by someone already working in the field.
  • Another third of the points were reasonable, but overly generic.

Unfortunately, I can’t share the specifics, but I can give an example of what would have been an embarrassing error if not caught:

  • Perplexity said that Person A made a very specific, important contribution to the field.
  • I knew that it actually was Person B, not Person A, who made the contribution.
  • Following the link to the source material, I saw it was an article entirely about Person B. Person A appeared once in the article, commenting on the importance of Person B’s contribution, but not explicitly mentioning Person B by name.
  • From context, it obviously was Person B’s accomplishment, but the AI completely missed the context and attributed it Person A, presumably because of the proximity of Person A to the accomplishment in the text.

If I hadn’t have caught the error, it would have been bad for a couple of reasons:

  • Both Person A and Person B would have been in the room when the results were presented, so an error would have fatally damaged the presenter’s credibility.
  • The info sheets may eventually be published, so any uncorrected errors would have become misleading source material for future AI analyses.

The sobering part is that the results were so well presented by Perplexity that I think a lot of people with casual knowledge of the subject would have been comfortable forwarding them as official documents with only minor corrections.

Overall, Perplexity saved me some time, especially since it provided links to source material that helped me to “fact check” easily, but it did take some real time and effort.

Bottom line: if the material matters, and especially if it is going to be shared on the Internet, DON’T SKIMP ON THE FACT CHECKING.

9 Likes
22

Just a caveat here for others - I got back into my Logi Options+ to look at my settings (I have a MX Master 3S), and the Perplexity promotion is available there. No indication of how long it might be there, but if anyone else were interested, and have a Logitech device, it might be worth checking to see if you can see the coupon code there. The MX Master 3S has been one of my better purchases. I absolutely love the way I can easily move back and forth between my work and personal Macs using the input select button on the bottom.
To be honest, I probably would have ignored the promotion if not for Adam’s article, as I’ve been avoiding AI for the most part (Microsoft keeps trying to force copilot across their entire product line that I use at work, and so far I’ve only found value in the ability to provide transcripts and reasonably accurate summarizations of video meetings). I use the Kagi search engine (and I’m happy to pay the $10/month for the increased accuracy and lack of marketing), and I’m finding that Perplexity works well in addition to Kagi. I still use Kagi for most of my targeted searches, but Perplexity is useful for more wide ranging searches where I don’t have enough info to form a targeted search.
I’m also using the Monarch launcher, and it was easy to set up a superlink for Perplexity - see the pictures below:

Superlink_Perplexity
Superlink_Perplexity750×440 75.7 KB

And in use:

Superlink_Perplexity_InUse
Superlink_Perplexity_InUse752×134 21.3 KB

23

oh sorry, I didn’t realize we were including ownership as pertinent to this topic.

25

In this case, I think it’s something many people would appreciate knowing.

4 Likes
26

I wonder what the consequences of this will be in a few years, assuming that web sites lose a significant number of their human viewers that turn to AI bots instead.

Sites like the New York Times. With many fewer human visitors to be bombarded with ads and enticed into subscriptions, it will have to rely more on AI licensing fees for survival. Increasing licensing fees will then cause consolidation and reduced competition among AI providers (which will probably happen anyway). Or will the owners scrap the whole news-producing part of the company and turn it into New York Wordle?

What about educational institutions like universities and museums that today make the effort to provide attractive and informative sites? Will they continue to do so simply to support the minority of web users who take @josehill’s wise advice to do fact checking?

What about sites produced by one or a few people that rely on some combination of ads and subscriptions? A site like the Online Etymology Dictionary, which I enjoy reading. Will they continue to make their information available to an audience of primarily AI bots?

27

I view social media and social media recommendation algorithms to be a bigger threat to general audience news publishers than generative AI “Answer Engines”. I also believe specialized news sources that serve professionals in a field, such as Bloomberg, will continue to operate in ways similar to today.

More broadly, I don’t think Answer Engines will affect website traffic and usage habits that much. Answer Engines are not that different from traditional search engines. As such, I don’t think clickthrough and follow-up habits will change to a great extent. People who rarely go beyond a Google result summary will simply read an Answer Engine summary and move on to another activity. Similarly, people who love to dig into results, look at pictures, and watch videos will continue to follow hyperlinks.

1 Like
28

It’s pertinent to some of us. Bob, thanks for pointing it out.

1 Like
29

Honestly, I had no idea that xAI’s Grok could do Web searches, which is why it’s not included. Your example, while compelling, is just standard chatbot behavior these days because it doesn’t require access to information retrieved from the Web. I wrote about doing that sort of thing with ChatGPT back in January.

That said, had I known that Grok could do Web searches, I wouldn’t have included it because I refuse to promote anything associated with Musk and X/Twitter, which is now owned by xAI in a corporate shell game.

8 Likes
30

One thing worth keeping in mind is that sites like The New York Times receive a significant amount of direct traffic. If you actually want to read the news, you’ll go to the site and see what’s there.

I suspect we’ll see a lot more emphasis on subscriptions and direct connections rather than relying on search traffic, which I always thought was a mistake anyway. It’s all too common for search providers to update their algorithms, causing previously popular destinations to suddenly see significantly less traffic.

2 Likes
31

I also expect news sources to have their own AI that will run through their own data to give a NYT or other answer. Medical journals are looking into this

1 Like
32

Thanks, Adam, for the excellent discussion. FWIW, I’d support Scott Kuntzelman’s use of Kagi as a default search engine. I too will frequently use Perplexity Pro for complicated search questions (and use ChatGPT, Claude, Gemini, and Consensus as well) and like others have found that the recent LLM iterations have become less prone to hallucinations - though they definitely still do make mistakes.

My wife and I gave up on Google almost 2 years ago, since their search results had become increasingly cluttered with advertising. We’ve been using Kagi for well over a year via the duo professional option ($160/year for 2 people); a single is $10/month. They continue to incorporate some AI (see their AI Assistant). We’ve found it blissfully free of the forced ranking from advertisers that has made Google a trial. It’s powerful, fast, and very customizable (restricting searches by recency, advanced searching with OR, excluding words, academic sources only, etc.). Queries are anonymous, there’s no data retention and Kagi translate is excellent. If you are into privacy when browsing, their Orion browser is worth a try. John Gruber and Cory Doctorow are fans.

From Jason Koebler’s 404 site:

"When you search on Kagi, the service makes a series of “anonymized API calls to traditional search indexes like Google, Yandex, Mojeek, and Brave,” as well as a handful of other specialized search engines, Wikimedia Commons, Flickr, etc. Kagi then combines this with its own web index and news index (for news searches) to build the results pages that you see. So, essentially, you are getting some mix of Google search results combined with results from other indexes.

All of this is completely invisible to you as a user, but my general, nonscientific impression from three months of searches is that using Kagi feels like using Google before Google enshittified and bloated itself…

Kagi has a few nice features that are similar to ones also included in the DIY open source search engine Stract and Brave Search, though in practice I personally do not use them. These include the ability to upvote, downvote, prioritize, and block specific domains from search results. The most commonly blocked sites are Pinterest, Fox News, Breitbart, Facebook, and Quora. The most commonly boosted are Github, Stack Overflow, Reddit, Wikipedia, and Hacker News, which perhaps explains the types of people who are most commonly using Kagi. Kagi also has a feature called “Lens,” which lets you search the Fediverse, message boards, PDFs, and Usenet. “

Kagi is our default browser search engine on all our computers, iPad, and phones. As for us, we are both very happy to pay $160/year to get a vastly improved search experience (I’ve no connection whatsoever to Kagi except as a happy user).

3 Likes
33

What blows me away is there was NO mention at all about something a pal turned me onto a few weeks ago. My friend and I also fed the same query to it and ChatGPT and both of us felt the answers far better in… Gemini. Maybe it’s the branded version of the GoogleAI mentioned. The only difference I spotted was ChatGPT can provide an image as the answer while Gemini can’t.

One example, I put in a general query about specifics of the monitor (1440p, mini led backlight, 27" size) I wanted and it came up with exact matches including the one I had searched for on Amazon for the last year or so (which probably also speaks to how god-awful their search engine is, I mean they should be embarrassed it’s so bad).

In the past 2 weeks it has actually answered a lot of queries I have had that the regular (google, tried duckduck, found it lacking) search engines could not find. Gemini’s answers are very well written, much more so than even software instruction manuals… yes, he and I have asked it questions about how to perform certain actions in various software programs and it always hit the nail on the head with complete and very thorough answers. Gemini not only supplies an answer, but explains how/why it came up with that answer. It’s 100% amazing.

34

Interesting, isn’t it, that you left the ‘most controversial aspect of answer engines for the last.’

"So yes, I think business models predicated on eyeballs and attention will suffer, and companies that rely on such models will have to adjust their approaches to survive.”

Many businesses are already suffering deep damage due to answer engines, are they not? It is well known that many AI companies are justifying their theft of user data, if not doubling down on it, by simply ignoring ‘do not index’ directives in robot.txt files by small publishers and even finding ways of breaching paywalls of commercial publications. Yes, The NY Times and big publications with deep pockets may eventually get paid somehow, but small hobby publishers seem to have zero recourse as of now, other than to charge subscriptions. Hence the proliferation of the substacks and the mediums and such platforms, where you essentially have no ownership of your data (unlike if you had your own website or blog).

The generative AI industry is even more destructive because they are positioning to essentially wipe out large segments of creatives including writers, artists, movie-makers and more (hence, the recent Hollywood strike).

And yet, here we are. Your suggestion of ‘micro-royalties’ is interesting but I wonder - once the AI ‘groks’ all the data it wants to ‘grok’ with impunity, what incentive does the company have to pay the small creatives even a single penny for their creative work? Who is going to stop them from ‘grokking’?

P.S. Congratulations on the 35th anniversary. I sincerely wish you many more. Thank you for your excellent coverage of all-things-Apple over the years!

35

I used Perplexity for a bit a year or so ago while also using Kagi as a regular search engine. After a while I found that I preferred Kagi because it was more of a traditional search engine, and I felt like the results were not so much opinionated as they were on Perplexity.

At the time I did not have the Kagi Ultimate package (which includes a bunch of AI engines). If you add the Ultimate subscription, you get more or less the same toolset as you get with Perplexity, but (for me at least) I feel more in control of my searches.

36

Thanks so much, @ace, for this article. Only yesterday, contemplating how to get a handle on AI and reflecting back on how I got a handle on the internet in the first place, I found myself wishing that I had something like The Internet Starter Kit for Mac to ease me into AI. (I still have the first edition of your book, complete with its 3.5in floppy.)

I did give Perplexity a test, which it pretty well failed, but I guess we need to be mindful of the limits of these tools and careful in phrasing questions. Over the weekend I managed to wipe out all my current tabs. I imagined that these could be recovered from a Time Machine backup of Safari preferences, like I used to do when accidentally wiping out all my open windows. But where exactly are those preferences now? I eventually found the answer in a Reddit thread after scrolling past three screens worth of Google pages telling me about ways to use the Safari History menu. When I tried with Perplexity it was so convinced by all the pages about the Safari History menu that it never bothered to present recovering preferences from a backup as a solution.

I nonetheless expect that the time has come when these answer engines may be useful for the likes of me. In addition to wanting a Starter Kit hand up, I was thinking that a structured course might be good, so thanks also to @jeff2 for suggesting Coursera.

PS Happy anniversary!

37

Just FYI @pbinderup " we’re excited to announce that Kagi Assistant is now available to all users across all plans, expanding from its previous exclusivity to Ultimate subscribers, as an added value to all Kagi customers, without increasing the price."

1 Like
38

I hadn’t used Kagi Assistant until this week, so now I’m experimenting with it. The nice thing about Kagi Assistant is that you can choose the model from a single interface from a list (and if you have Ulitmate, you can choose premium models). I’m still fairly early in my experimenting with the tools, so I’ll continue comparing Perplexity Pro for the six months trial provided by Logitech, and experiment with the various models I can see from Kagi Assistant, and see how things go over time. It’s interesting when reading various blogs and comments on social media how many people seem to have a favorite model, but there’s quite a bit of variation. I suspect that part of it is that different people will phrase their questions differently, but it’s also likely to depend on the questions being asked and how well each model accesses the various sources that are relevant to the questions (I may not be stating this clearly, but LLM hasn’t been my focus due to my work responsibilities).

39

Deciding which chatbot and, within that, which model, you like most is tricky. As best I can figure out, the only real option is to see what feels best. In some respects, that shouldn’t be surprising, since working with an AI is much more like working with a person, and it’s difficult to know which person is “best” as well.

That said, I don’t have a feel for how the split between the Web search and what the model does with it breaks down. If an answer engine’s search is on the weaker side, that will affect the results, as does what the model decides to do with the results it has found.

1 Like
40

There is a way to add arbitrary search sites to Firefox, but it is now hidden in the about:config options. Add an entry for browser.urlbar.update2.engineAliasRefresh with the boolean value true.

That will expose a new “Add” button in Firefox’s regular Search Preferences. You can then add a custom engine by name and URL as you can for other browsers, with the %s substitution. It will appear as an option in the URL/Search field menu like any other engine.

(Ironically, my main use for this is to add an AI-free version of Google; the search string for that is https://www.google.com/search?udm=14&client=firefox&q=%s )

1 Like
41

Please be careful what you use AI tools at their current level of sophistication (training?) for. For financial advice, for example, they (to use a technical term) suck rocks. Finance Agent

3 Likes