SEO vs AI Search: a not-so-epic battle

If you’ve been on this newsletter for the long haul–can 4 months even be considered “long”? 😆–then you know that I’m fully expecting the rush to optimize for AI search to make AI search basically worthless. 

As a card-carrying Grumpy AI Skeptic, I don’t have much of anything nice to say about AI search. In fact, I’ve had a long-held suspicion that optimizing for AI won’t look especially different from optimizing for search engines.

I'm strongly hoping that AI search proves to be a shorter-lived fad, but in order to give it a fair shake, I have to admit I do have some curiosity:

• Why do users turn to AI search?
• Does AI search find relevant results?
• Are they accurate results?

So, for you, dear reader, I held my nose and tested it out.

Test #1: are AI search results relevant?

To take a peek into this, I ran the question “who are the best wordpress website designers in Portland, Oregon?” through Perplexity, ChatGPT, and a regular-degular Google search.

Perplexity delivered a set of results that included star ratings for each. It seemed to be weighted based on customer reviews from sites such as Yelp, and Top 10 lists from online publications and directory sites. In other words, they're pulling from the types of sites that are often hyper-optimized for organic search.

Knowing that, it’s not surprising that there was a fair amount of overlap between the results Perplexity pulled up and what the Google search showed me. Still, it’s not great news if you’re an agency that doesn’t push for online reviews.

ChatGPT pulled up a completely different set of results from Perplexity and Google, and it was hard to tell how it arrived at those agencies being the “best” ones–many of the cited sources were the agencies’ own websites, which didn’t rank Google’s search results.

Still, all three gave me a grip of Wordpress website designers to consider, and they seemed to be of decent quality. It is worth pointing out that Google’s organic search results were dominated by Top 10 lists, and included a link to Upwork (famously not an agency).

Test #2: does AI deliver accurate search results?

Since “best” is a very subjective question, I switched gears and tested for accuracy by asking about something I know backwards and forwards: my own agency.

The results for “what does Upswept Creative do?” were… a bit mixed.

• Perplexity, ChatGPT, and Google’s AI summaries all included our client base and our major services.
• ChatGPT didn’t seem to “get” us as well as Perplexity or Google–it had multiple direct quotes of phrases from our website awkwardly stuffed into its summary.
• Perplexity and ChatGPT both picked up on our DEIJ commitments and support of positive impact organizations, which I appreciated! 
• ChatGPT noted “transparency and collaboration” among our differentiators. Perplexity mentioned our “positive, human-centered work environment.”

But, most notably, all three AI results pulled up outdated information.

At the time of this writing, Perplexity listed “branding and logo design,” and, “graphic design and print collateral” which are not core offerings of ours, and haven’t been since the pre-COVID days (5+ years ago).

ChatGPT also cited “Branding creative / copy.” I could be kind and give that a pass, but while we do create copy as part of our projects, "branding copy" isn’t the service.

What about the appeal to users?

There were a couple of flourishes in the AI searches that did seem like they could be appealing to the average user. 

Perplexity included a map of the online agencies’ respective locations. This didn’t feel especially relevant in our test case–many agencies take clients across the nation or even the world. However, it could be nice for businesses that rely on physical storefronts, or for more open-ended searches that don’t specify a location.

ChatGPT did best on anticipating the user’s needs–it included several bullet points about “How to Decide on the ‘Best’ One for You,” with a decent summary of considerations for selecting a website designer. It was accurate in this case, but as a platform that’s notoriously inaccurate, we can only assume that won’t be the case every time.

It’s also worth noting: Google’s search results page is A MESS these days. In my test, I had to scroll past multiple Sponsored results, their AI overview, and their Google-Maps-Powered results before I got to any organic search results. And then, the organic search results were interrupted partway through with a hide-show list of what Google deemed “relevant” questions. It's honestly pretty user-hostile.

What should I do to perform well in AI search?

Here's the good news, everyone: if you’ve already dialed in your SEO, you aren’t likely to need any big, sweeping changes to do well in AI search. AI searches prioritize many of the same sources that organic search does, so just keep on keeping an eye on relevant content, keywords, backlinks, and technical SEO.

The one big thing you may have missed? Make sure your information is up-to-date–in all of the places you show up.

AI search will happily pull in details from old pages on your website, that blog interview you did back in 2019, directory listings that are years out of date… you get the idea. You may not have control over all of those sources, but it’s a good idea to update the ones you do have control over. Once you've done that, all you'll need to do is give it time.