Why your AI Visibility results need a Research Plan

AI visibility has quickly become one of the most common topics in conversations with communicators. Almost every team I talk with is either measuring it already or trying to figure out how to start.

I’ve seen plenty of shiny objects that gripped communicators over the course of my career, including the early days of Netscape and the World Wide Web, the dawn of social media, and now the age of AI. People get excited about the newest thing and tend to overemphasize it for a while.

LLMs are genuinely important, and I’m not suggesting otherwise. But they’re one piece of the communications mix, and they behave differently from one another. ChatGPT, Claude, Gemini, and Perplexity each have their own quirks, and the AI answers built into Google Search are different again.

I recently sat down with Orla Graham, Insights and Consulting Director at CARMA in London, to talk through how teams can get results they can act on. Orla works with CARMA clients to go through their AI visibility results, put them in context, and decide what to do next.

AI visibility is a research exercise

Orla starts from a research mindset, treating AI visibility the way you would any other research program or media type.

“The most important thing is that we take a research mindset into it,” she told me, “making sure that we’ve got the right data going in, so the right prompts, that then you can analyse coming out with the right kind of information and that you can get really genuinely actionable insights out of it.”

That matters because there’s a lot we don’t know about how people use these tools. The available data on what people actually type into LLMs is limited, and it mostly comes from panels where users share their prompts or self-report what they’ve asked.

Orla is skeptical of both. “I certainly can’t remember the last kind of prompts that I’ve put in,” she said. “And also I would not give permission for anyone to be able to see the sort of stuff that I’m putting in.”

The models don’t hold still either. One month a model leans heavily on Reddit as a source, and the next month it doesn’t. A research-oriented program helps you find the insights that will hold up over time rather than chasing whatever today’s data point says.

Your prompts should start with your audience

This will sound familiar to anyone who has worked on SEO. My advice there was always to focus on the audience first. Chase the latest algorithm and you run into problems, because the algorithm is designed to chase the audience. LLMs are doing the same thing as they change their data sources to meet the needs of their users.

That puts the prompts at the heart of any AI visibility research, whether your team is typing questions into ChatGPT every morning or you’re working with a tool or a consultant. If the prompts aren’t at least close to right, the results won’t be either.

“It comes back to your audience again,” Orla said. “Know your audience.”

She draws on several sources to build that picture. Social media shows what people are talking about, as long as you’re looking where your audience actually spends time. Traditional search trends reveal the questions people typically ask, and if they’re asking a search engine, they’re probably asking AI too. Existing segmentation work and personas fill in the context.

From there, Orla maps prompts across different search intents so the research stays well rounded. Some are informational, from people who want to know more about a company or product. Some are commercial, from people weighing value for money or comparing options. Others are about action, like where to buy something or how to use it.

Start at the end and work backwards

Knowing who you’re trying to reach is half of it. You also need to know what you want them to do, think, feel, or believe, because otherwise you can’t figure out what content you want the LLMs to pick up.

“I always recommend to people they start at the end and they work backwards,” Orla said. Identify the final goal, then ask what needs to be in place for it to happen, and keep moving back from there.

That exercise reveals the signals you can measure along the way. It also acts as a check on the assumptions teams make when they jump straight into planning activities.

Orla doesn’t take a fundamentally different approach to LLMs than to any other channel, and she argues AI visibility belongs inside the wider measurement program. She also points out that the lack of control will feel familiar to PR people. “As much as you can have all of the best intentions and do your best to build the relationships, you don’t have a hundred percent control over what a journalist writes,” she said.

As Orla noted, the AMEC GEO Principles take a similarly holistic view, looking at the content that feeds AI answers, the answers themselves, and the likely impact on audiences.

Reacting without overreacting

A single result read in isolation can push you to overreact in either direction. It might show a competitor turning up everywhere you don’t, or it might make you look stronger than you are.

When I asked Orla how she helps clients react appropriately, she described three things.

The first is context. AI visibility results aren’t the same as a count of published articles. “This is likelihood,” Orla said. “It is not a hundred percent for sure what people are seeing.” She also brings in research on which models a client’s target audiences are most likely to use and how that varies across generations.

The second is integration. Orla often reports AI visibility alongside a client’s media and social analysis, which helps clients see AI as one important part of the picture.

The third is actionable insight. That can mean the technical work needed to make a website’s content discoverable to LLMs. It can mean spotting that a message isn’t cutting through in AI answers because it appears in outlets that the models aren’t drawing on, and recommending outlets that they are.

Old problems don’t fade the way they used to

Five years ago, the better response to a scandal or crisis was sometimes to say little and let it die down.

“Unfortunately, now LLMs have a long memory,” Orla said.

Her recommendation is to make sure there is dated content that addresses the old issue, explaining what the organization has done differently and what has changed. That gives the models current context, so an issue from five years ago isn’t presented as the current state of things.

Snapshots and direction

Orla recommends starting with a landscape analysis, a deep dive that establishes your baseline, much as you would with media measurement. From that baseline, pick the specific things you want to move, whether that’s a risk narrative or a website that isn’t showing up, and track them over time to see whether your actions are having an effect.

Rather than repeating the deep dive constantly, she suggests periodic deeper research, much like market research, with always-on tracking running in the background.

She also encourages experimentation. Adjust how the website is optimized, reach out to different outlets, and see what changes. “Play around with it, play with all of the different levers and see which ones have the results that you want,” she said.

What to do with this

Orla’s closing advice was the same point she started with, and it’s a good one to keep in mind before the next AI visibility report lands on your desk.

“The main thing is to think about this as research,” she said. “Fundamental research principles never die.”

Fale com um dos nossos experientes consultores sobre a sua monitorização de media e comunicação ainda hoje!