What Can AI Visibility Measurements Actually Tell You?

First identify the event you are counting

A brand name, source link, and product recommendation can appear in the same answer. They still do not say the same thing. Start reading a report by naming the event being counted.

Mention

The brand or site name appears in the answer.It does not mean there has been a positive assessment or a user preference.

Citation

The answer presents a page as a source.It does not prove that a user saw or clicked the link, or trusted the content.

Recommendation

The product or brand is presented as an option in the answer.The recommendation belongs to that question and context; it is not a general measure of expertise or market ranking.

The source of the data also changes its scope. A report a provider supplies from its own product and responses to prompts selected by a third-party tool are different observations. First-party data does not cover everything; you still need to know which products, queries, and events it covers.

Related article

In this article, I use this distinction to interpret Bing data

What Is AI Citation Share? What Bing's Data Actually Reveals

On sites I managed, I observed search performance and Bing citations move together. In the article, I explain the limit of that finding and why it cannot be generalised to all AI products or to a shared internal mechanism.

Opens in a new tab

Which user does a prompt test represent?

A defined test shows the answer to a specific question under specific conditions. Whether it represents a real user’s conversation needs separate justification. Running more prompts does not automatically correct a poorly represented context.

A hypothetical hosting search

We cannot assume the two paths will produce the same recommendation. A single-turn test is still useful, but its conclusion needs to fit its name and scope.

Ask for the test conditions before the score

Sample

Who selected the questions, and from which source? What supports the claim that they represent a particular user and decision stage?

Setup

Which product or API, model, language, region, web access, and conversation context were used?

Counting

Is the count a brand-name mention, a source link, or a recommendation? Ask what the denominator is and how repetitions are treated.

Comparison

Are the prompt set and conditions the same across periods? If the model, questions, or settings changed, the report should show it.

Related article

I explore the difference between this test and a user experience in a separate article

Do AI Visibility Tools Really Work? What They Actually Measure

Using the hosting example, I examine the limit of single-turn prompts, the tasks tools may be useful for, and the information an honest test report should contain. You can use it when assessing a report or tool proposal.

Opens in a new tab

What should you do with this signal?

When you see a decline under the same test conditions, first find the affected questions and pages. An access issue, outdated information, ambiguity in the brand name, or a change in the test may be investigated. The score can start that investigation; it cannot by itself tell you which explanation is correct.

Separate a technical experiment from a proven return

For a proposed new file, content representation, or markup, write down which behaviour of which product you expect to change. Keep the provider’s documentation, your own observation, and a hypothesis you have not yet tested separate. Requests for the file are not the same as being cited in an answer or achieving a business outcome.

An alternative content representation may be worth testing if your existing infrastructure can produce it at low cost and it has other uses too. If it requires a separate publishing pipeline and ongoing maintenance, you need to show more clearly why the expected benefit justifies that work.

Related article

I examine this boundary in my llms.txt experiment

llms.txt Was Never the Point

I explain the experiment I set up with knowledge bases, Markdown, and JSON representations, what I learned, and why I did not draw a visibility promise from it. You can weigh the conditions under which a technical idea is worth testing.

Opens in a new tab

Link the content decision to the reader’s need

It may make sense to answer a question that appears in a visibility test. First examine whether it corresponds to a real reader need. If your content explains a decision, solves a problem, or conveys a verifiable experience, it has value independent of the visibility metric.

Make the decision to participate in a community in the same way. Join when you can make a meaningful contribution to people’s discussion. Pages and messages made only so that a model recommends you more often can push the people you want to help into the background. Technical clarity and good information architecture should support that purpose.

Related article

I address content and channel choices more broadly in this article

Getting Your Brand Recommended by AI Is the Wrong Goal. What Is the Right One?

Using examples from Reddit, technical SEO, and content production, I separate a useful contribution from interventions made for visibility. You can reassess the goal of “having AI recommend us” by asking what you will help the reader with.

Opens in a new tab