# What Can AI Actually Automate in CRO Research?

> Can you have AI watch Clarity recordings and produce a ready-made optimization report? The limits of data access, automation, and human judgment in CRO research.

Our internal need was straightforward: nobody should have to watch Clarity session recordings one by one. AI should inspect them all, identify where visitors struggle, and tell us which pages need improvement.

Watching recordings takes time. With a large number of sessions, no one can give every replay the same attention. Reducing that burden could give the team more time to solve the problems uncovered in the recordings. Asking AI for help is reasonable.

But that request bundles together several different jobs: accessing the recordings, comparing behaviour across sessions, deciding whether a behaviour is actually a problem, and testing whether a proposed change works.

Conversion rate optimisation, or CRO, aims to make it easier for visitors to complete a target action such as a purchase or an application. Receiving a list of pages to improve does not complete that work. We still need to know which problem matters, for whom, and why.

[CXL's article on using AI in CRO research](https://cxl.com/blog/ai-cro-research/){.dofollow target="_blank" rel="noopener"} draws this line well. In the workshop it describes, AI is used to assess user journeys against usability principles, group research findings into themes, and generate test ideas. It does not hand the entire research process to one model. People review intermediate outputs, supply missing context, and decide what takes priority.

That is an account of a workshop, not evidence that this method will lift conversion at every company. One of the article's useful warnings is that AI can make detailed scores look like measured results. A low score for a page does not prove that the page is losing sales.

## Connecting to Clarity is not the same as accessing recordings

In this case, the first obstacle comes before interpretation: what data can the model actually reach?

Microsoft's [Clarity Data Export API documentation](https://learn.microsoft.com/en-us/clarity/setup-and-installation/clarity-data-export-api){.dofollow target="_blank" rel="noopener"} describes the data available externally as dashboard data. It includes aggregate measures such as visitor counts, scroll depth, and repeated clicks. The API does not make raw session recordings available for an external AI system to process in bulk.

It also matters what the recordings we watch on screen actually are. According to [Clarity's official FAQ](https://learn.microsoft.com/en-us/clarity/faq){.dofollow target="_blank" rel="noopener"}, it records the page's document object model, or DOM, together with interactions such as clicks and scrolling. Clarity reconstructs a session from that data in its own player. That replay is not a video file that can simply be handed to an external AI. The FAQ also says individual recordings cannot be downloaded.

The ability to [export a recording list as CSV](https://learn.microsoft.com/en-us/clarity/session-recordings/download-recordings){.dofollow target="_blank" rel="noopener"} does not remove that boundary. The file contains a summary and link for each recording. It does not give an external model everything that happened in those sessions.

Trying to extract the raw recordings by reverse-engineering the player's internal data flow is not an acceptable route either. The [Microsoft Clarity terms of use](https://clarity.microsoft.com/terms){.dofollow target="_blank" rel="noopener"} prohibit reverse engineering and attempts to circumvent technical limitations. An unofficial collection method could conflict with those terms.

It would also be fragile. If Clarity changes an internal service address or data format, the workflow breaks. Forcing access through internal data flows is not an acceptable solution for this need, either under the terms of use or as an operating model.

There is a substantial difference, then, between connecting to a dashboard and retrieving the data needed to study every visitor's behaviour. Aggregates can point to pages worth investigating. They cannot, by themselves, explain exactly what a visitor experienced there or why that visitor abandoned the task.

![A CRO lead studies aggregate indicators while session recordings remain separate in an inaccessible archive](/images/inline-cro-research/dashboard-is-not-recording-access.avif)

## Computer Use does not close that gap

Computer Use refers to AI that can see a screen and act through a mouse and keyboard. Opening a recording list, starting a replay in Clarity's player, and interpreting the visible activity could all be part of that kind of workflow.

Using the player does not give a model access to raw replay data. Nor does it turn thousands of sessions into comparable records ready for bulk analysis. Sessions still have to be opened, watched, and noted one after another.

Imagine asking an agent to watch 100,000 recordings this way. We would be putting an agent in front of the screen instead of a person. Opening, playing, and inspecting each session would not disappear. Nor could we reasonably expect a model to retain every relevant detail in one conversation and compare it reliably at the end.

That would require a separate system to store findings, classify them against consistent criteria, and combine them later. It would also need checks for missed moments and incorrect interpretations. Enabling Computer Use does not create that analysis system on its own.

Reviewing a small, selected set of sessions and analysing all traffic are jobs of a different scale. In the first case, a screen-based assistant may be useful. In the second, the team must first establish which data is genuinely accessible and how it can be processed. This is not a report of a completed recording-review experiment; it is an assessment of the limits of that approach.

## Comparing two summaries may not deliver the saving we need

If the remaining option is to place Clarity and GA4 summaries side by side and interpret them, the value of automation changes as well. The time saving we were looking for was less time spent reviewing individual recordings. Comparing two dashboard summaries is a less frequent task that a person can do easily.

AI can turn those summaries into a well-written report. We still need to check that it compared the same date range and the same visitor group. A report that offers a convincing explanation does not make that explanation correct.

I would not expect a large efficiency gain from this narrower use. Preparation may take a little less time, but the checking remains. More importantly, it does not address the recording-review work that consumed the time in the first place. The value of an automation should be judged against the burden it was meant to reduce.

## The more useful job is turning findings into a testable idea

Different options open up when the research material supplied to AI is available in a form the system can actually process. Survey responses and interview notes can be examined. A model can group similar observations, bring together findings that support one another, and suggest questions worth investigating. For quantitative analysis of user behaviour, however, the data itself still has to be designed.

Take an application form that needs improvement. The CRO lead or owner of the form first defines the business question: which form errors are associated with people abandoning the application? They then decide which user events to record, when to record them, and how. Seeing an error, correcting it, and completing the application may all belong in that plan.

![A CRO lead and software engineer review form errors, the measurement plan, and a hypothesis to test](/images/inline-cro-research/hypothesis-needs-measurement.avif)

The software team implements the measurement plan and sends those events to the analytics system. The records must be checked to ensure they reflect the expected behaviour. Responsibility remains with the owner of the page or flow. They do not hand the decision about what to measure to a software team or AI.

Only then can the team calculate whether the rate of abandoned applications rises after particular errors. That does not require replaying every session on screen. But without measurement, we should not assume that this detailed data exists.

AI can help interpret those results alongside user comments about the form. It may bring together reports that instructions were unclear with errors that occurred in the same field, then suggest a hypothesis. Someone still needs to check that the comments came from the relevant user group and that the events were recorded correctly.

In this example, software performs the calculation. The model organises the research material and develops possible explanations. The team compares a suggested explanation with the page's actual behaviour. The problem may be unclear wording, or it may be a faulty validation rule. The same abandonment signal cannot distinguish between them.

If the hypothesis is plausible, the owner of the page or flow defines the change and the success measure. The team runs an appropriate test and evaluates the result. A change suggested by AI earns the name improvement only when the result is better. Before that, it is a candidate.

The expectation that we can connect AI, stop doing CRO, and receive a ready-made improvement report skips over this work. We can speed up parts of the research. With accessible data, we may even run certain analyses regularly. But the responsibility for deciding what the result means and testing the recommendation remains with the team.

For this need, the useful starting point is not finding an agent to watch every recording. It is deciding what data is needed to understand where a visitor is struggling. Then we can see which part of the analysis workload AI can genuinely take on. That is where the work becomes useful: not when it produces a longer report, but when it helps the team verify a better hypothesis with less effort.

---

Language: English
License: CC BY 4.0
License URL: https://creativecommons.org/licenses/by/4.0/
Scope: Evren Bal-authored text, unless this article expressly states otherwise.
Excluded: Third-party material, quoted excerpts, logos, and separately marked images retain their own rights.
Attribution: Credit Evren Bal, link to the canonical source and license, and indicate changes.
Source: https://evrenbal.com/what-ai-can-automate-in-cro-research