Is Your AI System Actually Delivering Business Results?

💡 TL;DR: Key Takeaways

• Model output, execution path, completed work, and business impact must be measured separately. A single success rate can easily hide the difference.
• A trace explains why the system behaved as it did. The system of record proves what actually happened. A successful tool response does not establish that the work was completed.
• The evaluation effort should grow with the system's impact and cost of failure. Define success before choosing the model so that convenient metrics do not quietly become the goal.

I built a content system for Camiler.org using data from Diyanet, Google Maps, and the OpenAI APIs. It generated and published thousands of mosque pages. Within a few months, the site was receiving roughly one million Google impressions and more than 10,000 clicks per month.

The revenue was zero.

If I evaluated the content pipeline, the system worked. If I measured search visibility, it was successful again. If I judged the business I had intended to build by revenue, it failed.

I have written about the programmatic SEO experiment in detail. The point here is different: you can look at the same system and tell three different success stories depending on where you measure it.

AI systems make this mistake particularly easy. The model gives polished answers, tool calls appear to succeed, and transaction counts rise. The dashboard turns green. Yet we may still have no evidence that the customer's task was completed, the process improved, or the company achieved the result it expected.

Which record contains the success you are claiming?

Evaluating an AI system involves four questions that may look similar but are not the same:

All four questions can be asked at once. Their answers do not live in the same place.

A test set may show whether the model correctly understood a refund request. A log shows whether it called the right tool. The payment record confirms that the refund was actually created. First-contact resolution and repeat enquiries tell us whether the customer's problem was resolved.

A single success rate can easily collapse these distinctions.

An execution trace helps you find the failure

Some AI systems do more than produce one answer. They search for information, call a tool, read the result, and choose the next step. The record of that path is commonly called a trace.

A trace is valuable. It lets you see whether the model used the wrong source, called the same tool unnecessarily, bypassed human approval, or continued incorrectly after a connection failed.

The trace is not the outcome itself.

Anthropic's guide to evaluating AI agents explains the distinction with a flight-booking example. Instead of accepting the system's statement that a flight was booked, the evaluation checks whether the reservation exists in the database. The path explains why something went wrong. The system where the transaction lives proves the final state.

That is why logs should not be read as business outcomes. An email tool may run without an error, but sending the message to the wrong recipient is still a business failure. Even delivery to the correct recipient does not prove that the sales opportunity moved forward.

Verify the transaction where the change lives

A customer support system may select the right refund tool and fill in every field correctly. The tool may return a successful response. The payment could still remain pending, be rejected later, or be processed twice after a retry.

The model's response is not the place to check. The current payment record is. The same principle applies to an identity system for employee access, an ERP for a purchase order, and a shipping or order system for a delivery.

In the article about how much authority to give an AI system, I separated recommending an action from having the authority to execute it for the company. The measurement side adds another distinction: executing an action and verifying its result are separate responsibilities too.

Some results are not immediately known. The system may accept a request while the work continues in the background. In that case, “pending” or “outcome unknown” is more accurate than a premature success message.

A completed task and a business result are different measures

A confirmed refund is a real outcome. It still does not tell us why the company changed its customer support system.

If the goal was to resolve more problems on the first contact, repeat enquiries and resolution rates matter. If the goal was to increase agent capacity, response time alone is not enough; we need to measure how much work is resolved at the same quality. If the goal was to reduce customer churn, the observation period must be longer.

In Generative AI at Work, published in The Quarterly Journal of Economics in 2025, researchers examined data from 5,172 customer support agents. With AI assistance, the number of customer issues resolved per hour rose by 15% on average. The gains were larger for less experienced workers, while the most experienced workers saw small declines on some quality measures.

The study went beyond scoring the model's suggestions. It examined issues resolved, conversation duration, resolution rates, and customer experience. The researchers also limit their result to one company and one particular customer support system. We cannot take the 15% figure and treat it as the expected return from AI in every business.

The lesson is not the 15% figure. It is that measurement has to extend from what the system produces to the reason the work exists.

Not every use case needs a heavyweight evaluation system

If an internal knowledge assistant only answers employees with cited sources, you may not need hundreds of metrics at the start. A small set of real questions, checks for whether answers rely on the right source, regular human review, and errors reported by users can form a useful baseline.

A system that moves money, grants access, or makes commitments on behalf of a customer needs more evidence. In addition to output quality, you must track authorization, the real transaction outcome, retries, and recovery after failure.

Claims about business impact require a different standard again. If a KPI improves, you cannot immediately credit the AI system. Seasonality, a team change, another software release, or routing only easy work to the system may have changed the result. At minimum, you need a record of the previous state, a suitable comparison, and enough time for the effect to appear.

The evaluation effort should grow with the system's impact and the cost of failure. A small but real body of evidence may be enough for a simple use case. The same simplicity can become a serious blind spot in a high-impact system.

Define success before selecting the model

When success measures are chosen after an AI system is built, the numbers that happen to be available tend to become the goal. The number of responses generated, successful tool calls, acceptance rate, or tokens consumed can suddenly look like the project's outcome.

It is safer to complete four sentences before choosing the model or evaluation platform:

1. Which answer, recommendation, or classification must the system produce correctly?
2. What result must actually appear in an external system or in the user's task?
3. Which business measure should that result change, and within what period?
4. Which failure, authorization breach, or cost increase would invalidate the claim of success?

The answers will differ from one company to another. That is precisely why they are useful. They connect model comparison to the company's real decision.

It is easy to measure how intelligent a system appears. The harder task is showing what it actually completed and what changed in the business as a result.

I explain why this distinction should be made before model selection in the first article in this series.

You can find the other decisions about models, information, authority, and evaluation in the guide to designing an AI system for your business.