Start With the Business Problem, Not the AI Model
A company wants to improve customer support with AI. The first meeting turns to a familiar list: which LLM is most accurate, whether a more capable model will perform better, and whether the system needs several models.
Those questions matter, but they come later.
Start by deciding what should change in the operation. The goal may be faster answers, less time spent by agents searching for information, or more issues resolved on first contact. A system that drafts a response is also a different proposition from one that can initiate a refund. Until these choices are clear, a detailed model comparison remains technically interesting but disconnected from the business decision.
What it takes to resolve a support case
A language model can understand a customer message and draft a useful answer. A support operation needs more than that. It must identify the product involved, retrieve the current refund policy, read the order status from the right system, and tell an information request from a transaction request. If it is to start a refund, it must check the amount, the deadline, and the user's authority. Some cases must still reach a human agent.
That full, repeatable ability to produce an outcome is the business capability. The model may be an important part of it, but it is not the capability by itself. Data, business rules, existing systems, permissions, and human judgment all shape whether the work can be done reliably.
Architecture is therefore more than a diagram of connected technical boxes. It defines how models, data, software, and people work together to deliver the intended result. The distinction echoes the difference between managing technology and transforming the business: running technology and changing a business capability with it are different responsibilities.
Build only the system the work requires
General-purpose models can now understand text, inspect images, produce reasoned responses, and use software tools. That does not make a fleet of models and agents the default design.
An agent gives a model more discretion to choose tools and sequence its work. A workflow with known steps may not need that discretion. Anthropic's production guide recommends beginning with the simplest solution that works. Google Cloud's architecture guide makes a similar distinction between fixed processes and open-ended work where the model must choose a path.
Often, one capable model, a narrow set of tools, and software-defined workflow steps are enough. Add a second model, routing layer, or agent only when it demonstrably improves quality, cost, speed, or risk. The aim is not to accumulate AI components; it is to achieve the result without adding complexity the operation does not need.
Sometimes the best answer is no AI
Determining whether a refund window has expired may not need a language model. If order date, product type, and contract terms are recorded, a SQL query and explicit business rules may be more accurate, cheaper, and easier to audit.
Optimization software may fit shift planning or route creation, where objectives and constraints are explicit. For predictions from structured tables, classical machine learning can be a strong starting point.
Choosing a simpler method is not a failed AI project. It is sound system design. In a real chatbot project, the hard part was not simply generating an answer: integration, reliable handover, and ongoing operations determined the outcome.
A boundary between suggestion and action
A model can prepare the right information for a refund transaction. That does not give it authority to execute the transaction.
For systems that affect money, customer records, or external communication, define two boundaries separately: the actions a model may recommend and the actions the system may actually perform. Telling a model to perform only authorized actions is not access control. The transaction must be checked against the real user's or service's permissions, routed to human approval where needed, and recorded for later audit. OWASP's guidance on excessive agency likewise recommends actions be executed in the user's context with only the permissions required.
Authorization is not a security feature to bolt on at the end. It is a business decision that belongs at the start of the design.
Measure the work, not just the answer
A support system can tell a customer, “Your refund has been initiated.” The sentence may be clear and aligned with the policy. What matters first is whether the order system actually contains a refund record.
Even that record is not proof of business success. If first-contact resolution, customer satisfaction, losses, and agent workload do not change, the intended outcome may not have materialized.
Measure the model's response, the action completed by the system, and the effect on the business separately. The same distinction appears in the verification bottleneck in AI-generated code: producing an output does not prove the intended result was implemented correctly.
Three decisions before model selection
Before comparing models, settle three decisions:
- Business outcome: Which decision, workflow, or measurable result will change? What is the current baseline?
- Action boundary: Will the system provide information, prepare a recommendation, or execute a real transaction? Where is human approval required?
- Evidence: How will the model response, completed action, cost, and business result be measured?
Once those decisions are clear, the necessary data, rules, integrations, and models become much easier to identify.
Model selection is not trivial; it affects both what the system can do and its economics. But it becomes a useful choice only after the company understands what it wants to change and the boundaries within which the system must operate.
The next question may not be “Which model is better?” It may be “Does this problem need AI at all?”
I examine that decision through three real systems in When Does Process Automation Actually Need AI?
