An AI agent behaves differently every run because the model is probabilistic and the spec usually leaves the decisive behaviors unstated: when to stop, what to do when unsure, which instruction wins when two disagree. On the agentic health startup I’m building, our discovery agent — the one that interviews a family caregiver about their situation — asked anywhere from 3 to 27 questions across test runs. Some runs stopped after two or three questions. Some never stopped. The model was fine. The spec told it what to ask and never told it when it was done.
What does inconsistent agent behavior look like?
It rarely looks like an error. Each run, read on its own, seems reasonable. A short interview reads as efficient. A long one reads as thorough. You only see the problem when you run the same scenario many times and look at the spread.
We found ours through agentic QA: automated test walks through the same journey, repeated. The spread was the finding. A single passing walk would have told us nothing.
This matters for any agent inside a product or a process, not just in healthcare. A support agent that sometimes escalates and sometimes doesn’t, a triage agent that sometimes asks for documents and sometimes skips them, a scheduling agent that confirms some bookings and not others. Each is the same failure: behavior nobody specified tightly enough to repeat.
Definition — behavioral consistency: the share of runs, for the same scenario, in which an agent exhibits the specified behavior. It is measured across many runs, never from one.
Why does it happen?
Three causes, in the order I’d check them.
The spec states a goal, not a rule. “Stop when you understand the person’s needs” is a goal. The agent has to decide what understanding means, every run, from scratch. Sometimes it decides early. Sometimes it never decides.
The instructions disagree. When two sources give different guidance, the agent’s choice between them becomes close to a coin flip. A 2026 preprint testing agents across customer support, healthcare claims triage and legal drafting found that instruction consistency predicted instruction following (correlation of 0.57), and that structured context cut critical failures from 4.11 to 1.33 per evaluation (Bousetouane, arXiv 2607.14275). Earlier on our own build, agents reading three spec layers with no rule for which one wins produced output that varied run to run.
Nobody decided the edges. What happens when the user gives one-word answers? When they want to stop? When they describe something outside the product’s scope? If the spec is silent, each run improvises.
What counts as reliable enough?
Not perfect. Our target for the first release is that the discovery agent does the right thing in 90 percent of runs. Perfection is the wrong bar for a probabilistic system, and chasing it delays the release without helping the user much.
The important part is defining “right” in business terms before testing, across all four dimensions:
User experience: too many questions and caregivers drop off. Too few and they don’t feel heard.
Market value: too little information and the recommendations that follow are generic, which is the one thing the product can’t be.
Operations: every extra turn is processing time and a longer session to support.
Revenue: every turn has a token cost, and cost per conversation is a unit-economics number.
We haven’t locked the target question range yet. But the range is now a product decision, made deliberately, instead of a side effect of the model.
How do you specify an agent so it behaves consistently?
Replace goals with stopping conditions. List what the agent must know before it stops: the situations covered, the facts captured. Stopping becomes a check, not a judgment.
Set bounds. A minimum and maximum, with what to do at each. At the maximum, summarize what you have and confirm it with the user.
Write the uncertainty rule. When unsure, ask one clarifying question, then move on. Never fill a gap with a plausible assumption.
Write the exit rule. If the user wants to stop, summarize, confirm and stop. Don’t push.
Name the precedence. When two sources conflict, say which one wins. If none does, the agent raises the conflict instead of choosing.
Test the distribution. Run each scenario many times. Report the share of runs that meet the spec, not whether one run passed.
Each of these is a behavior card: one rule, a test, its dependencies. None of them is about code.
Is this a model problem or a product problem?
Mostly a product problem. A better model narrows the spread a little. A clear spec narrows it a lot, and it keeps working when the model changes underneath you. The decisions that make an agent consistent — when it stops, what it refuses, what it does at the edges — are product decisions. If product doesn’t make them, the model makes them, differently each time.
How does PBA specify agent behavior?
Product Behavior Architecture (PBA) is an AI-native product operating system that specifies how a product behaves at runtime, including the agents inside it. It starts at the business model — market viability, user experience, operational feasibility and revenue model integrity — and writes behavioral contracts with decision rules, stopping conditions, failure modes and “when uncertain, do X” rules. Conventional handoff artifacts describe screens and flows. An agentic product needs its judgment specified too.
Takeaway
If your agent behaves differently every run, look at the spec before the model. The behaviors that decide consistency are usually the ones nobody wrote down. Specify when it stops, what it does when unsure, and which instruction wins, then measure across many runs.
AI builds fast. How are your teams defining and deciding before the build?
Frequently Asked Questions
Why do AI agents give different results for the same input?
Models are probabilistic, and specs usually leave the decisive behaviors unstated: when to stop, what to do when unsure, which instruction wins. Each run improvises those decisions differently.
How do you measure AI agent consistency?
Run the same scenario many times and report the share of runs that meet the specified behavior. A single passing run proves nothing.
What reliability target should an MVP agent have?
Not perfection. Our target for the first release is that the agent does the right thing in 90 percent of runs, with “right” defined in business terms across user experience, value, operations and cost.
If you want to compare notes on an agent of your own, book a discovery call: calendly.com/besserhealth/discovery
Related: What should your product refuse to do? · AI Found Gaps We Didn’t Know to Look For.
