← Back to all posts

Practitioner Notes

What should your product refuse to do? Writing must-not rules an AI agent will follow

Craig Hankins, Besserhealth

Your product needs a written set of must-not rules: what it must never do, in business, user and operating terms, stated as behaviors an agent can build and test. Coding agents already get rules about the repository: which files not to touch, which commands not to run. Almost nobody writes the rules about the product. On the agentic health startup I’m building, the spec carries nine governing constraints. None of them is about code. All of them are about what the product must never do to the person using it or to the business behind it.

What is a product must-not rule?

It is a prohibition the product has to honor in every situation, written precisely enough that an agent can build it and a test can check it. Three from our build:

  • No product or service is mentioned until the person’s needs are understood.

  • No fear or urgency language, anywhere.

  • A partner’s commission is never an input to ranking.

None of these would appear in a typical user story. Each one protects something the business depends on. The first protects the user experience. The second protects trust. The third protects revenue model integrity: if commission could move a recommendation, the product would quietly become an ad.

Definition — prohibition set: the written list of behaviors a product must never exhibit, across market, user, operations and revenue, specified as testable rules rather than left as shared understanding.

Why won’t an agent infer them?

Because people used to supply them silently. An experienced engineer building a recommendation engine knows, without being told, that ranking by referral fee would embarrass the company. A coding agent doesn’t know your company. It picks the most plausible implementation, and for a ranking engine with a commission field in the data, plausible means using it.

Wherever agents execute work, the tacit norms a team carries have to become explicit. The agent receives the artifact, not the culture the artifact pointed at.

How is a product rule different from a repository rule?

  • Where it lives: a repository rule lives in AGENTS.md, CLAUDE.md or a constitution file. A product rule lives in the product spec, alongside the behaviors.

  • Who writes it: engineering writes repository rules. Product writes product rules, with the business owners.

  • What it protects: repository rules protect the codebase and the build. Product rules protect the user, the brand, the revenue model and compliance.

  • Example: “Don’t modify the migrations folder” is a repository rule. “Never use urgency language to drive a sign-up” is a product rule.

  • How it’s checked: repository rules by linting, CI and code review. Product rules by behavior tests against the spec.

Both are needed. Spec-driven development tooling has a natural home for the first kind. The second kind has no default home, so it usually doesn’t get written.

What should a product refuse to answer yet?

Must-not rules also set scope. Late in our build, the team agreed to limit the first release’s AI to vetted knowledge sources only. As one of our leaders put it, “We can’t answer every single possible question.”

That decision is a prohibition: the product must not answer from outside the vetted set. Written down, it let the team say no to a whole class of edge cases without reopening the vision. In my experience, the strongest products in year one aren’t the smartest. They are the most disciplined about what they refuse to do yet.

How do you write must-not rules an agent will follow?

  1. Start at the business, not the code. For each of the four dimensions — market, user, operations, revenue — ask what the product could do that would hurt the person or the business. Those answers are the candidate rules.

  2. Write each as a testable behavior. Name the trigger, the prohibited behavior and what to do instead. “Be sensitive” is not a rule. “Never show pricing before the needs summary is confirmed” is.

  3. Keep the set small and material. A broad rule gets applied everywhere at full weight. On our build, one unbounded instruction to flag all clinical content cascaded into a 600-line issue log we had to abandon. Rules need edges.

  4. Tell the agent what to do when unsure. Our spec instructs agents: do not fill a gap with a plausible assumption, and when two sources conflict, do not pick a side; raise it.

  5. Test them in the build, not just the spec. Our prohibitions survived every rewrite of the spec. The build didn’t honor all of them on the first pass. Independent review caught it. A rule on paper is a hypothesis until a test checks it.

Does this apply to operations work too?

Yes, and arguably more. When an agent runs an operations process — intake, scheduling, claims triage, customer follow-up — the must-not rules are where compliance and brand risk live. The cheapest compliance control is a well-placed constraint inside the process, not a review after the agent has acted.

In healthcare this is sharper. Intended use, consent and oversight are often documented for a regulator but never encoded as behavior the product must exhibit. That gap is one of the most important things I’m still learning about, and the place I’d look first on any regulated build.

How does PBA handle prohibitions?

Product Behavior Architecture (PBA) is an AI-native product operating system that treats the prohibition set as a first-class artifact. It starts at the business model, derives must-not rules from market viability, user experience, operational feasibility and revenue model integrity, and carries them into the behavior cards agents build from. Most product methods have an artifact for what the product should do. Very few have one for what it must never do.

Takeaway

Agents build what is stated and skip what isn’t. The rules that protect your users and your business are usually the ones nobody states, because people used to supply them without being asked. Write them down, keep them few, and test them.

AI builds fast. How are your teams defining and deciding before the build?

Frequently Asked Questions

What is a prohibition set?
The written list of behaviors a product must never exhibit, across market, user, operations and revenue, specified as testable rules rather than left as shared understanding.

How is a product rule different from an AGENTS.md rule?
AGENTS.md and similar files hold repository rules that protect the codebase and the build. Product rules protect the user, the brand, the revenue model and compliance, and they belong in the product spec as testable behaviors.

How many must-not rules should a product have?
Few, and each one material. A broad rule gets applied everywhere at full weight; on our build, one unbounded instruction cascaded into a 600-line issue log. Our spec carries nine governing constraints.

If you want to compare notes on your own product’s must-not rules, book a discovery call: calendly.com/besserhealth/discovery

Related: What is a behavior card? · Business Product Definition Matters More in AI Projects, Not Less.

Written by Craig Hankins, Besserhealth LLC

← Back to all posts