← Back to all posts

Practitioner Notes

AI Found Gaps We Didn't Know to Look For.

Craig Hankins, Besserhealth

The common concern about using AI in high-stakes product development is that it produces confident-sounding output that misses nuance. That it generates plausible-seeming specifications for things that won't actually work, and that errors only surface when domain experts review the work — or, more expensively, when engineering tries to build it.

A recent healthcare AI engagement ran exactly counter to that concern. During the specification phase — before any engineering had begun, before any clinical review had been formally scheduled — the specification process itself surfaced 47 open gaps. Six of them were critical. Twenty-two were high priority. Each one came with a named owner, a priority classification, and an impact assessment.

None of the 47 were discovered reactively. They weren't findings from a review cycle or a sprint retrospective. They were surfaced during the act of specifying — because systematic specification at depth finds things that summary planning misses.

The Right Place for Discovery

In a traditional product process, where does gap discovery actually happen?

Clinical and regulatory gaps surface during formal review cycles — scheduled after design work is considered complete. Technical gaps surface during engineering sprints, when implementation hits a missing specification. Business model gaps surface during financial modeling, often late in the process when they're most expensive to correct. The pattern is consistent: discovery happens at the wrong phase, because the discovery mechanism is applied after production rather than during it.

The cost of late discovery is well understood. A gap caught during specification requires a documented decision and a specification update. The same gap caught during engineering requires stopping implementation work, convening a clarifying conversation, revising the specification, and restarting. The gap caught after launch requires a product change, a user communication, and potentially a compliance remediation.

The expensive discovery is consistently the one in the wrong place.

What Systematic Specification Actually Catches

Gaps surface during specification when specification is done at depth — when the process of defining behavioral contracts for each feature forces the product team to confront questions they haven't answered yet.

On the engagement above, the 47 gaps spanned multiple categories. Some were clinical: what constitutes safe guidance versus guidance that requires professional oversight? What escalation protocol applies when a user's responses indicate elevated clinical risk? Some were technical: what does the system do when a required data element is unavailable? What are the boundary conditions for AI-generated content in a regulated context? Some were operational: what is the eligibility definition for the product's target population? How does the product handle users who present outside that definition?

In each case, the gap was not exotic or edge-case. It was a core question about how the product must behave in a scenario the product will encounter regularly. The gaps weren't discovered because AI was particularly clever. They were discovered because the specification process asked, systematically, "what must the product do when this happens?" — and the answer wasn't in the specification yet.

Gap Discovery vs. Gap Reporting

There's a meaningful distinction between discovering a gap and reporting it usefully.

Discovering a gap means identifying that the specification is silent on a required behavior. Reporting it usefully means providing the information that allows the right person to resolve it: what the gap is, why it matters, who needs to resolve it, what the resolution will unblock, and how urgent it is relative to other open items.

The 47-gap log on this engagement structured every item consistently: gap description, priority classification, the team or individual responsible for resolution, and the downstream specification elements that depended on that resolution. Clinical gaps were routed to clinical stakeholders with structured review packages. Technical gaps were routed to engineering with impact assessments that allowed sequencing against the build plan.

This structured routing is what converts gap discovery from a problem into a project management asset. A gap with an owner, a priority, and a clear description of what it unblocks is a trackable item. A gap discovered during engineering, without any of that context, is an interruption.

Why AI Changes the Gap Discovery Calculus

The scale and speed of AI-assisted specification creates an opportunity for gap discovery that doesn't exist in traditional product processes. When specification is produced quickly and at depth, the product team is covering more behavioral territory per session than human-only processes allow. More behavioral territory means more opportunities for the specification process to surface gaps — as long as the specification process is designed to surface them.

That design choice matters. An AI tool that produces fluent, complete-seeming output without flagging what it doesn't know will miss gaps that a more structured process would catch. An AI tool operating within a methodology that treats gap identification as a specification output — a deliverable, not a side effect — produces a different kind of specification.

On the engagement above, the gap log was a deliverable. It was produced during specification, updated continuously as new gaps surfaced, and managed as a structured backlog with the same discipline as the specification documents themselves. The result: clinical stakeholders received structured review packages organized by priority before engineering began. Six critical items were resolved — with decisions documented — before the engineering team wrote a line of code.

The expensive discoveries happened in the right place.

The Question to Ask at Every Specification Stage

Before proceeding from any specification workstream to the next, one question should have a documented answer: what are the open gaps in this workstream, and who is responsible for resolving each one?

Not "are there any gaps?" — because there always are. The question is whether they're documented, prioritized, owned, and tracked. A specification that's complete except for known, tracked, owned gaps is a significantly better engineering input than a specification that appears complete but carries untracked gaps as silent assumptions.

AI surfaced 47 gaps during specification on one engagement. The alternative — discovering them during engineering — would have converted 47 tracked items into 47 mid-sprint interruptions. The conversion cost compounds with each gap discovered late.

The principle: AI's most valuable contribution in specification isn't generating output faster. It's surfacing the gaps that a traditional process discovers at the wrong phase, at the wrong cost.

Frequently Asked Questions

Why does AI-assisted specification surface more gaps than traditional processes?
Because systematic specification at depth — covering more behavioral territory per session — creates more opportunities for the specification process to encounter questions that haven't been answered yet. A process designed to surface those gaps as explicit deliverables (rather than allowing AI to fill them with plausible assumptions) produces a gap inventory that allows structured resolution before engineering begins.

What types of gaps surface during specification on healthcare AI engagements?
Clinical boundary conditions (what guidance requires professional oversight), technical edge cases (system behavior when required data is unavailable), regulatory constraints (what the product can and cannot claim or recommend), operational gaps (eligibility definitions, population edge cases), and escalation protocols (what happens when user signals indicate elevated risk). These are typically core questions, not exotic edge cases.

What is the difference between gap discovery and gap reporting in IDPD?
Gap discovery is identifying that the specification is silent on a required behavior. Gap reporting is providing the information that enables resolution: what the gap is, why it matters, who resolves it, what it unblocks, and what priority it carries relative to other open items. IDPD treats gap reporting as a specification deliverable — a structured log managed with the same discipline as the specification documents themselves.

What happens when clinical and technical gaps are discovered during engineering rather than specification?
A gap discovered during specification requires a decision and a document update. The same gap discovered during engineering requires stopping implementation work, convening a resolution conversation, revising the specification, and restarting. In regulated domains, the cost compounds further if the gap affects compliance architecture. Late discovery is always more expensive than early discovery; the question is what mechanism causes discovery to happen early.

Written by Craig Hankins, Besserhealth LLC

← Back to all posts