Deterministic AI Code Modernization
Coverage measures conversion, not completeness, and buyers find the gap once testing stalls. What disposition-based verification requires before cutover.
Lindsay Britt
7 min read
A modernization vendor sells a buyer on modernizing their codebase. Months into the engagement, testing stalls, timelines slip, and the buyer discovers a distinction the proposal never drew: reported coverage has stalled well short of complete, 80 percent on this engagement, maybe 90 on another, and finding that out mid-project costs far more than knowing it before signing. What the vendor actually means is a pace metric: how much source code has been carried into a new language so far. What the buyer needs is a completeness metric: how much of the resulting change has been demonstrated against the source, intentionally changed, or explicitly flagged for review. Those are two different claims, and only one of them shows up in the sales conversation. The distance between them is where modernization budgets and timelines quietly come apart. The real question is not what percentage of the code an AI can convert. It is whether one hundred percent of the resulting change can be accounted for: what was preserved, what changed intentionally, what is new, and what remains explicitly unresolved.
What is AI code modernization coverage? In most vendor reporting, coverage measures how much of a legacy codebase has been converted or generated into modern code, not how much of that output has been checked against the original system's behavior. A high coverage number can describe a codebase that translates cleanly on the surface while carrying forward undocumented business rules, edge-case control flow, and hidden dependencies that no one has verified yet.
Most enterprise teams evaluating vendors right now are working from that same pitch: a proposal that promises modernization, not a disclosed ceiling. Few buyers sign expecting a partial outcome, so the question that would surface the remainder rarely gets asked before the engagement starts. Whatever its size on a given codebase, 20 percent on one, single digits on another, the remainder is not simply the hardest slice to translate. It is where the risk that was always present in the system finally becomes visible, and it stays invisible for exactly as long as coverage is treated as an implicit promise instead of a number a buyer can ask for directly.
Why the Remainder Is Structurally Different, Not Just Harder
The easy majority of a legacy migration is easy because the logic is legible. Straightforward COBOL paragraphs, well-scoped subroutines, and business rules that match their documentation translate predictably, whether the tool doing the translating is a large language model or a rules-based transpiler. The remaining fraction is not simply harder to translate; it is harder to accept. The behavior lives in the source: a decades-old edge case still executes on every run, and the code records exactly what the system does. The rationale does not. Whether that behavior is intentional, obsolete, or safe to preserve lives in institutional memory, in a support ticket from a decade ago, or in nobody at all, because the engineer who understood the rule retired years before the migration started. A generator can produce a plausible implementation of what it sees. What it cannot establish on its own is whether the evidence supports that implementation. And where neither the source nor supplied evidence establishes an answer, the honest output is not an inference. It is an explicitly flagged boundary.
Acceptance is only half the difficulty; generation itself degrades with scale, and the benchmark data shows how badly. RepoMod-Bench, a 2026 benchmark evaluating coding agents against real-world repository modernization tasks, found that agent pass rates fall from 91.3 percent on codebases under 10,000 lines to 15.3 percent once a codebase exceeds 50,000 lines, a drop of 76 points.¹ Note what that number measures: behavioral correctness, not coverage. The agents produced output at scale, and the output failed the hidden behavioral tests. It is evidence that generation degrades, not proof of a universal eighty-twenty law. It is not a gradual slope but a cliff, and it arrives at repository sizes still small next to the systems modernization programs actually face: sprawling COBOL cores, decades-old batch jobs, and business logic distributed across programs no single engineer has read start to finish. A probabilistic model trained on public code patterns has no reliable way to distinguish the rule it recovered from the rule it guessed, so it reports both the same way, as coverage, without saying which parts it verified and which parts it inferred.
The Gap Between Converted and Demonstrated
Coverage, reported honestly, describes conversion and test-readiness progress: how much of a codebase has been carried into modern code and is ready to test against. It says nothing about disposition, meaning whether each piece of that converted code has actually been checked, changed intentionally, or left untouched because nobody looked at it yet. Vendors that report only the first number are not lying. They are answering a question the buyer did not ask.
A fintech running a core banking migration learned this distinction the hard way during a competitive evaluation. Their central concern was avoiding what their engineering team called "JOBOL," Java that compiles and runs but is structurally still COBOL underneath, carrying the same undocumented liabilities forward in a new syntax. In a head-to-head bake-off, Holonic's CodeIntent applied semantic intent mapping followed by full recomposition and produced something the buyer had not seen in any coverage report: an explicit disposition for every artifact in the evaluation sample, delivered within hours, verified against original behavior, intentionally changed, or unresolved and flagged for review.² A competing modernization platform, working from the same source over months of effort, reported conversion progress with no artifact-level disposition attached to the remainder. One hundred percent accounted for does not mean one hundred percent automatically verified; some artifacts were flagged, not cleared. But nothing was silent, and CodeIntent was selected. A senior agentic engineering leader at a global systems integrator later summarized the pattern more broadly: "Each of these tools, ours, our competitors', our partners', gets us to about 80, 85% coverage. Holonic's deterministic approach fits into that 20% where we're missing." He is describing the artifact-level accounting that the percentage omits.
Coverage as reported by most vendors | What a buyer actually needs before cutover | |
|---|---|---|
What it measures | Percentage of code converted or generated | Disposition of every artifact against original behavior |
How gaps surface | Silently, discovered during testing | Explicitly, flagged before testing begins |
What "80%" means | 20% still unconverted or unchecked | Every artifact accounted for; some flagged for review |
Where risk hides | In the unreported remainder | Never in silence; unresolved evidence is named instead of being counted as success |
Every artifact dispositioned · zero unexplained is the standard that closes this gap. Instead of a single coverage percentage, a governed model assigns every artifact in the governed scope one of six explicit states: Verified, Preserved, Allowed Change, Net-New, Review Boundary, or Orphaned. Verified is an evidence state, not a confidence score: it is computed from governed evidence traceable to source, not asserted by a model's confidence or a reviewer's click. And the accounting starts with scope itself: copybooks, JCL, schedulers, and other adjacent artifacts are explicitly included, excluded, or flagged as unresolved. None sits silently outside the denominator. A buyer reading that disposition list knows exactly what changed, what stayed the same on purpose, and what still requires a human to look at it before cutover. A buyer reading an 80% coverage number knows none of that.
What Real Coverage Requires
Getting from a pace metric to a completeness claim requires a different kind of system underneath the conversion work, not just a more capable model. Specifically, it requires:
A deterministic, source-derived model of the codebase, so the same source, evidence, and governed rules produce the same derived result every time, and results are re-runnable and checkable rather than dependent on a model's confidence at generation time. The determinism lives in the accounting, not in the translation itself.
Explicit failure instead of silent gaps. When a rule cannot be verified against the source, the system flags it explicitly instead of letting it pass silently. In disposition terms, the rule lands as a Review Boundary instead of being silently counted as Verified.
Traceability from every output back to specific source lines, not to a model's general impression of what the original code probably did.
A record of what was preserved, changed, removed, inferred, or flagged, so a reviewer can audit the disposition list without re-deriving it from scratch.
None of this requires abandoning large language models. It requires putting a governed layer underneath them that checks their output against the source system before anyone calls the job done. Testing should confirm behavior, not discover what the LLM code generator forgot, and that distinction only holds if the disposition work happens before testing starts, not during it. Disposition does not replace the validation toolkit either: differential testing, characterization tests, trace replay, and parallel runs remain how behavior gets demonstrated. Disposition establishes what that evidence covers, and what remains unresolved.
What Stays After Cutover
The coverage question matters most during the migration itself, but the model built to answer it does not have to disappear once the code ships into production. A governed representation of source, intent, target, and evidence, built once during modernization, can persist as the reference against which a team, a systems integrator, or an AI coding agent checks future changes. During modernization the question is simple: can we accept this proposed implementation? That is the same question eighteen months later, when an AI agent proposes a change to a payment rule, and two years after that, when a new team rewrites a service. A major source of technical debt is the loss of any durable record of what a system is supposed to do once the people who built it leave the team. A disposition model built during modernization is one way to keep that record intact going forward, a durable reference for governing the changes that come after cutover.
Close
Coverage will keep being reported as a conversion percentage, because it is the number a generation pipeline can produce about itself. Buyers evaluating those numbers from scratch have a straightforward way to test any claim in the room: ask what happens to the remaining percentage, and ask for it in writing before signing, conversion as the throughput metric, disposition as the acceptance metric, both in the statement of work. The remaining percentage will shrink as generation improves. The obligation to account for one hundred percent of the change will not.
LLMs propose. Holonic verifies.
If you want to see what disposition-based coverage looks like against your own codebase's actual shape, rather than a vendor's aggregate percentage, book a demo and bring a representative sample.
Share this post
Stay in the loop
Get new writing on deterministic modernization, evidence, and governed AI.
Related articles

Why COBOL to Java Migrations Stall at Sign-Off, and How Bounded Verification Fixes It
AI can generate COBOL to Java migrations fast, but production sign-off still requires a human reviewer's certification. This piece explains why unbounded review breaks down at scale, and how a bounded, source-traced evidence trail makes sign-off possible again.

Evaluating AI Modernization Vendors? Ask What Replaces the Two-Year Parallel Run
Prompt-derived AI can translate legacy code fast, but it can't replace the multi-year parallel run enterprises rely on for proof. Here's what can.

AI Can Rewrite Your Legacy System. It Can't Tell You What It Changed.
LLMs can generate legacy code migrations fast, but they can't demonstrate what they changed or account for what broke along the way. Here's why sampling and coverage reports fall short, and what deterministic verification actually requires.
RepoMod-Bench, 2026. Agent pass rate declines from 91.3% (codebases under 10,000 LOC) to 15.3% (codebases over 50,000 LOC). https://arxiv.org/abs/2602.22518
Holonic customer proof point: fintech core banking modernization competitive evaluation, CodeIntent vs. IBM watsonx (internal case data).
HOLONIC
The deterministic evidence layer underneath legacy modernization and the agentic enterprise.
© 2026 Holonic Technologies, Inc. · Atlanta, GA · Tucson, AZ
CodeIntent® is a registered trademark of Holonic Technologies.