Deterministic AI Code Modernization
Vendors sell full modernization, not an 80% ceiling, and buyers only discover the gap once testing stalls. Here's why coverage is a pace metric, not a completeness claim, and what disposition-based verification actually requires before cutover.
Lindsay Britt
6 min read
A modernization vendor sells a buyer on modernizing their codebase, full stop. Months into the engagement, testing stalls, timelines slip, and the buyer discovers what the proposal never mentioned: the tool only ever reaches about 80% coverage, and finding that out mid-project costs far more than knowing it before signing. What the vendor actually means is a pace metric: how much source code has been carried into a new language so far. What the buyer needs is a completeness metric: how much of that converted code can be trusted to behave the way the original system did. Those are two different claims, and only one of them shows up in the sales conversation. The distance between them is where modernization budgets and timelines quietly come apart.
What is AI code modernization coverage? In most vendor reporting, coverage measures how much of a legacy codebase has been converted or generated into modern code, not how much of that output has been checked against the original system's behavior. A high coverage number can describe a codebase that translates cleanly on the surface while carrying forward undocumented business rules, edge-case control flow, and hidden dependencies that no one has verified yet.
Most enterprise teams evaluating vendors right now are working from that same pitch: a proposal that promises modernization, not a disclosed ceiling. Nobody signs expecting an 80% outcome, so nobody asks the question that would surface it before the engagement starts. The last 15 to 20% of a legacy codebase is not simply the hardest fifth to translate. It is where the risk that was always present in the system finally becomes visible, and it stays invisible for exactly as long as coverage is treated as an implicit promise instead of a number a buyer can ask for directly.
Why the Last 15 to 20% Is Structurally Different, Not Just Harder
The easy 80% of a legacy migration is easy because the logic is legible. Straightforward COBOL paragraphs, well-scoped subroutines, and business rules that match their documentation translate predictably, whether the tool doing the translating is a large language model or a rules-based transpiler. The remaining fraction resists translation for a different reason entirely: the information a model would need to translate it correctly does not live in the source code. It lives in institutional memory, in a support ticket from a decade ago, or in nobody at all, because the engineer who understood the rule retired years before the migration started.
This is a scale problem as much as a complexity problem, and the data backs that up plainly. RepoMod-Bench, a 2026 benchmark evaluating coding agents against real-world repository modernization tasks, found that agent pass rates fall from 91.3% on codebases under 10,000 lines to 15.3% once a codebase exceeds 50,000 lines, a drop of 76 points.¹ That is not a gradual degradation. It is a cliff, and it lands exactly where enterprise legacy systems live: sprawling COBOL cores, decades-old batch jobs, and business logic distributed across programs no single engineer has read start to finish. A probabilistic model trained on public code patterns has no reliable way to infer a rule that was never written down, so it does the next best thing: it generates plausible code and reports it as coverage without saying which parts it is confident about and which parts it inferred.
The Gap Between Converted and Demonstrated
Coverage, reported honestly, describes conversion and test-readiness progress: how much of a codebase has been carried into modern code and is ready to test against. It says nothing about disposition, meaning whether each piece of that converted code has actually been checked, changed intentionally, or left untouched because nobody looked at it yet. Vendors that report only the first number are not lying. They are answering a question the buyer did not ask.
A fintech running a core banking migration learned this distinction the hard way during a competitive evaluation. Their central concern was avoiding what their engineering team called "JOBOL," Java that compiles and runs but is structurally still COBOL underneath, carrying the same undocumented liabilities forward in a new syntax. In a head-to-head bake-off, Holonic's CodeIntent applied semantic intent mapping followed by full recomposition and reached 85% effective coverage in hours. IBM watsonx, working from the same source, reached 40% coverage after months of effort.² CodeIntent was selected, and a senior agentic engineering leader at a global systems integrator later summarized the pattern more broadly: "Each of these tools, ours, our competitors', our partners', gets us to about 80, 85% coverage. Holonic's deterministic approach fits into that 20% where we're missing."
Coverage as reported by most vendors | What a buyer actually needs before cutover | |
|---|---|---|
What it measures | Percentage of code converted or generated | Disposition of every artifact against original behavior |
How gaps surface | Silently, discovered during testing | Explicitly, flagged before testing begins |
What "80%" means | 20% still unconverted or unchecked | Every artifact accounted for; some flagged for review |
Where risk hides | In the unreported remainder | Nowhere; it is named |
Every artifact dispositioned, zero unexplained is the standard that closes this gap. Instead of a single coverage percentage, a governed model assigns every piece of the source codebase one of six explicit states: Verified, Preserved, Allowed Change, Net-New, Review Boundary, or Orphaned. A buyer reading that disposition list knows exactly what changed, what stayed the same on purpose, and what still requires a human to look at it before cutover. A buyer reading an 80% coverage number knows none of that.
What Real Coverage Requires
Getting from a pace metric to a completeness claim requires a different kind of system underneath the conversion work, not just a more capable model. Specifically, it requires:
A deterministic, source-derived model of the codebase, so the same input produces the same disposition every time, and results are re-runnable and checkable rather than dependent on a model's confidence at generation time.
Explicit failure instead of silent gaps. When a rule cannot be verified against the source, the system says so out loud instead of quietly passing it through. Buyers in this market trust systems that admit what they don't know more than systems that claim to know everything.
Traceability from every output back to specific source lines, not to a model's general impression of what the original code probably did.
A record of what was preserved, changed, removed, inferred, or flagged, so a reviewer can audit the disposition list without re-deriving it from scratch.
None of this requires abandoning large language models. It requires putting a governed layer underneath them that checks their output against the source system before anyone calls the job done. Testing should confirm behavior, not discover what the LLM code generator forgot, and that distinction only holds if the disposition work happens before testing starts, not during it.
What Stays After Cutover
The coverage question matters most during the migration itself, but the model built to answer it does not have to disappear once the code ships into production. A governed representation of source, intent, target, and evidence, built once during modernization, can persist as the reference a team, a systems integrator, or an AI coding agent checks future changes against. Technical debt accumulates in the first place because nobody keeps a durable record of what a system is supposed to do once the people who built it leave the team. A disposition model built during modernization is one way to keep that record intact going forward, a topic worth its own piece rather than a paragraph here.
Close
Vendors will keep reporting coverage as a conversion percentage because it is the number that makes their pipeline look fastest on a slide. Buyers evaluating those numbers from scratch have a straightforward way to test any claim in the room: ask what happens to the remaining percentage, and ask for it in writing before signing. LLMs propose. Holonic verifies. If you want to see what disposition-based coverage looks like against your own codebase's actual shape, rather than a vendor's aggregate percentage, book a demo and bring a representative sample.
Share this post
Stay in the loop
Get new writing on deterministic modernization, evidence, and governed AI.
Related articles

AI Can Rewrite Your Legacy System. It Can't Tell You What It Changed.
LLMs can generate legacy code migrations fast, but they can't prove what they changed or account for what broke along the way. Here's why sampling and coverage reports aren't proof, and what deterministic verification actually requires.

The evidence package is the product
Modernization only matters if every generated line can be traced back to the system it replaced. Holonic treats evidence as a first-class deliverable, not an afterthought.
RepoMod-Bench, 2026. Agent pass rate declines from 91.3% (codebases under 10,000 LOC) to 15.3% (codebases over 50,000 LOC). https://arxiv.org/abs/2602.22518
Holonic customer proof point: fintech core banking modernization competitive evaluation, CodeIntent vs. IBM watsonx (internal case data).
HOLONIC
The deterministic evidence layer underneath legacy modernization and the agentic enterprise.
© 2026 Holonic Technologies, Inc. · Atlanta, GA · Tucson, AZ
CodeIntent® is a registered trademark of Holonic Technologies.