Consider a hypothetical interest-accrual batch job that moves from COBOL to Java. For months the two systems run in parallel, and the totals match every night. Then the first year-end after cutover arrives, the converted year-end path runs in production for the first time, and the Java code rounds the year-end adjustment differently from the COBOL it replaced. Nothing in the parallel runs was wrong; the year-end path was simply outside what they exercised.

Gartner predicts that more than 70% of mainframe exit projects started in 2026 will fail to deliver their intended benefits because teams overestimate what generative AI tooling can do. Announcing the prediction, Gartner VP Analyst Alessandro Galimberti pointed to “a widening gap between the marketing promise of GenAI and its real-world ability to transform and migrate complex legacy code.”[1] Whatever a program decides about the mainframe, that gap is a reason to verify a COBOL to Java migration on evidence, starting with a clear definition of what it has to preserve.

What does it mean for a migration to preserve behavior? A migration preserves behavior when the migrated system produces the same observable results as the original, including outputs, data changes, return codes, and interface messages, within an agreed scope. Every intentional difference is approved and recorded.

This guide covers how to build the COBOL baseline, the traps your tests need to catch, where testing reaches its limit, what to require before sign-off, and an exit-criteria checklist you can adopt as written.

COBOL to Java migration testing starts with a baseline

Differential testing against the running COBOL system is the foundation of a credible test plan. Start with a versioned baseline, the reference for what’s sometimes called golden master testing: inputs, outputs, data state before and after each run, return codes, and interface messages for every workload in scope. Record the source and copybook versions behind it, plus execution context that can affect semantics: compiler options, JCL, code page and collation, record definitions, and runtime configuration.

Then build a harness that runs both systems on the same inputs and compares results at the field level, not as totals or row counts. Normalize only differences known to be irrelevant, such as timestamps or generated IDs, and treat every other mismatch as a finding until someone explains it. Captured workloads aren’t the only source of inputs, either. IBM researchers have described a framework that automates functional equivalence testing: it uses symbolic execution to generate unit tests for COBOL programs and then checks the translated Java against the same tests, which reaches paths production traffic may not.[2]

Finally, don’t wait for the calendar to test the calendar. Replay historical month-end, quarter-end, and year-end inputs, or run controlled date scenarios that force those paths, and then confirm the results with a parallel run of both systems before cutover. A team that does all of this will catch the year-end rounding problem in the opening example long before December.

The COBOL-to-Java traps your tests need to catch

COBOL and Java handle numbers, text, record layouts, control flow, and job orchestration differently, and each difference produces a predictable mismatch. The table below, written for the engineers building the harness, pairs each trap with the inputs that trigger it.

Where parity testing stops

Parity testing, the umbrella term for comparing the two systems’ results through differential testing, replay, and parallel runs, has a ceiling that better test design raises but doesn’t remove. A passing result describes the paths that ran. Error handling, rare data combinations, and conditions nobody thought to construct can stay untested, and a green report can’t tell you which ones they are. For more on that limit, see our piece on the coverage trap.

A parity-only acceptance standard can also preserve legacy defects unless someone explicitly decides otherwise. If the COBOL program mishandles an edge case, a faithful conversion reproduces it, and the harness reports a match. Each identified defect needs a recorded decision: keep it for compatibility, or fix it as an approved change. The same gap appears in any AI-assisted code change, where unintended behavior changes can pass every existing test.

What to require before sign-off

Tests answer one question: did these runs produce the same results? Acceptance needs an answer to a second one: for each business rule in scope, did the Java keep what the COBOL did, change it on purpose, or leave it unclear? Answering that means reading the code rather than running it. Take each calculation, validation rule, state transition, and interface contract in the COBOL, work out what it does from the source, find its counterpart in the Java, and compare the two. Where they differ, trace the difference to the specific code on both sides and to the downstream records and interfaces it feeds.

Manual code review can do this for a handful of programs, but across thousands of programs and copybooks it becomes the bottleneck, which is the gap purpose-built tooling helps reviewers close. Whatever does the work, each rule should end up with one of a few clear outcomes: it carried over unchanged, it changed and someone approved the change, it’s new, or it couldn’t be established from the code and needs a person to look at it. Spotting a possible difference, confirming a rule carried over, and approving an intentional change are different results, and the record should keep them separate.

Here is what that record could look like for the hypothetical year-end scenario from the opening:

• Scope: Convert the interest-accrual job while keeping the existing year-end calculation behavior.

• What the COBOL does: The year-end path stores the calculated adjustment in a two-decimal receiving field using ROUNDED.

• What the Java does: The converted calculation reduces the result to two decimal places using RoundingMode.HALF_EVEN.

• The difference: For an exact intermediate value of 123.445, the COBOL calculation produces 123.45, while HALF_EVEN produces 123.44.

• What it affects: The year-end accrual result and the downstream records and interfaces that consume it.

• What testing showed: The parallel-run workload never executed the year-end path, so the test results neither confirm nor contradict this difference.

• Decision needed: Change the Java to match the COBOL rounding rule, or approve and validate the new behavior.

• Status: Open, awaiting the program owner’s decision.

The record doesn’t pretend the question is settled. It hands the program owner, with risk sign-off where required, a specific decision that a clean test report can’t surface for a path it never ran. Our sign-off guide covers what risk officers will ask for in the audit trail. Hold vendor and SI equivalence claims to the same standard by asking what each claim covers and what evidence supports it; our guide to evaluating vendor equivalence claims goes further.

A 10-point exit-criteria checklist for accepting the migration

Agree on these criteria with your vendor or SI before conversion starts and write them into the acceptance terms, so both sides measure the same thing and acceptance is settled before disputes start. Review them at each migration wave and again before cutover. When an item isn’t met, it either holds acceptance until it’s resolved or gets an explicit, approved exception that states what is still uncertain. Assigning an owner records who will resolve an item; it doesn’t resolve it.

1. A versioned COBOL baseline is captured, with its source, copybook, and execution-context versions recorded.

2. Critical batch cycles reconcile at the field level across every material business-cycle boundary in scope, using historical replay or date scenarios where waiting for the calendar isn’t practical.

3. Zero unexplained differences: every difference between the two systems is either explained and approved or listed as open.

4. Every intentional difference has an owner, a rationale, and an approval.

5. Every identified legacy defect has a recorded keep-or-fix decision.

6. Every open item is either resolved or covered by an approved exception, recorded in the sign-off package, before cutover.

7. The traps in the table above have explicit tests.

8. Someone other than the party that performed the conversion can reproduce the evidence.

9. SI and vendor acceptance terms are based on reconciliation and explained differences, not percent converted.

10. The rollback, fallback, or recovery plan has been tested.

Where CodeIntent fits

Tests tell you whether selected runs produced the same results, and a diff tells you what text changed. Holonic’s CodeIntent answers the third question: which business rules, calculations, and interfaces carried over, which changed, and which still need someone to look at them, with each finding traced back to the COBOL source. Where it can’t establish something from the code, it says so instead of guessing, so every rule it analyzes ends with a recorded outcome and nothing is left unexplained. CodeIntent flags, checks, and traces; the acceptance decision stays with your team.

That record keeps working after cutover. Once the migration is accepted, the same reference for what the system’s rules are becomes the standard later changes are checked against, whether a developer writes it or an AI coding assistant does. The year-end rounding rule your team signs off on during the migration is the rule against which the next change to that job is measured. The same approach applies to any language-to-language migration.

See how CodeIntent works