The Pre-Release Scan Was Checking Only One Direction

A scan we run before every release meeting reported the board and the code in sync, no discrepancies, nothing to chase down. That verdict was about to go straight into the meeting.

What the scan believed

The report read clean: every ticket the release diff touched had a matching entry on the board, no leftover references, nothing missing. It was a real check, and it had passed. The person about to run the meeting pushed back anyway, from memory: last time, something in a different part of the codebase hadn’t been flagged properly, and they wanted it checked again before trusting a clean read.

What was actually true

The pushback was right, and the scan’s own design explained why. Every check it ran started from the pending code changes and asked whether the board agreed with them. That direction can catch a ticket wrongly marked or missing from the diff, but it structurally cannot catch a ticket the meeting would read out loud that the diff never contained at all, because there was nothing on the “code changes” side to compare against. A scan built entirely around “does the board match this diff” is blind to “is there a ticket on the board this diff should have touched and doesn’t.”

Rebuilding the check the other way, starting from every ticket on the board and asking what the code history actually says about it, immediately surfaced seven real discrepancies on a release the narrower scan had called clean minutes earlier: tickets carrying a release label whose code had never actually reached the branch that ships, and at least one ticket whose work lived in the codebase under a different label than the board’s own field claimed.

The fix’s own blind spot

The rebuilt check went in as three new rules. One of them was itself wrong. It flagged two tickets as anomalies because their code lived only in a separate pipeline the main release doesn’t touch, treating that as a mismatch. It wasn’t. That separate pipeline is supposed to carry the same release label, by design, and ships on its own cadence a little later. The false flag was caught about an hour after it shipped, once that pipeline’s actual release ran and gave something real to check the new rule against, and it was demoted from an anomaly to a plain note in the same meeting summary.

What shipped

The release meeting now reads from a check that looks both directions: board to code, and code to board. Seven real gaps got found the same morning they were finally visible, and the one new rule that cried wolf on correct data got corrected before it could do that every single week going forward. The clean report that started this had already earned trust it hadn’t verified; the only reason the actual gaps surfaced was that someone declined to accept a clean read from memory of what had gone wrong before.

dxdev LESSON · paste into your AI coding agent
LESSON: A Coherence Check Built to Read One Ship List Can't See a Ticket That List Never Contained
SOURCE: dxdev.com/blog/2026-08-18_the-pre-release-scan-was-checking-only-one-direction
WHAT HAPPENED: A pre-release scan reported the board and the code in sync, and that "in sync" verdict was about to go straight into a release meeting unchallenged. It was wrong: every one of its checks started from the pending code changes and asked whether the board agreed with them, which structurally cannot catch a ticket the meeting would read out that the pending changes never touched at all. Rebuilding the check to instead start from every ticket on the board and ask what the code actually says surfaced seven real discrepancies the old, narrower scan had been calling clean. The very first version of that rebuilt check then produced its own false positive, flagging two tickets as anomalies that had, in fact, shipped correctly through a different, valid pipeline; that was caught and corrected about an hour later, once the real release for that pipeline had actually run and could be checked against.
THE RULE: A coherence or reconciliation check that only walks in one direction (from the change set to the source of truth) cannot see a discrepancy that never produced a change in the first place; build it to walk from the source of truth outward instead, and expect its first version to have its own false positives worth checking rather than trusting immediately.
CHECK MY CODE, then report PASS or FAIL with file:line for each:
  1. Any reconciliation, drift, or coherence check that starts from a diff or change set and asks "does the source of truth agree," rather than starting from the source of truth and asking "does the diff account for this."
  2. Any "all clear" or "in sync" report that would be relied on in a meeting or decision without a spot check against the underlying data first.
  3. Any newly added anomaly-detection rule that hasn't yet been run against a real, known-good case to confirm it doesn't fire on correct data.
THEN PRINT: a table (check, PASS/FAIL, evidence, fix) + a verdict (applies / partially / OUT_OF_SCOPE / no) + the single most important next action.