Six 3.366.x hotfixes were live on master, customers were already seeing the fixes, and the ticket that tracked them was still open. I found that while running the pre-release scan for 3.367 on the morning of the release.

What the scan was built to do

The scan answers one question before a release meeting: what should ship, and is the board honest about it? It checks that each ticket’s repos are stamped, that its status matches its code, and that anything unmerged is flagged as waiting. That morning it did its job. Two tickets got repo stamps, one closed, one moved to NO CODE.

It only looked forward, though. It compared the board against what was pending. It never asked the reverse question: which fixes are already on master while the ticket still says open?

That gap matters because hotfixes skip the normal flow. A fix goes out as a 3.366.x tag, the customer-visible problem stops, and everyone moves on. Nobody goes back to close the ticket. The next scan then sees an open ticket with a branch and can treat it as pending work, so the same fix may get queued again for the next release.

Finding the open ticket

I noticed it while checking ticket state by hand against master. The umbrella ticket for one stretch of fixes still read open, yet all six hotfixes from 3.366.x were on master. I verified each one against the tag history before closing the ticket on 3.367, then filed a follow-up with a verified handoff in its dev notes.

Closing one ticket by hand fixes nothing about the next one, so I added a check to the scan. The logic is simple:

for each commit reachable from master since the last minor tag:
    if it came from a hotfix branch:
        ticket = key parsed from the branch name (hotfix/<ticket>)
        if ticket status is not closed/released:
            flag: hotfixed but unclosed

The release merge that looked like a hotfix

My first version of that check was wrong, and I acted on it before I saw why. It identified hotfixes by looking at what had landed on master, and it counted merge commits as evidence of a hotfix. A release merge is also a merge onto master. So the release itself showed up as a hotfix, and every ticket the release carried became a candidate for “shipped but unclosed.”

The cost was a noisy flag list, exactly what a scan is supposed to prevent. A flag list you have to second-guess is worse than no flag list, because the first false positive teaches you to skim. I had added the check to fix a trust problem with the board, and it began producing its own. I spent part of the afternoon going through its output ticket by ticket, checking each against the tag history to see which flags were real, before the pattern showed itself: the false ones all traced back to the release merge.

The fix was a second commit. The scan now excludes release merges and counts only commits that come from hotfix branches. The first commit added the detection, and the second made it usable.

What the check catches now

After both commits, the scan reports hotfixed-but-unclosed tickets and leaves release merges out. That day the release itself continued as hotfixes: one for a stats page error from the morning’s deploy, and a later one for a sweep that marked about 32 domains gone instead of at risk. Those are the tickets I would want flagged if they were left open, and a dashboard that counted the release as a hotfix would have buried them.

I also tightened the post-release step so it stays on the release week instead of surfacing old backlog. That keeps the reverse check from turning into a general cleanup crawler. Weekly after-release work now lives in its own Aftercare section, and post-release runs in a fresh session so it starts without the release day’s assumptions.

A scan can only see in one direction

The direction of a scan decides what it can see. Ours could see everything pending and nothing shipped. If your release tooling only asks what is left to ship, add the mirror query: what already shipped that the tracker doesn’t know about. And when you write it, test the boundary first. Ask what else lands on the same branch that looks like the thing you are counting. For us it was the release merge itself, and we only learned that by shipping the wrong version first.