Three Repos, One Release, No Coordinator
A ten-minute review was supposed to be a ten-minute review. I opened a ticket to check on a staff-login follow-up, and it surfaced release-flow drift instead. Three hours and forty-eight minutes later, that staff-login ticket was still sitting untouched in the backlog, but a brand new one, opened along the way, had shipped a full multi-repo release system across all three repos and closed to staging.
Here’s the shape of the problem that ticket exposed. The product is split across three repos: the main web app, an API service, and a zones service that handles regional routing. A single feature routinely touches two of the three, sometimes all three. JIRA had no way to know which repos a ticket actually belonged to, and the release process leaned on people remembering. That’s fine until the day someone doesn’t remember, and a hotfix ships to one repo’s main branch without a matching merge back into develop.
The env grid that came first
Earlier the same day, a separate ticket had finished a 6-environments-by-3-repos isolation matrix: dev, staging, and three named-cell environments, plus production, each fully isolated per repo. That work took 35 hours and change across two days and fixed a bad service DACL and a couple of 2016-era relics along the way. The mental model that came out of it was a grid: rows are repos, columns are environments, and each cell is an independent, lockable unit.
Once that grid existed for infrastructure, the release problem looked like the same shape. Rows are repos, but now the columns are release stages, and each cell needs to know whether it’s clear to ship without checking with the other cells first.
Repo flags: automation, not discipline
The fix for “which repos does this ticket touch” turned out to be boring on purpose. JIRA labels: repo:app, repo:app-api, repo:app-zones. No schema change, no custom field admin work, fully JQL-filterable, and a ticket that spans two repos just carries two labels.
The part that matters is that the labels aren’t manually applied. The branch-cut command stamps the label from whichever repo it cuts the branch in, and the close command verifies the stamp is there before it’ll let a ticket transition. If you cut a branch in the API repo, the ticket gets repo:app-api whether you remember to add it or not. That’s the difference between a convention and a guard: a convention degrades the first time someone’s in a hurry, a guard doesn’t.
The wrong turn: scripting the JIRA admin UI
Before I landed on labels, I tried to add a native “Other” option to a JIRA custom field so repo could be a real dropdown instead of free-text labels. That meant driving the admin UI through Playwright, since JIRA’s field-option admin has no API for this. I grabbed the “Edit Options” link off the page and had the script click it:
page.goto("https://[internal-host]" + href)It blew up:
playwright._impl._errors.Error: Page.goto: net::ERR_NAME_NOT_RESOLVED athttps://[internal-host]editcustomfieldoptions!default.jspa/?atl_token=...&fieldConfigSchemeId=11802&fieldConfigId=11802&customFieldId=11202&returnUrl=...The href wasn’t a path, it was already a full query string missing its leading slash, so string-concatenating it onto the host produced garbage and the browser tried to resolve a hostname that was actually half a URL. The failure was useful, though: the stack trace leaked the real fieldConfigId (11802) and customFieldId (11202) I needed. I used those to build a direct URL and skip the click-through entirely. But by then I’d burned real time on the premise that a relative href behaves like a path, and the actual dropdown-option approach was still more fragile than free-text labels would ever be, since every teammate with admin rights could rename or delete an option and silently break the JQL filters downstream. Labels won by being harder to break, not by being elegant.
Cell-unit locking
With repo flags in place as data, the release side needed a way to stop two repos from stepping on each other mid-release. The local cell grid gives every repo-and-release-stage pair its own lock. A release for app-api can be in flight while app-zones sits untouched, and neither one blocks the other from cutting, testing, or shipping. What it does block is two people trying to advance the same cell at once, which is the actual failure mode multi-repo releases have: not “repos conflict with each other” but “two people race to close the same repo’s release.”
The close path got hardened to match. The close script now does sibling detection: if a ticket carries more than one repo: label, it has to reconcile all of them, not just the one the caller happened to be sitting in. Hotfix tickets get special handling, since a hotfix ships through a master-plus-tag flow instead of the normal feature-into-develop merge:
if category == 'hotfix': if not args.no_merge: args.no_merge = True feature_branch = f'hotfix/{key}'There’s a guard right after that, added after an earlier ticket where a hotfix got marked staging/live without the merge actually landing: it asserts that origin/master really contains the hotfix commits before the transition is allowed to fire. That guard exists because the tool used to trust the branch name over the git history, and once, back on an earlier hotfix ticket, that trust was wrong.
Develop-first, enforced by CI, not by memory
The last piece is the rule that makes all of this durable instead of a one-time cleanup: develop-first. Every repo hotfixes into develop before it touches main, and the back-merge from main into develop is never optional. That rule already existed as a written convention. What the release-system ticket added was a CI check on push to main in both repos that fails if main has commits missing from develop. The same day this shipped, that check caught real drift: develop had fallen behind main in both repos, ancestry-only, same content, from the env-grid ticket’s hotfix earlier in the week. It fast-forwarded clean, but it was drift the old system would never have flagged until it turned into a real conflict weeks later.
Three repos now ship independently, each with its own lock, its own label, and its own guard rail, and none of them have to ask a person which one they’re allowed to touch. The follow-up tickets from that session (a cross-repo guard gap, an organize-standard enforcement gap, three older tickets whose work this one had already absorbed) are the honest tally of what a three-hour tangent into “why am I reviewing this ticket again” actually cost, and what it bought back.