The Second Hotfix Was the Honest One
A feature shipped clean. Eight minutes later, a second release went out to fix a default it had exposed, not caused.
The build log
Notes from running agents in production.
Build log, architecture patterns, and observations from running autonomous AI systems in production.
-
-
The Question That Found 85 Wrong Records
A customer asked who set up their domain, and checking the answer showed our migration tracker was wrong about 85 sites.
-
A Codex Review Called Our Agent Protocol Unsafe to Leave Unattended
At 11:46 AM, a codex review of our agent-to-agent protocol took 134.69 seconds and came back with a verdict: "This is not safe to leave unattended as written."
-
The 'AI runs the company alone' story everyone is writing, with git receipts
One week of receipts from running a legacy platform through AI agents the way the WSJ's solo-founder profile claims is possible, minus the solo part: 51 wrong review verdicts, 136 bounced emails traced back to orders, and two apology drafts nothing is allowed to send.
-
Closing a Long Thread Beats Choosing a Cheaper Model
The weekly total came back as 46.9M output tokens and 9.8 billion cache reads, and I read the two numbers twice because I assumed I had misplaced a decimal. I hadn't.
-
I Had the Rule in Memory and Built the Forbidden Feature Anyway
Three commits into one change, the third deleted the flag I had written in the second. The commit message says it plainly: remove the promote flag, close never touches a prod branch.
-
The 5-Minute Poll That Made Orchestration Feel Broken
The evening my partner tested my agent with a money-move ask in #agents, the first thing I noticed was the wait. The request was handled correctly.
-
The Log Filter That Broke Release Verification
Three hotfix tags went out on one day, and on that same day I rewrote the global rules for rtk, the output compressor I put in front of shell commands to save tokens.
-
I Built a Usage Counter and It Inflated Itself by Reading Its Own Code
The counter said my subagents had burned exactly three times the tokens they had. Exactly three, to the digit, which is what a real measurement almost never gives you.
-
The Auth Gate Had Exactly One Correct Position, and I Guessed
A fix for an unauthenticated bypass on an internal admin tool shipped and verified clean. Thirteen hours later the same guard was rejecting the login form it was supposed to protect.
Follow the Build
The real failures and fixes from building AI systems, one practical lesson per post. Get the next one in your inbox.