Squeeze the Quota Before Sunday
At 12:19 AM a failure note said the spend gate had deferred a model call. It was not a code bug. The gate was rationing a weekly quota that was about to reset.
The build log
Build log, architecture patterns, and observations from running autonomous AI systems in production.
At 12:19 AM a failure note said the spend gate had deferred a model call. It was not a code bug. The gate was rationing a weekly quota that was about to reset.
The alert fired at 6:46 AM. By 8:03 AM I was still staring at a blank brief that had refused to render, and the cron that generated it had already exited zero an hour earlier like nothing was wrong.
At 8:04 AM the brief fired into Discord five times in under two minutes, all five for the same ticket. That was the flap I sat down to fix.
GA4's viewsearchresults report was logging about 205,000 site-search events a day, and a chunk of them weren't searches at all.
At 12:19 AM the cron-fail triage agent read a failed dxdev run and answered in 9 seconds, 451 tokens: spendgate deferred the dxdev model call (band orange, discretionary), not a code bug.
The overnight work log for one night shows more than 50 headless claude -p sessions between 12:09 and 4:07 AM.
Six agent windows opened on top of the one I was typing in, and every one of them got its own taskbar button.
Four pipeline bugs were blocking the blog. After they were fixed, 217 blocked drafts dropped to 109 and 43 posts still failed two gates, because the problem was what they said.
The clone-cleanup job was supposed to be routine: 30 repos, fast-forward the stale base branches, delete dead leftover branches, close out the tickets sitting open against branches that no longer m...
984 of the 1,024 pages we imported from the old staff wiki had not been edited in five years, and the oldest was last touched in 2009. That number shaped the whole feature.
The real failures and fixes from building AI systems, one practical lesson per post. Get the next one in your inbox.