I put my own card through the live payment account, the first payment it had ever taken. Walking the client billing flow screen by screen surfaced four defects, and the test suite could not see any of them. All four are fixed and deployed.
Everything had been proven on localhost. I also built my partner’s 22-step walkthrough and moved it off a chat link and onto the internal tool itself, linked from the billing ticket.
The gap I caught before closing
The session ran 13h 27m. I caught the last gap: everything had been proven on localhost, and the live site still ran the payment processor’s test keys.
So the real-money payment said nothing about the deployed site’s configuration. “A payment succeeds against the live account” and “the site can take a payment” are separate claims. I filed tickets for the follow-ups.
The same night, a second time
Around the same time, I built a multi-tenant CRM from my partner’s plan, in an org of its own beside the main one, so my partner owns the asset. It had a full schema, row-level security and an adversarial isolation suite. I ran the suite against the live database project, and it passed 28 assertions. The suite also fails when one expectation is falsified, so it measures something.
Going live caught four defects a container could not show. One was a real leak: commission_balances ran with definer rights and would have handed a rep the whole org’s commission totals.
Proven on localhost is proven on localhost
A billing flow proven on localhost is a billing flow proven on localhost. The live site still runs test keys, so its key configuration needs its own check.
AI Skills
Use this lesson with the AI assistant you already use
I walked a client billing flow with a real card through the live payment account and found four defects that the test suite could not see. Just before closing the ticket, I noticed that the deployed site still ran the payment processor's test keys.
Paste the prompt, share only the context needed to answer it, and treat the result as a draft for your review. Do not include confidential information or let an AI assistant make changes without your approval.
Optional: for a visual report and saved memory, run /dxdev first.
Don’t have it? Get it at dxdev.com/skills/dxdev. The prompt works without it.
dxdev LESSON · paste into your AI coding agent
LESSON: A Passing Test Suite Only Vouches For The Environment It Ran In, So Verify Against The Live Target Too
SOURCE: dxdev.com/blog/2026-08-28_proven-on-localhost-production-still-test-keys
WHAT HAPPENED: The billing flow counted as built because it worked on localhost, but the suite could not say what a real card, a real account and real screens would do. Walking the flow with live money surfaced four defects, and all four were fixed and deployed. Before closing the ticket, I realized the real payment proved nothing about the deployed site, which was still configured with test credentials. The same night, an adversarial isolation suite for a multi-tenant CRM passed 28 assertions against the live database project, and it also failed when an expectation was falsified. Going live still caught four more defects, including a view running with definer rights that would have handed a rep the whole org's commission totals.
THE RULE: Evidence only covers the environment where it was produced, so a passing suite or a successful payment from one place says nothing about another. Check the deployed target's own configuration and behavior separately before calling work done.
CHECK MY CODE, then report PASS or FAIL with file:line for each:
1. For each claim of done, name the environment where the evidence was produced and confirm it is the same environment that ships, including its credentials and keys.
2. Before closing a release ticket, run at least one end-to-end pass with real accounts and real data against the deployed target, not only against localhost or a container.
3. Audit deployed configuration, such as payment keys and database view permissions, directly, and file a follow-up ticket for any gap instead of treating a passing suite as proof.
THEN PRINT: a table (check, PASS/FAIL, evidence, fix) + a verdict (applies / partially / OUT_OF_SCOPE / no) + the single most important next action.