Thirteen Hours to Learn Which Number Was Ours

Thirteen hours. That was how long I spent to learn that a payment receipt had been trying to find an order with the wrong number.

On the surface, it sounded impossible. Someone paid with a card. The payment service sent back the result. The receipt page should have been able to find the order and say thank you. Instead, it was getting a number that looked official and going nowhere with it.

I started where many people would start: on a computer at my desk. I tried to copy the message that the payment service was supposed to send and feed it into the page by hand. It did not settle the question. The service would only send the real message to a public web address, not to a computer sitting on my desk. Every pretend version was based on what I thought the message looked like. That was the very thing I needed to check.

So I put a small, temporary relay page into the live site. A payment callback is simply the message a payment service sends back after a card is processed. This relay page had one job: receive the real message and let me see what was actually in it.

That move was not elegant. It was, however, faster than spending another hour arguing with a made-up version of the facts.

The real message contained two numbers that were easy to confuse. One was the payment service’s transaction number, much like the number printed at the bottom of a store receipt. The other was the invoice number that had been attached to the order before the card was charged. The receipt page had been using the transaction number to search for the order.

That number was real. It just was not ours.

The order number was in the invoice field. The transaction number belonged in a separate place, as a record of what the payment service had done. The old rule tried one number and then the other. It had appeared to work in testing, but it failed when real payments arrived. Once I could see the actual message, the fix was plain: use the invoice number to find the order, and keep the payment service’s number as its own piece of information.

Seeing the real message also exposed a second problem. The order number from that public form was being placed directly into a request for the records system. A public payment address is still public. Anyone can send a form to it, not only the payment service. A stranger could type anything into the order-number box and send it along.

I added a basic guard. Before the number could be used, it had to be a real number greater than zero. It is the same sort of check you would make before writing a house number on an envelope. If the address is empty or made of letters, you do not put it in the mail and hope for the best.

There was one more quiet failure. The receipt was sometimes showing a blank merchant name. The page already had the account name available, but it was asking for it again from the wrong place. That extra request was wrapped in a safety net that swallowed the error, so the result was not a loud break. It was an empty line where a name should have been. I removed the unnecessary request and used the information already on the page, with sensible fallbacks if it was missing.

I also had one assumption left over from a different payment service. I had treated this system as if it needed a second conversation with the payment provider before the charge could count. It did not. By the time the message reached the receipt page, the payment service had already processed the card. The page needed to read the result, mark the order paid when the result said success, and keep the transaction number. Nothing more.

The work moved in seven small releases, from 08:06 in the morning to 20:52 that night. Each release had a label, like a bookmark in a long book. One changed which number found the order. Another fixed the name on the receipt. Another removed a broken detail from the page.

I could have gathered everything into one large change. That would have made it harder to tell which change caused trouble if something went wrong. Small releases gave me a known good place to return to after every step. They also made the temporary relay page less frightening. It was disposable scaffolding, and it left behind cleanup work, including backup and scratch copies. But no single release made the site impossible to undo.

The thirteen hours came down to one field swap: an invoice number replacing a transaction number in a single lookup, rolled out across seven small releases between 08:06 and 20:52. The receipt page now checks the right number before it checks anything else, and the relay page that proved it is gone, its job finished the moment the message showed its true shape.

dxdev LESSON · paste into your AI coding agent
LESSON: Stop guessing what a webhook sends. Capture the real payload.
SOURCE: dxdev.com/blog/2026-01-28_thirteen-hours-to-learn-which-number-was
WHAT HAPPENED: A payment receipt page couldn't locate the order it was supposed to confirm, because it searched using the payment gateway's own transaction number instead of the merchant invoice number the gateway actually returned in a separate field. Thirteen hours were spent before a temporary relay page, deployed to the live site for the sole purpose of capturing the gateway's real callback payload, revealed the field confusion directly. The fix also added input validation (must be a positive integer) on the order-number field, since a public callback endpoint can be hit by anything, not only the real gateway.
THE RULE: When a third-party callback's exact payload is in question, stop guessing from documentation or memory and capture what it actually sends in the real environment. Any identifier pulled from a public inbound endpoint needs validation before it's used to look up records.
CHECK MY CODE, then report PASS or FAIL with file:line for each:
  1. Any payment or webhook handler that assumes a specific field name for an order or transaction identifier without a comment citing the exact captured payload it was verified against.
  2. Any handler that takes an identifier from a public inbound request and uses it in a database lookup without validating its type or range first.
  3. Any place two similarly-named IDs (a gateway transaction ID vs a merchant invoice ID) could be swapped, with no test exercising both against production-shaped data.
THEN PRINT: a table (check, PASS/FAIL, evidence, fix) + a verdict (applies / partially / OUT_OF_SCOPE / no) + the single most important next action.