A webhook retry must not apply the order twice

A receiver can commit an order update and still fail to deliver its acknowledgement. The sender sees a failure and retries. That second delivery needs to find the original result without applying the order again. Integration Lab makes that sequence visible through a small, working application.

Start with the ambiguous failure

The usual outage example is straightforward: the receiver is unavailable, several attempts fail, and the delivery eventually needs intervention. A lost response is harder to reason about. The sender cannot tell from a timeout or failure response whether the receiver changed its data.

The lab includes a control called Lose response after committing. The receiver persists an actual receipt and order effect, then deliberately returns a synthetic 503. The failure is injected, but the database commit is real. Sending the event again exposes whether the implementation can recover safely.

Keep the receipt and business effect together

The receiver stores its receipt, the business effect, delivery status and audit transition in one optimistic SQLite update. A repeated delivery can return the stored success response. It does not need to apply the order a second time.

This is also why a successful HTTP status alone is weak evidence for a recovery demonstration. The useful checks are the stored receipt, the order ledger and the attempt history. Together, they show what the receiver committed and what the sender observed.

The public implementation uses a Cloudflare Worker and D1. A separate, runnable ASP.NET Core API uses SQLite locally. The two implementations share the same HTTP contract and acceptance scenarios. The public host does not run the .NET application.

Make recovery inspectable

The outage scenario sends four actual HTTP attempts to an unavailable receiver. An explicit virtual clock advances the retry schedule. Once the delivery becomes a dead letter, the operator can inspect its correlation ID, responses and history, restore the receiver and replay the event.

Earlier failed attempts remain visible after recovery. The operator can then queue a duplicate and check that the ledger still contains one order effect. Preserving that history matters: a final green status explains less than a sequence showing the original failure, corrective action and resulting state.

Check the boundaries of the claim

The recorded version 1 validation runs the same 25 HTTP scenarios against the local .NET API, local Worker and public custom domain. Each run records 216 requests, including its test operations. The coverage includes duplicates, conflicting bytes, concurrent operations, retry exhaustion and lost responses after a commit. Separate checks cover persistence after actual process restarts.

Those results describe bounded synthetic cases. They do not establish production throughput, availability or a distributed exactly-once guarantee. The browser initiates deliveries, the retry clock is virtual, and closing the tab stops further sends. Keeping those limits visible makes the demonstration easier to assess and reproduce.

Try the public Integration Lab, inspect the HTTP contract, or read the recorded validation and its limits.