ENGINEERING NOTES
Idempotency and Webhooks: Duplicate-Safe Processing
Use scoped keys, authenticated webhooks and transaction boundaries to handle ambiguous delivery safely.
What problem does this solve?
A client times out after a payment or order write commits, then retries. A provider redelivers the same webhook. Without a duplicate policy, a normal delivery behavior can create a second business side effect.
What is it?
Idempotency makes repeating one logical operation produce a stable business outcome. A request key identifies intent; a webhook event identifier identifies delivery content. Neither replaces authentication or a database transaction.
Mental model
A claim ticket belongs to one operation. Reusing it should retrieve the same outcome, while using it for different input should be rejected. Receiving a webhook is like accepting a parcel: receipt and completion are separate milestones.
How it works internally
Claim a key atomically under a uniqueness constraint scoped to actor and operation. Store a normalized payload fingerprint and explicit processing state. Where possible, commit the business mutation and saved result together. Verify webhook signatures against the exact raw body before parsing; do not recreate the body from JSON. Deduplicate authenticated event IDs and distinguish retryable failures from permanent rejection.
Production example
For a payment webhook, authenticate the delivery, durably record its identifier, then apply the payment transition atomically or enqueue durable processing. An acknowledgement must match the chosen durability boundary. An external payment and a local SQL write cannot be committed in one local transaction.
Node.js / TypeScript implementation
A small HMAC-SHA256 verifier for providers using a hex digest. The raw body and header format must follow the provider contract. Freshness, replay controls and business-state deduplication are additional checks.
import { createHmac, timingSafeEqual } from "node:crypto";
function verify(raw: Buffer, signature: string, secret: string) {
if (!/^[a-f0-9]{64}$/i.test(signature)) return false;
const expected = createHmac("sha256", secret).update(raw).digest();
const received = Buffer.from(signature, "hex");
return received.length === expected.length
&& timingSafeEqual(received, expected);
}
// Supply the original request bytes; never log the secret or signature.Common mistakes
A read-then-insert check races under concurrency. A valid signature does not make an event unique or guarantee it is recent. Returning 200 before durable acceptance can lose work. Blind retries after ambiguous external side effects can double-charge or double-send.
Performance / reliability considerations
Define key retention, in-progress responses and replay behavior. Reconcile unknown payment outcomes using provider references. Test concurrent duplicates, changed payloads, post-commit timeouts and webhook retries. Avoid claiming exactly-once external delivery from a local deduplication table.
Interview questions
Why does a uniqueness constraint matter? How does an idempotency key differ from a webhook event ID? Why verify the raw body? When is it safe to acknowledge a webhook? How do you reconcile an unknown external result?
Quick Revision
Authenticate before trusting delivery. Scope and atomically claim keys. Reject key reuse with different input. Commit mutation and result together where possible. Verify HMAC over raw bytes and compare safely. Persist before acknowledging according to your contract. Expect duplicate delivery and reconcile ambiguous external outcomes.