Operational Experience
Idempotency and Duplicate Events
Most webhook providers guarantee at-least-once delivery. When a provider can't confirm that a listener received an event — a timeout, a dropped connection, a 5xx — it retries. If the listener actually processed the first attempt but responded too slowly, it now receives the same event twice.
Providers say this plainly in their docs: Stripe endpoints "might occasionally receive the same event more than once", Okta delivery is "best-effort and at least once", and Twitch "sends you a notification at least once". The exceptions trade duplicates for lost events: Asana describes its delivery as at-most-once and recommends a fallback polling system for critical use cases.
An operation is idempotent when performing it once or many times has the same result. Consumers make webhook processing idempotent by recording which events they've handled and skipping repeats — and that's only practical when the provider sends a stable ID.
Unique event IDs
| Provider | Where the ID lives | Notes |
|---|---|---|
| GitHub | X-GitHub-Delivery header | Redeliveries reuse the original value |
| GitLab | webhook-id header (also Idempotency-Key) | Stays the same across retries and manual resends |
| Shopify | X-Shopify-Webhook-Id and X-Shopify-Event-Id headers | Webhook-Id is unique per subscription delivery; Event-Id is shared by every delivery triggered by the same merchant action |
| Stripe | id field in the payload | Track event IDs rather than the created timestamp |
| Twitch | Twitch-Eventsub-Message-Id header | Also part of the signed content |
| Okta | eventId field in each event | Compare against IDs already received |
| Standard Webhooks (OpenAI, GitLab, and others) | webhook-id header | The specification defines it as the idempotency key and includes it in the signature |
Deduplicating on the consumer
The simplest approach is a table (or cache) of processed IDs with a unique constraint:
app.post('/webhook', express.raw({ type: 'application/json' }), async (req, res) => {
if (!isValidSignature(req)) {
return res.status(401).send('Unauthorized')
}
const eventId = req.get('webhook-id')
// INSERT ... ON CONFLICT DO NOTHING returns 0 rows if we've seen this ID
const inserted = await db.query(
'INSERT INTO processed_events (id) VALUES ($1) ON CONFLICT DO NOTHING',
[eventId]
)
if (inserted.rowCount === 0) {
// Duplicate: acknowledge it so the provider stops retrying
return res.status(200).send()
}
await queue.publish(req.body)
res.status(202).send()
})
A few details matter:
- Acknowledge duplicates with a
2xx. Returning an error for a duplicate triggers more retries. - Record the ID together with the work. If you record the ID and then crash before processing, the event is lost; if you process and then fail to record the ID, it will be processed again. Recording both in the same transaction avoids either outcome.
- Keep IDs at least as long as the retry window. Stripe and PayPal retry for up to three days, OpenAI for 72 hours. See Retries and Timeouts.
Duplicates without duplicate IDs
Event IDs only catch redeliveries of the same event. Stripe notes that in some cases two separate events are generated for the same change, and recommends using the ID of the object in data.object along with the event type to identify them.
The most robust defense is making the handler itself idempotent: set an order's status to paid rather than incrementing a counter, and upsert records by the provider's object ID rather than inserting new ones. Fetching current state from the provider's API — the approach behind thin events — also makes repeated processing harmless.
Event IDs and replay prevention
Unique IDs also strengthen replay prevention. A signed timestamp limits how long a captured request is useful; remembering processed IDs rejects a replay even inside that window. GitHub recommends X-GitHub-Delivery for exactly this purpose, and Mailgun suggests caching the token sent with each webhook and rejecting reuse.
Recommendations for providers
- Send a unique ID with every event and keep it identical across retries and manual redeliveries
- Include the ID in the signed content so it can't be altered — as Standard Webhooks and Twitch do
- Document your delivery guarantee (at-most-once, at-least-once) and how long consumers should retain IDs