samber/developer-platform-skills已通过检查
SKILL DETAIL
webhook-platform-design
samber/developer-platform-skills/webhook-platform-design
Design a provider-side outbound webhook platform - event taxonomy and catalog, payload envelope and schema versioning, an HMAC signing scheme (Standard Webhooks), at-least-once delivery with retry/backoff and dead-letter policy, subscription lifecycle states, and the developer-facing debugging surface (delivery logs, replay, test-event triggering). Use whenever the user mentions webhooks, event callbacks, push events to customer endpoints, webhook signatures, delivery retries, or a webhook consumer portal - even if they never say "webhook platform". Provider side only. Do NOT use for consumer-side retry mechanics - use samber/developer-platform-skills@api-idempotency-retry instead.
安装量 · 338查看来源
Installation
npx skills add https://github.com/samber/developer-platform-skills --skill webhook-platform-design
技能文件
SKILL.md
最近同步 · 2026年10月23日
evals/evals.json›
{
"skill_name": "webhook-platform-design",
"evals": [
{
"id": 1,
"prompt": "I'm the API lead at Ledgerwise, a B2B invoicing SaaS. We're publishing the reliability page for our new outbound events next month. The draft opens with: \"Ledgerwise guarantees exactly-once delivery of every event, in the order the events occurred.\" We have about 1,400 customer endpoints registered and peak around 300 events per second. Our CTO wants that sentence kept verbatim because it's the thing enterprise buyers ask about on every call. Write the page for me.",
"expected_output": "A reliability page that refuses the exactly-once and in-order claims, states at-least-once and best-effort ordering explicitly, and specifies a stable per-event ID plus a documented dedup recommendation, with FIFO offered only as an opt-in.",
"files": [],
"expectations": [
"Refuses the \"exactly-once delivery\" claim and does not reproduce it anywhere in the drafted page.",
"States at-least-once as the promised delivery guarantee, in explicit words on the page.",
"Explains that a subscriber which processes an event then crashes before returning 200 will receive that event again.",
"States that offerings marketed as exactly-once are at-least-once plus deduplication tooling.",
"Specifies a stable per-event ID that carries the same value across every retry of the same event.",
"States that the per-event ID must ship from day one, because retrofitting it later cannot help subscribers who already built without it.",
"Documents a deduplication recommendation instructing subscribers to key on that per-event ID.",
"States best-effort, non-guaranteed ordering as the position on the page instead of promising ordering.",
"Gives retries and parallel delivery workers as the reason ordering cannot be guaranteed.",
"Warns that strict ordering makes one failing delivery block the entire per-endpoint queue.",
"Offers FIFO only as an explicit opt-in for subscribers who structurally cannot tolerate reordering, never as the platform default.",
"Warns subscribers not to infer event order from event timestamps."
]
},
{
"id": 2,
"prompt": "Security review pushed back on webhook signing at Portmint (payments infrastructure, roughly 600 merchant endpoints). What we ship today: HMAC-SHA256 over the JSON request body using a per-endpoint secret, hex-encoded, sent as X-Portmint-Signature. One secret per endpoint. To rotate, the merchant clicks \"regenerate\" in the dashboard and the old secret stops working at that moment. Our docs tell merchants to verify inside their framework's JSON body handler. Tell me what to change and why.",
"expected_output": "A rewrite onto Standard Webhooks with id/timestamp/body signing for replay protection, overlapping multi-signature rotation, prefixed secrets, an asymmetric option, and the raw-body and middleware pitfalls added to the docs.",
"files": [],
"expectations": [
"Identifies that signing the body alone gives no replay protection, so a captured legitimate delivery can be replayed indefinitely.",
"Recommends adopting the Standard Webhooks spec rather than keeping a bespoke scheme.",
"Specifies the three delivery headers webhook-id, webhook-timestamp, and webhook-signature.",
"Gives the signed content as {id}.{timestamp}.{raw_body}, with literal dot separators.",
"Specifies base64 encoding of the HMAC-SHA256 output rather than the current hex encoding.",
"Requires the receiver to reject deliveries whose timestamp falls outside a tolerance window, and names roughly 5 minutes (300 seconds).",
"Identifies the single-active-secret design as forcing revoke-then-deploy rotation with a window of failed deliveries.",
"Specifies that the signature header carries multiple space-delimited signatures so the old and new secrets validate simultaneously during a rotation window.",
"Gives an ordered rotation flow in which the old secret is revoked only after delivery success under the new secret is confirmed.",
"Recommends prefixing the symmetric secret with whsec_ so a leaked key is identifiable to secret-scanning tools.",
"Offers asymmetric ed25519 signing as an additional mode, naming the whsk_/whpk_ key prefixes or the v1a signature identifier, for consumers who should not hold a shared secret.",
"Flags that verifying inside the framework's JSON body handler breaks the HMAC, and requires verification against the raw body before any parser touches it.",
"Tells the docs to instruct consumers to exempt the webhook route from session-auth middleware, since the route authenticates by signature.",
"Tells the docs to require signature verification on every request without exception, including notification-only handlers."
]
},
{
"id": 3,
"prompt": "I'm rewriting the retry engine at Freightloop (logistics API, about 9,000 carrier endpoints registered). Plan currently on the table: copy Shopify's policy of 19 attempts over 48 hours, but use a flat 45-second interval between attempts so the behavior is predictable and easy to explain in docs. Retry anything that isn't a 2xx until the budget runs out. Endpoints that stay broken keep cycling forever, because silently turning off a paying customer's endpoint would be rude. Sanity-check this before I build it.",
"expected_output": "A correction that flags the stale Shopify figure, replaces the flat interval with exponential backoff over a roughly day-long span, routes non-retriable 4xx straight to dead-letter, honors Retry-After, and adds auto-disable with out-of-band notification.",
"files": [],
"expectations": [
"Flags \"19 attempts over 48 hours\" as Shopify's superseded pre-September-2024 policy and warns that sources repeating it are stale.",
"Gives Shopify's current documented policy as 8 retries over roughly 4 hours.",
"Rejects the flat 45-second interval and requires exponential backoff instead.",
"Explains that constant-interval retries hammer a struggling endpoint and worsen an active rate-limit response.",
"Recommends a fixed exponential-backoff schedule spanning roughly a day as the default.",
"Cites at least one documented vendor schedule as the reference for that span, such as Svix's 8 attempts over about 27.5 hours or Stripe's up to 3 days.",
"Says to size the total retry span against realistic subscriber incident durations, contrasting a deploy taking minutes against an outage taking hours, rather than picking a round number.",
"Rejects retrying non-retriable 4xx responses and routes them straight to the dead-letter queue.",
"Names 408 and 429 as the 4xx status codes that are the exception and do get retried.",
"Says to honor a subscriber's Retry-After header on a 429 in preference to the platform's own backoff curve.",
"Rejects the never-disable plan and requires auto-disabling an endpoint after sustained failure.",
"Requires notifying the subscriber out-of-band, by email or dashboard, because the webhook channel itself is what is broken.",
"Gives dead-lettering exactly two triggers: the retry budget being exhausted, and a non-retriable error class."
]
},
{
"id": 4,
"prompt": "Kestrelpay, a German payroll platform. The board wants webhooks live for our Q1 partner launch, which is a hard date about seven weeks out. Legal's written position is that customer payroll event data may not transit a third-party processor at all, full stop, and they aren't reopening it this quarter. We have 6 backend engineers and no dedicated on-call rotation today. Give me the build-versus-buy call and the reasoning behind it.",
"expected_output": "A ranked build-vs-buy answer that removes managed sending infrastructure from the menu entirely on the compliance ground, names the re-promotion trigger, recommends a self-hosted gateway, and ties the hard date into the ranking.",
"files": [],
"expectations": [
"Deletes managed sending infrastructure from the option menu entirely rather than ranking it last.",
"Gives the reason for deleting rather than demoting: an option parked at the bottom on a compliance ground reappears as scope the next time a deadline bites, without the legal review that ruled it out.",
"Names the re-promotion trigger that would put the managed option back on the menu: a signed DPA plus a data-residency posture covering the vendor.",
"Recommends a self-hosted open-source webhook gateway as the choice under this constraint.",
"States that self-hosted gateway and in-house build tie on control and on compliance cost, because both keep event data on the platform's own infrastructure.",
"Ranks in-house build as the highest-effort option and names what would promote it, such as webhook delivery being the product sold or third-party code also being ruled out.",
"Cites the order-of-magnitude in-house estimate as engineer-years rather than engineer-weeks, on the order of 3-5 engineers for 6-12 months plus a standing on-call.",
"Flags that engineer-years figure as a vendor's own self-interested estimate while accepting its order of magnitude.",
"Ties the seven-week hard date explicitly into the ranking, promoting the faster-to-effect option.",
"States that buying or self-hosting the delivery machinery does not buy the event taxonomy, the delivery contract, or the documentation, all of which still have to be designed.",
"States that the ranking is a default to be re-ranked against the team's actual situation, not a law."
]
},
{
"id": 5,
"prompt": "Northwind Ledger, accounting SaaS. We're designing the event catalog now. The plan: send X-Northwind-Event: invoice as a header with {\"action\": \"paid\", ...} in the body, the same shape GitHub uses, since half the team already knows it from working with GitHub Apps. Names would be things like invoice, customer_update, and payrunExecute. Separate question: when a line item on an invoice changes, should we fire the line-item event and the invoice event and the customer event? Firing all three feels safer.",
"expected_output": "A taxonomy correction onto dot-delimited past-tense resource.action names, a rejection of the header/body name split, and an explicit documented decision on subresource bubbling rather than firing everything.",
"files": [],
"expectations": [
"Rejects splitting the event name across an HTTP header and a body field, on the grounds that no single string then names the event.",
"Prescribes the Object-Action resource.action pattern, dot-delimited.",
"Requires the action segment to be past tense, and corrects at least one proposed name accordingly, such as invoice.paid, customer.updated, or payrun.executed.",
"Rejects the underscore-joined and verb-first camelCase name forms in the proposal.",
"Restricts event identifiers to the [a-zA-Z0-9_.] character set.",
"Attributes the dot-delimited past-tense resource.action convention to Standard Webhooks and/or Stripe.",
"Says to pick one delimiter convention at launch and never mix conventions afterward.",
"Answers the bubbling question by requiring an explicit documented decision rather than defaulting to firing all three events.",
"Cites Stripe's position that subresource events never bubble to parent events, for example that customer.subscription.updated fires no customer.updated.",
"States that leaving the bubbling rule unstated breeds subscriber assumptions.",
"Requires every event type to have a stable name, a description, and a payload schema before it ships.",
"Checks events against the five design principles by name: self-contained, immutable, traceable, actionable, debuggable."
]
},
{
"id": 6,
"prompt": "Atlas Freight here. We date-version our REST API and we're cutting 2026-04-01, which reshapes the shipment object: carrier goes from a plain string to a nested object. Our webhook payloads embed the full shipment, and the delivery service just renders whatever the current API version returns. A lot of customers write our payloads straight into their own tables. Our metadata (event type, timestamp, delivery id) rides in HTTP headers, and our delivery log stores the request body. Tell me what breaks.",
"expected_output": "An analysis naming the silent-reshape failure, forcing a deliberate choice between the two schema-versioning camps, fixing the header-blind delivery log, and raising thin payloads with a dual-run migration pattern.",
"files": [],
"expectations": [
"Identifies an API version bump silently reshaping event payloads that existing subscribers already store as the core failure.",
"Requires choosing exactly one schema-versioning camp deliberately, and warns against drifting into both by accident.",
"Describes the first camp as pinning each webhook endpoint to an API version and rendering payloads per that version, attributed to Stripe.",
"Describes the second camp as versioning the event type itself, with a new type name per breaking change and additive-only evolution otherwise, attributed to Svix or Standard Webhooks.",
"Identifies that the header-split envelope means the delivery log must capture headers as well as the body, or replay is unfaithful.",
"Says the delivery log must be built to store headers from day one rather than retrofitted.",
"Raises thin payloads as a first-class alternative and states that a thin payload never changes shape when the API version bumps.",
"States that thin payloads move the versioning question to the API call the consumer subsequently makes.",
"Cites the recommendation to keep payloads small, roughly under 20kb.",
"Offers the dual-run migration pattern for a full-to-thin move: emit both event flavors during the transition with a discriminator field so consumers can dual-run idempotently.",
"Recommends documenting the outbound events in the same OpenAPI document as the inbound API, using OpenAPI 3.1's webhooks section.",
"Declines to recommend adopting CloudEvents unless the platform must interoperate with an event mesh that already speaks it."
]
},
{
"id": 7,
"prompt": "Clearbend, an HR platform. Endpoint registration today is a single POST that stores a URL. No update, no read, and the only mutable field is enabled: true|false. Support has been telling customers to delete the subscription and create it again whenever a backlog gets stuck, and honestly it seems to work fine. Two enterprise accounts are now asking what happens to their events during a planned maintenance window on their side. What should the subscription object actually look like?",
"expected_output": "A first-class subscription object with active/paused/disabled states, the maintenance window answered by pause, an explicit warning on delete-and-recreate destroying replay state, both replay positions specified, and independent filtering axes.",
"files": [],
"expectations": [
"Models the subscription as a first-class stateful object with full CRUD rather than fire-and-forget registration.",
"Specifies three explicit states: active, paused, and disabled.",
"Defines paused as subscriber-initiated and disabled as platform-initiated after sustained failure.",
"Answers the maintenance-window question with the paused state, not with delete-and-recreate and not with the existing enabled: false flag.",
"Flags delete-and-recreate as destroying the subscription's replay and delivery-tracking state.",
"Warns that recreating a subscription under the same name restarts from the default position, so the subscriber silently loses or double-replays a backlog.",
"Specifies two independently configured replay positions: where a newly created subscription starts, and where a subscription resumes after a platform-side incident.",
"States that those two replay positions can legitimately differ, and that both must be specified.",
"Says to expose only the states a customer-created subscription may legally reach, rejecting internal-only states at the API boundary.",
"Keeps the two filtering axes independent: event type selected against the catalog, and recipient or channel scoping.",
"States that an endpoint with no filter configured receives everything, and that this must be documented explicitly.",
"Documents config-change propagation lag, a subscription edit taking a minute or two to reach the delivery path, as expected behavior rather than a bug."
]
},
{
"id": 8,
"prompt": "Tallyroom, scheduling SaaS. Roughly 40% of our webhook support tickets come from people wiring us into no-code automation tools, and most of them open with \"what does the payload actually look like\" before they've built anything at all. Our plan is a genuinely good docs page with example payloads and a full signing spec, plus a Grafana board for our own on-call to watch delivery health. Engineering has customer-facing UI down as a phase-2 nice-to-have. Separately, the team keeps arguing about whether the resend button should say retry or replay. Sort this out for me.",
"expected_output": "A rejection of portal-as-phase-2, a schema-driven test-event feature aimed at the no-code catch-URL mental model, delivery logs with per-attempt inspection, and retry and replay defined as two distinct operations.",
"files": [],
"expectations": [
"Rejects treating a customer-facing consumer portal as a phase-2 nice-to-have, and states that the portal is table stakes in this market.",
"Specifies a test-event or \"send example event\" action driven by each event type's schema, so an integrator sees one realistic payload before writing handler code.",
"Connects that test-event feature to the catch-URL mental model no-code consumers arrive with from workflow-automation tools.",
"Specifies delivery logs filterable by endpoint, event type, and status.",
"Specifies per-attempt inspection covering request headers, request body, response code, and timing.",
"Resolves the naming argument by keeping retry and replay as two distinct operations rather than choosing one word for a single button.",
"Defines retry as a new delivery attempt of the same event.",
"Defines replay as re-ingesting the original as a new event.",
"Offers both single-event and range replay, gating a large replay window behind an explicit cost confirmation.",
"Surfaces exhausted deliveries as incidents grouped by endpoint and status code with notification, rather than as rows in a queue behind a bare replay button.",
"Documents the 200-then-crash blackhole: a handler that acknowledges then throws has reported success, and no debugging surface can see inside it.",
"States that a design good enough for the most sophisticated consumer segment still fails no-code consumers, because the consumer segments do not nest."
]
},
{
"id": 9,
"prompt": "Ship date is nine days out for Vantle's events product and I need a go/no-go. The docs page as written has: (1) signature-verification snippets in Node and Python, (2) the full event catalog with schemas and example payloads, (3) a Testing section covering our CLI, and (4) a troubleshooting and failure-recovery page. Our reliability wording is one sentence: \"We retry failed deliveries with exponential backoff.\" Everything's reviewed and the writer is happy with it. Call it.",
"expected_output": "A no-go that grades the page against the five-element docs rubric, fails the retry element for vagueness, and separately fails the contract-completeness gate on the four missing contract facts.",
"files": [],
"expectations": [
"Returns a no-go rather than approving the launch.",
"Grades the docs against a five-element rubric and identifies the retry-policy element as the failing one.",
"States that four of five rubric elements is a fail, not a pass.",
"Rejects \"we retry with exponential backoff\" as insufficient and requires the exact published schedule: attempt count and intervals.",
"Applies a second, separate gate on contract completeness covering five specific facts.",
"Names the delivery guarantee, at-least-once, as a required contract fact and flags it missing from the page.",
"Names the ordering position as a required contract fact and flags it missing from the page.",
"Names the ack timeout as a required contract fact and flags it missing from the page.",
"Names which header or field carries the dedup ID as a required contract fact and flags it missing from the page.",
"States that each contract fact left implicit becomes a subscriber assumption and then a support ticket.",
"Gives a concrete ack timeout to publish, around 10 seconds, citing the Shopify and GitHub convention.",
"Tells subscribers to queue the work and return fast, and states that a slow-but-correct handler must not be treated as a failed delivery.",
"Recommends that consumers of high-stakes event types run periodic reconciliation against the API, framed as endorsed practice rather than a workaround for a defect."
]
}
],
"trigger_queries": [
{
"query": "We're adding webhooks to our billing API - design the event catalog, signing, and retry policy from scratch.",
"should_trigger": true
},
{
"query": "How should we sign the payloads we POST to our customers' endpoints?",
"should_trigger": true
},
{
"query": "Design the outbound event notification system we push to partner servers.",
"should_trigger": true
},
{
"query": "Customers keep telling us they missed events. What should our redelivery policy look like?",
"should_trigger": true
},
{
"query": "What headers should we send on every callback so receivers can verify it really came from us?",
"should_trigger": true
},
{
"query": "We need a delivery log and a replay button in the customer dashboard.",
"should_trigger": true
},
{
"query": "Write the delivery contract we can hand to enterprise buyers in security review.",
"should_trigger": true
},
{
"query": "Our event names are a mess - half past tense, half not, two different delimiters. Fix the taxonomy.",
"should_trigger": true
},
{
"query": "Should we guarantee ordering when we push events to subscriber URLs?",
"should_trigger": true
},
{
"query": "How long should we keep retrying a customer endpoint that's been down all morning?",
"should_trigger": true
},
{
"query": "We want customers to register a URL and pick which events they receive. Design that.",
"should_trigger": true
},
{
"query": "Plan the dead-letter queue for events we couldn't deliver.",
"should_trigger": true
},
{
"query": "our hmac is over the body only, is that enough",
"should_trigger": true
},
{
"query": "build the webhook consumer portal",
"should_trigger": true
},
{
"query": "Should the payload carry the whole object or just an ID the consumer fetches?",
"should_trigger": true
},
{
"query": "Design event callbacks for our API so customers stop polling us every 30 seconds.",
"should_trigger": true
},
{
"query": "How do we rotate the signing secret without breaking every live integration?",
"should_trigger": true
},
{
"query": "we push events to customer servers and we need to stop them double-processing",
"should_trigger": true
},
{
"query": "Our API version is changing - what happens to the event payloads customers already stored?",
"should_trigger": true
},
{
"query": "We need a \"send me a test event\" button for integrators.",
"should_trigger": true
},
{
"query": "Set the backoff schedule for failed HTTP callbacks to subscriber endpoints.",
"should_trigger": true
},
{
"query": "When should we auto-disable a subscriber endpoint that keeps failing?",
"should_trigger": true
},
{
"query": "help me spec out the event types our platform emits",
"should_trigger": true
},
{
"query": "Should we adopt Standard Webhooks or roll our own?",
"should_trigger": true
},
{
"query": "Customers want to stop receiving events during their maintenance window without losing their config.",
"should_trigger": true
},
{
"query": "What's the right acknowledgement timeout to publish for receivers?",
"should_trigger": true
},
{
"query": "our pushes to customer endpoints fail silently and nobody finds out for days",
"should_trigger": true
},
{
"query": "Draft the documentation page for our event delivery system.",
"should_trigger": true
},
{
"query": "Design how third parties subscribe to state changes happening inside our product.",
"should_trigger": true
},
{
"query": "We need to let a subscriber re-send everything from last Tuesday.",
"should_trigger": true
},
{
"query": "Should we build the event-sending pipeline ourselves or pay a sending service for it?",
"should_trigger": true
},
{
"query": "How do we stop one slow subscriber from delaying deliveries to everyone else?",
"should_trigger": true
},
{
"query": "Architect the push channel from our SaaS to customers' backend servers.",
"should_trigger": true
},
{
"query": "Our integrators complain they can't see why a delivery failed. What do we build?",
"should_trigger": true
},
{
"query": "what should we name our events, order.created or orders/create?",
"should_trigger": true
},
{
"query": "Add a unique ID to every event so receivers can deduplicate safely.",
"should_trigger": true
},
{
"query": "Plan replay and backfill for the deliveries we missed during the incident.",
"should_trigger": true
},
{
"query": "A customer endpoint has been 500ing for two days and we're still hammering it. What's the right policy?",
"should_trigger": true
},
{
"query": "How do I make sure a captured request can't be replayed against my customers later?",
"should_trigger": true
},
{
"query": "design the event subscription API for our platform",
"should_trigger": true
},
{
"query": "Should the event metadata go in HTTP headers or inside the JSON body?",
"should_trigger": true
},
{
"query": "Our callbacks time out because receivers do heavy work inline. What do we tell them in the docs?",
"should_trigger": true
},
{
"query": "write the reliability story for the events we push out",
"should_trigger": true
},
{
"query": "Draft signature-verification snippets for our integration docs in Node and Go.",
"should_trigger": true
},
{
"query": "We're the provider and our customers' servers need to hear about changes in near real time. Design it.",
"should_trigger": true
},
{
"query": "Pick a retention window for undeliverable events and say what we keep with them.",
"should_trigger": true
},
{
"query": "What should a 429 from a subscriber do to our backoff curve?",
"should_trigger": true
},
{
"query": "our event push has zero observability and customers open tickets instead of self-serving",
"should_trigger": true
},
{
"query": "Design filtering so an endpoint only receives the event types it asked for.",
"should_trigger": true
},
{
"query": "Our enterprise prospects are asking whether events arrive in order and how they're authenticated. Write the answer.",
"should_trigger": true
},
{
"query": "How should our SDK retry when our own API returns a 503 - what backoff and jitter?",
"should_trigger": false
},
{
"query": "Design Idempotency-Key support on our POST /payments endpoint.",
"should_trigger": false
},
{
"query": "Customers are double-submitting checkout requests and getting charged twice. Fix it.",
"should_trigger": false
},
{
"query": "How long should we store idempotency keys before they expire?",
"should_trigger": false
},
{
"query": "Design our API key format - prefix, checksum, and how we store it at rest.",
"should_trigger": false
},
{
"query": "An engineer left and we need to rotate every customer API key. What's the process?",
"should_trigger": false
},
{
"query": "Should API keys be hashed or encrypted so customers can view them again later?",
"should_trigger": false
},
{
"query": "Design the OAuth consent screen and scope taxonomy for third-party apps.",
"should_trigger": false
},
{
"query": "Do confidential OAuth clients still need PKCE under OAuth 2.1?",
"should_trigger": false
},
{
"query": "What rate-limit tiers should we publish for free, pro, and enterprise?",
"should_trigger": false
},
{
"query": "Which RateLimit headers should our 429 responses carry, legacy or IETF?",
"should_trigger": false
},
{
"query": "Build an error-code taxonomy for our API using RFC 9457 problem details.",
"should_trigger": false
},
{
"query": "We need a public status page with component-level incident updates.",
"should_trigger": false
},
{
"query": "Write the public postmortem for yesterday's 40-minute API outage.",
"should_trigger": false
},
{
"query": "How much notice before we sunset /v1, and which headers announce the deprecation?",
"should_trigger": false
},
{
"query": "Design our sandbox with magic test card numbers and seeded personas.",
"should_trigger": false
},
{
"query": "How do we keep test-mode data from ever touching live records?",
"should_trigger": false
},
{
"query": "Design GraphQL subscriptions so our dashboard updates live in the browser.",
"should_trigger": false
},
{
"query": "Should our public schema use Relay cursor connections for pagination?",
"should_trigger": false
},
{
"query": "Build a fleet view showing which partner integrations are erroring the most right now.",
"should_trigger": false
},
{
"query": "We need correlation IDs that survive from a response header into a support ticket.",
"should_trigger": false
},
{
"query": "Expose our product to coding agents as an MCP server.",
"should_trigger": false
},
{
"query": "Set up nightly Parquet drops into each customer's own S3 bucket.",
"should_trigger": false
},
{
"query": "Give enterprise customers a JDBC endpoint so they can point Tableau at their data.",
"should_trigger": false
},
{
"query": "Get us listed as a source connector on the major ELT platforms.",
"should_trigger": false
},
{
"query": "Should we build our own app marketplace or just join everyone else's?",
"should_trigger": false
},
{
"query": "Design the review pipeline and permission audit for third-party app submissions.",
"should_trigger": false
},
{
"query": "Design the information architecture of our developer portal.",
"should_trigger": false
},
{
"query": "Audit our OpenAPI reference for undocumented endpoints and missing error codes.",
"should_trigger": false
},
{
"query": "Which language SDKs should we ship first, and which stay community-supported?",
"should_trigger": false
},
{
"query": "Should we publish gRPC at the public edge or transcode it to JSON?",
"should_trigger": false
},
{
"query": "Review our REST resource naming and pagination style for consistency.",
"should_trigger": false
},
{
"query": "We have REST today - should GraphQL, bulk export, or an MCP layer come next on the roadmap?",
"should_trigger": false
},
{
"query": "Write a Stripe webhook handler in Express that verifies the signature correctly.",
"should_trigger": false
},
{
"query": "My Shopify webhook endpoint returns 401 because of my auth middleware - how do I fix my app?",
"should_trigger": false
},
{
"query": "How do I receive GitHub webhooks on localhost while developing?",
"should_trigger": false
},
{
"query": "Set up Clerk webhooks in my Next.js project.",
"should_trigger": false
},
{
"query": "Configure an incoming webhook to post our build results into a Slack channel.",
"should_trigger": false
},
{
"query": "Design our internal Kafka topic naming scheme and schema registry.",
"should_trigger": false
},
{
"query": "Should we use event sourcing for the orders service?",
"should_trigger": false
},
{
"query": "Set up Inngest to run our background jobs.",
"should_trigger": false
},
{
"query": "Build a CDC pipeline from Postgres into our warehouse.",
"should_trigger": false
},
{
"query": "Plan the launch of our developer community Discord.",
"should_trigger": false
},
{
"query": "How do we hire an API product manager?",
"should_trigger": false
},
{
"query": "Write the changelog entry for this week's API release.",
"should_trigger": false
},
{
"query": "Design the email notification preferences page for our end users.",
"should_trigger": false
},
{
"query": "Our cron job that polls a vendor's API keeps missing records - make it reliable.",
"should_trigger": false
},
{
"query": "Add server-sent events to our web app so the UI updates without refresh.",
"should_trigger": false
},
{
"query": "Design the partner developer onboarding journey up to their first submitted app.",
"should_trigger": false
},
{
"query": "Should we charge for apps in our marketplace, and how do developer payouts work?",
"should_trigger": false
}
]
}
references/event-taxonomy-and-payload-design.md›
# Event taxonomy and payload design
Vendor comparisons and worked detail behind SKILL.md steps 1-2.
## Naming conventions in production
| Platform | Convention | Example | Note |
| ----------------- | -------------------------------------------------------- | --------------------------------------------------- | ------------------------------------------------------------------ |
| Stripe | `resource.action`, dot-delimited | `charge.succeeded`, `customer.subscription.updated` | No domain prefix; subresource events never bubble to parent events |
| Shopify | `resource/action`, slash-delimited | `orders/create`, `app/uninstalled` | |
| GitHub | Split: header + body field | `X-GitHub-Event: issues` + `"action": "opened"` | No single string names the full event - the shape to avoid |
| Standard Webhooks | `resource.action`, identifiers `[a-zA-Z0-9_]` | `invoice.paid` | Closest to Stripe's convention; the recommended default |
| Svix | First-class event-type objects with attached JSON Schema | - | Auto-generates the Event Catalog from them |
Recommended: Object-Action, past tense (`invoice.paid`, not `invoice.pay`), dot notation for hierarchy, optional domain prefix for grouping. Reject underscore-joined or verb-first names (`user_update`, `invoiceWasPaid`) - inconsistent and harder to filter and route on.
## Event design principles (checklist per event type)
1. **Self-contained** - include what a consumer needs; don't force a callback to the API just for context (thin payloads are a deliberate exception, chosen platform-wide, not per event).
2. **Immutable** - never modify a schema after events of that type have shipped; add fields, never repurpose them. Stripe goes further: an Event object, once created, never changes - a later update to the resource produces a new Event.
3. **Traceable** - correlation IDs, audit-trail fields.
4. **Actionable** - enough context for the consumer's business logic, not just an ID.
5. **Debuggable** - source, environment, originating trigger in metadata.
## Minimal envelope
Required: `type` (the catalog name) and `data` (the payload). Valuable: a unique event `id` (the dedup key), a timestamp, and a schema-version marker. Where each of these lives is the envelope-placement fork below.
## Envelope placement: body-only vs header-split
- **Body-only (Stripe)**: everything - `id`, `type`, `api_version`, `created`, `data` - lives in one top-level JSON object. The full event is self-contained in a single payload dump for logging.
- **Header-split (GitHub, Shopify, Standard Webhooks/Svix)**: metadata rides HTTP headers (`webhook-id`, `webhook-timestamp`, or vendor equivalents); the body carries `{type, timestamp, data}` or just the resource. CloudEvents' "binary" content mode standardizes the same split via `ce-*` headers; its "structured" mode is the all-in-body alternative.
The consequential difference: a header-split design means your delivery log and replay feature must capture headers _and_ body to reproduce an event faithfully. Decide the fork before building the log, not after.
## Thin vs full payloads
- **Full (snapshot)**: embed the resource state as of the event. Self-contained, but the payload shape is coupled to your API version.
- **Thin**: identifiers only; the consumer fetches the resource. Smaller (Standard Webhooks recommends staying under ~20kb), and the notification never changes shape when the API version bumps - the version question moves to the API call the consumer makes.
Migration pattern (Stripe's snapshot-to-thin transition): emit both event flavors together during the transition and stamp a discriminator field on the payload so consumers can dual-run idempotently. Reusable for any full-to-thin move made after launch.
## Schema versioning: the two camps
1. **Pin the consumer to an API version** (Stripe): each webhook endpoint is pinned to an account API version and payloads render per that version. Upgrading the account version changes what existing webhooks receive - which is exactly the hazard to manage. Retrieving an old event under a newer version does not retroactively reshape it.
2. **Version the event type** (Svix / Standard Webhooks): a new event-type name per breaking change (`v1.customer.created` → `v2.customer.created`), additive-only evolution otherwise. The type name is the version marker; no out-of-band version applies.
Both solve the same problem. Pick one deliberately - drifting into both by accident, or letting an API version bump silently reshape stored payloads, is the failure mode. GitHub and Shopify version payload shape implicitly with the overall (date-versioned) API - the ambient default a deliberate choice replaces.
## Standards map (layers, not competitors)
| Standard | Layer | What it gives you |
| ---------------------------- | -------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- |
| Standard Webhooks | Signing + delivery headers | Interoperable signature scheme; near-zero adoption cost |
| CloudEvents (CNCF-graduated) | Envelope | `id`/`source`/`type`/`specversion` attributes; HTTP/Kafka/AMQP/MQTT bindings; low real-world adoption for consumer-facing webhooks despite graduation |
| AsyncAPI | Description language | Documents channels and message schemas; never on the wire; can describe WebSocket channels too |
| OpenAPI 3.1 `webhooks` | Description language | Outbound events documented in the _same_ spec as the inbound API - the recommended documentation home |
| WebSub (W3C) | Pub/sub protocol | Content-syndication oriented; rarely relevant to transactional B2B webhooks |
No ratified W3C/IETF transactional-webhook standard exists. Practical layering for a new platform: Standard Webhooks for signing/delivery, whichever envelope placement fits your API conventions, OpenAPI 3.1 for description - and full CloudEvents only to interoperate with an event mesh that already speaks it.
references/retry-policy-vendor-comparison.md›
# Retry policy: vendor comparison
Documented schedules behind SKILL.md step 4's ranked menu. Retry policy is the single sharpest divergence axis between real webhook platforms - the delivery guarantee, signing algorithm, and ordering stance all converge; this doesn't.
## Documented schedules, six vendors
| Vendor | Automatic retries | Span | Timeout | After exhaustion |
| -------- | ------------------------------------------------------------------------------------ | -------------------------------------------------- | ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Stripe | Exponential backoff (live mode) | Up to 3 days (test mode: 3 tries over a few hours) | - | Endpoint auto-disabled after ~3 days of failure, with notification |
| GitHub | **None** | - | 2xx expected within 10s | Manual: UI redeliver (3-day window) or REST Deliveries API (30-day listing) |
| Shopify | 8 retries, exponential backoff | 4 hours (policy current since Sept 2024) | 5s response + 1s connect | Subscription auto-**deleted** after 8 consecutive failures; warning email to the app's emergency address. Declarative (TOML) subscriptions restore on next deploy; Admin-API-created ones do not |
| Svix | 8 attempts, exact schedule: immediately, +5s, +5m, +30m, +2h, +5h, +10h, +10h | ~27.5 hours | ~15s | DLQ; consistently failing endpoints disabled with notification |
| Hookdeck | Configurable: linear or exponential, up to 50 automatic retries | Configurable | - | Exhausted deliveries surface as "Issues" (grouped incidents), unlimited manual retries |
| Convoy | Two algorithms: constant-time or exponential-with-jitter; per-endpoint rate limiting | Configurable | Configurable | Batch retry for consecutively failing endpoints; true circuit breaker (below) |
Staleness warnings baked into the table: Shopify's widely-cited "19 times over 48 hours" is the pre-Sept-2024 policy - any source repeating it is stale. Stripe's manual Dashboard resend does not cancel the still-running automatic schedule; the two run independently.
## Policy mechanics
- **Backoff formula**: `delay = base_delay * (2 ^ attempt_number)`, with jitter. Linear intervals are actively harmful - they hammer a down endpoint continuously and worsen an active rate-limit response.
- **Failure classes differ in duration**, so size the total span against them: a network blip resolves in milliseconds, a deployment in minutes, an infrastructure outage in hours. A schedule ending before a realistic outage ends dead-letters events the subscriber would have recovered.
- **Not everything retries**: non-retriable 4xx (anything except 408/429) skips the budget and dead-letters immediately - retrying a permanent failure wastes resources and can look like an attack to the receiving endpoint. A 429 with `Retry-After` is retried at the receiver's stated delay, overriding your own curve.
- **Queue-first is the prerequisite**: POSTing webhooks from the main application thread collapses under load; delivery decouples from the event-producing request path via a durable queue before any retry policy matters.
## Dead-letter policy
- Exactly two triggers: retry budget exhausted, or a non-retriable error class.
- Retention: 7-30 days, preserving full event context (headers included) for investigation and replay.
- **Dead-letter rate is the headline operational metric** - a spike signals an endpoint in sustained failure, distinct from background transient retries.
- The "Issues" framing (Hookdeck): treat an exhausted delivery as an operational incident to investigate - grouped by endpoint and status code, surfaced to a human with notification - rather than a message sitting in a queue behind a bare replay button.
- Test the recovery path on a schedule: inject failures, verify events land in the DLQ, practice the replay - the path decays if untested.
## Circuit breaker (the promoted rung's reference design)
Convoy ships the only true half-open circuit breaker among the six: a volume threshold and error-ratio threshold trip it, a timeout parameter holds it open, and an automatic probe delivery tests recovery before resuming full traffic. Contrast with the common simpler mechanisms - Stripe's auto-disable and Shopify's auto-delete are one-way doors requiring subscriber action to reopen. The circuit breaker matters at scale: without it, every sustained-failing endpoint consumes its full retry budget on every event, and a platform with thousands of endpoints pays that continuously.
## Why "no retries" is a named anti-pattern, precisely
GitHub ships zero automatic retries and survives it only because two compensations ship alongside: a deliveries-list API with a redeliver endpoint (so consumers can build a scheduled poll-and-redeliver script), and documentation that explicitly assigns reconciliation to the consumer. That pushes 100% of reliability engineering onto every integrator - defensible for GitHub's developer-heavy audience, and still the sharpest outlier surveyed. A platform copying the "no retries" half without the API-and-docs half is not shipping a minimalist design; it is shipping unreliable delivery.
## Reconciliation as an endorsed practice
For high-stakes event types (payments is the canonical example), recommend consumers periodically diff processed events against your API's own records - if the API shows state changes the consumer never processed, events were lost in a way retries and the DLQ didn't catch. Shopify's docs recommend exactly this as the primary defense rather than treating gaps as a bug to fix - set the same honest expectation.
references/signing-scheme-reference.md›
# Signing scheme reference
Worked detail behind SKILL.md step 3. Spec: Standard Webhooks (`standard-webhooks.com`, spec at `github.com/standard-webhooks/standard-webhooks`) - community-driven and vendor-neutral, with self-reported adoption by OpenAI, Anthropic, Google Gemini, Kong, Svix, Supabase, Twilio, PagerDuty, and others. Treat that list as directional (claimed, not audited), but it is the closest thing this domain has to a standard.
## Standard Webhooks construction
Three headers on every delivery:
```
webhook-id: msg_2KWPBgLlAfxdpx2AI54pPJ85f4W
webhook-timestamp: 1706100000
webhook-signature: v1,K5oZfzN95Z9UVu1EsfQmfVNQhnkZ2pj9o9NDN/H/pI4=
```
Signature computation:
```
signed_content = "{webhook-id}.{webhook-timestamp}.{raw_body}" # literal dots
signature = base64( HMAC-SHA256( secret, signed_content ) )
header value = "v1," + signature
```
Verification on the receiving side recomputes the HMAC from the raw body and, separately, rejects deliveries whose `webhook-timestamp` is older than a tolerance window - the spec recommends 300 seconds. Without the timestamp check, a captured legitimate request replays indefinitely: the signature proves authenticity, not freshness.
## Secret formats
- Symmetric HMAC secret: 24-64 random bytes, base64-encoded, prefixed `whsec_`; signature id `v1`.
- Asymmetric ed25519: private key prefixed `whsk_`, public key prefixed `whpk_`; signature id `v1a`.
The prefix makes a leaked key immediately identifiable as a webhook secret by scanning tools - a pattern worth copying even without adopting the full spec. Offer asymmetric signing when you don't control both ends: the consumer verifies with a non-secret public key instead of holding your shared secret. A single provider can support both modes simultaneously.
## Zero-downtime rotation
`webhook-signature` carries multiple space-delimited `<version>,<signature>` pairs; the receiver verifies against any of them. Rotation flow:
1. Issue the new secret; sign every delivery with both old and new.
2. Subscriber deploys the new secret and confirms verification passes.
3. Revoke the old secret only after confirming delivery success under the new one.
There is no cutover moment where a request fails to verify. Because the scheme is uniform, verification can also live once at an API-gateway layer instead of per-integration in application code.
## How the major non-adopters sign (contrast table)
| Provider | Header | Content signed | Encoding | Replay protection | Rotation |
| -------- | -------------------------------------- | ----------------------------------------------------------- | -------- | --------------------------------------------------------------- | --------------------------------------------------- |
| Stripe | `Stripe-Signature` (`t=<ts>,v1=<sig>`) | `"{timestamp}.{raw_body}"` | hex | Yes (timestamp) | Roll secret, up to 24h dual-validity |
| GitHub | `X-Hub-Signature-256` (`sha256=<sig>`) | raw body only | hex | **None** - consumers track the `X-GitHub-Delivery` GUID instead | None - manual reconfigure with a gap |
| Shopify | `X-Shopify-Hmac-SHA256` | raw body only | base64 | **None** | Client-secret change with up to ~1h propagation lag |
| Convoy | `X-Convoy-Signature` | "simple" (body) or "advanced" (Stripe-style with timestamp) | hex | Opt-in via mode | Rolling secrets with explicit expiry |
The pattern to learn from the weak rows: a body-only signature has no replay protection and usually no overlap rotation. GitHub's and Shopify's schemes predate the convergence - don't copy them for a new platform.
## Consumer pitfalls your signing docs must cover
These dominate real integration failures; document them explicitly rather than assuming they're obvious:
1. **Verify every request, no exceptions** - a handler that "only" posts to chat still triggers arbitrary side effects on a spoofed event.
2. **Verify against the raw body** - a framework's JSON body-parser re-serializes the body before the check runs and breaks the HMAC (e.g. Express needs `express.raw({ type: 'application/json' })` on the webhook route specifically).
3. **Exempt the webhook route from session-auth middleware** - the route authenticates by signature; uniform auth middleware 401s it before the handler runs.
references/subscription-lifecycle-comparison.md›
# Subscription lifecycle and debugging surface: vendor comparison
Detail behind SKILL.md steps 5-6. The clearest pattern in this data: the three infrastructure-only vendors (Svix, Hookdeck, Convoy) converge on richer states, deeper filtering, true replay, and a full portal, while the platforms where webhooks are one feature among many (Stripe partially, Shopify, GitHub) ship the thinner model. Treat the infrastructure-vendor shape as the target state; the thinner shape is what most teams launch with and outgrow.
## Endpoint states
| Vendor | States | Notes |
| ---------------------- | -------------------------- | ------------------------------------------------------------------------------------------------------ |
| Stripe | enabled / disabled | Auto-disabled after the ~3-day retry window |
| GitHub | active / inactive | Binary |
| Shopify | active until auto-deleted | 8 consecutive failures deletes an Admin-API-created subscription; TOML-declared ones restore on deploy |
| Svix, Hookdeck, Convoy | active / paused / disabled | Plus auto-disable-with-notification on sustained failure - the model to default to |
Case-study lesson on state design (Salesforce's managed event subscriptions): valid states are run and stop; an internal pause state exists but user-created subscriptions are rejected from setting it. The transferable rule: choose the states a customer-created subscription may legally reach, and reject internal-only states at the API boundary rather than exposing them.
## Replay positions (same case study)
Two independently configured positions, each taking "latest" (new events only) or "earliest" (full retained backlog):
- **Default replay** - where a newly created subscription starts reading.
- **Error-recovery replay** - where a subscription resumes after a platform-side incident, as distinct from a normal restart.
These can legitimately differ; specify both.
- Deleting a subscription permanently destroys its replay-tracking state - recreating under the same name restarts from the default position, so a subscriber deleting-to-reset silently loses or double-replays a backlog.
- Activating a full-backlog replay on a high-volume channel can mean days of replay traffic - gate it behind an explicit confirmation.
- Config changes can take a minute or two to propagate to the delivery path; document that as expected, not a bug.
## Replay and backfill capability
The sharpest capability split in the domain:
- **Stripe**: Events API lists/retrieves 30 days, supports resend; documents polling the resource API to backfill missed objects.
- **Svix**: portal replay of individual messages or ranges; polling endpoints iterate the entire event history.
- **Hookdeck**: permanent event log, bulk pause/cancel/replay via API. Source of the terminology worth adopting verbatim: **retry** = a new delivery attempt of the same event; **replay** = re-ingesting the original request as a brand-new entity that produces new events.
- **Convoy**: persists all events; UI + API manual and batch retry.
- **Shopify**: no native replay or backfill - the documented mitigation is a consumer-side reconciliation job.
- **GitHub**: UI redeliver (3-day window) or Deliveries API (30-day listing); no true backfill.
## Filtering and fan-out
Two independent filtering axes - keep them separate in the design:
1. **Event type** - selection against the catalog (Stripe's `enabled_events`, Shopify's one-subscription-per-topic).
2. **Recipient/channel scoping** - an orthogonal dimension for sub-grouping deliveries (per-tenant, per-repo, per-project). Svix models this as "channels", off by default, filtered independently of event types; Hookdeck and Convoy route on richer rule/payload-structure criteria.
Filtering is producer-side: an endpoint with no filter configured receives everything - state that explicitly. Multi-endpoint fan-out is universal (Standard Webhooks recommends multiple endpoints per consumer as a resilience pattern); one event fans out to every matching endpoint, each delivered and retried independently, so one slow or broken endpoint never blocks the others.
## Portal and testing tooling benchmark
What the A-grade debugging surface actually ships, per vendor:
- **Stripe CLI** - the reference for code-first consumers: `stripe listen --forward-to localhost:PORT` (stable CLI-issued signing secret) plus `stripe trigger <event>` firing a realistic example event on demand.
- **Svix App Portal** - the most complete consumer portal, embeddable and brandable: event catalog with schemas and example events, a schema-driven "send example webhook" button, filterable logs, per-attempt inspection, message replay.
- **Hookdeck** - real-time visual delivery traces; exports latency/error-rate metrics to an external observability stack; config-as-code via API/Terraform.
- **Convoy** - embeddable customer-facing dashboard (iframe or white-label) with email and chat failure notifications built in.
- **GitHub** - no first-party trigger command; sends an automatic "ping" event on webhook creation (a pattern worth copying: the subscriber verifies wiring without waiting for a real event).
Local tunneling and inspection (ngrok-class tunnels, request-inspection sites) are a shared consumer-side ecosystem - your docs should point at the category rather than build it, unless you ship a first-party CLI tunnel.
No-code consumers arrive with the catch-URL mental model from workflow-automation tools: a temporary URL captures one real payload so they can map fields before building logic, and a frequent production bug is a sender left pointed at the test URL after setup. Your test-event feature is the provider side of that same job - design and document it assuming this mental model, and make test deliveries clearly distinguishable from live ones.
SKILL.md›
---
name: webhook-platform-design
description: Design a provider-side outbound webhook platform - event taxonomy and catalog, payload envelope and schema versioning, an HMAC signing scheme (Standard Webhooks), at-least-once delivery with retry/backoff and dead-letter policy, subscription lifecycle states, and the developer-facing debugging surface (delivery logs, replay, test-event triggering). Use whenever the user mentions webhooks, event callbacks, push events to customer endpoints, webhook signatures, delivery retries, or a webhook consumer portal - even if they never say "webhook platform". Provider side only. Do NOT use for consumer-side retry mechanics - use samber/developer-platform-skills@api-idempotency-retry instead.
license: MIT
metadata:
author: Samuel Berthe
version: "1.0.1"
---
# Webhook Platform Design
You are an outbound-webhook platform designer. Design how a platform notifies its customers' servers of events - what the events are, how they are signed, how delivery is guaranteed and retried, how subscriptions live and die, and how an integrator debugs a delivery that never arrived.
Svix frames the real scope: "Sending an HTTP POST is easy. Doing it reliably at scale is not." The gap between those two sentences - slow-consumer isolation, retries with backoff, dead-lettering, signing, replay protection - is what this skill designs.
A whole-surface consistency review across every surface, including this one, belongs to `samber/developer-platform-skills@public-api-design-review`; this skill owns the webhook surface's own design. Whether webhooks are the right surface to build at all, against the other integration-surface shapes, is `samber/developer-platform-skills@api-integration-surface-strategy`'s call.
**Memory (advised):** When memory lives in a file, consider using `developer-platform-context.md`; if a different memory system is in use, rely on that instead. The file is an advisory reference, not a mandatory requirement. Separate task info in different sections. Remove finished tasks. Add a date to a task; no date for general project context. Some interview responses may differ between 2 tasks.
## Clarifying questions
Ask these before designing anything; batch them - this is a tactical design task, not a strategy interview.
1. Greenfield or retrofit? If retrofit, request the current event list, retry behavior, signing scheme, and whatever delivery logging exists.
2. Scale, order of magnitude: how many subscriber endpoints (tens, thousands, more), and peak events per second? (drives fan-out isolation and whether FIFO is even discussable)
3. Which consumer types will integrate: enterprise integrators, typical API developers, no-code workflow tools, or a mix? (see next section)
4. Effort ceiling and data constraints: how much engineering capacity - headcount, standing on-call, operating budget - can you commit to delivery infrastructure, and may customer event data transit a third-party processor at all? (see Build vs buy)
5. Deadline and payoff shape: by what date must webhooks ship, and is this a quick feature launch or the platform's compounding integration surface for years?
6. Is any of the stated need actually interactive, client-facing streaming (live dashboards, chat)? That's GraphQL-subscription/WebSocket territory - complementary to webhooks, not competing. Webhooks own durable server-to-server delivery where the consumer may be offline when the event fires; subscriptions own sub-second client push where occasional loss is tolerable. A platform can legitimately ship both.
Questions 4-5 exist because the build-vs-buy and retry menus diverge sharply on time-to-effect, durability, and effort - neither ranking can be picked without them. A hard ship date promotes managed infrastructure and the shorter retry span; a compounding-surface mandate promotes the pipeline you operate and tune yourself.
If your harness has persistent memory, store the delivery contract's core decisions (guarantee, ordering position, retry schedule, ack timeout, dedup ID location, signing scheme) so a later run - a new event type, a retry-policy change, the docs pass - starts from the contract instead of re-deriving it.
## Consumer sophistication
Webhook consumers are servers, whoever bought the product. The design splits by consumer sophistication, and unlike most audience splits the segments don't nest - each adds its own requirements:
- **Enterprise integrators** - demand the delivery contract in writing: guarantee semantics, retry schedule, signing spec, ack timeout. They will ask for FIFO and for asymmetric signing (verifying with a public key instead of holding your shared secret).
- **Typical API developers** - self-serve through docs and the portal. They need verification snippets that handle the raw-body pitfall, a stable per-event ID for dedup, and replay from the dashboard when their handler was broken for an hour.
- **No-code consumers (workflow-automation tools)** - think in catch URLs: "show me one real payload before I build against it." They never read the signing spec; they need test-event sending and visible recent deliveries far more than prose.
Serve every segment present. A design good enough for enterprise integrators still fails no-code consumers if it ships without test events and a delivery log.
## Build vs buy
Decide this before step 1 - it determines whether steps 4-6 are built or merely configured. Three options, ranked:
- effort: `in-house build > self-hosted open-source webhook gateway > managed sending infrastructure`
- value (working delivery pipeline on day one): `managed > self-hosted gateway > in-house`
- control (data residency, custom semantics): `in-house == self-hosted gateway > managed` - both keep event data on your infrastructure; in-house adds custom semantics at much higher cost
- compliance cost: `managed > self-hosted gateway == in-house` - managed triggers DPA and data-residency review, and reversing it later means re-papering contracts and migrating live endpoints; the other two keep event data on your own infrastructure equally
- efficiency: `managed > self-hosted gateway > in-house`
- **Default rung: managed sending infrastructure**, when webhooks are a feature of your product rather than the product itself - the Svix/Hookdeck/Convoy class of vendor (names are citations, not requirements; evaluate whatever equivalent your ecosystem offers). Svix's own build-vs-buy estimate for replicating the feature set in-house is engineer-years, not engineer-weeks - 3-5 engineers for 6-12 months plus a standing on-call. Treat the exact figure as vendor self-interest, but the order of magnitude matches what the feature list in this skill implies.
- **Move up to a self-hosted gateway** when compliance rules out third-party transit but you can operate the software. In that case the managed rung is **deleted, not demoted**: question 4 answering "customer event data may never transit a third-party processor" removes it from the menu instead of ranking it last, because a rung parked at the bottom on a compliance ground reappears as scope the next time a deadline bites - and re-appears without the legal review that ruled it out. Re-promotion trigger: a signed DPA and a residency posture that covers the vendor.
- **In-house is the starved option**: highest control, highest effort, loses every efficiency round. Promoted when webhook delivery is itself the product you sell, or when both third-party transit and third-party code are ruled out. Whatever you pick, every design decision in steps 1-7 still has to be made - buying infrastructure buys the machinery, not the taxonomy, contract, or docs.
- This ranking is a default, not a law. Re-rank against the interview: a hard ship date promotes managed; an in-house team already operating durable queues at scale gets the build far cheaper than the estimate, which flips the effort line.
## Workflow
1. Define the event taxonomy and catalog.
2. Design the payload envelope and schema-versioning story.
3. Adopt the signing scheme.
4. Set the delivery contract: guarantees, retry policy, dead-lettering.
5. Model the subscription lifecycle.
6. Build the debugging surface: logs, replay, test events.
7. Document everything against the five-element rubric.
Each step has a section below, in order.
## 1. Event taxonomy and catalog
- Name events with the Object-Action pattern: `resource.action`, action in past tense (`invoice.paid`, never `invoice.pay`), dot-delimited, identifiers restricted to `[a-zA-Z0-9_.]`. This is the Standard Webhooks recommendation and the Stripe convention. Production platforms diverge on delimiter and on where the discriminator lives, so pick one convention at launch and never mix. Splitting the event name across a header and a body field, GitHub-style, means no single string names the event - avoid it.
- The catalog is the contract: every event type gets a stable name, a description, and a payload schema before it ships. Subscribers select what to receive by catalog name, so an undocumented event type is effectively unsubscribable.
- Check each event against five design principles: self-contained, immutable (add fields, never repurpose), traceable, actionable, debuggable.
- Decide explicitly whether subresource events bubble to parent events (Stripe's answer: never - `customer.subscription.updated` fires no `customer.updated`). Whichever way, state it; silence here breeds subscriber assumptions.
See [references/event-taxonomy-and-payload-design.md](references/event-taxonomy-and-payload-design.md) for vendor naming comparisons and the design-principle checklist applied.
## 2. Payload envelope and schema versioning
- Choose envelope placement deliberately: all metadata in the JSON body (Stripe), or metadata in HTTP headers with only resource data in the body (GitHub, Shopify, Standard Webhooks). Header-split designs force your delivery log to capture headers and body to replay an event faithfully - build the log for that from day one, not as a retrofit.
- Choose thin vs full payloads as a first-class axis: full payloads embed resource state; thin payloads carry identifiers and force an API fetch. Standard Webhooks recommends keeping payloads small, usually under ~20kb. Thin payloads also sidestep versioning: the notification never changes shape when the API version bumps.
- Pick one of the two schema-versioning camps deliberately: pin each webhook endpoint to an API version and render payloads per that version (Stripe's model), or version event types themselves - a new type name per breaking change, additive-only evolution otherwise (the Svix/Standard Webhooks model). Drifting into both by accident is the failure; an API version bump must never silently reshape event payloads a subscriber already stores.
- Know the standards layering - these are layers, not competitors: Standard Webhooks covers signing and delivery headers, CloudEvents covers the envelope, AsyncAPI and OpenAPI 3.1's `webhooks` section cover description. Document outbound events in the same OpenAPI document as the inbound API; only adopt CloudEvents if you must interoperate with an event mesh that already speaks it.
See [references/event-taxonomy-and-payload-design.md](references/event-taxonomy-and-payload-design.md) for the envelope fork in detail, the Stripe thin/snapshot dual-run migration pattern, and the standards map.
## 3. Signing scheme
You own the signing scheme - what is signed, how, and how consumers verify. Signing-key issuance, storage, and rotation infrastructure belong to sibling `samber/developer-platform-skills@api-auth-key-management`.
- Adopt the Standard Webhooks spec instead of inventing a bespoke scheme - it's the closest thing this fragmented domain has to a standard, with real (self-reported) adoption across major API platforms. Three headers on every delivery: `webhook-id`, `webhook-timestamp`, `webhook-signature`.
- Signature construction: HMAC-SHA256 over `{id}.{timestamp}.{raw_body}`, base64-encoded, algorithm-version-prefixed (`v1,...`). Signing the timestamp is what buys replay protection - the receiver rejects deliveries whose timestamp falls outside a tolerance window (the spec recommends 5 minutes). A body-only signature (GitHub's and Shopify's shape) validates forever, so a captured request replays indefinitely.
- Prefix secrets so leaked keys are identifiable by scanners: `whsec_` for symmetric HMAC secrets, `whsk_`/`whpk_` for asymmetric ed25519 private/public keys (signature id `v1a`). Offer asymmetric signing for consumers who shouldn't hold a shared secret - the recommended choice when you don't control both ends.
- Design zero-downtime rotation in: the `webhook-signature` header carries multiple space-delimited signatures, so old and new secrets validate simultaneously during a rotation window. A platform with a single active secret forces subscribers into revoke-then-deploy, with a window of failed deliveries.
- Your signing docs must cover the three consumer-side pitfalls that dominate integration failures:
- Verify every request without exception, even notification-only handlers.
- Verify against the raw body before any JSON parser touches it.
- Exempt the webhook route from session-auth middleware, since it authenticates by signature, not session.
See [references/signing-scheme-reference.md](references/signing-scheme-reference.md) for the worked construction, the four major non-adopters' schemes contrasted, and the rotation flow.
## 4. Delivery contract: guarantees, retry, dead-letter
- Promise at-least-once delivery, explicitly, as a deliberate choice: you retry until acked, so a subscriber that processes then crashes before responding 200 sees the event twice. Never claim exactly-once - every "exactly-once" claim in this market is at-least-once plus dedup tooling. Ship the honest version of that: a stable per-event ID (same value across retries of the same event) from day one, plus a documented dedup recommendation. Retrofitting the ID later can't help subscribers who already built without it.
- State the ordering position explicitly: best-effort, non-guaranteed ordering is the correct default - retries and parallel workers reorder events, and strict ordering means one failing delivery blocks the whole per-endpoint queue. Every major platform ships non-FIFO by default; offer FIFO only as an explicit opt-in for consumers who structurally cannot tolerate reordering. Warn subscribers not to infer order from timestamps.
- Decouple delivery from the request path via a durable queue, with per-endpoint isolation - one slow or broken subscriber must never delay delivery to the others. This fan-out isolation is the core value the platform sells to its own engineering team.
- Choose the retry policy - this is where real platforms diverge most:
- effort: `configurable per-endpoint schedules + circuit breaker > fixed day-scale schedule + DLQ + auto-disable > fixed hours-scale schedule > manual redeliver only`
- value (deliveries recovered without human action): `configurable + circuit breaker > fixed day-scale > fixed hours-scale > manual redeliver only`
- efficiency: `fixed day-scale schedule > fixed hours-scale schedule > configurable + circuit breaker > manual redeliver only`
- **Default rung: a fixed exponential-backoff schedule spanning roughly a day** - the shape Svix ships (8 attempts over ~27.5 hours) and Stripe approximates (up to 3 days); Shopify's 8 tries over 4 hours marks the short end. Size the total span against real subscriber incident durations - a deploy takes minutes, an outage can take hours - not a round number. Exponential, never linear: constant-interval retries hammer a struggling endpoint.
- **Promote to configurable schedules plus a circuit breaker** (half-open probing before resuming, Convoy-style) when endpoint count grows large enough that sustained-failing endpoints materially load the pipeline, or when enterprise consumers negotiate delivery SLAs. High value, high effort - the starved option that those two conditions promote.
- **Manual redeliver only (GitHub's model: zero automatic retries) is a named anti-pattern**, not a minimalist rung - it works for GitHub only because a deliveries-list-plus-redeliver API and explicit "build your own reconciliation" docs ship with it. Copying the zero without the API is just shipping unreliable delivery.
- This ranking is a default, not a law - re-rank on the interview: a mostly no-code consumer base devalues configurability nobody will touch; a payments-grade domain promotes the longer span and the circuit breaker.
- Classify subscriber responses instead of treating them alike:
- 2xx acks and stops retries. Expect the ack within ~10 seconds, the Shopify/GitHub convention.
- Treat "handler is slow" as distinct from "handler errored" - a subscriber that queues the work and returns fast is doing exactly what you should tell it to.
- Honor a `Retry-After` header on 429 over your own backoff curve - most senders don't, and it's a differentiator.
- Send non-retriable 4xx (anything but 408/429) straight to the dead-letter queue - retrying a permanent failure wastes the budget and looks like an attack.
- Dead-letter on exactly two triggers:
- Retry budget exhausted.
- A non-retriable error class.
Retain dead-lettered events 7-30 days with full context (headers included) for investigation and replay. After sustained failure, auto-disable the endpoint and notify the subscriber out-of-band - email or dashboard, since the webhook channel itself is what's broken.
- Recommend consumers of high-stakes event types run periodic reconciliation against your API - vendor-endorsed practice, not a workaround. Your docs setting that expectation is more honest than implying delivery is stronger than at-least-once/best-effort actually promises.
See [references/retry-policy-vendor-comparison.md](references/retry-policy-vendor-comparison.md) for the six documented vendor schedules, timeout values, auto-disable behaviors, and the circuit-breaker reference design.
## 5. Subscription lifecycle
- Model the subscription (endpoint registration) as a first-class stateful object with full CRUD - never fire-and-forget registration. Give it three explicit states:
- active
- paused - subscriber-initiated stop without losing configuration
- disabled - platform-initiated after sustained failure
The three infrastructure-focused vendors converge on this richer model; binary enabled/disabled is the thinner shape platforms launch with and outgrow. Expose only states a customer-created subscription can legally reach - reject internal-only states at the API boundary.
- Keep filtering's two axes independent: event type (against the catalog) and recipient/channel scoping. An endpoint with no filter receives everything - say so.
- Specify both replay positions: where a newly created subscription starts (new events only vs retained backlog), and where a subscription resumes after a platform-side incident. They can legitimately differ. Document loudly that deleting a subscription destroys its replay/delivery state - recreating it under the same name starts fresh, and a subscriber deleting-to-reset will silently lose or double-replay a backlog.
- Document config-change propagation lag as expected behavior (a subscription edit taking a minute or two to reach the delivery path is normal), so integrators don't file it as a bug.
See [references/subscription-lifecycle-comparison.md](references/subscription-lifecycle-comparison.md) for the six-vendor state/filtering/replay comparison and the state-machine case study.
## 6. Debugging surface
A self-service consumer portal is table stakes in this market, not a nice-to-have - flag "no portal" the way you'd flag a missing signature-verification doc. The portal carries:
- **Delivery logs**: filterable by endpoint, event type, and status, with per-attempt inspection (request headers, body, response code, timing). If your envelope splits metadata into headers (step 2), the log must store them or replay is unfaithful.
- **Replay and retry, as distinct operations** - adopt Hookdeck's terminology verbatim, because the market uses the words interchangeably and your docs shouldn't: a _retry_ is a new delivery attempt of the same event; a _replay_ re-ingests the original as a new event. Offer single-event and range replay; gate a large replay window behind a cost confirmation, since replaying a big backlog is an expensive operation the subscriber may not intend.
- **Test-event triggering**: a "send example event" action driven by each event type's schema, so an integrator sees one realistic payload before writing a line of handler code - the same job a no-code tool's catch-URL solves from the consumer side. Test triggering in a sandbox environment (fake data, test-mode keys) is sibling `samber/developer-platform-skills@api-test-mode-design`'s ground; this skill owns the delivery-side button and the example payloads it sends.
- **Failure visibility**: surface exhausted deliveries as incidents to investigate - grouped by endpoint and status code, with notification - rather than rows in a queue. Document the 200-then-crash blackhole: a handler that acks then throws has told you delivery succeeded, and no debugging surface can see inside it.
See [references/subscription-lifecycle-comparison.md](references/subscription-lifecycle-comparison.md) for what each vendor's portal and testing tooling actually ships, as the benchmark to match.
## 7. Documentation
Grade your webhook docs against the five-element rubric the market's best docs are graded on, all five before launch:
1. Signature verification with runnable snippets.
2. The event catalog with schemas and example payloads.
3. The retry policy - exact schedule, not "we retry with backoff".
4. Troubleshooting and failure recovery.
5. Testing tools.
State the delivery contract as explicit, findable sentences:
- The guarantee (at-least-once).
- The ordering position.
- The retry schedule.
- The ack timeout.
- Which header or field carries the dedup ID.
Every one of these left implicit becomes a subscriber assumption, then a support ticket. Publish the event catalog from the same spec that drives delivery (OpenAPI 3.1 `webhooks` section) so docs and payloads can't drift apart silently.
## Failure modes
Anti-pattern checklist - each is a direct audit finding:
- Zero automatic retries without a deliveries-list-plus-redeliver API and reconciliation docs shipping alongside.
- "Exactly-once delivery" claimed anywhere in the docs.
- Ordering guaranteed implicitly (or subscribers left to assume it) instead of an explicit best-effort statement.
- Linear retry intervals, or retrying non-retriable 4xx responses until the budget exhausts.
- A signature over the body only - no timestamp, so no replay protection.
- One active signing secret, forcing revoke-then-deploy rotation.
- No stable per-event ID across retries of the same event.
- A delivery log that stores only the body while the envelope carries metadata in headers.
- An API version bump silently reshaping event payloads existing subscribers already store.
- Delete-and-recreate of a subscription silently resetting its replay position.
- Slow-but-correct handlers (queue fast, ack fast) treated as failures because the timeout and the queue-fast advice were never documented.
- No consumer portal: no logs, no replay, no test events.
## Measurement
Two binary gates - iterate the design until both pass:
- **Contract completeness**: the guarantee, ordering position, retry schedule, ack timeout, and dedup ID location are each explicitly documented. Five facts, all present - a missing one fails the gate, because each absence is a subscriber assumption waiting to become an incident.
- **Documentation rubric**: all five rubric elements exist before launch. Any missing element fails the gate.
Operational metrics to stand up, watched as trends rather than pass thresholds:
- Per-endpoint delivery success rate - the platform's headline health number.
- Dead-letter rate - a spike means an endpoint in sustained failure, distinct from background transient retries.
- Time-to-detect a failing endpoint - bounded by your auto-disable/notification window. Set the baseline from your first month and improve against it rather than against an external threshold.
## Invocation examples
- "We're adding webhooks to our billing API - design the event catalog, signing, and retry policy from scratch."
- "Our webhook system retries three times in a row then drops the event, and customers keep missing deliveries - redesign the retry and dead-letter policy."
- "Enterprise customers are asking how our webhooks are signed and whether we guarantee ordering - write the delivery contract we should be able to show them."
## References
- [references/event-taxonomy-and-payload-design.md](references/event-taxonomy-and-payload-design.md) - naming conventions across vendors, envelope placement fork, thin vs full payloads, schema-versioning camps, standards map.
- [references/signing-scheme-reference.md](references/signing-scheme-reference.md) - Standard Webhooks construction worked example, non-adopter scheme contrast, rotation flow, consumer-pitfall checklist for your docs.
- [references/retry-policy-vendor-comparison.md](references/retry-policy-vendor-comparison.md) - six vendors' documented retry schedules, timeouts, auto-disable behavior, dead-letter and circuit-breaker patterns.
- [references/subscription-lifecycle-comparison.md](references/subscription-lifecycle-comparison.md) - endpoint states, filtering, replay/backfill, and portal/testing tooling across six vendors, plus a subscription state-machine case study.
- `samber/developer-platform-skills@api-idempotency-retry` - general retry and idempotency mechanics (backoff algorithms, idempotency keys) on the API's request side; this skill applies those ideas to outbound delivery only.
- `samber/developer-platform-skills@integration-error-observability` - how integration errors surface to the developer beyond the webhook delivery log.
- `samber/developer-platform-skills@api-status-communication` - incident and status-page communication when delivery failures are the platform's fault.
- `samber/developer-platform-skills@api-versioning-policy` - the versioning machinery behind step 2's event-schema evolution choice.
- `samber/developer-platform-skills@api-error-design` - the error surface of the subscription-management API itself.
- `samber/developer-platform-skills@public-graphql-api-design` - GraphQL subscriptions, the complementary push mechanism named in question 6: first-party live UI over sub-second push, versus this skill's durable server-to-server delivery.