---
title: "Webhook evidence"
description: "Signed vendor webhooks terminate at the gateway, which verifies the original bytes before parsing, records immutable pre-correlation evidence, and never lets control synthesize a conclusion."
---
Some effects are only provable by a later event: the vendor answered `202`, the object was not immediately readable, and the thing that establishes delivery is a signed webhook that arrives minutes later.
Webhook receipt and signature verification are gateway-owned ([ADR-0021](/concepts/invariants)). Native vendor webhooks terminate at the gateway, not at the control plane, because the shared webhook secret is encrypted per-subscription connection material and plaintext exists only inside the gateway process.
## The ingress
```text Endpoints
POST /webhooks/{workspace_id}/{connection_id}/{subscription_id}
POST /hooks/{workspace_id}/{connection_id}/{subscription_id}
```
Both paths are public and both reach the same handler. The endpoint loads only local and Redis snapshot state, and decrypts signing keys only inside the gateway.
The order of checks is the security property, so it is worth stating exactly:
The workspace, connection, and subscription come from the path. An unknown subscription is `404`. A subscription whose signature profile does not match the connection's tier, custody, and connector slug is refused — a Nango or Composio profile requires federated custody with a broker reference, and every other profile requires a native, Obol-custody connection.
Known-subscription traffic is capped at 120 requests per minute per workspace, connection, and subscription. Over that is `429`.
If the subscription pins a `notification_url`, the request path must match it. That URL is configuration and is never derived from `Host` or forwarded headers.
Bodies are limited to 256 KiB. Larger is `413`.
The gateway HMACs the raw body — before parsing it — against the current key and, inside the rotation overlap, the previous key. Comparison is constant-time. A profile that signs a timestamp has it authenticated and bounded in both directions against the subscription's declared tolerance.
Deduplication is workspace + connection + subscription + provider event id. A repeated id carrying a different body digest is a `409` and a security log line, never evidence.
An invalid signature produces bounded security telemetry only. It never becomes evidence and never becomes a contradiction. Keys and raw payloads never enter the evidence stream, the receipt, or control-plane plaintext memory.
## Signature profiles
`SignatureProfile` is a closed, reviewed registry — never a config language, never a regex, never a per-vendor plugin. Adding one is a reviewed, security-sensitive change. `stripe_v1` (`t=…,v1=…` over `t.body`) and `github_sha256` (`X-Hub-Signature-256` over the raw body) are the two called out by name in the contract; the registry also covers Svix-shaped and standard-webhooks-shaped schemes, and a broad set of per-vendor HMAC variants.
Trust follows the profile:
- `nango_sha256` and `composio_sha256` establish `broker_attested`.
- Every other reviewed profile establishes `vendor_signed`.
A broker signature proves what the broker sent, not what the vendor signed. Broker-forwarded evidence is never promoted to `vendor_signed`.
### The GitHub caveat
GitHub authenticates neither the event-family header, nor the delivery id, nor a timestamp. So for that profile the gateway derives the supported event family from the **signed payload structure**, rejects a conflicting unsigned family header, requires a UUID-shaped delivery id, and deduplicates the signed body digest as well as the provider id.
This still does not prove freshness of a previously unseen GitHub payload, and no upstream correlation id is inferred from an ambiguous shared payload shape. Recording that limit is the point; papering over it would let a captured review event be relabeled as a pull request event and bypass the declared family.
## Evidence before correlation
A valid delivery becomes an immutable, versioned `WebhookEvidenceEvent` — distinct from the invocation-correlated `VerificationEvent`. It carries an evidence id, subscription and event type, safe correlation ids, trust, signature profile and key version, a body digest, an authenticated array position for providers that batch, and bounded assertion outcomes. It carries no raw body.
```json Webhook evidence event
{
"schema_version": 1,
"evidence_id": "evd_01931f4c9a7b7e2f8c1d0a5b6e3f9014",
"workspace_id": "ws_acme",
"connection_id": "conn_stripe",
"subscription_id": "wsub_stripe_refunds",
"event_type": "charge.refunded",
"provider_event_id": "evt_3AbcDefGhiJklMno",
"upstream_id": "re_3AbcDefGhiJklMno",
"trust": "vendor_signed",
"signature_profile": "stripe_v1",
"key_version": "v1",
"body_digest": "5f2b1c8e9d0a4f6b3c7e1a9d8b0f2e4c6a8d0b2f4e6a8c0d2f4b6e8a0c2d4f6b",
"results": [
{ "name": "event_says_refunded", "outcome": "passed", "trust": "vendor_signed" }
],
"received_at": "2026-08-29T12:00:00Z"
}
```
Control persists the envelope and correlates it by workspace, connection, subscription, declared event type and stage, and safe upstream id. An optional reflected idempotency key may narrow a match but never replaces the upstream id, and ambiguous evidence stays unmatched. Control then sends the immutable evidence **reference** — not a reconstructed payload — to the gateway's capability-bound evaluation endpoint, which independently rechecks every binding against pinned state.
Neither ingestion nor persistence changes a receipt or an effect conclusion. Control is scheduler, correlator, and persistence owner; the gateway stays the receipt authority. See [Verification](/receipts/verification) for how correlated evidence advances a plan.
Out-of-order unmatched evidence is retained for a bounded reconciliation window: plans carry a deadline of at most one day, late authoritative reconciliation stays available for a further day, and gateway verification state is retained for up to seven days.
## Provisioning a subscription
Signing material reaches the control plane already encrypted. Control never sees plaintext.
A local gateway command reads the secret from non-interactive stdin — never a terminal, never an argument — and seals it under the same KEK as the target gateway. It starts no service; the only potentially remote work is configured KMS access.
```bash Seal a webhook signing key
printf '%s' "$WEBHOOK_SIGNING_SECRET" \
| obol-gateway seal-webhook-key --key-version 2026-09-a --valid-for-days 30 \
> current-key.json
```
The secret must be 16 to 4096 bytes, and control characters are rejected except inside a PEM public-key block. `--valid-for-days` accepts 1 through 30 and defaults to 7. Output is the standard encrypted signing-key contract, including its masked `last4`.
```bash Provision a subscription
curl -X PUT \
"https://control.tryobol.dev/api/v1/workspaces/ws_acme/connections/conn_stripe/webhooks/wsub_stripe_refunds" \
-H "Authorization: Bearer $OBOL_OPERATOR_TOKEN" \
-H "Content-Type: application/json" \
--data @subscription.json
```
The endpoint requires the connection-manage operator action, accepts at most 128 KiB, and validates the body against a closed schema before storing it. Errors are deliberately opaque — this endpoint never echoes the value it rejected.
A replacement must be new secret material under a new version. Re-encrypting the same plaintext, reintroducing a key that has left the retained current/previous pair, or recreating it under a new subscription id is not cryptographically detected, so treat rotation as an operational discipline. Previous-key overlap is capped at 24 hours, replay markers are retained for 30 days, and `DELETE` on the same path removes the subscription.
Read the current configuration — versions and validity windows, never key material — with `GET` on the connection's `webhooks` path. See [Webhooks API](/api-reference/webhooks).
## Recovering receipt delivery
Signed evidence is only useful if the resulting receipt revisions actually reach the audit projection. The gateway's audit stream is delivered at least once, and a consumer can fail after receiving an entry and before committing it.
So the control consumer reclaims pending deliveries with bounded `XAUTOCLAIM` pages and persists the returned scan position under a workspace-, group-, and consumer-qualified Redis key, rather than restarting the scan from zero and stranding eligible entries behind a long prefix of recently renewed leases. A drain admits no more than its batch limit across reclaimed and fresh entries, commits before acknowledging, and rolls back one workspace's transaction without blocking another's ([ADR-0042](/concepts/invariants)).
Each receipt is projected with a single conditional upsert that requires the incoming revision to be strictly greater than the stored one and the workspace to match, so overlapping consumers cannot write a stale revision over a newer one. Exactly-once transport is not claimed; monotonic projection is what makes at-least-once safe.
Listing endpoints page with a versioned opaque cursor over `(occurred_at, invocation_id)` descending, so ordinary traffic inside a boundary second is traversable. Legacy RFC3339 `before` values are still accepted as timestamp bounds, and a malformed cursor is rejected rather than silently substituted with the current time.
## Related
How correlated webhook evidence advances a verification plan.
Where projected receipts and revisions are queried.