---
title: "Gateway overview"
description: "The Rust data plane: how a request is authenticated, authorized, dispatched, and metered on the way to a model or a tool."
---

The gateway (`apps/gateway`, crate `obol-gateway`) is the only component your
agents talk to. It is an axum service that terminates agent traffic on `/v1/*`
and `/mcp`, decides whether the call is allowed, injects your vendor credential
on the outbound hop, and records what happened.

Everything except `main.rs` lives in the library, so integration tests drive the
same router production serves.

## Surfaces

| Route | Method | Purpose |
|---|---|---|
| `/v1/chat/completions` | `POST` | OpenAI chat completions |
| `/v1/responses` | `POST` | OpenAI responses |
| `/v1/embeddings` | `POST` | OpenAI embeddings |
| `/v1/messages` | `POST` | Anthropic messages |
| `/v1/route` | `POST` | Advisory capability resolver — never dispatches |
| `/mcp` | `POST`, `GET`, `DELETE` | Streamable HTTP MCP server |
| `/webhooks/{workspace}/{connection}/{subscription}` | `POST` | Vendor webhook ingest (also mounted at `/hooks/...`) |
| `/healthz` | `GET` | Liveness — always `200` |
| `/readyz` | `GET` | Redis reachable, at least one policy snapshot loaded, not draining |
| `/startupz` | `GET` | Redis reachable and not draining; empty snapshots OK |
| `/metrics` | `GET` | Prometheus text exposition |
| `/internal/...` | `POST` | Control-plane-only endpoints, authenticated by service JWT |

The `/v1/*` LLM surface is documented in [LLM routes](/gateway/llm-routes); the
MCP surface in [MCP surface](/gateway/mcp).

## The five hot-path rules

The gateway is deliberately narrow. Five constraints shape every route.

<Steps>
<Step title="Authentication is a cached Redis read, never Postgres">
A virtual key is hashed with SHA-256 and looked up at `key:{hash}` in Redis
(60-second TTL by default, `OBOL_KEY_TTL_S`). On a miss the gateway calls the
control plane's internal snapshot API with a short-lived service JWT and writes
the result back. The gateway holds no Postgres pool on the request path
(ADR-0006). A store failure fails closed as `not_ready` — never as
`unauthorized`, which a client would treat as final.
</Step>

<Step title="Policy is compiled data evaluated in-process">
CEL decides visibility, Cedar decides permission, and both run inside the
gateway process against a snapshot already in memory. There is no network hop
and no database query per decision (ADR-0003). See
[CEL prefilter](/policy/cel-prefilter) and [Cedar](/policy/cedar).
</Step>

<Step title="Vendor credentials are unwrapped at the outbound hop">
The encrypted credential travels in the workspace snapshot. `obol-vault`
decrypts it (AES-256-GCM under a KEK) immediately before the outbound request
and the plaintext exists only between that call and the socket. It is never
serialized, cached, or logged. See [Vault](/security/vault).
</Step>

<Step title="No product logic, and no Python, in the gateway process">
OAuth exchange orchestration, billing, approvals inboxes, and connector adapters
live elsewhere. Python connectors are out-of-process MCP workers (ADR-0002,
ADR-0007).
</Step>

<Step title="Usage emission is asynchronous and lossy; invocation evidence is not">
Usage events go through a bounded in-memory queue and a batching flusher onto a
Redis stream. If that queue is full, the event is dropped and a counter
increments — a response is never blocked on metering. Invocation receipts and
verification events do not use that queue: they are committed durably through
the idempotency store before the gateway exposes a result (ADR-0032).
</Step>
</Steps>

## Request lifecycle

Both surfaces share the same prelude: request id, authentication, workspace
snapshot, environment and freeze checks, scope check. They diverge after that.

```mermaid
flowchart TD
    A[Request] --> B[Assign request id, open access log span]
    B --> C[Authenticate: virtual key, MCP OAuth JWT, or service JWT]
    C --> D[Load pinned workspace + policy snapshot pair]
    D --> E{Frozen? Env mismatch? Missing scope?}
    E -->|yes| X[Error envelope with x-request-id]
    E -->|no| F{Route}
    F -->|/v1/*| G[Parse request, CEL model visibility]
    G --> H[Resolve model target and fallbacks]
    H --> I[Cedar 'complete' on the model]
    I --> J[Admission: RPM, budget, token reservation]
    J --> K[SSRF guard on the upstream URL]
    K --> L[Vault unwrap, render request, send with UsageGuard]
    L --> M[Passthrough JSON or SSE]
    F -->|/mcp| N[Verify optional approval token headers]
    N --> O[MCP server: session, JSON-RPC method]
    O --> P[GatewayHooks: resolve, claim idempotency, CEL, Cedar]
    P --> Q{Verdict}
    Q -->|deny or unknown| Y[Unknown tool]
    Q -->|approval required| Z[approval_required + Approval ticket]
    Q -->|allow| R[Vault unwrap, dispatch, verify effect]
    R --> S[Commit receipt + verification event, then respond]
```

## What the gateway proves

The guarantees the gateway makes *before* a call are the same everywhere: the
key is authenticated, the tool or model is visible to it under CEL, Cedar
allowed the action under the pinned policy publication, admission limits held,
and an idempotency key was assigned.

What the gateway can *prove after* the call is not uniform. A native route
executes through the gateway's own OpenAPI executor, so the receipt can record a
gateway-observed outcome. A federated catalog route executes at a broker the
gateway did not observe, so its evidence is `untrusted` or `broker_attested` and
never `verified`. The receipt always names which class it is — read
[Receipts](/receipts/overview) before relying on one.

## Error envelopes

Every error is one `ErrorEnvelope` rendered in the client's dialect. `/v1/*`
OpenAI-format routes get an `error` object with `message`, `type`, `code`, and
`param`; `/v1/messages` gets the Anthropic `type: "error"` shape; `/mcp` gets a
JSON-RPC error object carrying the envelope as `error.data`. Every error
response carries `x-request-id`, and `retry-after` when the code supplies one.

## Configuration

Configuration is environment-only, parsed once at startup. `Debug` redacts every
secret.

| Variable | Default | Effect |
|---|---|---|
| `OBOL_LISTEN_ADDR` | `0.0.0.0:8080` | Data-plane bind address |
| `OBOL_METRICS_ADDR` | `0.0.0.0:9091` | Separate metrics/health listener |
| `OBOL_ENV` | `dev` | Environment name; also the default for private-upstream policy |
| `REDIS_URL` | required | Hot-path store; connection is verified at boot |
| `OBOL_CONTROL_URL` | unset | Control plane, for snapshot fallback |
| `OBOL_CONNECTORS_URL` | `http://connectors:8090` | Trusted worker base URL. The binary still carries this default, but **no deployment target sets it and nothing listens there** — there is no worker deployable ([Trusted workers](/connectors/trusted-workers)) |
| `OBOL_SERVICE_JWT_SECRET` | required | Shared secret for gateway/control service JWTs |
| `OBOL_KEY_TTL_S` | `60` | Cached `KeyContext` TTL |
| `OBOL_SESSION_TTL_S` | `1800` | MCP session idle TTL |
| `OBOL_IDEMPOTENCY_TTL_S` | `86400` | Invocation record retention |
| `OBOL_UPSTREAM_TIMEOUT_S` | `120` | Per-upstream call timeout |
| `OBOL_DRAIN_TIMEOUT_S` | `30` | Total graceful-shutdown budget |
| `OBOL_ALLOW_PRIVATE_UPSTREAMS` | `true` only when `OBOL_ENV=dev` | Loopback/private upstream guard, see [egress and SSRF](/security/egress-and-ssrf) |

`OBOL_IDEMPOTENCY_TTL_S` is validated at startup: it must exceed the upstream
plus drain timeouts, and must not exceed the store maximum. A bad value fails
the process at boot rather than at the first call.

## Startup and drain

`run` connects Redis and fails fast if it is unreachable, preloads up to 1000
workspace snapshots, spawns the invalidation subscriber, then binds. Shutdown on
`SIGTERM` or `SIGINT` flips `/readyz` and `/startupz` to draining, stops accepting connections,
waits for in-flight requests and metering reconciliation, and closes the meter
queue — all inside one `OBOL_DRAIN_TIMEOUT_S` deadline. See
[Sessions and drain](/gateway/sessions).

## Metrics

The gateway registers these Prometheus series:

| Metric | Labels |
|---|---|
| `obol_requests_total` | route, status |
| `obol_mcp_requests_total` | JSON-RPC method, status |
| `obol_llm_tokens_total` | model, direction |
| `obol_cost_micros_total` | — |
| `obol_policy_decisions_total` | action |
| `obol_meter_dropped_total` | — |
| `obol_invocation_finalize_failures_total` | — |
| `obol_snapshot_loaded` | — |

JSON-RPC method labels are drawn from a fixed allowlist; an unrecognized method
name is labeled `unknown` so an attacker cannot grow the metric cardinality. See
[Telemetry](/observability/telemetry).

## Next

<Columns cols={2}>
<Card title="LLM routes" href="/gateway/llm-routes">
The OpenAI-compatible surface, translation, streaming, and error mapping.
</Card>
<Card title="MCP surface" href="/gateway/mcp">
Tool namespacing, list filtering, invocation, and approvals.
</Card>
<Card title="Sessions and drain" href="/gateway/sessions">
Opaque session ids, snapshot consumption, graceful shutdown.
</Card>
<Card title="Crate reference" href="/gateway/crates">
What every crate in `apps/gateway/crates` is responsible for.
</Card>
</Columns>