Architecture

Obol is two planes in one monorepo. ADR-0005 named four deployables; three of them ship — apps/gateway, apps/control, and apps/web — alongside apps/landing, which the ADR predates and which only Compose builds. The fourth, apps/connectors, is a reserved design with no source and no service in any deployment target. The split exists because the two halves have opposite profiles: the data plane is a hot proxy where tail latency and memory matter, and the control plane is a product where iteration speed and library ecosystem matter (ADR-0002). ADR references throughout these docs point at the decision records in docs/adr/ in the Obol repository. Accepted ADRs are immutable — a reversal supersedes one rather than editing it.

System shape

Clients (apps, Cursor, Claude, OpenAI SDK)
                    |
                    |  Bearer ob_live_...  /  MCP OAuth JWT
                    v
         +---------------------+
         |  DATA PLANE (Rust)  |  apps/gateway
         |  /v1/*   and  /mcp  |
         |  auth, CEL, Cedar   |
         |  stream, mux, vault |
         +----------+----------+
                    | snapshots only
         Redis <----+----> KMS (KEK)
                    |
         +----------v----------+
         | CONTROL (Python)    |  apps/control
         | keys, OAuth, Cedar  |
         | billing, approvals  |
         +----------+----------+
                    |
              Postgres + Stripe Billing
                    |
         +----------v----------+
         | WEB (Next.js)       |  apps/web
         +---------------------+

The gateway’s outbound edge fans out to LLM providers under the customer’s own key, an in-process OpenAPI executor for generic REST, and federated remote-MCP or catalog backends. A worker:// arm is compiled in but has nothing to dial: no worker is deployed in any target.

The deployables

App	Language	Ships as	Owns
`apps/gateway`	Rust	one static binary	Data plane: LLM and MCP proxy, OpenAPI executor, auth, CEL and Cedar, vault unwrap, metering emit
`apps/control`	Python (FastAPI)	API and worker images	Tenants, key mint, OAuth orchestration, Cedar compile and publish, approvals, Stripe Billing, audit
`apps/connectors`	—	nothing; no image, no deployed service	Reserved design, not built (parked by ADR-0058). Intended for reviewed request planners and response normalizers for messy vendors; observations only, never receipt authority
`apps/landing`	TypeScript (Next.js)	web app	Marketing site; built by Compose only, no backend dependency
`apps/web`	TypeScript (Next.js)	web app	Dashboard: keys, connectors, policy editor, receipts, approvals, usage

Data plane — `apps/gateway`

The gateway is one binary with two listeners. It is never split into an LLM proxy and an MCP proxy, because sessions, auth, policy, and metering have to share memory. It listens on:

POST /v1/chat/completions, /v1/responses, /v1/embeddings, /v1/messages
POST | GET | DELETE /mcp (Streamable HTTP, Mcp-Session-Id)
POST /v1/route for capability resolution
POST /hooks/{workspace}/{connection}/{subscription} for signature-verified vendor webhooks
GET /healthz, /readyz (Redis reachable, ≥1 policy snapshot, not draining), and /startupz (Redis reachable and not draining; empty snapshots OK)
An internal surface for reload, credential exchange, refresh, seal, catalog session, verification readback and evaluation, and approval resume

It does:

Authenticate

Bearer ob_live_... resolves by SHA-256 hash against Redis, falling back to control’s snapshot API; MCP OAuth 2.1 JWTs are validated on iss, aud, and scopes.

Admit

Key active, workspace not frozen, model or tool allowed, budget remaining, RPM and TPM within limits.

Decide

CEL prefilter for visibility, then Cedar is_authorized as the decision of record. A forbid on a destructive tool in production emits an approval ticket and returns a structured approval_required error.

Dispatch

Unwrap the outbound credential in-process, inject it on the outbound hop only, and route to an LLM provider, a remote or catalog MCP server, or the openapi:// executor. A worker:// executor is compiled in but has nothing to dial; it still speaks ADR-0014, not ADR-0024.

Receipt

Normalize every executor result into a DispatchObservation, run the deterministic EffectSpec verifier, and mint an InvocationReceipt plus a VerificationEvent — committed together as one InvocationAuditEnvelope before any terminal result or receipt metadata is exposed.

It does not run OAuth consent, render UI, talk to Stripe Billing, embed CPython, or hold connector business logic. The crate layout is deliberately thin and leaf-heavy — leaf crates depend only on obol-types, and composition happens in obol-gateway through store, executor, and hook traits (ADR-0018). See Gateway crates for the full map. One structural detail worth noting here: capability discovery lives in obol-discover, which may depend on obol-types alone. The absence of an edge from obol-route is what keeps retrieval off the dispatch path, and a DAG check script enforces it (ADR-0029).

Control plane — `apps/control`

Control owns Postgres and everything that is a product rather than a proxy: workspaces (each carrying an org_id column — there is no organizations table), memberships and invitations, virtual keys, the connector catalog and connections, OAuth orchestration, policy validation and publication, the approvals inbox, receipts, usage, and the audit log, all under /api/v1. Background workers (arq) handle OAuth exchange and refresh scheduling, Cedar compile and snapshot publish, key-cache invalidation on revoke, usage and receipt flush to Postgres and Stripe meters, verification scheduling, webhook correlation, and deadline reconciliation. ConnectorBundle validation and publication are not among them. Control reads a reviewed pack straight off disk (services/tools.py::load_bundle → packages/connectors/<slug>/bundle.json); there is no connector_bundles table and no publisher. Bundle validation exists in Rust (obol-types/src/bundle.rs).

Cedar validation runs through a small vendored Rust helper binary, obol-policy-check, invoked as a subprocess. Cedar is never reimplemented in Python, and that binary is the only Rust inside the control image. The subprocess boundary is a versioned contract: the CLI emits one PolicyCompileReport on stdout for both outcomes, and its stderr is never parsed and never reaches an operator response.

Connectors — `apps/connectors` (reserved, not built)

Reviewed Python workers are designed to run out of process and be called over MCP (ADR-0007). Embedding CPython in the gateway was rejected: the GIL fights Tokio, one bad connector takes the process down, and connectors could not scale independently. Nothing is deployed here. The trusted-worker tier is parked (ADR-0058): apps/connectors/ holds one README — no package, no image — and no deployment target defines a connector-worker service. Every shipped connector runs in the gateway’s in-process OpenAPI executor or federates over remote MCP. ADR-0024 specifies that a worker returns a bounded, bundle-declared request plan, and that the gateway validates it, injects the credential, and performs the HTTP itself — so that a worker never sees a vendor credential and never calls a vendor. That protocol is not implemented. The compiled WorkerExecutor still runs ADR-0014’s hand-off and would pass the vendor credential to a worker in x-obol-upstream-authorization. No worker exists to receive it, so no credential crosses that hop today. A worker never mints a receipt or assigns evidence trust on either path. See Trusted workers.

Dashboard — `apps/web`

Screens only. The browser talks to control and never to the gateway, and there is deliberately no browser-side playground: the dashboard hands over client configuration and shows the receipt that results. Human authentication is Clerk (ADR-0033); control still owns workspaces, membership, and every workspace_id, and Cedar remains solely the authorization engine for agent traffic.

How state flows

Postgres is the sole source of truth and is written only by control. The gateway holds no Postgres pool on the request path (ADR-0006).

State flow

Postgres (truth, control-owned)
   |  compile: keys + connections + tools + Cedar policy set and entities
   v
Snapshot publication  --(one Redis transaction, read back)-->  Redis
   |                                                             |
   |  invalidation on channel obol:invalidate                    |
   v                                                             v
Gateway snapshot cache  <-- MGET workspace bytes + policy bytes + fingerprint
   |
   |  request pins one immutable pair for its whole lifetime
   v
CEL visibility -> Cedar permission -> dispatch -> receipt
   |
   |  async: usage + invocation-audit envelopes on Redis streams
   v
Control workers flush to Postgres and Stripe meters

Three properties matter:

Snapshots are the only way state reaches the data plane. Control compiles a per-workspace bundle of keys, connections, tools, and the Cedar policy set with entities, publishes it to Redis, and the gateway hot-reloads. On a cache miss the gateway calls control’s internal snapshot API with a service JWT.
Publication is monotonic and fingerprinted. One Redis operation compares the workspace version in the candidate pair, refuses a lower version outright, and writes both snapshot keys, both invalidations, and the pair fingerprint together. A gateway reads the bytes and the fingerprint in one MGET and validates the exact-byte digest; a missing or mismatched fingerprint is not an admissible publication. Each cached pair is revalidated after 30 seconds, and once that window expires, reads fail closed until validation succeeds.
Writes off the hot path are async and batched. The gateway emits usage events and invocation-audit envelopes to Redis streams; control workers drain them into Postgres and Stripe meters. A token stream never blocks on a database write.

The accepted cost is eventual consistency: budgets lag by seconds, and revoke propagation is bounded at roughly 60 seconds, which is the product’s stated SLO rather than an implementation accident.

What lives in Redis

Key	Holds
`key:{sha256}`	`KeyContext` JSON, ~60s TTL, purged on revoke
`rpm:{key_id}:{yyyyMMddHH}`	Request-rate counter
`spend:{key_id}:{yyyyMM}`	Spend counter in micro-USD
`sess:{mcs_...}`	MCP `SessionRecord`, TTL equal to the idle timeout
`policy:{workspace_id}`	Compiled Cedar text, entities, and hash
`snap:workspace:{id}`	Connections, versioned connector bundles, tools
`idem:{workspace_id}:...`	Invocation ownership and terminal replay
`stream:usage`	Bounded usage event stream
`stream:invocation-audit:{workspace_id}`	Untrimmed receipt and verification envelopes
`stream:approvals`	Approval tickets, awaited on write, never gateway-trimmed

Every Redis key carries workspace_id, exactly as every Postgres row does. See Tenancy and identity.

These multi-key scripts target the current single-Redis deployment. Redis Cluster would need a later ADR and a live-key migration plan (ADR-0032).

Repo layout

Repository

apps/gateway      Rust data plane — axum; /v1/* + /mcp; auth, CEL, Cedar,
                  vault unwrap, session mux, metering emit
apps/control      Python control plane — FastAPI; tenants, keys, OAuth,
                  Cedar compile/publish, approvals, billing, audit
apps/connectors   reserved for Python MCP workers — README only, nothing built
apps/web          Next.js dashboard — talks only to control
apps/landing      Next.js marketing site — static, no backend deps
packages/proto    wire contracts (JSON schemas) — freeze before handlers
packages/cedar    Cedar schema + policies + fixture tests
packages/connectors  reviewed connector packs, plus the frozen catalog index
packages/fixtures synthetic connector traces; no live credentials
infra/            docker-compose (dev == Enterprise self-host), fly/, helm/
docs/             PRD, ARCHITECTURE, ADRs, GLOSSARY

packages/proto is the contract pack, and it is contract-first: freeze the schema before writing handlers, and if Rust and Python disagree on a field name, the contract pack is wrong. The JSON schemas are generated from apps/gateway/crates/obol-types with schemars and drift-checked in CI (ADR-0017).

Where the boundaries hold

Two boundaries are load-bearing enough to restate:

No Python and no product logic in the gateway. Generic OpenAPI and remote MCP execute in Rust. Messy first-party vendors are designed as out-of-process workers under ADR-0024; that protocol is not implemented, and the trusted-worker tier is parked. A feature that smells like product goes to control by default.
The gateway owns invocation identity, evidence classification, verification, and receipts. Executors report observations. A worker, a remote MCP server, or a catalog response is evidence, never a receipt, and no vendor or broker payload can promote its own evidence class (ADR-0019).

Invariants

The nine rules these boundaries encode.

Gateway overview

Request paths, listeners, and dispatch in detail.

Policy publishing

Compile, publish, fingerprint, hot-reload.

Deploy

Compose, Fly, and Helm targets.

Get started

Concepts

Gateway

Policy and authorization

Credentials and security

Connectors

Receipts and verification

Routing and catalog

Billing and observability

Enterprise self-host

Architecture

System shape

The deployables

Data plane — `apps/gateway`

Control plane — `apps/control`

Connectors — `apps/connectors` (reserved, not built)

Dashboard — `apps/web`

How state flows

What lives in Redis

Repo layout

Where the boundaries hold

Invariants

Gateway overview

Policy publishing

Deploy

​System shape

​The deployables

​Data plane — apps/gateway

​Control plane — apps/control

​Connectors — apps/connectors (reserved, not built)

​Dashboard — apps/web

​How state flows

​What lives in Redis

​Repo layout

​Where the boundaries hold

Invariants

Gateway overview

Policy publishing

Deploy

System shape

The deployables

Data plane — `apps/gateway`

Control plane — `apps/control`

Connectors — `apps/connectors` (reserved, not built)

Dashboard — `apps/web`

How state flows

What lives in Redis

Repo layout

Where the boundaries hold