---
title: "Metering"
description: "How a usage event travels from the gateway's meter, through the bounded Redis usage stream, into control's immutable usage facts — and what happens when that transport loses data."
---

Metering is the record of what your traffic consumed. The gateway records completion usage per upstream attempt and a separate first-dispatch fact per logical call, hands them to a bounded async queue, and returns. Control drains that queue into Postgres out of band. Nothing in this path is allowed to block a token stream — that is [ADR-0006](/concepts/invariants), and it shapes every design choice below.

<Note>
Metering is not the same thing as Obol's invoice. The cost figures in a usage event are your estimated *vendor* cost from your own pricebook. The design-partner offer charges per startup plus logical dispatched tool/API and LLM requests, on the grandfathered **$1 / $3 / $10 / $1** per 1,000 meters — not the public **$0.30 / $1.00 / $3.00 / $0.10** ladder. Both schedules are named on [Stripe Billing](/billing/stripe).
</Note>

## What is metered

A usage event has one of two kinds, from `packages/proto/usage_event.schema.json`:

| Kind | Emitted for | Cost basis |
|---|---|---|
| `llm` | One inbound model request on `/v1/chat/completions`, `/v1/responses`, `/v1/embeddings`, `/v1/messages` | Token counts × the workspace pricebook's per-million-token rates |
| `tool` | One logical tool invocation over `/mcp` | A configured fixed per-call rate × `tool_calls` |

Every event carries `workspace_id`, `key_id`, `request_id`, `kind`, `usd_micros`, `pricing_status`, `latency_ms`, HTTP `status`, and a `partial` flag; `llm` events add `model`, `provider`, and the token block; `tool` events add `tool`, `tool_calls`, and the `receipt_id` when the call produced a [receipt](/receipts/overview).

Token counts are normalized to the "input excludes cached" convention: `tokens_in` never double-counts `tokens_cached`, and `tokens_reasoning` is informational only — providers already fold it into `tokens_out`, so it is never billed twice.

### Priced versus unpriced

`pricing_status` is explicit, because a numeric zero is ambiguous. `known` means the workspace pricebook had a rate for this model or tool; `unknown` means it did not, and the `usd_micros` zero is a sentinel rather than a measurement. Aggregates keep the two apart ([usage queries](/billing/usage) report a known subtotal plus an `unpriced_count`).

## In the gateway: the `obol-meter` crate

`apps/gateway/crates/obol-meter` owns pricing arithmetic, admission counters, and event emission. It never touches a credential — events carry ids and counts only.

### Cost

`Pricing::cost` works in integer micro-USD. Each term is computed in `i128`, rounded half-up, and the sum saturates at `i64::MAX`. Cached tokens use the pricebook's `cached_input_per_mtok` when present and the input rate otherwise. There is no floating point anywhere on this path.

### Admission and reservation

Before an upstream is opened, `Limiter` charges rate limits and reserves spend against Redis fixed windows. Any store error fails closed — Redis unreachable means `not_ready`, not "allow and hope".

| Counter | Window | Key shape |
|---|---|---|
| Requests | UTC hour | `rpm:{key_id}:{yyyyMMddHH}` |
| Tokens | UTC minute | reservation window `{yyyyMMddHHmm}` |
| Spend | UTC calendar month | `spend:{key_id}:{yyyyMM}` |

<Warning>
The request limit is enforced as `rpm × 60` requests per UTC hour, not per rolling minute. A burst can spend the whole hour's allowance early, and `retry_after_s` is then the time to the next hour boundary.
</Warning>

A model request reserves a pessimistic token bound and maximum cost atomically (`admit_llm`), then reconciles down to the observed usage when the request finishes. A tool call reserves its configured fixed price immediately before the first upstream dispatch (`reserve_tool`, ADR-0046) and reserves no tokens at all. A budgeted tool call with no configured price fails closed with `PricingUnavailable` rather than executing for free.

Reconciliation is tracked, not fire-and-forget: the drain sequence waits for outstanding reconciliations before closing the meter queue. If reconciliation fails, the pessimistic reservation stays charged and `reconciliation_failures` increments — the safe direction.

### The `UsageGuard`

One `UsageGuard` exists per upstream attempt and emits exactly one usage event on `Drop`. That covers normal completion, panic unwind, and future cancellation alike, so a client that disconnects mid-stream still produces a usage event — flagged `partial: true`, since the upstream never signalled completion.

### Dispatch facts

A `UsageDispatchEvent` records the gateway-owned logical identity, first dispatch timestamp, and reviewed tool effect class. It is emitted when dispatch starts, before completion, so a long-running request does not defer ordinary billing ingestion to the next period. Completion events retain the same metadata for recovery. LLM fallbacks retain one logical identity while their token and estimated vendor cost events remain separate.

This fact uses the same asynchronous transport and its loss limitations. It is not a durable proof that the source is complete. Usage-bearing design-partner invoices require operator review even after observed totals reconcile.

### The async emitter

`Meter` is a bounded `mpsc` channel feeding a batching flusher:

| Parameter | Value |
|---|---|
| Queue capacity | 4096 events |
| Batch size | 64 events |
| Batch linger | 50 ms |

Enqueue is a non-blocking `try_send`. When the queue is full the event is dropped, counted, and logged — a response is never delayed to make room. Emission retries once; a second failure drops the event and increments the same counter. That counter surfaces as `obol_meter_dropped_total` on `/metrics` (see [Telemetry](/observability/telemetry)), and it is the one usage-loss mode that happens *before* the transport and is therefore invisible to control's loss latch.

### The transport

The flusher writes each event to the Redis stream `stream:usage` with `XADD ... MAXLEN ~ 1000000`. The entry envelope is three fields — `type` (`usage` or `dispatch`), `workspace_id`, and the JSON `payload`. The stream is shared infrastructure; tenant identity rides in every envelope.

Because the stream is capped, it is a *bounded* transport. Sustained ingest lag can trim entries that control has not yet consumed. Detecting that is the whole point of the next section.

## In control: ingest

`apps/control/app/services/usage_ingest.py` runs inside the arq worker as `drain_usage_events`, scheduled every five seconds (`second={0,5,10,...,55}`, `run_at_startup=True`).

<Steps>
  <Step title="Sample the stream for loss">
    Read the consumer group's position and compare the max deleted entry id against the last delivered id. Any gap latches immediately.
  </Step>
  <Step title="Reclaim stalled entries">
    `XAUTOCLAIM` on the `control-usage` group for entries idle longer than `OBOL_USAGE_CLAIM_IDLE_MS` (default 60000), resuming from an in-process scan position so old pending entries are not starved behind fresh ones.
  </Step>
  <Step title="Read fresh entries">
    `XREADGROUP` fills the remainder of the batch (`OBOL_USAGE_BATCH`, default 256), using the same atomic sampling-and-latching script so a trim between claim and read cannot hide loss.
  </Step>
  <Step title="Project and store">
    Each entry is validated against the frozen JSON Schema and projected into an immutable `usage_events` row. Legacy unenrolled tool traffic retains its `usage_meter_delivery` path. A design-partner dispatch fact creates a separately deduplicated immutable billing allocation; completion metadata can recover the same fact. Enrolled traffic never enters both meter paths.
  </Step>
  <Step title="Commit, then acknowledge">
    The SQL commit happens before `XACK`. A crash between them replays the entry, and the event id's primary key makes the replay a no-op duplicate.
  </Step>
</Steps>

### Per-entry outcomes

| Outcome | Cause |
|---|---|
| `stored` | Valid, tenant-scoped, first sight of this `event_id` |
| `duplicate` | The `event_id` already exists for the same workspace |
| `quarantined` | Unknown workspace, bad envelope, invalid JSON, schema or bounds violation, or a payload `workspace_id` that disagrees with the envelope |
| `discarded_unscoped` | The envelope carries no `ws_…` workspace id at all, so there is no tenant to attribute it to |

Only the trusted *outer* envelope attribute decides the tenant. A quarantine row stores the stream entry id, the workspace, a reason code, and a fixed diagnostic string — never the rejected event id and never the payload, because an id-shaped field in a rejected payload could carry injected content.

### Loss latching (ADR-0045)

Redis `XAUTOCLAIM` removes deleted pending entries from the pending list while returning their ids. If the SQL marker commit then failed, the only evidence that usage went missing would be gone. So the evidence is made durable in Redis *before* it is consumed.

The latch is a sentinel consumer group named `control-usage:loss-detected` on the existing `stream:usage` key. It holds only shared transport health: no tenant identifier, no payload, no credential, no receipt. A Redis script samples stream and group metadata and latches any gap before returning entries; the claim script creates the sentinel before `XAUTOCLAIM`'s destructive pending-list cleanup, and only a clean claim that found no loss may remove a temporary sentinel it created itself.

Once latched, the state survives every later healthy call, every SQL failure, and every worker restart. Each drain propagates it into a sticky `usage_ingestion_gap_at` stamp on every existing workspace, with two visible consequences:

- Usage queries report `"completeness": "gap_detected"` instead of `"complete"`.
- Stripe meter delivery for that workspace pauses (`paused_gap`) rather than reporting a knowingly incomplete count.

<Warning>
Ingestion never clears the latch or the workspace gap markers. Clearing them requires reconciliation against a durable usage source, which does not exist yet — there is no automatic recovery and no operator mutation endpoint for it.
</Warning>

The latch's guarantees stop at the transport. Deleting or replacing the Redis instance, and events dropped by the gateway's bounded queue before they ever reached the stream, are both outside what it can observe.

## Configuration

```bash Control worker environment
OBOL_USAGE_CONSUMER_GROUP=control-usage
OBOL_USAGE_CONSUMER_NAME=control-worker-1
OBOL_USAGE_BATCH=256
OBOL_USAGE_CLAIM_IDLE_MS=60000
```

These are set for the `control-worker` service in `infra/docker-compose.yml`; see [Docker Compose](/deploy/docker-compose).

## What this is, and is not

This pipeline produces a **bounded-source projection**, not invoice-complete accounting. It is eventually consistent by construction: budgets lag by seconds, the usage read model lags by a drain cycle, and the read model tells you when it knows it is incomplete rather than presenting a gap as a total.

<Columns cols={2}>
  <Card title="Stripe Billing" icon="credit-card" href="/billing/stripe">
    What actually reaches Stripe, and why it is never your payment volume.
  </Card>
  <Card title="Querying usage" icon="chart-line" href="/billing/usage">
    The usage summary endpoint, its windows, and budget posture.
  </Card>
</Columns>