---
title: "Model providers"
description: "Connect BYOK model keys to a workspace, map client model names onto upstream models, and understand how the gateway selects and falls back."
---

A model provider connection binds one **model pattern** that clients ask for to one **upstream model** at one provider, using a key you supply. The key is sealed by the gateway into the vault and is never returned by any read.

Model connections are ordinary rows in the same `connections` table as tool connections, distinguished by a non-null `model_config`. They are managed through their own routes in `apps/control/app/api/model_providers.py`.

## Supported providers

| `provider` | Display name | `provider_kind` | Base URL | Auth scheme |
|---|---|---|---|---|
| `openai` | OpenAI | `openai_compat` | `https://api.openai.com/v1` | Bearer |
| `anthropic` | Anthropic | `anthropic` | `https://api.anthropic.com` | `x-api-key` header |
| `openrouter` | OpenRouter | `openai_compat` | `https://openrouter.ai/api/v1` | Bearer |
| `openai_compatible` | OpenAI-compatible | `openai_compat` | You supply it | Bearer |

The base URL is fixed by the provider definition for the first three; only `openai_compatible` reads `base_url` from the request, and it is validated hard: HTTPS only, no userinfo, no query, no fragment, a port in range if present, and a path ending in `/v1`. The authority is lowercased and the trailing slash removed before storage.

`provider_kind` is what the gateway's translation table keys on. `obol-types` also defines `openai_responses`, but no control-plane provider definition produces it today.

## Create a connection

```bash Connect a model provider
curl -X POST https://control.tryobol.dev/api/v1/workspaces/ws_123/model-providers \
  -H "Authorization: Bearer $OPERATOR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "provider": "anthropic",
    "api_key": "sk-ant-...",
    "model_pattern": "claude-*",
    "upstream_model": "claude-sonnet-4-5"
  }'
```

Requires the `connection.manage` operator action. The response projects the connection without the credential:

```json Response
{
  "schema_version": 1,
  "connection": {
    "schema_version": 1,
    "connection_id": "conn_...",
    "workspace_id": "ws_123",
    "provider": "anthropic",
    "display_name": "Anthropic",
    "model_pattern": "claude-*",
    "upstream_model": "claude-sonnet-4-5",
    "base_url": "https://api.anthropic.com",
    "status": "active",
    "last4": "...",
    "credential_revision": 1,
    "publication": { "state": "published", "requested_version": 12, "published_version": 12 }
  }
}
```

### What happens in order

<Steps>
<Step title="Validate">
The pattern and upstream model are checked for length, control characters, and whitespace. A pattern may contain at most one `*`, and only as the whole string, a prefix, or a suffix. An upstream model may contain none.
</Step>
<Step title="Refuse a duplicate pattern">
Under the workspace lock, a second non-disabled connection claiming the same `model_pattern` is a `409`. Two rows matching one pattern would make selection order-dependent.
</Step>
<Step title="Insert pending and publish">
The row is inserted with `status: pending` and no credential, and the snapshot is republished. A pending row publishes no model, so nothing is routable yet.
</Step>
<Step title="Seal">
Only once the published version covers the workspace's current version does control ask the gateway to seal the key (`POST /internal/v1/credentials/seal`), under a single-use capability token bound to the workspace, the connection, the auth-profile digest, and the expected credential revision.
</Step>
<Step title="Activate and republish">
The sealed envelope is applied, the row goes `active`, and the snapshot is published again — this time carrying the model.
</Step>
</Steps>

If the first publication does not settle, the route returns the pending projection and no key is ever sent to the gateway. Plaintext never touches the control plane's database, and the only durable trace is `last4`.

<Warning>
A key beginning `sk_live_` is refused at every sealing entry point, including this one. Invariant 8 forbids storing a full live vendor key; use a restricted key.
</Warning>

## List and disable

```bash List model providers
curl https://control.tryobol.dev/api/v1/workspaces/ws_123/model-providers \
  -H "Authorization: Bearer $OPERATOR_TOKEN"
```

```bash Disable one
curl -X POST https://control.tryobol.dev/api/v1/workspaces/ws_123/model-providers/conn_abc/disable \
  -H "Authorization: Bearer $OPERATOR_TOKEN"
```

Listing requires `workspace.read`; disabling requires `connection.manage`. Disable is idempotent: an already-disabled row is projected back unchanged with no audit row and no republish. Disabling sets `status: disabled`, which drops the model from the next snapshot; the credential ciphertext stays on the row. To destroy the stored credential without destroying the record, see [Credential brokering](/security/credential-brokering).

Every mutation writes an audit row — `MODEL_CONNECTION_START`, `MODEL_CONNECTION_CREDENTIAL_STORE`, `MODEL_CONNECTION_DISABLE` — carrying the provider, the pattern, the status, and `last4`, never the key.

## How a model reaches the gateway

`models_from_graph` publishes a snapshot entry only for a connection that is `active`, has a `model_config`, has a `model_provider` target, and holds a credential envelope. The entry is:

```json Snapshot model entry
{
  "name": "claude-*",
  "provider": "anthropic",
  "base_url": "https://api.anthropic.com",
  "upstream_model": "claude-sonnet-4-5",
  "credential": { "...": "sealed envelope" },
  "connection_id": "conn_...",
  "fallbacks": []
}
```

The snapshot's `name` is the **pattern**, not a model name. The gateway matches the client's `model` string against it.

## Selection

`ModelRouter::resolve` in `apps/gateway/crates/obol-route/src/lib.rs`:

1. An exact match on `name` wins.
2. Otherwise the first entry whose `name` contains `*` and glob-matches, in snapshot order.
3. Otherwise `ModelNotFound`, rendered to the client as "model is not available in this workspace".

The price entry is looked up for the matched pattern first, then for the literally requested model name.

<Note>
This is not cost-and-quality scoring. Deterministic scoring on price, quality, coverage, auth mode, and surface applies to **tool routing** through capabilities, not to model selection. See [Capabilities](/routing/capabilities).
</Note>

Before the router runs, the CEL prefilter decides whether the key may see the model at all, and Cedar's `complete` action decides whether it may call it. A model the key cannot see is refused with `ModelNotAllowed` — the same code an unavailable model produces, so the surface is not a catalogue oracle.

## Fallbacks

`ModelSnapshot.fallbacks` is a list of other model names in the same snapshot. The gateway builds an ordered candidate list from the primary plus every fallback that passes **its own** CEL visibility check and **its own** Cedar `complete` decision, then attempts them in order in `apps/gateway/crates/obol-gateway/src/routes/v1.rs`.

A candidate is skipped and the next tried when:

- The upstream host fails the netguard egress check.
- Its connection is missing, inactive, or carries no credential envelope.
- The vault unwrap fails.
- The upstream call fails with a retryable error and this is not the last candidate.

A non-retryable upstream error returns immediately rather than shopping the request around. Admission reserves the **maximum** price across every eligible candidate, so a cheaper primary cannot understate what a fallback would cost, and a budgeted key is refused with `NotReady` if any eligible candidate has no published price.

<Warning>
The control plane writes `"fallbacks": []` on every model connection and exposes no route to set them, so the fallback chain is always empty in practice. The gateway-side machinery above is implemented and exercised, but nothing populates it today.
</Warning>

{/* TODO: confirm whether a fallback-configuration endpoint is planned, or whether fallbacks are intended to be operator-authored some other way. Nothing in `apps/control` or `apps/web` writes the field. */}

Model fallback is a distinct mechanism from capability fallback. `allow_fallbacks` on a route qualifier is reserved and acted on by nothing (ADR-0027's acceptance note); see [Capabilities](/routing/capabilities).

## Pricing and budgets

Model prices live in the workspace pricebook (`PUT /api/v1/workspaces/{id}/pricing`) and ride in the snapshot as `pricing`. They feed admission reservation and the usage record, not selection. See [Metering](/billing/metering).

## Related pages

<Columns cols={2}>
  <Card title="LLM routes" href="/gateway/llm-routes">
    The `/v1/chat/completions`, `/v1/responses`, `/v1/embeddings`, and `/v1/messages` surfaces.
  </Card>
  <Card title="Vault" href="/security/vault">
    How the sealed credential is stored and unwrapped.
  </Card>
</Columns>