---
title: "Credential vault"
description: "Envelope encryption for customer vendor credentials: the AES-256-GCM wire format, KEK providers, and why plaintext exists only inside the gateway process."
---

The vault holds customer vendor credentials under envelope encryption. Two
planes touch it and their roles are asymmetric: control seals and persists
ciphertext, and **the gateway is the only process that ever decrypts**
(ADR-0004, ADR-0016).

<Warning>
**Plaintext decrypt happens only inside the gateway process.** A plaintext
vendor secret exists between `Vault::unwrap` and the outbound socket, and
nowhere else. Control's process, task logs, and crash dumps never contain a
vendor token; neither does the web app, an agent, or a worker.
</Warning>

## What is stored

An `EncryptedCredential` is the only credential shape that reaches Postgres or a
snapshot (`apps/gateway/crates/obol-vault`, `obol-types::snapshot`):

| Field | Contents |
|---|---|
| `ciphertext_b64` | base64 (standard, padded) of `nonce(12) ‖ ciphertext ‖ tag(16)` |
| `dek_wrapped_b64` | base64 of the KEK-wrapped 32-byte DEK; inner format depends on the provider |
| `kek_provider` | `local`, `aws`, or `gcp` — must equal the gateway's configured provider |
| `kms_key_id` | Optional KMS key id override (AWS); `null` for `local` |
| `scheme` | `bearer`, `basic`, `header{name,prefix}`, or `query{param}` |
| `last4` | Last four characters of the plaintext — the only fragment ever logged |

The plaintext counterpart, `OutboundCredential`, deliberately implements neither
`Serialize` nor `JsonSchema`. That is enforced at compile time with a
`compile_fail` doctest, so it cannot reach a snapshot, a log line, or a response
body by accident.

## Wire format

AES-256-GCM with a fixed layout, shared byte-for-byte with control's Python side
so that either language can produce a credential the other reads:

- Nonce: 96 bits, freshly random per seal.
- Tag: 128 bits.
- AAD: empty.
- Plaintext: the UTF-8 bytes of the credential string.
- One fresh DEK per seal.

```python
from cryptography.hazmat.primitives.ciphers.aead import AESGCM
dek = os.urandom(32)
nonce = os.urandom(12)
ciphertext_b64 = b64encode(nonce + AESGCM(dek).encrypt(nonce, secret.encode(), None))
```

A known-answer test pins the layout: key = 32 × `0x01`, nonce = 12 × `0x02`,
plaintext `"obol"` produces
`0202…02 68b4a625 739201b722fde75876f1b38a0103f51d`
(`obol-vault/src/aead_tests.rs::aead_wire_format_matches_python_reference`).

Inputs shorter than 28 bytes are rejected as `InvalidFormat` before the cipher
runs; any authentication failure is `DecryptionFailed`.

## KEK providers

The DEK is wrapped by a key-encryption key behind
`trait Kek { provider, unwrap_dek, wrap_dek }`, selected by
`OBOL_KEK_PROVIDER` through `kek_from_env`.

| Provider | Wrapped DEK | Status |
|---|---|---|
| `local` (`LocalAgeKek`, feature `kek-local`, default) | Binary (non-armored) age v1 file encrypted to an x25519 recipient; identity read from `OBOL_AGE_KEY_FILE` | Shipped |
| `aws` (`AwsKmsKek`, feature `kek-aws`) | KMS `Encrypt`/`Decrypt` `CiphertextBlob`; key id from the credential's `kms_key_id`, else the configured default | Shipped |
| `gcp` (`GcpKmsKek`) | — | Not implemented; construction always returns `KekUnavailable` |

Every misconfiguration fails closed: `local` without `OBOL_AGE_KEY_FILE`, `aws`
without `OBOL_KMS_KEY_ID` or without the feature compiled in, and `gcp` in all
cases return `VaultError::KekUnavailable`. A credential whose `kek_provider`
differs from the configured provider is refused with `WrongProvider` **before
any decrypt is attempted**.

{/* TODO: AWS KMS behaviour is verified by construction only — ADR-0016 records no live KMS test. Confirm before documenting operational guidance for the `aws` provider. */}

## Unwrap path

`Vault::unwrap` performs, in order:

1. Provider match, or `WrongProvider`.
2. Base64 decode of the wrapped DEK and the sealed credential; a sealed value
   shorter than nonce plus tag is `Malformed`.
3. DEK resolution through the cache, else a KEK unwrap.
4. AES-256-GCM open.
5. UTF-8 decode; a failure zeroizes the bytes and returns `Malformed`.
6. A `last4` comparison against the stored `last4`. A mismatch logs one `warn!`
   carrying the expected `last4` only, and refuses.

`Vault::unwrap_checked` additionally refuses a credential whose `expires_at` is
at or before `now`, checked before any KEK work. Call sites pass the
connection's `expires_at` (`obol-gateway/src/hooks.rs`,
`routes/v1.rs`, `verification.rs`).

## Caching

Unwrapped DEKs are cached in `moka`, keyed by `sha256(dek_wrapped)`, with
`DEK_CACHE_TTL = 5 minutes` and a 10,000-entry ceiling. The cache holds
`Arc<SecretBox<[u8; 32]>>` and nothing else.

<Note>
A plaintext credential is never cached. Two test hooks exist solely to prove it:
`cache_entry_lens` (every entry is exactly 32 bytes) and `cache_contains_bytes`,
used by `dek_cache_never_holds_plaintext_credential`.
</Note>

The five-minute TTL is a deliberate trade: it bounds KEK revocation lag while
avoiding one KMS `Decrypt` per tool call.

## What never leaves the gateway process

- No plaintext credential is serialized, cached, or returned to control.
- Key material is held in `SecretBox`/`SecretString` and zeroized on drop; every
  `Debug` implementation is redacted (`Aes256Gcm([REDACTED])`,
  `LocalAgeKek([REDACTED])`, `OutboundCredential { secret: "[REDACTED]" }`).
- `VaultError` maps to a generic 5xx without echoing details.
- Logs carry the connector or connection id and `last4` only.

## Injection on the outbound hop

`OutboundCredential::apply` is the single place header and query mutation
happens (`obol-types/src/credential.rs`). It writes one of:

- `Authorization: Bearer <secret>`
- `Authorization: Basic <secret>`
- a named header, optionally with a prefix
- a query parameter

Header values are marked `sensitive` so the HTTP stack keeps them out of debug
output. Callers drop the plaintext immediately after.

## Sealing

Control never seals a credential itself. It calls the gateway's internal
`POST /internal/v1/credentials/seal` with a short-lived, single-use capability;
the gateway seals and returns ciphertext plus a `credential_revision`, and
control persists only that envelope
(`obol-gateway/src/routes/credentials.rs`, `apps/control/app/services/connections.py`).

Two custody rules are enforced at that boundary rather than in a runbook:

- Control refuses any sealable secret beginning with `sk_live_`
  (`validate_sealable_secret`), and the gateway refuses a Stripe API-key seal
  beginning with `sk_live_` again on its own side.
- A federated connection cannot accept a vendor secret at all: sealing is
  rejected unless the auth profile is `federated_broker`.

That is invariant 8 in code — no pooled vendor accounts, no full `sk_live_`
storage; restricted keys, OAuth, or Connect only. See
[/security/oauth](/security/oauth) and [/concepts/invariants](/concepts/invariants).

## Enterprise self-host

For development and [Enterprise self-host](/get-started/self-hosting), `local`
is the default: an age x25519 identity file at `OBOL_AGE_KEY_FILE`. `packages/fixtures/keys/age-test.key` is a
committed test identity and must never guard real data. See
[/get-started/self-hosting](/get-started/self-hosting).