The vault holds customer vendor credentials under envelope encryption. Two planes touch it and their roles are asymmetric: control seals and persists ciphertext, and the gateway is the only process that ever decrypts (ADR-0004, ADR-0016).
Plaintext decrypt happens only inside the gateway process. A plaintext vendor secret exists between Vault::unwrap and the outbound socket, and nowhere else. Control’s process, task logs, and crash dumps never contain a vendor token; neither does the web app, an agent, or a worker.

What is stored

An EncryptedCredential is the only credential shape that reaches Postgres or a snapshot (apps/gateway/crates/obol-vault, obol-types::snapshot): The plaintext counterpart, OutboundCredential, deliberately implements neither Serialize nor JsonSchema. That is enforced at compile time with a compile_fail doctest, so it cannot reach a snapshot, a log line, or a response body by accident.

Wire format

AES-256-GCM with a fixed layout, shared byte-for-byte with control’s Python side so that either language can produce a credential the other reads:
  • Nonce: 96 bits, freshly random per seal.
  • Tag: 128 bits.
  • AAD: empty.
  • Plaintext: the UTF-8 bytes of the credential string.
  • One fresh DEK per seal.
A known-answer test pins the layout: key = 32 × 0x01, nonce = 12 × 0x02, plaintext "obol" produces 0202…02 68b4a625 739201b722fde75876f1b38a0103f51d (obol-vault/src/aead_tests.rs::aead_wire_format_matches_python_reference). Inputs shorter than 28 bytes are rejected as InvalidFormat before the cipher runs; any authentication failure is DecryptionFailed.

KEK providers

The DEK is wrapped by a key-encryption key behind trait Kek { provider, unwrap_dek, wrap_dek }, selected by OBOL_KEK_PROVIDER through kek_from_env. Every misconfiguration fails closed: local without OBOL_AGE_KEY_FILE, aws without OBOL_KMS_KEY_ID or without the feature compiled in, and gcp in all cases return VaultError::KekUnavailable. A credential whose kek_provider differs from the configured provider is refused with WrongProvider before any decrypt is attempted.

Unwrap path

Vault::unwrap performs, in order:
  1. Provider match, or WrongProvider.
  2. Base64 decode of the wrapped DEK and the sealed credential; a sealed value shorter than nonce plus tag is Malformed.
  3. DEK resolution through the cache, else a KEK unwrap.
  4. AES-256-GCM open.
  5. UTF-8 decode; a failure zeroizes the bytes and returns Malformed.
  6. A last4 comparison against the stored last4. A mismatch logs one warn! carrying the expected last4 only, and refuses.
Vault::unwrap_checked additionally refuses a credential whose expires_at is at or before now, checked before any KEK work. Call sites pass the connection’s expires_at (obol-gateway/src/hooks.rs, routes/v1.rs, verification.rs).

Caching

Unwrapped DEKs are cached in moka, keyed by sha256(dek_wrapped), with DEK_CACHE_TTL = 5 minutes and a 10,000-entry ceiling. The cache holds Arc<SecretBox<[u8; 32]>> and nothing else.
A plaintext credential is never cached. Two test hooks exist solely to prove it: cache_entry_lens (every entry is exactly 32 bytes) and cache_contains_bytes, used by dek_cache_never_holds_plaintext_credential.
The five-minute TTL is a deliberate trade: it bounds KEK revocation lag while avoiding one KMS Decrypt per tool call.

What never leaves the gateway process

  • No plaintext credential is serialized, cached, or returned to control.
  • Key material is held in SecretBox/SecretString and zeroized on drop; every Debug implementation is redacted (Aes256Gcm([REDACTED]), LocalAgeKek([REDACTED]), OutboundCredential { secret: "[REDACTED]" }).
  • VaultError maps to a generic 5xx without echoing details.
  • Logs carry the connector or connection id and last4 only.

Injection on the outbound hop

OutboundCredential::apply is the single place header and query mutation happens (obol-types/src/credential.rs). It writes one of:
  • Authorization: Bearer <secret>
  • Authorization: Basic <secret>
  • a named header, optionally with a prefix
  • a query parameter
Header values are marked sensitive so the HTTP stack keeps them out of debug output. Callers drop the plaintext immediately after.

Sealing

Control never seals a credential itself. It calls the gateway’s internal POST /internal/v1/credentials/seal with a short-lived, single-use capability; the gateway seals and returns ciphertext plus a credential_revision, and control persists only that envelope (obol-gateway/src/routes/credentials.rs, apps/control/app/services/connections.py). Two custody rules are enforced at that boundary rather than in a runbook:
  • Control refuses any sealable secret beginning with sk_live_ (validate_sealable_secret), and the gateway refuses a Stripe API-key seal beginning with sk_live_ again on its own side.
  • A federated connection cannot accept a vendor secret at all: sealing is rejected unless the auth profile is federated_broker.
That is invariant 8 in code — no pooled vendor accounts, no full sk_live_ storage; restricted keys, OAuth, or Connect only. See /security/oauth and /concepts/invariants.

Enterprise self-host

For development and Enterprise self-host, local is the default: an age x25519 identity file at OBOL_AGE_KEY_FILE. packages/fixtures/keys/age-test.key is a committed test identity and must never guard real data. See /get-started/self-hosting.