Helm

Self-hosted deployments are available on the Enterprise plan. See self-hosting.

infra/helm/obol is the Kubernetes self-host path (ADR-0009). The default posture is bring-your-own Postgres, Redis, and KMS; optional Bitnami subcharts plus a local age key give a self-contained install. The topology mirrors the Compose stack: gateway ×2, control API ×1, control worker ×1, web ×1. There is no connectors workload — see Trusted workers.

The chart deploys no connectors workload. templates/connectors.yaml, its connectors values block and ServiceAccount, and the NetworkPolicy that guarded it were all removed: apps/connectors/ ships no package and no image, so the template rendered a Deployment that could never pull one, and a NetworkPolicy whose workload did not exist. The chart sets no OBOL_CONNECTORS_URL; an operator running a worker of their own supplies it through gateway.extraEnv.

Chart structure

infra/helm/obol/
├── Chart.yaml              # obol 0.1.0, appVersion 0.1.0
├── Chart.lock
├── values.yaml             # BYO Postgres/Redis/KMS
├── values-selfhost.yaml    # in-cluster Postgres + Redis, local age KEK
├── charts/                 # vendored postgresql 15.5.x, redis 20.1.x
├── files/wait-for-database.py
└── templates/
    ├── gateway.yaml        # Deployment, Service, HPA, PDB
    ├── control.yaml        # API Deployment, worker Deployment, migration Job
    ├── web.yaml
    ├── bootstrap.yaml      # ConfigMap holding wait-for-database.py
    ├── secret.yaml         # rendered only when secrets.values is set
    ├── ingress.yaml
    ├── networkpolicy.yaml
    ├── serviceaccount.yaml
    ├── _helpers.tpl
    └── NOTES.txt

Both subcharts are conditional: postgresql.enabled and redis.enabled, both false by default.

Installing

Build chart dependencies

Only needed when you enable the in-cluster PostgreSQL or Redis subcharts.

helm dependency update infra/helm/obol

Create the application secret

The chart expects an existing Secret with these keys: DATABASE_URL, REDIS_URL, OBOL_SERVICE_JWT_SECRET, STRIPE_SECRET_KEY, STRIPE_WEBHOOK_SECRET, CLERK_SECRET_KEY. Stripe and Clerk keys are optional lookups; the service JWT secret is not.

kubectl -n obol create secret generic obol-secrets \
  --from-literal=DATABASE_URL='postgresql+psycopg://obol:...@db:5432/obol' \
  --from-literal=REDIS_URL='redis://redis:6379/0' \
  --from-literal=OBOL_SERVICE_JWT_SECRET="$(openssl rand -hex 32)"

Point the chart at it with secrets.existingSecret. Use External Secrets or SOPS in production. secrets.values renders an inline Secret and is labelled dev convenience only.

Install

helm upgrade --install obol infra/helm/obol -n obol --create-namespace \
  --wait --wait-for-jobs --timeout 15m \
  --set secrets.existingSecret=obol-secrets \
  --set control.clerk.issuer=https://YOUR_INSTANCE.clerk.accounts.dev \
  --set urls.web=https://app.example.com \
  --set kek.aws.keyId=arn:aws:kms:us-east-1:...:key/... \
  --set kek.aws.region=us-east-1 \
  --set ingress.hosts.gateway=gateway.example.com

Use --wait --wait-for-jobs and a Helm timeout longer than control.migrations.timeoutSeconds, so a failed bootstrap is reported rather than left running.

Check readiness

kubectl -n obol get pods -l app.kubernetes.io/instance=obol

Self-contained install

values-selfhost.yaml enables the in-cluster PostgreSQL and Redis subcharts, switches the KEK to a local age identity, and uses .obol.local hosts without TLS.

kubectl -n obol create secret generic obol-age-key --from-file=age.key
helm dependency update infra/helm/obol
helm upgrade --install obol infra/helm/obol -n obol --create-namespace \
  -f infra/helm/obol/values-selfhost.yaml \
  --set secrets.values.OBOL_SERVICE_JWT_SECRET="$(openssl rand -hex 32)"

Set kek.local.recipient to the public age recipient matching that identity, so control can seal credentials it cannot open.

Values

URLs and images

Value	Default	Notes
`global.imageRegistry`	`ghcr.io/obol`
`global.imageTag`	`""`	Falls back to `.Chart.AppVersion`
`global.env`	`prod`	Becomes `OBOL_ENV`
`urls.gateway`	`https://gateway.obol.example`	Also the MCP JWT audience
`urls.control`	`https://control.obol.example`	Also the MCP JWT issuer
`urls.web`	`https://app.obol.example`	Default for `control.clerk.authorizedParties`

Key-encryption key

kek:
  provider: aws          # aws | gcp | local
  aws: { keyId: "", region: "" }
  gcp: { keyName: "" }
  local:
    existingSecret: ""   # Secret with key `age.key`; mounted only in gateway
    recipient: ""        # public recipient exposed to control, encrypt-only

kek.aws.keyId is required when the provider is aws, and kek.gcp.keyName when it is gcp; rendering fails otherwise. With provider: local, the age Secret is mounted at /run/secrets/age.key with mode 0400 in the gateway pod only. Control receives OBOL_AGE_RECIPIENT and nothing more. See vault and key management.

Workloads

Value	Default
`gateway.replicaCount`	`2`
`gateway.port` / `gateway.metricsPort`	`8080` / `9091`
`gateway.idempotencyTtlSeconds`	`86400`
`gateway.autoscaling.enabled`	`false` (min 2, max 10, 70% CPU)
`gateway.pdb`	enabled, `minAvailable: 1`
`gateway.terminationGracePeriodSeconds`	`30`
`control.api.replicaCount` / `control.worker.replicaCount`	`1` / `1`
`control.migrations.enabled`	`true`
`control.migrations.backoffLimit`	`3`
`control.migrations.timeoutSeconds`	`600`
`control.clerk.issuer`	`""` — set it, or `/api/v1` answers `503`
`control.forwardedAllowIps`	`127.0.0.1,::1`
`web.replicaCount`	`1`

Every workload accepts extraEnv, nodeSelector, tolerations, affinity, and resources.

web.clerkPublishableKey is a runtime value and cannot change a compiled client bundle. Build the web image with the matching NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY and public URLs.

Security context

Applied to every pod and container:

podSecurityContext:
  runAsNonRoot: true
  runAsUser: 10001
  fsGroup: 10001
  seccompProfile: { type: RuntimeDefault }
containerSecurityContext:
  allowPrivilegeEscalation: false
  readOnlyRootFilesystem: true
  capabilities: { drop: [ALL] }

Because the root filesystem is read-only, each pod mounts an emptyDir at /tmp.

Service accounts

serviceAccounts:
  gateway: { create: true, name: "", annotations: {} }
  control: { create: true, name: "", annotations: {} }
  web:     { create: true, name: "", annotations: {} }

Three separate identities, so a gateway KMS decrypt role cannot flow to another workload. Attach KMS IAM (IRSA or Workload Identity) only to gateway and control, and scope each independently — gateway needs decrypt, control needs encrypt. When create: false, name is required for that component.

Migration lifecycle

The migration is a normal, revision-named Job, not a pre-install hook (ADR-0043). Helm can therefore create the ServiceAccount, Secret, ConfigMap, and optional PostgreSQL subchart before the Job’s Pod starts.

The Job’s init container runs wait-for-database.py database — a bounded, read-only connectivity check.
The Job container runs alembic upgrade head.
New control API and worker pods run wait-for-database.py schema as their own init container, waiting until the database’s complete Alembic head set matches their image’s. An empty initial database is a waiting state, not a failure.

control.migrations.timeoutSeconds (default 600) becomes the Job’s activeDeadlineSeconds and also bounds each application bootstrap attempt. PostgreSQL connections and queries have five-second timeouts. The Job retries up to backoffLimit (default 3), then stays failed for inspection; it is retained as a normal Helm resource with no TTL deletion, so its result survives until the next release revision replaces it. Every Helm revision names a new Job and bumps the API and worker pod-template annotation obol.dev/migration-revision, so an upgrade retries a failed bootstrap even with an unchanged image tag.

Rolling upgrades keep old application pods while new schema-gated pods wait. Migrations must stay compatible with those old pods; a breaking schema change needs a separate staged migration plan. Inspect a failed Job before retrying — never erase a database to repair a deployment.

Setting control.migrations.enabled: false also disables the startup gate and makes schema preparation your responsibility.

Networking

Ingress

ingress.enabled is true by default, class nginx, with three hosts.

Host	Paths routed
`ingress.hosts.gateway`	`/` (prefix)
`ingress.hosts.control`	`/api`, `/oauth`, `/.well-known` (prefix), `/healthz` (exact)
`ingress.hosts.web`	`/` (prefix)

The control host uses an allowlist of paths precisely so /internal/* is never exposed by the public edge. This is the boundary the Fly configuration cannot yet enforce — see Fly.io. TLS is on by default with ingress.tlsSecretName: obol-tls. Long LLM streams may need a controller annotation such as nginx.ingress.kubernetes.io/proxy-read-timeout: "600".

NetworkPolicy

networkPolicy.enabled is true by default and adds one ingress policy:

gateway accepts traffic from control and control-worker pods, plus a namespaceSelector: {} for the ingress controller and monitoring. That selector is deliberately broad; tighten it to your controller’s namespace.

Observability

observability:
  otlpEndpoint: ""
  serviceMonitor:
    enabled: false

Setting otlpEndpoint injects OTEL_EXPORTER_OTLP_ENDPOINT into every workload. serviceMonitor.enabled creates a prometheus-operator ServiceMonitor for the gateway’s /metrics on gateway.metricsPort. See telemetry.

Validating without a cluster

make -C infra helm-lint

The target adds the Bitnami repository, builds chart dependencies, then lints and renders both the default and the self-host profile. It supplies a synthetic KMS key ARN so the required guard on kek.aws.keyId does not block rendering. It starts nothing and contacts no cluster.

Self-hosting

The environment contract shared by all three deploy targets.

Invariants

Why the gateway has no database and state reaches it only as snapshots.

Get started

Concepts

Gateway

Policy and authorization

Credentials and security

Connectors

Receipts and verification

Routing and catalog

Billing and observability

Enterprise self-host

Chart structure

Installing

Self-contained install

Values

URLs and images

Key-encryption key

Workloads

Security context

Service accounts

Migration lifecycle

Networking

Ingress

NetworkPolicy

Observability

Validating without a cluster

Self-hosting

Invariants

​Chart structure

​Installing

​Self-contained install

​Values

​URLs and images

​Key-encryption key

​Workloads

​Security context

​Service accounts

​Migration lifecycle

​Networking

​Ingress

​NetworkPolicy

​Observability

​Validating without a cluster

​Related

Self-hosting

Invariants

Chart structure

Installing

Self-contained install

Values

URLs and images

Key-encryption key

Workloads

Security context

Service accounts

Migration lifecycle

Networking

Ingress

NetworkPolicy

Observability

Validating without a cluster

Related