> ## Documentation Index
> Fetch the complete documentation index at: https://docs.jacobpevans.com/llms.txt
> Use this file to discover all available pages before exploring further.

# OpenBao

> The self-hosted T2 runtime manager — the primary AI/machine credential interface, the SSH certificate authority for automation, and the global flow-lease authority. AppRole auth, native leases, Raft-snapshot DR.

<Note>
  Status: **live.** A 2-node Raft cluster (OpenBao 2.5.5, on-prem static-key auto-unseal) is running; the Ansible role provisions the KV hierarchy, the domain-split RBAC, and the snapshot daemon. A 3rd Raft voter is planned.
</Note>

> The day the homelab needs more than per-host SOPS, this is what catches it — and it is what an always-on agent talks to instead of a human at a keyboard.

## Its job in the four-tier model

OpenBao is **T2** in the [four-tier secrets model](/security/comparison): the self-hosted source of truth for live machine credentials, and the **primary AI/machine interface**. An always-on agent authenticates to it as itself and gets scoped, expiring credentials with **zero interactive prompts** — the property the macOS keychain can never offer a headless process. It does three jobs:

* **Credential broker.** AppRole (and, for the most sensitive services, TLS cert) auth methods issue short-lived, per-service credentials from a KV v2 engine and dynamic secrets engines.
* **SSH certificate authority.** It signs short-TTL SSH certificates for automation principals (`ai-agent`, `ansible`, `ci`) — a hybrid model where machines carry expiring certs and humans keep their static keys. No long-lived automation key sits on disk waiting to be stolen.
* **Global flow-lease authority.** A single lease in its KV store, taken via compare-and-set so two writers cannot both win, gates every mutating flow (a tofu apply, an ansible run, an infra-changing agent). The rule is credential-gating: **no lease, no credentials** — an agent that cannot take the lease cannot obtain the creds it would need to mutate anything, so serialization is enforced where secrets are issued.

It is the MPL-licensed fork of HashiCorp Vault 1.14.x — the last release before HashiCorp's BSL transition in 1.15.

## Architecture

A single OpenBao service holds the truth; one lightweight agent per consumer renews leases and renders the secrets it needs into a memory-only file.

| Layer                                  | What lives here                                                                              | Why                                                                    |
| -------------------------------------- | -------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------- |
| OpenBao host (privileged)              | Raft storage encrypted at rest, AppRole / cert auth methods, KV v2 + dynamic engines, SSH CA | Privileged is acceptable for the vault host; `mlock`/`memlock` need it |
| Consumer: agent                        | Holds the consumer's own AppRole role-id + secret-id                                         | Each consumer authenticates as itself; the server never pushes         |
| Consumer: `/run/openbao/*.env` (tmpfs) | Rendered secrets used by the service binary                                                  | tmpfs is memory-only — destroyed on reboot, never persists to disk     |

## Auth model

* **AppRole per service.** Each consumer has a role ID (durable) and a secret ID (rotating). The Ansible role distributes role IDs in cleartext but obtains secret IDs from OpenBao — ideally as a **response-wrapped, single-use, short-TTL token** at provision time, so an intercepted bootstrap is self-evident.
* **SSH CA for automation.** Automation principals present a public key and receive a short-TTL signed certificate; the target host trusts the CA, not a pile of `authorized_keys`.
* **Periodic tokens** for long-running services where re-authentication would be operationally heavy.
* **TLS cert-based auth** for the most sensitive services as a defense-in-depth layer.

## RBAC — split by resource domain

Access is split into one least-privilege AppRole **per resource domain**, so a
compromise of any single credential is scoped to that domain's secrets. The
sharpest boundary is between the two IaC identities: `terraform-apply`
(human-triggered, writes `secret/infra/*`) and `terrakube-plan`, which runs
VCS-driven — potentially untrusted — plans and is therefore read-only and
confined to `secret/platform/terrakube`, **never** `secret/infra/*`, so a
malicious plan cannot rewrite secrets Ansible later trusts.

| Identity                      | Reads                                                        | Writes |
| ----------------------------- | ------------------------------------------------------------ | ------ |
| `terraform-apply`             | `secret/infra/*`, `secret/platform/{dns,traefik}`            | same   |
| `terrakube-plan`              | `secret/platform/terrakube` only                             | —      |
| `ansible-converge`            | `secret/platform/*`, `secret/apps/*`                         | —      |
| `observability`               | `secret/platform/{splunk,cribl}`                             | —      |
| `local-cloud`                 | `secret/platform/{object-storage,compute}`                   | —      |
| `monitoring` / `media`        | `secret/apps/{monitoring,media}`                             | —      |
| `local-llm`                   | `secret/ai/*` (LLM serving stack)                            | —      |
| `ai-readonly` / `ai-elevated` | `secret/ai/*`, `secret/apps/*` (+ `platform/*` for elevated) | —      |
| `snapshot`                    | `sys/storage/raft/snapshot` only                             | —      |
| `public`                      | `secret/public/*` (non-exploitable facts, ambient creds)     | —      |

Each AppRole carries a `secret_id_bound_cidrs` scoped to its consumer's subnet.
AI **agents** (`ai-readonly` / `ai-elevated`) are read-only and walled off from
the infra kernel; the LLM **serving** stack is the separate `local-llm`
identity, not an agent identity.

## Secret-zero — what bootstraps the bootstrapper

OpenBao cannot start sealed-and-empty on its own. Two pieces of *secret-zero* live one tier up, in the [Doppler strict cloud tier](/security/tools/doppler) (T3): the **static seal key** that auto-unseals the service, and the **AppRole `secret_id`** that agents present to authenticate. T3 is small and only AI-reachable under explicit human approval precisely because it holds these. The human-only **recovery shares** live in the [Bitwarden vault](/security/tools/bitwarden) (T4) as the break-glass path if both T2 and T3 are lost.

On an **always-on Mac agent host**, each domain's AppRole `role_id`/`secret_id` is delivered as its own item in a **dedicated, auto-locking [keychain](/security/tools/macos-keychain)** (72-hour auto-lock). The keychain's **lock state is the access boundary**: a human unlocks once every \~3 days, and between unlocks a user-domain LaunchAgent reads each credential and publishes it into the session environment, so an agent's read is promptless — the "zero prompts at 3 a.m." property holds because the unlock is a periodic action, not a per-read one.

## Boundary with the other tiers

* **SOPS + age (T1):** stays the source of truth for at-rest, in-repo config. OpenBao is for runtime-resolved secrets where lease/renew/rotation matters.
* **Doppler (T3):** holds secret-zero and the rare keys-to-the-kingdom values; OpenBao holds the day-to-day machine credentials that agents actually consume.
* **Bitwarden (T4):** human-only cold storage and OpenBao's recovery shares.

The tiers compose; OpenBao is the primary machine path, not a replacement for the layers around it.

## Sealed posture, audit trail, and recovery

* **Sealed at rest.** OpenBao restarts sealed. Routine restarts auto-unseal from the Doppler-held seal key; a full recovery from N-of-M shares is a [Golden law #3](/security/golden-laws#3-human-approval-gates-every-potentially-destructive-action) human-in-the-loop event.
* **Audit log to Splunk.** Every read, write, and policy change appears in Splunk via the standard OpenBao audit-device → file → Cribl Edge path.
* **Raft-snapshot DR.** An on-box systemd timer snapshots the active node on a schedule (leader-gated at runtime, authenticated with the least-privilege `snapshot` AppRole), integrity-checks each snapshot, and keeps them on a ZFS/PBS-backed volume that replicates off-box; a second copy to an encrypted S3 bucket is a tracked follow-up. Recovery is: restore the snapshot onto a fresh node, auto-unseal from secret-zero, and let the leases re-issue. Rehearsed with a drill on a non-production instance ([Golden law #15](/security/golden-laws#15-backup-the-vault-itself-and-test-recovery)).

## See also

* [Tools comparison](/security/comparison) — where T2 sits, and the bootstrap / read-path / DR flows drawn out.
* [Agent secrets](/autonomous-agents/secrets) — the headless read path that makes OpenBao the primary interface.
* [Golden laws](/security/golden-laws) — every posture above maps to a law.
* [`dryvist/ansible-proxmox-apps`](https://github.com/dryvist/ansible-proxmox-apps) — the bring-up role.
* [`docs.dryvist.com`](https://docs.dryvist.com) — dryvist-internal specifics (topology, policy paths, lease TTLs) live there once this lands.