OpenBao

The day the homelab needs more than per-host SOPS, this is what catches it — and it is what an always-on agent talks to instead of a human at a keyboard.

​

Its job in the four-tier model

OpenBao is T2 in the four-tier secrets model: the self-hosted source of truth for live machine credentials, and the primary AI/machine interface. An always-on agent authenticates to it as itself and gets scoped, expiring credentials with zero interactive prompts — the property the macOS keychain can never offer a headless process. It does three jobs:

• Credential broker. AppRole (and, for the most sensitive services, TLS cert) auth methods issue short-lived, per-service credentials from a KV v2 engine and dynamic secrets engines.
• SSH certificate authority. It signs short-TTL SSH certificates for automation principals (ai-agent, ansible, ci) — a hybrid model where machines carry expiring certs and humans keep their static keys. No long-lived automation key sits on disk waiting to be stolen.
• Global flow-lease authority. A single lease in its KV store, taken via compare-and-set so two writers cannot both win, gates every mutating flow (a tofu apply, an ansible run, an infra-changing agent). The rule is credential-gating: no lease, no credentials — an agent that cannot take the lease cannot obtain the creds it would need to mutate anything, so serialization is enforced where secrets are issued.

It is the MPL-licensed fork of HashiCorp Vault 1.14.x — the last release before HashiCorp’s BSL transition in 1.15.

​

Architecture

A single OpenBao service holds the truth; one lightweight agent per consumer renews leases and renders the secrets it needs into a memory-only file.

​

Auth model

• AppRole per service. Each consumer has a role ID (durable) and a secret ID (rotating). The Ansible role distributes role IDs in cleartext but obtains secret IDs from OpenBao — ideally as a response-wrapped, single-use, short-TTL token at provision time, so an intercepted bootstrap is self-evident.
• SSH CA for automation. Automation principals present a public key and receive a short-TTL signed certificate; the target host trusts the CA, not a pile of authorized_keys.
• Periodic tokens for long-running services where re-authentication would be operationally heavy.
• TLS cert-based auth for the most sensitive services as a defense-in-depth layer.

​

RBAC — split by resource domain

Access is split into one least-privilege AppRole per resource domain, so a compromise of any single credential is scoped to that domain’s secrets. The sharpest boundary is between the two IaC identities: terraform-apply (human-triggered, writes secret/infra/*) and terrakube-plan, which runs VCS-driven — potentially untrusted — plans and is therefore read-only and confined to secret/platform/terrakube, never secret/infra/*, so a malicious plan cannot rewrite secrets Ansible later trusts. Each AppRole carries a secret_id_bound_cidrs scoped to its consumer’s subnet. AI agents (ai-readonly / ai-elevated) are read-only and walled off from the infra kernel; the LLM serving stack is the separate local-llm identity, not an agent identity.

​

Secret-zero — what bootstraps the bootstrapper

OpenBao cannot start sealed-and-empty on its own. Two pieces of secret-zero live one tier up, in the Doppler strict cloud tier (T3): the static seal key that auto-unseals the service, and the AppRole secret_id that agents present to authenticate. T3 is small and only AI-reachable under explicit human approval precisely because it holds these. The human-only recovery shares live in the Bitwarden vault (T4) as the break-glass path if both T2 and T3 are lost. On an always-on Mac agent host, each domain’s AppRole role_id/secret_id is delivered as its own item in a dedicated, auto-locking keychain (72-hour auto-lock). The keychain’s lock state is the access boundary: a human unlocks once every ~3 days, and between unlocks a user-domain LaunchAgent reads each credential and publishes it into the session environment, so an agent’s read is promptless — the “zero prompts at 3 a.m.” property holds because the unlock is a periodic action, not a per-read one.

​

Boundary with the other tiers

• SOPS + age (T1): stays the source of truth for at-rest, in-repo config. OpenBao is for runtime-resolved secrets where lease/renew/rotation matters.
• Doppler (T3): holds secret-zero and the rare keys-to-the-kingdom values; OpenBao holds the day-to-day machine credentials that agents actually consume.
• Bitwarden (T4): human-only cold storage and OpenBao’s recovery shares.

The tiers compose; OpenBao is the primary machine path, not a replacement for the layers around it.

​

Sealed posture, audit trail, and recovery

• Sealed at rest. OpenBao restarts sealed. Routine restarts auto-unseal from the Doppler-held seal key; a full recovery from N-of-M shares is a Golden law #3 human-in-the-loop event.
• Audit log to Splunk. Every read, write, and policy change appears in Splunk via the standard OpenBao audit-device → file → Cribl Edge path.
• Raft-snapshot DR. An on-box systemd timer snapshots the active node on a schedule (leader-gated at runtime, authenticated with the least-privilege snapshot AppRole), integrity-checks each snapshot, and keeps them on a ZFS/PBS-backed volume that replicates off-box; a second copy to an encrypted S3 bucket is a tracked follow-up. Recovery is: restore the snapshot onto a fresh node, auto-unseal from secret-zero, and let the leases re-issue. Rehearsed with a drill on a non-production instance (Golden law #15).

​

See also

• Tools comparison — where T2 sits, and the bootstrap / read-path / DR flows drawn out.
• Agent secrets — the headless read path that makes OpenBao the primary interface.
• Golden laws — every posture above maps to a law.
• dryvist/ansible-proxmox-apps — the bring-up role.
• docs.dryvist.com — dryvist-internal specifics (topology, policy paths, lease TTLs) live there once this lands.