Skip to main content
Everything with a model in the loop, in one repo — extracted out of the general apps repo once the AI stack outgrew being one more application among many.
ansible-proxmox-ai deploys the homelab’s AI/LLM stack onto VMs and LXC containers that tofu-proxmox provisioned and ansible-proxmox configured. It was extracted from ansible-proxmox-apps once the AI-specific roles grew large enough to warrant their own repo and release cadence; pre-split commit history for each role lives in ansible-proxmox-apps’s git log.

​
What it does

  • LLM serving — Ollama, llama.cpp + llama-swap (GPU-tier serving), the LiteLLM router that fronts the large/light tiers, and Open WebUI
  • RAG — a LlamaIndex embeddings pipeline and the Qdrant vector database
  • Agents — the Hermes agent gateway, sandboxed agent execution, an agent gateway, and an isolated Codex CLI execution user
  • LLM app platforms — Dify, LangFlow, LangGraph, and Langfuse, each Docker-in-LXC

​
How it fits

​
Getting started

1

Confirm the hosts are configured

Run ansible-proxmox first. Hosts need their ZFS, networking, and monitoring agents in place before the AI stack lands on them.
2

Clone and enter the dev shell

git clone <repo-url> ansible-proxmox-ai && cd ansible-proxmox-ai && direnv allow
3

Install Galaxy dependencies

ansible-galaxy install -r requirements.yml
This repository currently ships roles only — there is no site playbook or dynamic tofu-inventory loader yet. Deploy orchestration is a tracked follow-up.

ansible-proxmox

Host config. Must run first.

ansible-proxmox-apps

The general-app deploy tier this repo was extracted from.

Self-hosted ChatGPT

The serving fabric this repo’s LLM-serving roles stand up.

Source on GitHub

Roles, molecule scenarios, full README.