Skip to main content
Everything with a model in the loop, in one repo. Extracted out of the general apps repo once the AI stack outgrew being one more app among many.
ansible-proxmox-ai deploys the homelab’s AI/LLM stack onto VMs and LXC containers that tofu-proxmox provisioned and ansible-proxmox configured. The AI-specific roles grew large enough to warrant their own repo and release cadence, so this repo split out of ansible-proxmox-apps. Pre-split commit history for each role lives in ansible-proxmox-apps’s git log.

​
What it does

  • LLM serving. Ollama, llama.cpp + llama-swap (GPU-tier serving), the LiteLLM router that fronts the large/light tiers, and Open WebUI
  • RAG. A LlamaIndex embeddings pipeline and the Qdrant vector database
  • Agents. The Hermes agent gateway, sandboxed agent execution, an agent gateway, and an isolated Codex command-line tool execution user
  • LLM app platforms. Dify, LangFlow, LangGraph, Langfuse, and Arize Phoenix, each Docker-in-LXC

​
How it fits

​
Getting started

1

Confirm the hosts are configured

Run ansible-proxmox first. Hosts need their ZFS, networking, and monitoring agents in place before the AI stack lands on them.
2

Clone and enter the dev shell

git clone <repo-url> ansible-proxmox-ai && cd ansible-proxmox-ai && direnv allow
3

Install Galaxy dependencies

ansible-galaxy install -r requirements.yml
This repository currently ships roles only. It has no site playbook or dynamic tofu-inventory loader yet. Deploy orchestration is a tracked follow-up.

ansible-proxmox

Host config. Must run first.

ansible-proxmox-apps

The general-app deploy tier this repo was extracted from.

Self-hosted ChatGPT

The serving fabric this repo’s LLM-serving roles stand up.

Source on GitHub

Roles, molecule scenarios, full README.