Skip to main content
The AI/LLM stack lives in this repository.
Language: Python  ·  Status: active  ·  Last active: today  ·  Source on GitHub
ansible-proxmox-ai deploys the homelab’s AI/LLM stack onto VMs and LXC containers that tofu-proxmox provisioned and ansible-proxmox configured.

What it does

  • LLM serving. Ollama, llama.cpp + llama-swap (GPU-tier serving), the LiteLLM router that fronts the large/light tiers, and Open WebUI
  • RAG. A LlamaIndex embeddings pipeline and the Qdrant vector database
  • Agents. The Hermes agent gateway, sandboxed agent execution, an agent gateway, and an isolated Codex command-line tool execution user
  • LLM app platforms. Dify, LangFlow, LangGraph, Langfuse, and Arize Phoenix, each Docker-in-LXC

How it fits

The AI-specific deploy tier. General app payloads (n8n, Cribl, Vikunja, Zammad, and the rest) stay in ansible-proxmox-apps. This repo owns only what has a model, an embedding, or an agent loop in it.

Getting started

1

Confirm the hosts are configured

Run ansible-proxmox first. Hosts need their ZFS, networking, and monitoring agents in place before the AI stack lands on them.
2

Clone and enter the dev shell

git clone <repo-url> ansible-proxmox-ai && cd ansible-proxmox-ai && direnv allow
3

Install Galaxy dependencies

ansible-galaxy install -r requirements.yml
This repository includes its own site playbook and dynamic OpenTofu inventory loader. It converges the AI fleet without ansible-proxmox-apps. See the site playbook. The inventory loader lives here too.

Release process

This repo follows git-flow; release and promotion follow Branch conventions and CI/CD policy. Nothing merges back from main to develop.

ansible-proxmox

Host config. Must run first.

ansible-proxmox-apps

The general-app deploy tier.

Self-hosted ChatGPT

The serving fabric this repo’s LLM-serving roles stand up.

Source on GitHub

Roles, molecule scenarios, full README.