How the Nix stack serves a model

One host file names a couple of numbers. Nix turns them into a sysctl, a set of serve flags, a swap policy, and a build-time safety check. It then launches the worker. This page follows that path from top to bottom. Two Apple Silicon Macs run local models. Neither is an agent. Each exposes an OpenAI-compatible API on loopback. The agent layer calls in over HTTP. See Local LLM for the strategy; this page is the mechanism.

​

Two hosts, two roles

The split is by job, not by hardware. One host is headless; the other runs an interactive desktop.

• The headless host provides large-memory capacity for serial service of models that only fit in 128 GB. The measured comparison used one request at a time; it does not establish live resident weights or multi-request saturation.
• The interactive laptop self-serves: localhost clients only, so serving never competes with the network host. It is sized to coexist with an interactive desktop and the user’s own work.

The role: use the headless host for models that need its larger memory. It provides serial capacity without the laptop’s battery and memory constraints. The dated model roster, verdicts, and performance numbers live in Mac Studio serving; per-host memory numbers live in the gated operational reference.

​

Ports

The headless host adds a LAN-facing gate.

​

How the repos compose

Two repos meet to launch one worker. nix-darwin owns the host: the OS memory ceiling and power knobs. nix-ai owns the serving stack: which models exist, how each is served, and the command that starts it. A host file picks values; the modules turn them into a running process.

​

The nix-darwin side: the host ceiling

modules/darwin/apple-silicon-tunables.nix owns the machine’s memory ceiling and inference power knobs. Its main option is wiredLimitMb: the iogpu.wired_limit_mb sysctl, the OS ceiling on wired GPU memory. The sysctl is volatile (it resets on reboot), so the module re-applies it from a launchd daemon at boot and again at every rebuild. The cluster ceiling changes during the manual lifecycle. cluster-join applies 90,000 MiB on the MacBook Pro and 102,400 MiB on the Mac Studio; cluster-detach reverses the clustered state. The operator starts the join on the coordinator, which drives the worker over SSH before joining locally. The full preflight and formation sequence is in Two-Mac MLX cluster operations. Each host’s standalone ceiling remains declared separately. The values are not typed by hand; each host derives its declared ceiling from a memory budget, wiredLimitMb = maxLocalLlmGb × 1024. The historical memory-ceilings guide covers retired vllm-mlx controls; the current mlx-lm wrapper appears below. The per-host budgets live in the gated operational reference. The module also handles power posture: Low Power Mode off, Power Nap off, and App Nap off for the inference daemon. It also sets a Metal debug-env guard. None of that picks a model; it shapes the box the model runs in.

​

The nix-ai side: the serving stack

nix-ai holds four kinds of file under modules/mlx/:

​

The launch path, end to end

1. A host file picks model entries, assigns each a class, and sets host-scoped posture: the preload list, the swap policy, the concurrency limit.
2. catalog-data.nix supplies each chosen model’s family args and its class profile; the host’s knob values merge on top.
3. The serve-command renderer builds one mlx-lm server command per resident model, folding in the merged flags.
4. Each host’s proxy routes requests to the model servers selected by its host configuration.
5. A warmup agent faults the preload list into memory at boot, so the first request never pays a cold start.
6. nix-darwin has already set the wired ceiling the worker allocates under.

The registry that records which physical model is the current default is written at activation and read as ~/.config/ai-stack/registry.json. Docs describe the strategy; the live id is always a registry value, never hard-coded. For the packaging view of these repos, see AI tooling and macOS host.

​

Why the launch agents run under the platform shell

Every shell payload in this stack is launched by the operating system’s own /bin/bash, not the newer shell the package manager provides. That looks like a mistake and is not. It is the design, and it constrains how the scripts get written. macOS keys its privacy grants (Local Network access, in particular) to a binary’s code-signing identity. A package-manager binary’s identity is derived from its content hash, so it changes every time the package set moves forward. Grant the permission today, bump the dependency tomorrow, and the OS sees an unknown binary and silently revokes the grant. Every prior approval evaporates on an unrelated upgrade. The platform shell’s identity is stable across every upgrade the machine ever takes, so a grant made once holds. Two real failures, both found and fixed on 2026-07-27, show the shape: The watcher path in this historical incident was removed on 2026-10-18 in favour of manual cluster-join and cluster-detach. Both failures had been chased down the wrong path for a long time, because neither produced an error anyone saw. The durable fix is not the two patches. It is the mutation-tested compatibility check now in the build. It parses every shipped script under the platform shell, and the test suite proves the check itself fails when a violating construct is introduced. A guard that has never been shown to fail is not a guard. One consequence is worth naming here, because this page is where the swapping proxy is described. A response from any endpoint described earlier cannot tell you which weights answered it. The model field is echoed from the request, and a /v1/models listing enumerates the local cache rather than what is loaded. Read the model argument off the listening process before trusting any number taken through these ports. See Verifying the instrument.

​

The memory-safety control plane

The model worker uses a wired ceiling plus per-process memory and cache limits. The former link-watcher control was removed on 2026-10-18. The cluster ceiling changes only during an explicit join or detach; cable movement does not change it. Nix’s build-time job is narrow:

• the host ceilings: each host declares its standalone ceiling and its cluster-mode value.
• the lifecycle controls: the manual join checks the link, sets the cluster-mode ceiling, quiesces declared workloads, checks room and versions, and starts aligned ranks. Detach reverses the joined state.
• the launch wrapper and the model data: the wrapper that sets MLX’s in-process memory and cache limits, and the catalog fields it reads.

Runtime applies the declared values at the required boundaries:

• the manual lifecycle applies cluster-mode ceilings during join and restores the standalone state during detach.
• the launch wrapper sets the MLX memory and cache limits from the live ceiling at each worker start. Swapping a model needs no nix change.

A build-time assert remains, demoted to a policy check. It verifies the declared budgets are internally consistent and each leaves an OS floor, not that any one model fits.

​

Laptop local serving

For the small and medium fallback order, see Fast-subagent tier. For Cribl Edge observability, see Mac pack.

​

The role map and host derivation

model-roles.json in homelab-contracts maps each role to a model, an egress class, and a host class. Hosts and the router both read it; see Fast-subagent tier for the roles and chains. The shared model-catalog.json owns per-model limits and backend profiles. A model’s router limits can differ from its MLX serving profile while remaining in the same catalog entry. nix-ai reads the locked catalog for MLX values; the router and serving roles project root or backend-profile values from the same source. Their checks fail if rendered limits disagree. The Nix hosts derive their serving from these shared inputs:

• the catalog is the 27B dense model, MiMo-9B, and an OCR swap model on the server class;
• the preload list, resident-worker count, and role-to-id registry come from the map; per-model concurrency defaults come from the shared catalog, with host-specific modelConcurrency overrides;
• the MacBook’s per-worker memory limit derives to 30 GiB;
• vllm-mlx is removed, and its options now fail with a removal message.

​

Related