lovey/docs/nixos-guest.md

NixOS guest (VM 200 llm-nixos, 192.168.1.67)

NixOS 26.05, systemd-boot, kernel 6.18, NVIDIA open driver 595.71 (CUDA 13.2). The complete config is in nixos/ and is the source of truth: /etc/nixos on the guest should match it.

What the config sets up

• Network: static 192.168.1.67/24 on ens18, gateway .1, DNS 1.1.1.1/8.8.8.8 (NetworkManager removed). Static because lovey-fans only trusts GPU temps from this address. Firewall on: TCP 22 (SSH) and 8080 (Strata).
• Users: jason and root are SSH-key only; sudo needs no password.
• NVIDIA: open kernel module, headless, nvidia-persistenced. nvidia-container-toolkit + podman (--device nvidia.com/gpu=all).
• CUDA tooling: cuda_nvcc and the cuda-maintainers cachix substituter, so nothing CUDA-heavy compiles locally. programs.nix-ld with config.hardware.nvidia.package in its libraries: without that, pip/uv torch and vLLM can't find libcuda and torch.cuda.is_available() is False.
• Memory: memlock unlimited (PAM and systemd default), needed for pinned host memory.
• /models: virtiofs from lovey's /mnt/models, nofail.
• Services: nvidia-power-limit, gpu-temp-push, strata (see strata.md).

GPU tuning

• Power cap: nvidia-power-limit runs nvidia-smi -pl 250 after persistenced on every boot (stock is 350 W).
• gpu-unlock (nixos/gpu-unlock.nix): the Zagitta/gpu-unlock kernel driver, pinned at af28cbb, enables full-rate FP16/BF16/TF32 tensor math on GA102. It runs inside the guest and flashes nothing. The Nix file builds the module for the guest kernel and generates its firmware from pinned NVIDIA 610.43.03 inputs (all SHA-256 checked), loads gpu_unlock from the initrd before nvidia, and sets gpu_unlock.targets=all. To disable it, drop that kernel parameter.
  • Check: journalctl -k -b | grep -E "gpu_unlock|entry:". unlocked appears only on the first boot after lovey powers off. After a VM reboot it logs unlock failed (-114) (-EALREADY): the cards are still unlocked, which is harmless.
  • The unlock survives VM reboots and host bus resets; only a host power-off clears it.

Dense 8192³ GEMM, torch 2.14 / cu130:

Power limit	FP16	BF16
250 W	77–81 TFLOPS	80–86 TFLOPS
350 W	112–114 TFLOPS	115–116 TFLOPS

A stock 3090 peaks at about 71 TFLOPS. The 250 W cap costs ~30% of matmul (prefill) throughput; decode is bandwidth-bound and barely changes.

The cap is 275 W since 2026-10-27. In Strata's benchmark, 275 W vs 250 W gave +4.6% prompt tok/s at 16K, unchanged decode, and +2–4°C (see benchmarks/standard). Strata's real gain is far smaller than the GEMM numbers above suggest, because much of each request isn't matmul-bound.

Temps

4-minute dual-GPU BF16 burn at 250 W: GPU0 34→45°C, GPU1 44→55°C (still creeping), fans ~36%. GPU1 runs ~10°C warmer than GPU0 (loop order or block contact). Plenty of headroom.

Rebuild

1. Create VM 200 (proxmox-host.md) and boot the NixOS 26.05 minimal ISO.
2. Partition: 1 GB ESP (vfat) + ext4 root on the 100 GB disk; mount at /mnt and /mnt/boot.
3. nixos-generate-config --root /mnt, then copy configuration.nix and gpu-unlock.nix from this repo. Keep the generated hardware-configuration.nix (its disk UUIDs are new) but add the /models virtiofs entry from the copy here.
4. nixos-install, reboot, detach the ISO.
5. Check: nvidia-smi shows both 3090s at 275 W, ls /models, systemctl status gpu-temp-push and journalctl -u lovey-fans on the host shows a GPU temperature.
6. Install Strata: strata.md.

Applying a config change: edit here, copy to /etc/nixos/, sudo nixos-rebuild switch, commit.