Keep 275 W power cap at boot
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
0c519cd9a3
commit
cff8cf640e
@@ -12,7 +12,7 @@ serving Qwen3.8-Flash-Next IQ3_S on an OpenAI/Anthropic-compatible API, used fro
|
||||
┌──────────────────┐ LAN ┌──────────────────────────────────────────────────────┐
|
||||
│ pi coding agent │──────────────▶│ VM 200 llm-nixos 192.168.1.67 (NixOS 26.05) │
|
||||
│ ~/.pi/agent │ :8080 + key │ strata.service ── Qwen3.8-Flash-Next IQ3_S │
|
||||
└──────────────────┘ │ 2× RTX 3090 (VFIO), gpu-unlock, 250 W cap │
|
||||
└──────────────────┘ │ 2× RTX 3090 (VFIO), gpu-unlock, 275 W cap │
|
||||
│ /models ◀── virtiofs ── /mnt/models (NVMe XFS) │
|
||||
│ gpu-temp-push ──UDP :7777──▶ lovey-fans.service │
|
||||
└──────────────────────────────────────────────────────┘
|
||||
@@ -37,7 +37,7 @@ serving Qwen3.8-Flash-Next IQ3_S on an OpenAI/Anthropic-compatible API, used fro
|
||||
|---|---|---|
|
||||
| Host | Proxmox VE 9.2.21, kernel 7.0.14-pve, pve-no-subscription repos | [docs/proxmox-host.md](docs/proxmox-host.md) |
|
||||
| VM 200 | `llm-nixos`: NixOS 26.05, kernel 6.18, NVIDIA open 595.71, CUDA 13.2 | [docs/nixos-guest.md](docs/nixos-guest.md) |
|
||||
| GPU tuning | 250 W power cap + [gpu-unlock](https://github.com/Zagitta/gpu-unlock) (full-rate FP16/BF16) | [docs/nixos-guest.md](docs/nixos-guest.md#gpu-tuning) |
|
||||
| GPU tuning | 275 W power cap + [gpu-unlock](https://github.com/Zagitta/gpu-unlock) (full-rate FP16/BF16) | [docs/nixos-guest.md](docs/nixos-guest.md#gpu-tuning) |
|
||||
| Inference | [Strata](https://github.com/Niko1221/Strata) @ `4b9e05a2`, Qwen3.8-Flash-Next IQ3_S | [docs/strata.md](docs/strata.md) |
|
||||
| Client | pi coding agent 1.4.0 | [docs/pi-client.md](docs/pi-client.md) |
|
||||
| Fans | `lovey-fans.py` on the host, fed GPU temps by the guest | [docs/proxmox-host.md](docs/proxmox-host.md#fan-control) |
|
||||
|
||||
+3
-2
@@ -6,8 +6,9 @@ Newest first. Each entry: what broke, the cause, and what fixed it.
|
||||
|
||||
With the case closed, alternating runs (250/275/250/275, fresh restart each) gave +4.6% prompt speed at
|
||||
16K at 275 W (2,892 → 3,023 tok/s) and +1.6% at 4K; decode was unchanged within noise. Peak temps went
|
||||
from 51/59°C to 53/63°C. The cap is **runtime only** (`nvidia-smi -pl 275`), and
|
||||
`nvidia-power-limit.service` still sets 250 W at boot. Results:
|
||||
from 51/59°C to 53/63°C. **Kept:** `nvidia-power-limit.service` in
|
||||
[`nixos/configuration.nix`](../nixos/configuration.nix) now sets 275 W at boot (applied with
|
||||
`nixos-rebuild switch`; Strata kept running). Results:
|
||||
[benchmarks/standard](../benchmarks/standard/README.md#power-cap-250-w-vs-275-w).
|
||||
|
||||
## 2026-10-26 (night): first standard benchmark and tuning arms
|
||||
|
||||
+6
-1
@@ -43,6 +43,11 @@ Dense 8192³ GEMM, torch 2.14 / cu130:
|
||||
A stock 3090 peaks at about 71 TFLOPS. The 250 W cap costs ~30% of matmul (prefill) throughput;
|
||||
decode is bandwidth-bound and barely changes.
|
||||
|
||||
The cap is **275 W** since 2026-10-27. In Strata's benchmark, 275 W vs 250 W gave +4.6% prompt tok/s at
|
||||
16K, unchanged decode, and +2–4°C (see
|
||||
[benchmarks/standard](../benchmarks/standard/README.md#power-cap-250-w-vs-275-w)). Strata's real gain is far smaller
|
||||
than the GEMM numbers above suggest, because much of each request isn't matmul-bound.
|
||||
|
||||
## Temps
|
||||
|
||||
4-minute dual-GPU BF16 burn at 250 W: GPU0 34→45°C, GPU1 44→55°C (still creeping), fans ~36%.
|
||||
@@ -57,7 +62,7 @@ GPU1 runs ~10°C warmer than GPU0 (loop order or block contact). Plenty of headr
|
||||
repo. Keep the **generated** `hardware-configuration.nix` (its disk UUIDs are new) but add the
|
||||
`/models` virtiofs entry from the copy here.
|
||||
4. `nixos-install`, reboot, detach the ISO.
|
||||
5. Check: `nvidia-smi` shows both 3090s at 250 W, `ls /models`, `systemctl status gpu-temp-push`
|
||||
5. Check: `nvidia-smi` shows both 3090s at 275 W, `ls /models`, `systemctl status gpu-temp-push`
|
||||
and `journalctl -u lovey-fans` on the host shows a GPU temperature.
|
||||
6. Install Strata: [strata.md](strata.md#rebuild).
|
||||
|
||||
|
||||
@@ -41,7 +41,7 @@
|
||||
nvidiaPersistenced = true;
|
||||
nvidiaSettings = false;
|
||||
};
|
||||
# Cap both 3090s at 250 W (default 350 W); reapplied on every boot
|
||||
# Cap both 3090s at 275 W (default 350 W); reapplied on every boot. 275 vs 250: +4.6% prompt tok/s, see benchmarks/standard
|
||||
systemd.services.nvidia-power-limit = {
|
||||
description = "Set NVIDIA GPU power limit";
|
||||
after = [ "nvidia-persistenced.service" ];
|
||||
@@ -50,7 +50,7 @@
|
||||
serviceConfig = {
|
||||
Type = "oneshot";
|
||||
RemainAfterExit = true;
|
||||
ExecStart = "${config.hardware.nvidia.package.bin}/bin/nvidia-smi -pl 250";
|
||||
ExecStart = "${config.hardware.nvidia.package.bin}/bin/nvidia-smi -pl 275";
|
||||
};
|
||||
};
|
||||
# lovey's fan daemon (lovey-fans) can't see passed-through GPUs; send it the hottest one every 2 s
|
||||
|
||||
Reference in New Issue
Block a user