Keep 275 W power cap at boot
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
0c519cd9a3
commit
cff8cf640e
@@ -12,7 +12,7 @@ serving Qwen3.8-Flash-Next IQ3_S on an OpenAI/Anthropic-compatible API, used fro
|
|||||||
┌──────────────────┐ LAN ┌──────────────────────────────────────────────────────┐
|
┌──────────────────┐ LAN ┌──────────────────────────────────────────────────────┐
|
||||||
│ pi coding agent │──────────────▶│ VM 200 llm-nixos 192.168.1.67 (NixOS 26.05) │
|
│ pi coding agent │──────────────▶│ VM 200 llm-nixos 192.168.1.67 (NixOS 26.05) │
|
||||||
│ ~/.pi/agent │ :8080 + key │ strata.service ── Qwen3.8-Flash-Next IQ3_S │
|
│ ~/.pi/agent │ :8080 + key │ strata.service ── Qwen3.8-Flash-Next IQ3_S │
|
||||||
└──────────────────┘ │ 2× RTX 3090 (VFIO), gpu-unlock, 250 W cap │
|
└──────────────────┘ │ 2× RTX 3090 (VFIO), gpu-unlock, 275 W cap │
|
||||||
│ /models ◀── virtiofs ── /mnt/models (NVMe XFS) │
|
│ /models ◀── virtiofs ── /mnt/models (NVMe XFS) │
|
||||||
│ gpu-temp-push ──UDP :7777──▶ lovey-fans.service │
|
│ gpu-temp-push ──UDP :7777──▶ lovey-fans.service │
|
||||||
└──────────────────────────────────────────────────────┘
|
└──────────────────────────────────────────────────────┘
|
||||||
@@ -37,7 +37,7 @@ serving Qwen3.8-Flash-Next IQ3_S on an OpenAI/Anthropic-compatible API, used fro
|
|||||||
|---|---|---|
|
|---|---|---|
|
||||||
| Host | Proxmox VE 9.2.21, kernel 7.0.14-pve, pve-no-subscription repos | [docs/proxmox-host.md](docs/proxmox-host.md) |
|
| Host | Proxmox VE 9.2.21, kernel 7.0.14-pve, pve-no-subscription repos | [docs/proxmox-host.md](docs/proxmox-host.md) |
|
||||||
| VM 200 | `llm-nixos`: NixOS 26.05, kernel 6.18, NVIDIA open 595.71, CUDA 13.2 | [docs/nixos-guest.md](docs/nixos-guest.md) |
|
| VM 200 | `llm-nixos`: NixOS 26.05, kernel 6.18, NVIDIA open 595.71, CUDA 13.2 | [docs/nixos-guest.md](docs/nixos-guest.md) |
|
||||||
| GPU tuning | 250 W power cap + [gpu-unlock](https://github.com/Zagitta/gpu-unlock) (full-rate FP16/BF16) | [docs/nixos-guest.md](docs/nixos-guest.md#gpu-tuning) |
|
| GPU tuning | 275 W power cap + [gpu-unlock](https://github.com/Zagitta/gpu-unlock) (full-rate FP16/BF16) | [docs/nixos-guest.md](docs/nixos-guest.md#gpu-tuning) |
|
||||||
| Inference | [Strata](https://github.com/Niko1221/Strata) @ `4b9e05a2`, Qwen3.8-Flash-Next IQ3_S | [docs/strata.md](docs/strata.md) |
|
| Inference | [Strata](https://github.com/Niko1221/Strata) @ `4b9e05a2`, Qwen3.8-Flash-Next IQ3_S | [docs/strata.md](docs/strata.md) |
|
||||||
| Client | pi coding agent 1.4.0 | [docs/pi-client.md](docs/pi-client.md) |
|
| Client | pi coding agent 1.4.0 | [docs/pi-client.md](docs/pi-client.md) |
|
||||||
| Fans | `lovey-fans.py` on the host, fed GPU temps by the guest | [docs/proxmox-host.md](docs/proxmox-host.md#fan-control) |
|
| Fans | `lovey-fans.py` on the host, fed GPU temps by the guest | [docs/proxmox-host.md](docs/proxmox-host.md#fan-control) |
|
||||||
|
|||||||
+3
-2
@@ -6,8 +6,9 @@ Newest first. Each entry: what broke, the cause, and what fixed it.
|
|||||||
|
|
||||||
With the case closed, alternating runs (250/275/250/275, fresh restart each) gave +4.6% prompt speed at
|
With the case closed, alternating runs (250/275/250/275, fresh restart each) gave +4.6% prompt speed at
|
||||||
16K at 275 W (2,892 → 3,023 tok/s) and +1.6% at 4K; decode was unchanged within noise. Peak temps went
|
16K at 275 W (2,892 → 3,023 tok/s) and +1.6% at 4K; decode was unchanged within noise. Peak temps went
|
||||||
from 51/59°C to 53/63°C. The cap is **runtime only** (`nvidia-smi -pl 275`), and
|
from 51/59°C to 53/63°C. **Kept:** `nvidia-power-limit.service` in
|
||||||
`nvidia-power-limit.service` still sets 250 W at boot. Results:
|
[`nixos/configuration.nix`](../nixos/configuration.nix) now sets 275 W at boot (applied with
|
||||||
|
`nixos-rebuild switch`; Strata kept running). Results:
|
||||||
[benchmarks/standard](../benchmarks/standard/README.md#power-cap-250-w-vs-275-w).
|
[benchmarks/standard](../benchmarks/standard/README.md#power-cap-250-w-vs-275-w).
|
||||||
|
|
||||||
## 2026-10-26 (night): first standard benchmark and tuning arms
|
## 2026-10-26 (night): first standard benchmark and tuning arms
|
||||||
|
|||||||
+6
-1
@@ -43,6 +43,11 @@ Dense 8192³ GEMM, torch 2.14 / cu130:
|
|||||||
A stock 3090 peaks at about 71 TFLOPS. The 250 W cap costs ~30% of matmul (prefill) throughput;
|
A stock 3090 peaks at about 71 TFLOPS. The 250 W cap costs ~30% of matmul (prefill) throughput;
|
||||||
decode is bandwidth-bound and barely changes.
|
decode is bandwidth-bound and barely changes.
|
||||||
|
|
||||||
|
The cap is **275 W** since 2026-10-27. In Strata's benchmark, 275 W vs 250 W gave +4.6% prompt tok/s at
|
||||||
|
16K, unchanged decode, and +2–4°C (see
|
||||||
|
[benchmarks/standard](../benchmarks/standard/README.md#power-cap-250-w-vs-275-w)). Strata's real gain is far smaller
|
||||||
|
than the GEMM numbers above suggest, because much of each request isn't matmul-bound.
|
||||||
|
|
||||||
## Temps
|
## Temps
|
||||||
|
|
||||||
4-minute dual-GPU BF16 burn at 250 W: GPU0 34→45°C, GPU1 44→55°C (still creeping), fans ~36%.
|
4-minute dual-GPU BF16 burn at 250 W: GPU0 34→45°C, GPU1 44→55°C (still creeping), fans ~36%.
|
||||||
@@ -57,7 +62,7 @@ GPU1 runs ~10°C warmer than GPU0 (loop order or block contact). Plenty of headr
|
|||||||
repo. Keep the **generated** `hardware-configuration.nix` (its disk UUIDs are new) but add the
|
repo. Keep the **generated** `hardware-configuration.nix` (its disk UUIDs are new) but add the
|
||||||
`/models` virtiofs entry from the copy here.
|
`/models` virtiofs entry from the copy here.
|
||||||
4. `nixos-install`, reboot, detach the ISO.
|
4. `nixos-install`, reboot, detach the ISO.
|
||||||
5. Check: `nvidia-smi` shows both 3090s at 250 W, `ls /models`, `systemctl status gpu-temp-push`
|
5. Check: `nvidia-smi` shows both 3090s at 275 W, `ls /models`, `systemctl status gpu-temp-push`
|
||||||
and `journalctl -u lovey-fans` on the host shows a GPU temperature.
|
and `journalctl -u lovey-fans` on the host shows a GPU temperature.
|
||||||
6. Install Strata: [strata.md](strata.md#rebuild).
|
6. Install Strata: [strata.md](strata.md#rebuild).
|
||||||
|
|
||||||
|
|||||||
@@ -41,7 +41,7 @@
|
|||||||
nvidiaPersistenced = true;
|
nvidiaPersistenced = true;
|
||||||
nvidiaSettings = false;
|
nvidiaSettings = false;
|
||||||
};
|
};
|
||||||
# Cap both 3090s at 250 W (default 350 W); reapplied on every boot
|
# Cap both 3090s at 275 W (default 350 W); reapplied on every boot. 275 vs 250: +4.6% prompt tok/s, see benchmarks/standard
|
||||||
systemd.services.nvidia-power-limit = {
|
systemd.services.nvidia-power-limit = {
|
||||||
description = "Set NVIDIA GPU power limit";
|
description = "Set NVIDIA GPU power limit";
|
||||||
after = [ "nvidia-persistenced.service" ];
|
after = [ "nvidia-persistenced.service" ];
|
||||||
@@ -50,7 +50,7 @@
|
|||||||
serviceConfig = {
|
serviceConfig = {
|
||||||
Type = "oneshot";
|
Type = "oneshot";
|
||||||
RemainAfterExit = true;
|
RemainAfterExit = true;
|
||||||
ExecStart = "${config.hardware.nvidia.package.bin}/bin/nvidia-smi -pl 250";
|
ExecStart = "${config.hardware.nvidia.package.bin}/bin/nvidia-smi -pl 275";
|
||||||
};
|
};
|
||||||
};
|
};
|
||||||
# lovey's fan daemon (lovey-fans) can't see passed-through GPUs; send it the hottest one every 2 s
|
# lovey's fan daemon (lovey-fans) can't see passed-through GPUs; send it the hottest one every 2 s
|
||||||
|
|||||||
Reference in New Issue
Block a user