lovey/docs/changelog.md

Change log and fixes

Newest first. Each entry: what broke, the cause, and what fixed it.

2026-10-27: power cap 250 W vs 275 W

With the case closed, alternating runs (250/275/250/275, fresh restart each) gave +4.6% prompt speed at 16K at 275 W (2,892 → 3,023 tok/s) and +1.6% at 4K; decode was unchanged within noise. Peak temps went from 51/59°C to 53/63°C. Kept: nvidia-power-limit.service in nixos/configuration.nix now sets 275 W at boot (applied with nixos-rebuild switch; Strata kept running). Results: benchmarks/standard.

2026-10-26 (night): first standard benchmark and tuning arms

Ran Strata's standard-v1 suite on the idle server, then one arm per tuning flag with a fresh restart each. Baseline: 2,881–2,938 prompt tok/s at 16K, decode ~110–131. --adapt-async 1 neutral, --pipeline-windows 2 −10–12% decode, --prefill auto:32768 −22% prompt at 16K. No config change kept; the original strata-iq3_s.json was restored (byte-identical, backup at /var/lib/strata/strata-iq3_s.json.orig-2026-10-26). Results: benchmarks/standard.

2026-10-26 (evening): compaction loop again, summary snowball

Symptom: the same "generation hit the token cap" loop, now at the 32K cap, retrying every ~4 min on a long husky-walk session.

Cause: pi's update prompt makes every compaction preserve the whole previous summary. After 13 compactions it was 86.6K chars (~22K tokens); thinking (~11K, ending on its own) plus rewriting it exceeded 32,768 tokens. Not a model loop: the content was unique, just never trimmed. A ~22K summary plus the 20K kept tail also made compactions come sooner.

Fix: pi/extensions/budget-compaction.ts, a pi extension that writes a fresh checkpoint with thinking off and enforces a 2,500-word limit (condense passes, since the model ignored the limit when only asked). On a copy of the session: 44.5 s, 1,183 words, 23K tokens of context afterwards. See pi-client.md.

2026-10-26: pi auto-compaction loop

Symptom: on long pi sessions, "Auto-compaction failed: Summarization failed: generation hit the token cap and the summary is incomplete", retried every ~2 minutes, forever. /thinking off + /compact also failed.

Cause:

1. Pi caps the summary request at the model's maxTokens (was 16384). Strata gave thinking 75% of that (12288), leaving ~4K for the summary, and every attempt ran to 16384 tokens (length).
2. Pi's thinking level never reached Strata: the compat had supportsReasoningEffort: false and no thinkingFormat, so pi sent no thinking control and Strata thought at full effort on every request. That's why the thinking-off workaround did nothing.

Fix (in pi/):

• maxTokens 16384 → 32768
• compaction: { reserveTokens: 32768, keepRecentTokens: 20000 } (compact at ~98K instead of ~115K)
• compat thinkingFormat: "chat-template" + chatTemplateKwargs for enable_thinking and reasoning_effort

Verified: pi -p --thinking off produced no thinking; --thinking medium got a 24,576-token budget; the stuck session compacted with thinking off.

2026-10-26: Strata, firewall, fans, GPU tuning

• Strata installed and serving IQ3_S (strata.md). NixOS firewall turned on (22, 8080).
• lovey-fans fan control on the host, gpu-temp-push in the guest.
• 250 W cap and gpu-unlock in the guest (nixos-guest.md). configuration.nix.pre-unlock on the guest is the config from before; generation 1 is plain.
• Guest switched to static IP, NetworkManager removed.

2026-10-25: board swap and 4-DIMM instability

Moved to ASUS Maximus Z790 Hero + i9-14900KS, 4× 32 GB, both 3090s.

Memory speed	Result
5400 MT/s	Bootloop: kernel oopses at RIP 0x0 in pve-startall, qm/pveversion segfaults
5200 MT/s	Random pvesm status segfaults, host dropped off the network within minutes
4800 JEDEC, 1.1 V	Stable: 15-min soak (root/stab.sh) with 0 failures

• Every PVE/Perl file checksum-verified clean: the corruption was in RAM, not on disk. Cause is 4-DIMM speed, not CPU degradation.
• Cheap tell: "Initramfs unpacking failed: ZSTD-compressed data is corrupt" at boot means memory is unstable on this box. It disappears at 4800.
• vmbr0 moved to eno2 (Proxmox pins NIC names to MACs).
• NixOS 26.05 installed in VM 200, which got both GPUs and 112 GB.

2026-10-12: Z690 memory channel A fault

On the original MSI Z690 Carbon, memory channel A was contact-marginal: under load the box hard-froze (purple/static on the iGPU, no logs). Memtest froze with 4 DIMMs or one DIMM in A after ~4 min with 0 errors; channel B alone was stable. CPU and RAM were cleared on another board. Resolved by moving to the Z790 Hero. Crash-capture sysctls from this debugging are still on the host.