Jason WittyandClaude Opus 5.5
569081e8ff
Asking for 2,500 words was ignored on a real session (10,401 words, 21K tokens). Condense an over-limit previous summary first, give each section a word allowance, and condense the result until it fits. On a copy of husky-walk: 44.5 s, 1,183 words. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
95 lines
4.5 KiB
Markdown
95 lines
4.5 KiB
Markdown
# pi client (cosmic)
|
|
|
|
[pi](https://pi.dev) coding agent 1.4.0 on the workstation, pointed at Strata. Config lives in
|
|
`~/.pi/agent/`; the copies in [`pi/`](../pi) have the API key replaced with `<STRATA_API_KEY>`.
|
|
|
|
## models.json
|
|
|
|
[`pi/models.json`](../pi/models.json) defines provider `strata`:
|
|
|
|
- `baseUrl: http://192.168.1.67:8080/v1`, `api: openai-completions`
|
|
- `contextWindow: 131072` (matches Strata's `--max-context`)
|
|
- `maxTokens: 32768`
|
|
- `reasoning: true`, `input: text + image`
|
|
- `compat`:
|
|
- `supportsDeveloperRole: false`, `supportsReasoningEffort: false`
|
|
- `thinkingFormat: "chat-template"` with `chatTemplateKwargs`
|
|
`{ enable_thinking: {$var: thinking.enabled}, reasoning_effort: {$var: thinking.effort, omitWhenOff: true} }`.
|
|
**Without this, pi's thinking level never reaches Strata** and every request thinks at full effort.
|
|
|
|
## settings.json
|
|
|
|
[`pi/settings.json`](../pi/settings.json): default provider/model, thinking `medium`, and
|
|
|
|
```json
|
|
"compaction": { "reserveTokens": 32768, "keepRecentTokens": 20000 }
|
|
```
|
|
|
|
Pi compacts once context exceeds `contextWindow - reserveTokens`, so this starts at ~98K. Keep
|
|
`reserveTokens` ≥ `maxTokens`.
|
|
|
|
## Thinking levels
|
|
|
|
| pi level | What Strata does |
|
|
|---|---|
|
|
| off | No thinking |
|
|
| low | System note asking for brief thinking |
|
|
| minimal / medium | System note: think as much as the task needs |
|
|
| high / xhigh / max | Unrestricted thinking |
|
|
|
|
Use medium day to day, high for hard debugging or planning. Compaction reuses the session's level.
|
|
|
|
## How compaction works (and why it failed)
|
|
|
|
Pi sends one summary request containing everything except the last `keepRecentTokens`. Its output is
|
|
capped at the model's `maxTokens` and it uses the current thinking level. If the reply ends with
|
|
`stopReason: "length"`, pi refuses to save the partial summary ("generation hit the token cap and the
|
|
summary is incomplete") and tries again on the next turn. See the
|
|
[2026-10-26 changelog entry](changelog.md#2026-10-26-pi-auto-compaction-loop).
|
|
|
|
If it ever happens again: `/thinking off`, `/compact`, then set thinking back.
|
|
|
|
### Summary snowball and the budget-compaction extension
|
|
|
|
The default update prompt tells the model to "PRESERVE all existing information from the previous
|
|
summary", so every compaction copies the old summary forward and adds to it. On the `husky-walk`
|
|
session it grew 9.9K → 86.6K chars over 13 compactions (~22K tokens), and rewriting that plus ~11K of
|
|
thinking no longer fit in 32K: the loop came back. The text was legitimate (1,458 of 1,573 lines
|
|
unique), mostly an ever-growing `### Done` list.
|
|
|
|
[`pi/extensions/budget-compaction.ts`](../pi/extensions/budget-compaction.ts) hooks
|
|
`session_before_compact` and replaces the summary step:
|
|
|
|
1. If the previous summary is over the limit, condense it on its own first (a short call).
|
|
2. Merge it with the new messages into one fresh checkpoint, with a word allowance per section,
|
|
**2,500 words** in total (`PI_COMPACT_WORDS` overrides).
|
|
3. Count the words; if still more than 25% over, run up to two condense passes on the result.
|
|
|
|
All calls run with thinking off, whatever the session's level. If a call fails, pi's default compaction
|
|
runs instead. The read-only file list is capped at 25 entries (modified files are always listed), and
|
|
`/compact <text>` still adds a focus.
|
|
|
|
An instruction alone was not enough: the first version only asked for 2,500 words, and on a copy of
|
|
`husky-walk` the model produced 10,401 words (21K tokens, 129 s), copying the old summary forward. The
|
|
enforced version on the same copy:
|
|
|
|
| | Prompt-only version | Enforced version |
|
|
|---|---:|---:|
|
|
| Time | 129 s | 44.5 s |
|
|
| Output tokens | 21,158 | 4,719 (two calls) |
|
|
| Checkpoint | 10,401 words | 1,183 words |
|
|
| Context after compaction | 39.5K tokens | 23.1K tokens |
|
|
|
|
The previous 11,542-word summary condensed to 1,162 words and the checkpoint kept the current state,
|
|
contracts, decisions and next steps. 0 reasoning tokens in every call.
|
|
|
|
Install: copy it to `~/.pi/agent/extensions/` and restart pi (extensions load at startup).
|
|
|
|
Testing without touching real sessions: copy the session `.jsonl` and a pi config into a scratch dir,
|
|
then run `PI_CODING_AGENT_DIR=<scratch>/agent pi --mode rpc --session <copy>` and send
|
|
`{"id":"c1","type":"compact"}` on stdin. Project-level `.pi/settings.json` needs project trust, so use
|
|
the agent-dir override instead.
|
|
|
|
`maxTokens` costs no VRAM. Strata sizes the KV cache from `--max-context` / `--kv-resident` at engine
|
|
start; `max_tokens` is only a per-request stop, and Strata shortens it to the room left in the window.
|