Switch to UD-Q4_K_XL for testing; benchmark vs IQ3_S

Standard benchmark, two fresh-restart runs: prompt -51% at 4K, -37% at 16K,
decode -40% vs IQ3_S. 12/12 sanity checks. Kept active for husky-bench.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

benchmarks/standard/2026-10-27-standard-rtx-3090-3eb539d8/README.md

@@ -0,0 +1,73 @@
+# <user> standard benchmark: NVIDIA GeForce RTX 3090
+
+Protocol `strata-qwen38-thinking-v1-draft.1` (draft), run `20261027T192719Z-3eb539d8`, started 2026-10-27T19:27:19Z. **Status: nonconforming.**
+
+Written by `<user> bench run`. The machine-readable result is [results.json](results.json); every request, its raw usage and timings and the client event offsets are in [outputs.jsonl](outputs.jsonl).
+
+This run is not a complete standard result:
+
+- no model file could be hashed
+
+## Hardware and software
+
+- GPU: NVIDIA GeForce RTX 3090 (24.0 GiB dedicated)
+- CPU: Intel(R) Core(TM) i9-14900KS; RAM: 110.0 GiB
+- Storage of the model files: not measured
+- OS: NixOS 26.05 (Yarara) 6.18.55; drivers: gpu-driver 595.71.05; runtime: cuda 13.2
+- <user> commit: `4b9e05a2a806a3fe24f7b2560d98626c37dade35`; engine 0.1.51; source build
+- Background work: not measured
+
+## Model and configuration
+
+- Model: qwen3.8-flash-next-unsloth-ud-q4_k_xl; quantization UD-Q4_K_XL; revision not measured; source revision not measured
+- Files: not measured
+- Track: **custom** (UD-Q4_K_XL rep 2, 275 W, case closed)
+- Context 131072 tokens; server capacity 1 slot(s)
+- Sampling: temperature 1.0, top_p 0.95, top_k 20, min_p 0.0, presence_penalty 0.0, repetition_penalty 1.0, reasoning_effort xhigh; thinking on, preserved
+- Engine options: `config.json`; chat template: `provenance/chat_template.jinja`
+
+## Method
+
+The frozen requests of `bench/prompts/standard-v1` (corpus `e75eff8b9feea04f...`) were sent over loopback HTTP, streaming, one at a time, in this order: 1 warm-up (excluded), 3 repetitions of the short, 4K and 16K prompts, and the twelve sanity checks. Prompt and decode speed are the engine's own `timings`; time to first token is the client's monotonic clock from dispatch to the first nonempty reasoning, answer or tool-call delta. No request was retried. Time after the model loaded: 2 min 38 s.
+
+## Results
+
+| Case | Prompt tok/s | Decode tok/s | TTFT s | First answer s | Total s | Capped |
+| --- | --- | --- | --- | --- | --- | --- |
+| short | n/a | 65.7 (64.2-66.4) n=3 | 1.18 (0.95-1.18) n=3 | 2.51 (2.43-2.83) n=3 | 5.9 (5.5-6.1) n=3 | 0/3 |
+| prompt-4k | 847.4 (811.0-855.6) n=3 | 68.9 (68.0-72.1) n=3 | 4.85 (4.81-5.07) n=3 | 10.57 (8.45-11.29) n=3 | 12.3 (11.9-12.5) n=3 | 3/3 |
+| prompt-16k | 1917.8 (1917.3-1920.6) n=3 | 76.3 (72.3-76.4) n=3 | 8.57 (8.56-8.58) n=3 | 13.66 (12.00-13.85) n=3 | 15.3 (15.2-15.6) n=3 | 3/3 |
+
+Cells are median (min-max) n=samples. Short is a prompt of about 80 tokens: its prompt rate is dominated by the fixed cost of a request, so it is not shown; read prompt speed from prompt-4k and prompt-16k. Capped = requests that reached the 512-token cap, reasoning and answer together. A capped request's total time is the time to the cap, not to an answer, and it has no first-answer time when it was still reasoning.
+
+| Phase | Planned | Completed | Truncated | Failed | Skipped |
+| --- | ---: | ---: | ---: | ---: | ---: |
+| warmup | 1 | 0 | 1 | 0 | 0 |
+| performance | 9 | 3 | 6 | 0 | 0 |
+| sanity | 12 | 12 | 0 | 0 | 0 |
+| concurrency | 0 | 0 | 0 | 0 | 0 |
+
+Sanity checks: **12 of 12**.
+
+| Case | Result | Reason |
+| --- | --- | --- |
+| I1 | pass | the expected JSON value |
+| I2 | pass | the expected JSON value |
+| I3 | pass | the expected JSON value |
+| I4 | pass | the exact expected text |
+| R1 | pass | the exact expected text |
+| R2 | pass | the exact expected text |
+| R3 | pass | the exact expected text |
+| R4 | pass | the expected JSON value |
+| T1 | pass | the expected tool call |
+| T2 | pass | the expected tool call |
+| T3 | pass | the expected tool call |
+| T4 | pass | the expected tool call |
+
+## Failures, exclusions and limitations
+
+- No request failed.
+- Prompt-prefix reuse is the shared template prefix only; the cache was not emptied between requests, and the expert cache stays warm and evolves. These are not cold-model measurements.
+- Output caps include reasoning; a request that reaches its cap is `truncated` and fails its sanity check.
+- The runner cannot see other programs on the machine, the repeat-stop and loop-recovery switches of the server, or the quantization repository's revision; see `missing` in results.json for every value it could not read.
+- The server had answered no request before this run (a freshly loaded engine).

benchmarks/standard/2026-10-27-standard-rtx-3090-3eb539d8/config.json

@@ -0,0 +1,13 @@
+{
+  "engine_options": null,
+  "other_engine_options": {},
+  "server_defaults": {
+    "temperature": 1.0,
+    "top_p": 0.95,
+    "top_k": 20,
+    "n_predict": -1
+  },
+  "total_slots": 1,
+  "context": 131072,
+  "source": "the server was already running; its engine options are not exposed"
+}

benchmarks/standard/2026-10-27-standard-rtx-3090-3eb539d8/provenance/chat_template.jinja

@@ -0,0 +1,184 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- set sysns = namespace(count=0, text='') %}
+{%- for message in messages %}
+    {%- if sysns.count == loop.index0 and (message.role == 'system' or message.role == 'developer') %}
+        {%- set sys_content = render_content(message.content, false, true)|trim %}
+        {%- if sys_content %}
+            {%- set sysns.text = sysns.text + ('\n' if sysns.text else '') + sys_content %}
+        {%- endif %}
+        {%- set sysns.count = sysns.count + 1 %}
+    {%- endif %}
+{%- endfor %}
+{%- set num_sys = sysns.count %}
+{%- set merged_system = sysns.text %}
+{%- set reasoning_instructions = '' %}
+{%- if enable_thinking is undefined or enable_thinking is true %}
+    {%- set resolved_reasoning_effort = reasoning_effort|default('xhigh') %}
+    {%- if resolved_reasoning_effort == 'high' %}
+        {%- set resolved_reasoning_effort = 'xhigh' %}
+    {%- endif %}
+    {%- if resolved_reasoning_effort not in ('xhigh', 'medium', 'low') %}
+        {{- raise_exception('Unexpected reasoning effort ' ~ reasoning_effort ~ '. Supported types are xhigh (default), medium, and low.') }}
+    {%- endif %}
+    {%- if resolved_reasoning_effort == 'xhigh' %}
+        {%- set reasoning_instructions = 'Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.' %}
+    {%- elif resolved_reasoning_effort == 'low' %}
+        {%- set reasoning_instructions = 'Reasoning effort is set to low. Keep your thinking brief and focused, moving directly to the conclusion without unnecessary elaboration.' %}
+    {%- endif %}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {%- if reasoning_instructions %}
+        {{- reasoning_instructions + '\n\n' }}
+    {%- endif %}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if merged_system %}
+        {{- '\n\n' + merged_system }}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if merged_system %}
+        {{- '<|im_start|>system\n' + (reasoning_instructions + '\n\n' if reasoning_instructions else '')  + merged_system + '<|im_end|>\n' }}
+    {%- elif reasoning_instructions %}
+        {{- '<|im_start|>system\n' + reasoning_instructions + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- for message in messages %}
+    {%- if loop.index0 >= num_sys %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" or message.role == "developer" %}
+        {{- raise_exception('System message must be at the beginning.') }}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if preserve_thinking is undefined or preserve_thinking is true or loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if tool_call.name is not defined or tool_call.name is none %}
+                    {{- raise_exception('Tool call is missing a function name.') }}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is mapping %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | string if args_value is string else args_value | tojson | safe %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- elif tool_call.arguments is string %}
+                    {%- if tool_call.arguments|trim %}
+                        {{- raise_exception('Tool call arguments for function "' + (tool_call.name | string) + '" were passed as a JSON string. Parse them into an object before calling apply_chat_template.') }}
+                    {%- endif %}
+                {%- elif tool_call.arguments is defined and tool_call.arguments is not none %}
+                    {{- raise_exception('Tool call arguments for function "' + (tool_call.name | string) + '" must be an object/mapping or a JSON string.') }}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}
+{#- Unsloth fixes - developer role, merged system messages, tool calling #}

benchmarks/standard/2026-10-27-standard-rtx-3090-3eb539d8/provenance/corpus-build.json

@@ -0,0 +1,206 @@
+{
+  "protocol_id": "strata-qwen38-thinking-v1-draft.1",
+  "bundle": "standard-v1",
+  "profile": "qwen3.8-flash-next-thinking",
+  "tokenizer_files": {
+    "vocab.json": "4ba64f0332abcfb0b600b7df1537d1e836e68009d5fb7cdb77339738dc6365c4",
+    "merges.txt": "080cf95432173729d160346b7882bd2a644b07f7a3813a3eb73f4e66787bfde9",
+    "token_type.json": "5088c8c298fc06af8382ddb3b76c888703ac2634e82263b97eedcc2ad202738b"
+  },
+  "template": {
+    "path": "serve/chat_template.jinja",
+    "sha256": "ae9644304511ef71f28e036b6f65b4c709880bc5a23b7d0b42125ec63a8c70ca"
+  },
+  "template_kwargs": {
+    "enable_thinking": true,
+    "preserve_thinking": true,
+    "reasoning_effort": "xhigh"
+  },
+  "record_format": "record{:06d}: alpha beta gamma delta epsilon zeta eta theta.",
+  "short_task": "Explain how a hash table finds a value, including collisions and one small example.",
+  "summary_question": "Summarize the structure of these records and explain how to look up a record by its ID.",
+  "record_counts": {
+    "prompt-4k": {
+      "records": 211,
+      "target_tokens": 4096
+    },
+    "prompt-16k": {
+      "records": 858,
+      "target_tokens": 16384
+    }
+  },
+  "length_tolerance": 0.02,
+  "requests": [
+    {
+      "id": "warmup",
+      "input_tokens": 75,
+      "max_shared_prefix_tokens": 47
+    },
+    {
+      "id": "performance-short-r1",
+      "input_tokens": 77,
+      "max_shared_prefix_tokens": 50
+    },
+    {
+      "id": "performance-prompt-4k-r1",
+      "input_tokens": 4093,
+      "max_shared_prefix_tokens": 54
+    },
+    {
+      "id": "performance-prompt-16k-r1",
+      "input_tokens": 16387,
+      "max_shared_prefix_tokens": 55
+    },
+    {
+      "id": "performance-short-r2",
+      "input_tokens": 77,
+      "max_shared_prefix_tokens": 50
+    },
+    {
+      "id": "performance-prompt-4k-r2",
+      "input_tokens": 4093,
+      "max_shared_prefix_tokens": 54
+    },
+    {
+      "id": "performance-prompt-16k-r2",
+      "input_tokens": 16387,
+      "max_shared_prefix_tokens": 55
+    },
+    {
+      "id": "performance-short-r3",
+      "input_tokens": 77,
+      "max_shared_prefix_tokens": 50
+    },
+    {
+      "id": "performance-prompt-4k-r3",
+      "input_tokens": 4093,
+      "max_shared_prefix_tokens": 54
+    },
+    {
+      "id": "performance-prompt-16k-r3",
+      "input_tokens": 16387,
+      "max_shared_prefix_tokens": 55
+    },
+    {
+      "id": "sanity-I1",
+      "input_tokens": 82,
+      "max_shared_prefix_tokens": 49
+    },
+    {
+      "id": "sanity-I2",
+      "input_tokens": 90,
+      "max_shared_prefix_tokens": 49
+    },
+    {
+      "id": "sanity-I3",
+      "input_tokens": 85,
+      "max_shared_prefix_tokens": 49
+    },
+    {
+      "id": "sanity-I4",
+      "input_tokens": 71,
+      "max_shared_prefix_tokens": 49
+    },
+    {
+      "id": "sanity-R1",
+      "input_tokens": 4089,
+      "max_shared_prefix_tokens": 49
+    },
+    {
+      "id": "sanity-R2",
+      "input_tokens": 4089,
+      "max_shared_prefix_tokens": 49
+    },
+    {
+      "id": "sanity-R3",
+      "input_tokens": 16382,
+      "max_shared_prefix_tokens": 49
+    },
+    {
+      "id": "sanity-R4",
+      "input_tokens": 16390,
+      "max_shared_prefix_tokens": 49
+    },
+    {
+      "id": "sanity-T1",
+      "input_tokens": 325,
+      "max_shared_prefix_tokens": 61
+    },
+    {
+      "id": "sanity-T2",
+      "input_tokens": 335,
+      "max_shared_prefix_tokens": 62
+    },
+    {
+      "id": "sanity-T3",
+      "input_tokens": 352,
+      "max_shared_prefix_tokens": 61
+    },
+    {
+      "id": "sanity-T4",
+      "input_tokens": 339,
+      "max_shared_prefix_tokens": 62
+    },
+    {
+      "id": "concurrency-r1-c1",
+      "input_tokens": 78,
+      "max_shared_prefix_tokens": 51
+    },
+    {
+      "id": "concurrency-r1-c2",
+      "input_tokens": 78,
+      "max_shared_prefix_tokens": 51
+    },
+    {
+      "id": "concurrency-r1-c3",
+      "input_tokens": 78,
+      "max_shared_prefix_tokens": 51
+    },
+    {
+      "id": "concurrency-r1-c4",
+      "input_tokens": 78,
+      "max_shared_prefix_tokens": 51
+    },
+    {
+      "id": "concurrency-r2-c1",
+      "input_tokens": 78,
+      "max_shared_prefix_tokens": 51
+    },
+    {
+      "id": "concurrency-r2-c2",
+      "input_tokens": 78,
+      "max_shared_prefix_tokens": 51
+    },
+    {
+      "id": "concurrency-r2-c3",
+      "input_tokens": 78,
+      "max_shared_prefix_tokens": 51
+    },
+    {
+      "id": "concurrency-r2-c4",
+      "input_tokens": 78,
+      "max_shared_prefix_tokens": 51
+    },
+    {
+      "id": "concurrency-r3-c1",
+      "input_tokens": 78,
+      "max_shared_prefix_tokens": 51
+    },
+    {
+      "id": "concurrency-r3-c2",
+      "input_tokens": 78,
+      "max_shared_prefix_tokens": 51
+    },
+    {
+      "id": "concurrency-r3-c3",
+      "input_tokens": 78,
+      "max_shared_prefix_tokens": 51
+    },
+    {
+      "id": "concurrency-r3-c4",
+      "input_tokens": 78,
+      "max_shared_prefix_tokens": 51
+    }
+  ],
+  "note": "input_tokens counts the rendered thinking request with the generation prompt. max_shared_prefix_tokens is the longest token prefix this request shares with any other request of the bundle: cache reuse above it means the user payload was read from the cache."
+}

benchmarks/standard/2026-10-27-standard-rtx-3090-3eb539d8/provenance/corpus-manifest.sha256

@@ -0,0 +1,35 @@
+bc6c1f81d86863c3dc36a31f58010701d8ab05180f69f86c6978214409e43af2  build.json
+f724855774dfbea89fade712a70d534a085c88cda6abc123db419fd79eefab49  requests/concurrency-r1-c1.json
+2c85f877abbf144ca065c58efc2a7568aaf2c8dc317ad8266e78c149f8245abb  requests/concurrency-r1-c2.json
+264097f61645a0d5089562dc003b7748b1c20339c2df1290ad6664fc1834068a  requests/concurrency-r1-c3.json
+65895c5a5fab53c06e9358d88d921f1bfc9dc7cc8b6e8a2521ac90de080ff470  requests/concurrency-r1-c4.json
+2c08d50e0c95e345512ec918f58fa716780b79900f09bbfabc37519c8d86543a  requests/concurrency-r2-c1.json
+8aba49693701b9315b2bc5ad55e97a8b03496baedd05f370c49c2c77f45ce6cd  requests/concurrency-r2-c2.json
+7cfb4a5a8fb5702244ce42e0dba9ccd3064d4023d700b409a1a472790a82809c  requests/concurrency-r2-c3.json
+3ab56fbba5fd405945e53cff01b59c3ef966b0f08a0a1b0b50633d26e90cad62  requests/concurrency-r2-c4.json
+3adb684cf2cb84d4e514de30a4c6f8ccebdd8471160cfcdfc0cdb8ca19ac2137  requests/concurrency-r3-c1.json
+650379ef9a4da667f6581ae9573eb9af2db2a1fc617701e50536590b18f70077  requests/concurrency-r3-c2.json
+111415388c5e69ecbd9ab66efe85eb8f3db9142201e1548d03203d3acc05eca1  requests/concurrency-r3-c3.json
+691a8d27fd06e9beab57e6efb3d82c2a7ed9fc26635fd51b1b1d017117440c1e  requests/concurrency-r3-c4.json
+06776ef38485dc4bb7e18809534fe16a57d421d163f81d4d1532c4b08225968e  requests/performance-prompt-16k-r1.json
+4995c5f3c47c1682f4dc753a9f3a1fba6920fea14691e1ded2da6208ef5fe628  requests/performance-prompt-16k-r2.json
+7af4c49f99192007be27110296b5a0bcc2efbc91c7ff1c337a6cac6732050559  requests/performance-prompt-16k-r3.json
+bda0e60c2de0703899ac1bd5711d793b9b325c7f8ba6be6c967f22dd07d8dda8  requests/performance-prompt-4k-r1.json
+e898eec0ee22499473179a719505c9feb9d3772658833f607dffb34c3727150e  requests/performance-prompt-4k-r2.json
+6d3b3300eace318574490f4379119409920cbe7e88fdaff28eb6f5c79430c3e3  requests/performance-prompt-4k-r3.json
+483b4f18fcc64ca802fde3377c59bf7b114ec5a5c96fe781bd2be4665ef7b898  requests/performance-short-r1.json
+b0ec85f40117303d19c8c17a4813625c8554f59caddbe69e7d8aafdcf51d501a  requests/performance-short-r2.json
+e210da20b80c61147da5102fcba65131dafbbb4ceee952fa6b91ce2bb06581ed  requests/performance-short-r3.json
+70ebcd31af5ae1bc7573136d9a257469d8ad9e85b5d0233980998e957c36d6c2  requests/sanity-I1.json
+1cc6e4bc56b5a22043b6e384b3036170dc6d3edeb403be391a8b802b5613f4cb  requests/sanity-I2.json
+c29b13e6e31b674ff733202b3c0b1f9410768b9ea9368a4a6a13430b3e011fb8  requests/sanity-I3.json
+780fbd1d02f119323cf50ac4dabcfffbd23dd84115efbe18152e7ee9112b041a  requests/sanity-I4.json
+3c1e0718ad06e591e47cf8df9c36bab2d13332d44b05e440ec17d0402bf9facc  requests/sanity-R1.json
+352e054c48e08e11d339d3a02a14f4f50125071710679b8d04b77aba711fbafc  requests/sanity-R2.json
+36770434f1ea046ec84b32691d644e8416e89fbcbb04b8f6032da66016d78291  requests/sanity-R3.json
+a63df33c784732cc6b4d8a0f2258b7226c8df5a3b4f6818f9e886bbf09604999  requests/sanity-R4.json
+6b272ef684b43c1e0004a40b958c807b84dcd9de579d4b2d8819e39ae4196fb2  requests/sanity-T1.json
+8253ee4aa6b19e61e28d64406807218c9aa10cf2cf682101e8a2d6a50564de46  requests/sanity-T2.json
+aa2be8761793f51eab6ef04bc54c96e018f6981b7a6a26b9e50cc3c7ac856803  requests/sanity-T3.json
+04043bdec397df487bebd2999e280a9fae5925e5efec1a10fa96055ec3430faa  requests/sanity-T4.json
+d3365c57db8be1fd0dd9771f74f264b1f6994a396e472c3e5fa4ede2641ffb5c  requests/warmup.json

benchmarks/standard/2026-10-27-standard-rtx-3090-58540dcc/README.md

@@ -0,0 +1,74 @@
+# <user> standard benchmark: NVIDIA GeForce RTX 3090
+
+Protocol `strata-qwen38-thinking-v1-draft.1` (draft), run `20261027T192209Z-58540dcc`, started 2026-10-27T19:22:09Z. **Status: nonconforming.**
+
+Written by `<user> bench run`. The machine-readable result is [results.json](results.json); every request, its raw usage and timings and the client event offsets are in [outputs.jsonl](outputs.jsonl).
+
+This run is not a complete standard result:
+
+- no model file could be hashed
+
+## Hardware and software
+
+- GPU: NVIDIA GeForce RTX 3090 (24.0 GiB dedicated)
+- CPU: Intel(R) Core(TM) i9-14900KS; RAM: 110.0 GiB
+- Storage of the model files: not measured
+- OS: NixOS 26.05 (Yarara) 6.18.55; drivers: gpu-driver 595.71.05; runtime: cuda 13.2
+- <user> commit: `4b9e05a2a806a3fe24f7b2560d98626c37dade35`; engine 0.1.51; source build
+- Background work: not measured
+
+## Model and configuration
+
+- Model: qwen3.8-flash-next-unsloth-ud-q4_k_xl; quantization UD-Q4_K_XL; revision not measured; source revision not measured
+- Files: not measured
+- Track: **custom** (UD-Q4_K_XL rep 1, 275 W, case closed)
+- Context 131072 tokens; server capacity 1 slot(s)
+- Sampling: temperature 1.0, top_p 0.95, top_k 20, min_p 0.0, presence_penalty 0.0, repetition_penalty 1.0, reasoning_effort xhigh; thinking on, preserved
+- Engine options: `config.json`; chat template: `provenance/chat_template.jinja`
+
+## Method
+
+The frozen requests of `bench/prompts/standard-v1` (corpus `e75eff8b9feea04f...`) were sent over loopback HTTP, streaming, one at a time, in this order: 1 warm-up (excluded), 3 repetitions of the short, 4K and 16K prompts, and the twelve sanity checks. Prompt and decode speed are the engine's own `timings`; time to first token is the client's monotonic clock from dispatch to the first nonempty reasoning, answer or tool-call delta. No request was retried. Time after the model loaded: 2 min 39 s.
+
+## Results
+
+| Case | Prompt tok/s | Decode tok/s | TTFT s | First answer s | Total s | Capped |
+| --- | --- | --- | --- | --- | --- | --- |
+| short | n/a | 66.9 (66.6-70.1) n=3 | 1.14 (0.94-1.16) n=3 | 2.42 (2.33-2.93) n=3 | 5.9 (5.2-6.1) n=3 | 0/3 |
+| prompt-4k | 845.8 (811.7-855.6) n=3 | 69.7 (67.2-73.5) n=3 | 4.86 (4.81-5.07) n=3 | 9.05 (8.27-9.82) n=2 | 12.1 (11.8-12.7) n=3 | 3/3 |
+| prompt-16k | 1917.7 (1914.3-1919.4) n=3 | 67.8 (67.6-77.1) n=3 | 8.57 (8.57-8.60) n=3 | 15.11 (12.55-15.64) n=3 | 16.1 (15.2-16.1) n=3 | 3/3 |
+
+Cells are median (min-max) n=samples. Short is a prompt of about 80 tokens: its prompt rate is dominated by the fixed cost of a request, so it is not shown; read prompt speed from prompt-4k and prompt-16k. Capped = requests that reached the 512-token cap, reasoning and answer together. A capped request's total time is the time to the cap, not to an answer, and it has no first-answer time when it was still reasoning.
+
+| Phase | Planned | Completed | Truncated | Failed | Skipped |
+| --- | ---: | ---: | ---: | ---: | ---: |
+| warmup | 1 | 0 | 1 | 0 | 0 |
+| performance | 9 | 3 | 6 | 0 | 0 |
+| sanity | 12 | 12 | 0 | 0 | 0 |
+| concurrency | 0 | 0 | 0 | 0 | 0 |
+
+Sanity checks: **12 of 12**.
+
+| Case | Result | Reason |
+| --- | --- | --- |
+| I1 | pass | the expected JSON value |
+| I2 | pass | the expected JSON value |
+| I3 | pass | the expected JSON value |
+| I4 | pass | the exact expected text |
+| R1 | pass | the exact expected text |
+| R2 | pass | the exact expected text |
+| R3 | pass | the exact expected text |
+| R4 | pass | the expected JSON value |
+| T1 | pass | the expected tool call |
+| T2 | pass | the expected tool call |
+| T3 | pass | the expected tool call |
+| T4 | pass | the expected tool call |
+
+## Failures, exclusions and limitations
+
+- No request failed.
+- Excluded from speed summaries: performance-prompt-4k-r3: first_answer_s is not available for this request
+- Prompt-prefix reuse is the shared template prefix only; the cache was not emptied between requests, and the expert cache stays warm and evolves. These are not cold-model measurements.
+- Output caps include reasoning; a request that reaches its cap is `truncated` and fails its sanity check.
+- The runner cannot see other programs on the machine, the repeat-stop and loop-recovery switches of the server, or the quantization repository's revision; see `missing` in results.json for every value it could not read.
+- The server had answered no request before this run (a freshly loaded engine).

benchmarks/standard/2026-10-27-standard-rtx-3090-58540dcc/config.json

@@ -0,0 +1,13 @@
+{
+  "engine_options": null,
+  "other_engine_options": {},
+  "server_defaults": {
+    "temperature": 1.0,
+    "top_p": 0.95,
+    "top_k": 20,
+    "n_predict": -1
+  },
+  "total_slots": 1,
+  "context": 131072,
+  "source": "the server was already running; its engine options are not exposed"
+}

benchmarks/standard/2026-10-27-standard-rtx-3090-58540dcc/provenance/chat_template.jinja

@@ -0,0 +1,184 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- set sysns = namespace(count=0, text='') %}
+{%- for message in messages %}
+    {%- if sysns.count == loop.index0 and (message.role == 'system' or message.role == 'developer') %}
+        {%- set sys_content = render_content(message.content, false, true)|trim %}
+        {%- if sys_content %}
+            {%- set sysns.text = sysns.text + ('\n' if sysns.text else '') + sys_content %}
+        {%- endif %}
+        {%- set sysns.count = sysns.count + 1 %}
+    {%- endif %}
+{%- endfor %}
+{%- set num_sys = sysns.count %}
+{%- set merged_system = sysns.text %}
+{%- set reasoning_instructions = '' %}
+{%- if enable_thinking is undefined or enable_thinking is true %}
+    {%- set resolved_reasoning_effort = reasoning_effort|default('xhigh') %}
+    {%- if resolved_reasoning_effort == 'high' %}
+        {%- set resolved_reasoning_effort = 'xhigh' %}
+    {%- endif %}
+    {%- if resolved_reasoning_effort not in ('xhigh', 'medium', 'low') %}
+        {{- raise_exception('Unexpected reasoning effort ' ~ reasoning_effort ~ '. Supported types are xhigh (default), medium, and low.') }}
+    {%- endif %}
+    {%- if resolved_reasoning_effort == 'xhigh' %}
+        {%- set reasoning_instructions = 'Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.' %}
+    {%- elif resolved_reasoning_effort == 'low' %}
+        {%- set reasoning_instructions = 'Reasoning effort is set to low. Keep your thinking brief and focused, moving directly to the conclusion without unnecessary elaboration.' %}
+    {%- endif %}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {%- if reasoning_instructions %}
+        {{- reasoning_instructions + '\n\n' }}
+    {%- endif %}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if merged_system %}
+        {{- '\n\n' + merged_system }}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if merged_system %}
+        {{- '<|im_start|>system\n' + (reasoning_instructions + '\n\n' if reasoning_instructions else '')  + merged_system + '<|im_end|>\n' }}
+    {%- elif reasoning_instructions %}
+        {{- '<|im_start|>system\n' + reasoning_instructions + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- for message in messages %}
+    {%- if loop.index0 >= num_sys %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" or message.role == "developer" %}
+        {{- raise_exception('System message must be at the beginning.') }}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if preserve_thinking is undefined or preserve_thinking is true or loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if tool_call.name is not defined or tool_call.name is none %}
+                    {{- raise_exception('Tool call is missing a function name.') }}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is mapping %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | string if args_value is string else args_value | tojson | safe %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- elif tool_call.arguments is string %}
+                    {%- if tool_call.arguments|trim %}
+                        {{- raise_exception('Tool call arguments for function "' + (tool_call.name | string) + '" were passed as a JSON string. Parse them into an object before calling apply_chat_template.') }}
+                    {%- endif %}
+                {%- elif tool_call.arguments is defined and tool_call.arguments is not none %}
+                    {{- raise_exception('Tool call arguments for function "' + (tool_call.name | string) + '" must be an object/mapping or a JSON string.') }}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}
+{#- Unsloth fixes - developer role, merged system messages, tool calling #}

benchmarks/standard/2026-10-27-standard-rtx-3090-58540dcc/provenance/corpus-build.json

@@ -0,0 +1,206 @@
+{
+  "protocol_id": "strata-qwen38-thinking-v1-draft.1",
+  "bundle": "standard-v1",
+  "profile": "qwen3.8-flash-next-thinking",
+  "tokenizer_files": {
+    "vocab.json": "4ba64f0332abcfb0b600b7df1537d1e836e68009d5fb7cdb77339738dc6365c4",
+    "merges.txt": "080cf95432173729d160346b7882bd2a644b07f7a3813a3eb73f4e66787bfde9",
+    "token_type.json": "5088c8c298fc06af8382ddb3b76c888703ac2634e82263b97eedcc2ad202738b"
+  },
+  "template": {
+    "path": "serve/chat_template.jinja",
+    "sha256": "ae9644304511ef71f28e036b6f65b4c709880bc5a23b7d0b42125ec63a8c70ca"
+  },
+  "template_kwargs": {
+    "enable_thinking": true,
+    "preserve_thinking": true,
+    "reasoning_effort": "xhigh"
+  },
+  "record_format": "record{:06d}: alpha beta gamma delta epsilon zeta eta theta.",
+  "short_task": "Explain how a hash table finds a value, including collisions and one small example.",
+  "summary_question": "Summarize the structure of these records and explain how to look up a record by its ID.",
+  "record_counts": {
+    "prompt-4k": {
+      "records": 211,
+      "target_tokens": 4096
+    },
+    "prompt-16k": {
+      "records": 858,
+      "target_tokens": 16384
+    }
+  },
+  "length_tolerance": 0.02,
+  "requests": [
+    {
+      "id": "warmup",
+      "input_tokens": 75,
+      "max_shared_prefix_tokens": 47
+    },
+    {
+      "id": "performance-short-r1",
+      "input_tokens": 77,
+      "max_shared_prefix_tokens": 50
+    },
+    {
+      "id": "performance-prompt-4k-r1",
+      "input_tokens": 4093,
+      "max_shared_prefix_tokens": 54
+    },
+    {
+      "id": "performance-prompt-16k-r1",
+      "input_tokens": 16387,
+      "max_shared_prefix_tokens": 55
+    },
+    {
+      "id": "performance-short-r2",
+      "input_tokens": 77,
+      "max_shared_prefix_tokens": 50
+    },
+    {
+      "id": "performance-prompt-4k-r2",
+      "input_tokens": 4093,
+      "max_shared_prefix_tokens": 54
+    },
+    {
+      "id": "performance-prompt-16k-r2",
+      "input_tokens": 16387,
+      "max_shared_prefix_tokens": 55
+    },
+    {
+      "id": "performance-short-r3",
+      "input_tokens": 77,
+      "max_shared_prefix_tokens": 50
+    },
+    {
+      "id": "performance-prompt-4k-r3",
+      "input_tokens": 4093,
+      "max_shared_prefix_tokens": 54
+    },
+    {
+      "id": "performance-prompt-16k-r3",
+      "input_tokens": 16387,
+      "max_shared_prefix_tokens": 55
+    },
+    {
+      "id": "sanity-I1",
+      "input_tokens": 82,
+      "max_shared_prefix_tokens": 49
+    },
+    {
+      "id": "sanity-I2",
+      "input_tokens": 90,
+      "max_shared_prefix_tokens": 49
+    },
+    {
+      "id": "sanity-I3",
+      "input_tokens": 85,
+      "max_shared_prefix_tokens": 49
+    },
+    {
+      "id": "sanity-I4",
+      "input_tokens": 71,
+      "max_shared_prefix_tokens": 49
+    },
+    {
+      "id": "sanity-R1",
+      "input_tokens": 4089,
+      "max_shared_prefix_tokens": 49
+    },
+    {
+      "id": "sanity-R2",
+      "input_tokens": 4089,
+      "max_shared_prefix_tokens": 49
+    },
+    {
+      "id": "sanity-R3",
+      "input_tokens": 16382,
+      "max_shared_prefix_tokens": 49
+    },
+    {
+      "id": "sanity-R4",
+      "input_tokens": 16390,
+      "max_shared_prefix_tokens": 49
+    },
+    {
+      "id": "sanity-T1",
+      "input_tokens": 325,
+      "max_shared_prefix_tokens": 61
+    },
+    {
+      "id": "sanity-T2",
+      "input_tokens": 335,
+      "max_shared_prefix_tokens": 62
+    },
+    {
+      "id": "sanity-T3",
+      "input_tokens": 352,
+      "max_shared_prefix_tokens": 61
+    },
+    {
+      "id": "sanity-T4",
+      "input_tokens": 339,
+      "max_shared_prefix_tokens": 62
+    },
+    {
+      "id": "concurrency-r1-c1",
+      "input_tokens": 78,
+      "max_shared_prefix_tokens": 51
+    },
+    {
+      "id": "concurrency-r1-c2",
+      "input_tokens": 78,
+      "max_shared_prefix_tokens": 51
+    },
+    {
+      "id": "concurrency-r1-c3",
+      "input_tokens": 78,
+      "max_shared_prefix_tokens": 51
+    },
+    {
+      "id": "concurrency-r1-c4",
+      "input_tokens": 78,
+      "max_shared_prefix_tokens": 51
+    },
+    {
+      "id": "concurrency-r2-c1",
+      "input_tokens": 78,
+      "max_shared_prefix_tokens": 51
+    },
+    {
+      "id": "concurrency-r2-c2",
+      "input_tokens": 78,
+      "max_shared_prefix_tokens": 51
+    },
+    {
+      "id": "concurrency-r2-c3",
+      "input_tokens": 78,
+      "max_shared_prefix_tokens": 51
+    },
+    {
+      "id": "concurrency-r2-c4",
+      "input_tokens": 78,
+      "max_shared_prefix_tokens": 51
+    },
+    {
+      "id": "concurrency-r3-c1",
+      "input_tokens": 78,
+      "max_shared_prefix_tokens": 51
+    },
+    {
+      "id": "concurrency-r3-c2",
+      "input_tokens": 78,
+      "max_shared_prefix_tokens": 51
+    },
+    {
+      "id": "concurrency-r3-c3",
+      "input_tokens": 78,
+      "max_shared_prefix_tokens": 51
+    },
+    {
+      "id": "concurrency-r3-c4",
+      "input_tokens": 78,
+      "max_shared_prefix_tokens": 51
+    }
+  ],
+  "note": "input_tokens counts the rendered thinking request with the generation prompt. max_shared_prefix_tokens is the longest token prefix this request shares with any other request of the bundle: cache reuse above it means the user payload was read from the cache."
+}

benchmarks/standard/2026-10-27-standard-rtx-3090-58540dcc/provenance/corpus-manifest.sha256

@@ -0,0 +1,35 @@
+bc6c1f81d86863c3dc36a31f58010701d8ab05180f69f86c6978214409e43af2  build.json
+f724855774dfbea89fade712a70d534a085c88cda6abc123db419fd79eefab49  requests/concurrency-r1-c1.json
+2c85f877abbf144ca065c58efc2a7568aaf2c8dc317ad8266e78c149f8245abb  requests/concurrency-r1-c2.json
+264097f61645a0d5089562dc003b7748b1c20339c2df1290ad6664fc1834068a  requests/concurrency-r1-c3.json
+65895c5a5fab53c06e9358d88d921f1bfc9dc7cc8b6e8a2521ac90de080ff470  requests/concurrency-r1-c4.json
+2c08d50e0c95e345512ec918f58fa716780b79900f09bbfabc37519c8d86543a  requests/concurrency-r2-c1.json
+8aba49693701b9315b2bc5ad55e97a8b03496baedd05f370c49c2c77f45ce6cd  requests/concurrency-r2-c2.json
+7cfb4a5a8fb5702244ce42e0dba9ccd3064d4023d700b409a1a472790a82809c  requests/concurrency-r2-c3.json
+3ab56fbba5fd405945e53cff01b59c3ef966b0f08a0a1b0b50633d26e90cad62  requests/concurrency-r2-c4.json
+3adb684cf2cb84d4e514de30a4c6f8ccebdd8471160cfcdfc0cdb8ca19ac2137  requests/concurrency-r3-c1.json
+650379ef9a4da667f6581ae9573eb9af2db2a1fc617701e50536590b18f70077  requests/concurrency-r3-c2.json
+111415388c5e69ecbd9ab66efe85eb8f3db9142201e1548d03203d3acc05eca1  requests/concurrency-r3-c3.json
+691a8d27fd06e9beab57e6efb3d82c2a7ed9fc26635fd51b1b1d017117440c1e  requests/concurrency-r3-c4.json
+06776ef38485dc4bb7e18809534fe16a57d421d163f81d4d1532c4b08225968e  requests/performance-prompt-16k-r1.json
+4995c5f3c47c1682f4dc753a9f3a1fba6920fea14691e1ded2da6208ef5fe628  requests/performance-prompt-16k-r2.json
+7af4c49f99192007be27110296b5a0bcc2efbc91c7ff1c337a6cac6732050559  requests/performance-prompt-16k-r3.json
+bda0e60c2de0703899ac1bd5711d793b9b325c7f8ba6be6c967f22dd07d8dda8  requests/performance-prompt-4k-r1.json
+e898eec0ee22499473179a719505c9feb9d3772658833f607dffb34c3727150e  requests/performance-prompt-4k-r2.json
+6d3b3300eace318574490f4379119409920cbe7e88fdaff28eb6f5c79430c3e3  requests/performance-prompt-4k-r3.json
+483b4f18fcc64ca802fde3377c59bf7b114ec5a5c96fe781bd2be4665ef7b898  requests/performance-short-r1.json
+b0ec85f40117303d19c8c17a4813625c8554f59caddbe69e7d8aafdcf51d501a  requests/performance-short-r2.json
+e210da20b80c61147da5102fcba65131dafbbb4ceee952fa6b91ce2bb06581ed  requests/performance-short-r3.json
+70ebcd31af5ae1bc7573136d9a257469d8ad9e85b5d0233980998e957c36d6c2  requests/sanity-I1.json
+1cc6e4bc56b5a22043b6e384b3036170dc6d3edeb403be391a8b802b5613f4cb  requests/sanity-I2.json
+c29b13e6e31b674ff733202b3c0b1f9410768b9ea9368a4a6a13430b3e011fb8  requests/sanity-I3.json
+780fbd1d02f119323cf50ac4dabcfffbd23dd84115efbe18152e7ee9112b041a  requests/sanity-I4.json
+3c1e0718ad06e591e47cf8df9c36bab2d13332d44b05e440ec17d0402bf9facc  requests/sanity-R1.json
+352e054c48e08e11d339d3a02a14f4f50125071710679b8d04b77aba711fbafc  requests/sanity-R2.json
+36770434f1ea046ec84b32691d644e8416e89fbcbb04b8f6032da66016d78291  requests/sanity-R3.json
+a63df33c784732cc6b4d8a0f2258b7226c8df5a3b4f6818f9e886bbf09604999  requests/sanity-R4.json
+6b272ef684b43c1e0004a40b958c807b84dcd9de579d4b2d8819e39ae4196fb2  requests/sanity-T1.json
+8253ee4aa6b19e61e28d64406807218c9aa10cf2cf682101e8a2d6a50564de46  requests/sanity-T2.json
+aa2be8761793f51eab6ef04bc54c96e018f6981b7a6a26b9e50cc3c7ac856803  requests/sanity-T3.json
+04043bdec397df487bebd2999e280a9fae5925e5efec1a10fa96055ec3430faa  requests/sanity-T4.json
+d3365c57db8be1fd0dd9771f74f264b1f6994a396e472c3e5fa4ede2641ffb5c  requests/warmup.json

benchmarks/standard/README.md

@@ -5,8 +5,9 @@ short / 4K / 16K prompts with a 512-token cap at the Qwen model-card sampling (t
 on, effort xhigh), then 12 sanity cases (instruction, retrieval, tool calls). Each folder is the runner's
 own output: `README.md` (its report), `results.json`, `config.json`, `outputs.jsonl`, `provenance/`.
 
-All runs 2026-10-26, IQ3_S, engine 0.1.51 at `4b9e05a2`, 250 W cap, gpu-unlock, idle server, run from
-the guest over loopback.
+Engine 0.1.51 at `4b9e05a2`, gpu-unlock, idle server, run from the guest over loopback. Tuning arms and
+the power-cap runs use IQ3_S ([`strata-iq3_s.json`](../../strata/strata-iq3_s.json)); the quant comparison
+at the end says which config each run used.
 
 ## Tuning arms
 
@@ -72,10 +73,33 @@ While busy (samples above 150 W):
 - GPU1 still hits its power limit most of the time at 275 W, so a higher cap would probably buy a
   little more prompt speed.
 
+## Quant: IQ3_S vs UD-Q4_K_XL
+
+2026-10-27, 275 W, case closed, setup defaults for each quant
+([`strata-iq3_s.json`](../../strata/strata-iq3_s.json),
+[`strata-unsloth-ud-q4_k_xl.json`](../../strata/strata-unsloth-ud-q4_k_xl.json)), fresh restart before each
+run. IQ3_S rows are the two 275 W runs from the power-cap test.
+
+| Quant | Folder | Prompt 4K tok/s | Prompt 16K tok/s | TTFT 16K | Decode short | Decode 4K | Decode 16K |
+|---|---|---:|---:|---:|---:|---:|---:|
+| IQ3_S | `5c64d992` | 1,742 | 3,030 | 5.44 s | 125.3 | 115.3 | 116.1 |
+| IQ3_S | `16ea71d0` | 1,743 | 3,016 | 5.46 s | 114.2 | 111.6 | 111.0 |
+| UD-Q4_K_XL | `58540dcc` | 846 | 1,918 | 8.57 s | 66.9 | 69.7 | 67.8 |
+| UD-Q4_K_XL | `3eb539d8` | 847 | 1,918 | 8.57 s | 65.7 | 68.9 | 76.3 |
+
+- **UD-Q4_K_XL is much slower:** prompt −51% at 4K and −37% at 16K (+3.1 s to first token at 16K),
+  decode −40% (~66–76 vs ~111–125 tok/s). Both repeats agree to within 0.2% on prompt speed.
+- Why: its experts are 71.7 GiB vs 47 GiB, so only 5,328 fit in VRAM (15.6 GiB) vs 9,508 for IQ3_S, and
+  every miss moves more bytes. Load took 2 min 22 s; 82 GB of the guest's RAM in use, 27 GB available.
+- The GPUs work less hard: mean 211–227 W while busy (IQ3_S 228–239 W), peaks 54 / 63°C.
+- All 12 sanity checks passed in both runs. Quality differences these checks can't see are left to
+  husky-bench; Strata's yardstick puts UD-Q4_K_XL at 93.1% same-top-token as Q8_0 vs 87.7% for IQ3_S.
+
 ## Not tried yet
 
 - Huge pages for the expert arena: the engine logs `MAP_HUGETLB unavailable (needed 23983 2 MiB pages,
   vm.nr_hugepages=0)`. That would mean reserving ~47 GB of huge pages in the guest (a NixOS change and a
   reboot).
 - Power cap above 275 W (300–350 W). gpu-unlock GEMM was +40% at 350 W, but 275 W gave only +4.6% prompt speed.
-- Other quants.
+- Other quants: UD-IQ4_XS (barely better than IQ3_S on Strata's yardstick), IQ3_XXS / IQ2_XS (smaller,
+  faster, worse).

docs/changelog.md

@@ -2,6 +2,15 @@
 
 Newest first. Each entry: what broke, the cause, and what fixed it.
 
+## 2026-10-27: switched to UD-Q4_K_XL for testing
+
+`strataConfig = "strata-unsloth-ud-q4_k_xl.json"`, `nixos-rebuild switch`. Loads in 2 min 22 s with all
+71.7 GiB of experts in RAM (82 GB used, 27 GB available) and 5,328 experts in VRAM. Standard benchmark,
+two runs after fresh restarts: prompt 846 / 1,918 tok/s at 4K / 16K (IQ3_S: 1,742 / 3,023), decode
+~66–76 tok/s (IQ3_S ~111–125); 12/12 sanity checks. Much slower; kept active only to test quality with
+husky-bench. Results: [benchmarks/standard](../benchmarks/standard/README.md#quant-iq3_s-vs-ud-q4_k_xl).
+To go back: set `strataConfig` to `strata-iq3_s.json` and rebuild.
+
 ## 2026-10-27: UD-Q4_K_XL installed, configs tracked per model
 
 Downloaded Unsloth's UD-Q4_K_XL (111 GB, SHA-256 checked by setup) to test a bigger quant than IQ3_S: Strata's

docs/strata.md

@@ -12,7 +12,8 @@ pipeline, so it needs no NVLink.
 
 ## Decisions
 
-- IQ3_S quant
+- IQ3_S quant. UD-Q4_K_XL is installed too (better quality on Strata's yardstick, ~40% slower); the
+  active one is in [`strata/README.md`](../strata/README.md).
 - Source build at a reviewed commit. The prebuilt engine zip installs unverified if GitHub returns no
   digest, so we don't use it.
 - Whole LAN with an API key (default would bind 127.0.0.1)

nixos/configuration.nix

@@ -116,7 +116,7 @@
 
   # The API server (setup's run-<model>.sh) with offline mode and no downloads. The API key and engine args live in
   # the run config setup wrote; host 0.0.0.0, port 8080. strataConfig picks the active one (repo: strata/README.md).
-  systemd.services.strata = let strataConfig = "strata-iq3_s.json"; in {
+  systemd.services.strata = let strataConfig = "strata-unsloth-ud-q4_k_xl.json"; in {
     description = "Strata inference server (${strataConfig})";
     after = [ "network-online.target" "nvidia-persistenced.service" "nvidia-power-limit.service" ];
     wants = [ "network-online.target" ];

strata/README.md

@@ -5,12 +5,12 @@ One file per configuration: the run config `setup.sh` wrote on the guest
 every config it has run; the service starts whichever one `strataConfig` names in
 [`nixos/configuration.nix`](../nixos/configuration.nix).
 
-**Active:** `strata-iq3_s.json`
+**Active:** `strata-unsloth-ud-q4_k_xl.json` (since 2026-10-27, under test)
 
 | Config | Model / quant | Experts in RAM | Status | Benchmarks |
 |---|---|---|---|---|
-| [`strata-iq3_s.json`](strata-iq3_s.json) | Qwen3.8-Flash-Next, ISTA-DASLab GSQ-RCO IQ3_S | 47 GB arena + 29 GB PLE table | active since 2026-10-26 | [standard](../benchmarks/standard/README.md), husky-bench `*iq3_s*` |
-| [`strata-unsloth-ud-q4_k_xl.json`](strata-unsloth-ud-q4_k_xl.json) | Qwen3.8-Flash-Next, Unsloth UD-Q4_K_XL (pinned rev `38bb39e`), env `STRATA_HC_Q8=1`, `STRATA_ROUTE_TAIL_SKIP=0` | all 77 GB (setup's estimate) + 29 GB PLE table | installed 2026-10-27, testing | not run yet |
+| [`strata-iq3_s.json`](strata-iq3_s.json) | Qwen3.8-Flash-Next, ISTA-DASLab GSQ-RCO IQ3_S | 47 GB arena + 29 GB PLE table | active 2026-10-26 → 10-27, the fast one | [standard](../benchmarks/standard/README.md), husky-bench `*iq3_s*` |
+| [`strata-unsloth-ud-q4_k_xl.json`](strata-unsloth-ud-q4_k_xl.json) | Qwen3.8-Flash-Next, Unsloth UD-Q4_K_XL (pinned rev `38bb39e`), env `STRATA_HC_Q8=1`, `STRATA_ROUTE_TAIL_SKIP=0` | all 77 GB (setup's estimate) + 29 GB PLE table | active since 2026-10-27, testing: −37–51% prompt, −40% decode vs IQ3_S | [standard](../benchmarks/standard/README.md#quant-iq3_s-vs-ud-q4_k_xl) |
 
 ## Switching