lovey/benchmarks/standard/2026-10-27-standard-rtx-3090-53bbc5d8

standard benchmark: NVIDIA GeForce RTX 3090

Protocol strata-qwen38-thinking-v1-draft.1 (draft), run 20261027T000709Z-53bbc5d8, started 2026-10-27T00:07:09Z. Status: nonconforming.

Written by <user> bench run. The machine-readable result is results.json; every request, its raw usage and timings and the client event offsets are in outputs.jsonl.

This run is not a complete standard result:

• no model file could be hashed

Hardware and software

• GPU: NVIDIA GeForce RTX 3090 (24.0 GiB dedicated)
• CPU: Intel(R) Core(TM) i9-14900KS; RAM: 110.0 GiB
• Storage of the model files: not measured
• OS: NixOS 26.05 (Yarara) 6.18.55; drivers: gpu-driver 595.71.05; runtime: cuda 13.2
• commit: 4b9e05a2a806a3fe24f7b2560d98626c37dade35; engine 0.1.51; source build
• Background work: not measured

Model and configuration

• Model: qwen3.8-flash-next-iq3_s; quantization IQ3_S; revision not measured; source revision not measured
• Files: not measured
• Track: custom (arm combined: ["--adapt-async","1","--pipeline-windows","2","--prefill","auto:32768"])
• Context 131072 tokens; server capacity 1 slot(s)
• Sampling: temperature 1.0, top_p 0.95, top_k 20, min_p 0.0, presence_penalty 0.0, repetition_penalty 1.0, reasoning_effort xhigh; thinking on, preserved
• Engine options: config.json; chat template: provenance/chat_template.jinja

Method

The frozen requests of bench/prompts/standard-v1 (corpus e75eff8b9feea04f...) were sent over loopback HTTP, streaming, one at a time, in this order: 1 warm-up (excluded), 3 repetitions of the short, 4K and 16K prompts, and the twelve sanity checks. Prompt and decode speed are the engine's own timings; time to first token is the client's monotonic clock from dispatch to the first nonempty reasoning, answer or tool-call delta. No request was retried. Time after the model loaded: 1 min 42 s.

Results

Case	Prompt tok/s	Decode tok/s	TTFT s	First answer s	Total s	Capped
short	n/a	121.1 (107.1-121.8) n=3	0.48 (0.39-0.49) n=3	1.43 (1.27-1.57) n=3	3.9 (3.1-4.0) n=3	0/3
prompt-4k	1699.4 (1541.2-1740.4) n=3	121.7 (95.1-125.9) n=3	2.43 (2.37-2.67) n=3	4.20 (4.12-6.68) n=3	6.4 (5.9-8.0) n=3	2/3
prompt-16k	2242.5 (2163.6-2259.4) n=3	113.4 (101.7-116.8) n=3	7.34 (7.28-7.60) n=3	10.75 (10.70-10.81) n=2	11.8 (11.7-12.6) n=3	3/3

Cells are median (min-max) n=samples. Short is a prompt of about 80 tokens: its prompt rate is dominated by the fixed cost of a request, so it is not shown; read prompt speed from prompt-4k and prompt-16k. Capped = requests that reached the 512-token cap, reasoning and answer together. A capped request's total time is the time to the cap, not to an answer, and it has no first-answer time when it was still reasoning.

Phase	Planned	Completed	Truncated	Failed	Skipped
warmup	1	0	1	0	0
performance	9	4	5	0	0
sanity	12	12	0	0	0
concurrency	0	0	0	0	0

Sanity checks: 12 of 12.

Case	Result	Reason
I1	pass	the expected JSON value
I2	pass	the expected JSON value
I3	pass	the expected JSON value
I4	pass	the exact expected text
R1	pass	the exact expected text
R2	pass	the exact expected text
R3	pass	the exact expected text
R4	pass	the expected JSON value
T1	pass	the expected tool call
T2	pass	the expected tool call
T3	pass	the expected tool call
T4	pass	the expected tool call

Failures, exclusions and limitations

• No request failed.
• Excluded from speed summaries: performance-prompt-16k-r1: first_answer_s is not available for this request
• Prompt-prefix reuse is the shared template prefix only; the cache was not emptied between requests, and the expert cache stays warm and evolves. These are not cold-model measurements.
• Output caps include reasoning; a request that reaches its cap is truncated and fails its sanity check.
• The runner cannot see other programs on the machine, the repeat-stop and loop-recovery switches of the server, or the quantization repository's revision; see missing in results.json for every value it could not read.
• The server had answered no request before this run (a freshly loaded engine).