lovey/husky-bench/PLAN.md

husky-bench — a recreate-the-website benchmark

A benchmark for coding LLMs that produces a visually and physically judged artifact, not a unit- testable one. The task: rebuild husky-walk, an animated walking Siberian husky in a scrolling park, as a single static web page. Anatomy, gait direction, layering and secondary motion are the hard parts; they are also the parts a model gets plausibly, silently wrong.

Run label: husky-walk-recreate

---

1. What the model must produce

A directory containing exactly:

index.html      the whole scene as INLINE SVG (no <img>, no canvas, no WebGL, no build step)
styles.css      theming + all animation
app.js          state, counters, persistence, particle spawning
favicon.svg     day icon (explicit width AND height attributes)
favicon-night.svg
favicon.ico     binary fallback
README.md       user-facing description
tools/audit.js      headless regression guards (Playwright)
tools/poses.js      frozen-phase screenshots of every gait
tools/responsive.js viewport screenshots, day and night

Hard constraints:

• No runtime dependencies, no build step. index.html opened directly (file://) must work.
• Playwright is allowed only under tools/ (tools/package.json, local node_modules).
• The only image files in the project are the three favicons.
• No screenshots or PNGs committed as scene assets.
• Runs offline. No fonts, CDNs, or network calls.

The benchmark is run by a human or a script that opens the page and reads screenshots. A submission without working verification tooling is scored down even if it looks right, because it cannot be re-checked later.

---

2. Required behaviour (the contract)

2.1 Scene and coordinate system

Contract	Value
SVG viewBox	0 0 900 460 — never change it
Scenery strips	each exactly 900 units wide, cloned seamlessly with <use href="#strip" x="900"/>
Background motion	one keyframe per strip: translateX(0 → -900px); speed is a dial, never a re-authored path
Ground / paw line	y ≈ 340; paws land 336–344
Dog footprint	x ≈ 300–620, head centred near (560, 180)
Contact shadow	ellipse at (426, 352)
Lamppost	post at x = 664, glass centred (667.5, 166)
Parallax	four strips (far hills / mid trees / near bushes / dirt), four dials --glide-far, --glide-mid, --glide-near, --glide-dirt; ratio between them is the depth cue and must be preserved when the pace changes

2.2 Controls

• Four paces: stroll, walk, hop, zoomies (in that order). Buttons and ← / → key cycling must both derive from one ordered list, so they can never drift apart.
• Pause — stops the gait but not the animal: head still looks around, ear still flicks.
• Day / night — themes the entire page, not just the SVG: <html data-theme="day|night"> is the single source of truth for chrome and scene.
• Reduced motion — the scene renders as a correct still frame; an opt-in button (labelled Start the walk) is appended to the existing .controls container.
• Counters (distance walked, steps, elapsed, …) driven from the same pace table, and they must keep counting at a rate consistent with the stride frequency.

One dial per gait: --cycle. Every limb, bob, nod, wag and shadow derives from it. Stroll 1.06s, walk 0.78s, hop 0.62s, zoomies 0.38s. When a pace changes speed, the four parallax durations are divided by the same ratio.

2.3 Anatomy (profile view)

• The dog faces right, so the flank we see is his right side. *-near-* = right legs, *-far-* = left legs. Getting this backwards breaks the hop gag later.
• One eye, one ear, one brow visible. The near ear must be painted after the skull and the white face mask — an ear drawn behind the skull collapses into a horn.
• Ear: a broad-based triangle, base ≈42 units, height ≈53, leaning back ≈20°, pivoting at its base. Resize by scaling the whole group about that pivot, never by re-authoring points.
• A foreleg is a chain hinged at the shoulder on the side of the body — blade (scapula lying on the ribcage) → upper arm → elbow → forearm → wrist → paw. The elbow is the second joint, not the attachment point. Reference pivots, near leg: 498,214 → 482,252 → 485,340; far leg 478,212 → 464,250 → 466,340.
• The near foreleg is painted last, over the head/neck/chest fur. Overlap is the only depth cue a flat profile drawing gets; a leg painted under the neck coat has its shoulder sheared off.
• The scapula is a filled plane on the body (low-opacity fill), not a stroked outline or a hollow. A path with no matching fill rule renders black.
• Upper arm thicker than forearm: stroke-width 21 vs 14.5.
• The hind limb is not a mirrored foreleg: femur leans back, shank leans forward, so it needs its own fold numbers.
• Collar encircles neck + throat at the base of the neck, tilted ~17° forward, width ≈1/8 of skull width, ends flush with the body outline, front edge disappearing behind the skull.
• Underside is one continuous white bib from chest to belly (one visual piece from two paths that must overlap), deep enough to have area (~19 units of white above the underside), not a stripe.

2.4 Gait physics — the part most models fail

• A paw that is on the ground must travel backward (leftward, opposite the scroll). Foot lands in front of the shoulder, body passes over it, it leaves the ground behind him. Sign trap: in a y-down SVG/CSS space a positive rotation swings a downward-pointing limb backward, toward the tail. Flip the sign and the dog moonwalks — which looks finished and is wrong.
• Stance stops use linear timing (a planted foot cannot accelerate). Free-swing stops use cubic-bezier so the limb whips and the body dwells at the top of its arc — the dwell is what reads as suspension.
• Reference walk: strike at 0% (limb protracted, elbow near straight), toe-off at 46% (limb retracted, still straight), swing 46→100% with the elbow folded ~60°, and the limb only straightening in the last few percent so the reach finishes in the air and the descent is last.
• The paw is a third animated part (.pawgrp hinged at the wrist): flexed reaching, flat loaded, extended at toe-off, tucked through the swing. If a rig has three animated leg parts, all three must appear in every animation-delay rule or a paw runs on a different phase from its shin.
• Four contact-shadow dots on the path, one per paw, each with the exact computed animation-delay of its own leg, and declared outside the dog group so they never inherit its bob.
• Tail length must be identical across gaits: reach the low streamed tail with rotation only. A bounding-box diagonal is not rotation invariant; measure base-to-tip distance through the transform.
• Secondary action: tail tip lags the tail, collar tag swings, ears flick, eyes blink, body absorbs the landing.

2.5 Gaits, in detail

• stroll / walk — four-beat walk, diagonal pairs offset by half a cycle, far pair offset ~4% from the near pair to break mechanical lockstep.
• hop — a tripod: the dog's right foreleg (the near one) detaches, arcs backwards, lands once on the path behind him and stays gone while hop is selected. One-shot (fill-mode: both, never repeating). The detached group must pass far enough left that it clearly reads as separated, and the remaining three limbs beat together while the whole dog springs (peak ≈ −27 units, touchdown ≈ +2, paws still inside the path band). In still / reduced-motion mode the missing leg must be hidden by CSS, not only by the animation — a one-shot gag cannot be the only thing hiding a limb.
• zoomies — a gallop, not a fast trot: legs beat in pairs (near-front 0, far-front −0.1, near-rear −0.5, far-rear −0.6 of a cycle), knees fold in the air and limbs straighten at the ground beats, body bounce and shadow run at calc(var(--cycle) * .5) because full extension happens twice per stride. Tail streamed out low. Tongue blown back.

2.6 Extras that are part of the spec

• Shedding: at hop and zoomies only, clumps of undercoat come loose and tumble behind him (randomised anchor, size, spin, flight time, capped). Never while paused, never in a still scene.
• Hearts when the dog is happy: transform via custom properties (--x, --y, --s) inside the keyframes, because a CSS animation overrides an SVG transform attribute and the hearts then render off-canvas. All hearts must be inside the viewBox.
• Night: the animal's white underside stays high-value white. Do not dim the belly with the rest of the scene; lift/darken coat and trail instead. Reference: --fur-white:#f4f7fc, --coat-dark:#333d52, --coat-mid:#48556d, --trail:#464e66.
• Persistence: day/night remembered across reloads. Cookie husky-sky as primary, mirrored to localStorage because file:// rejects cookies and private modes can throw. A synchronous <script> in <head> applies the theme before first paint (no white flash). Pace is deliberately not persisted.
• Favicons: real files, swapped day/night by JS. An SVG with only a viewBox rasterises blank, so favicon.svg needs explicit width/height.

---

3. Verification tooling (also graded)

Headless Chromium via Playwright, screenshots in a temp directory, nothing committed.

• audit.js — asserts, at minimum: no page errors; running-animation count; pace/sky/paused attrs; hearts inside the viewBox; favicon files fetch and decode; contact-dot delays equal their leg's delays in every pace; paused-but-alive (gait paused, head still animating); tail drop and tail length across gaits; foreleg pivot pins have not drifted; blade stays under the topline and on the body; paw envelope stays within the path band; leg taper ratio; white-bib area via isPointInFill; elbow sweep per pace; grounded-paw drift (sample the cycle, find the frames each paw is down at the path, sum x travel — must be negative); cookie/localStorage persistence after a reload in a real HTTP context; and a second context with reducedMotion: 'reduce' because the default context never exercises that class of bug.
• poses.js — every gait frozen at chosen phases. Frozen sample times are coupled to --cycle: if the cycle changes, the sample times change too.
• responsive.js — full-page day and night on desktop and at ~390 px.

Tooling traps that cost real time in the reference build:

• page.screenshot({clip}) takes page pixels, not SVG user units. Convert with the scene's bounding rect and rect.width / 900.
• getScreenCTM() of a group under nested CSS transforms double-counts ancestor rotations and can report a paw 200 units underground. Measure getBoundingClientRect() and divide by the scene scale. If a probe disagrees with the screenshots, the probe is the bug.
• document.querySelector('[data-speed="walk"]') matches the scene, not the button — the click silently does nothing. Always button[data-speed="…"].
• str.replace() in a fix-up script does not fail when the pattern is missing. Every scripted edit must assert its pattern is present, then re-read the audit numbers.
• <use>-cloned scenery cannot be themed by class rules that depend on ancestors; theme with custom properties inherited from the SVG root. fill="url(#…)" paints black on clones.
• SVG transform / transform-origin need transform-box: view-box.

---

4. Run protocol

1. Record start time. Work until audit.js reports errors: none and the screenshots look right.
2. Do not copy from the reference repo. The reference exists only to score against.
3. Self-check before submitting:
   • open file:///…/index.html, cycle all four paces, pause, switch to night, reload (night must survive and the pace must reset to walk), resize to a phone, and re-run with prefers-reduced-motion emulated;
   • confirm the dog reads as walking forward: on any frame where a paw is on the path, that paw is behind where it landed;
   • confirm hop removes the right foreleg and the still frame still reads as a tripod;
   • confirm the belly is one continuous white piece, day and night.
4. Fill in the metadata footer (below) with the run's real numbers, then stop.

---

5. Required footer

Every submission must render, at the bottom of the page, a block that always prints exactly these five labels, filled in with the run's real values:

Time Taken:
Tokens Used:
Model:
Engine:
Thinking level:

• Time Taken — wall clock from run start to the final verification pass (e.g. 14m 20s).
• Tokens Used — total tokens consumed by the session (prompt + completion), as reported by the agent harness. Write unreported if the harness exposes no counter.
• Model — the exact model id under test.
• Engine — serving stack/version (e.g. llama.cpp 0.1.51, Anthropic API, vLLM 0.11).
• Thinking level — the reasoning budget the run was given (e.g. off, low, high).

Rules: the footer is always visible, it is never hidden or removed by any state (day, night, paused, reduced motion, any pace), and the values are honest — an empty or guessed field scores worse than unreported. In pi, these come from PI_MODEL, PI_PROVIDER, PI_REASONING_LEVEL, the session file, and the run's wall clock.

---

6. Scoring

Each row is pass / partial / fail, judged from screenshots plus audit.js output.

#	Check	Weight
1	Loads from file://, no console errors, no runtime deps	3
2	Gait direction — grounded paws sweep backward; no moonwalk	8
3	Foreleg hinged at the shoulder, elbow visibly moves, upper arm thicker than forearm	8
4	Near foreleg painted over neck/chest; blade reads as a plane on the body	4
5	White bib continuous and deep, day and night; night keeps the belly white	6
6	Ear reads as an ear, in front of the skull, one of each facial feature	4
7	Zoomies is a gallop (pair beats, fold in air, straight at ground); tail length unchanged	6
8	Hop tripod: right foreleg gone, once only, tripod still legible in a still frame	6
9	Paw articulation, contact dots in phase, secondary action (tail tip, tag, blink, ear flick)	5
10	One --cycle dial drives everything; parallax ratios preserved across paces	4
11	Page-wide night theme, persisted before first paint, pace not persisted	4
12	Reduced-motion still mode + opt-in button	3
13	Pause stops the gait but not the animal	2
14	Tooling: audit + poses + responsive run clean and assert the contracts above	6
15	Footer present with all five fields filled honestly	2
—	Total	75

Speed is recorded separately, not folded into quality: time taken, tokens used, tool calls made, and number of verification passes before the first clean audit (how long the model took to catch its own wrong answer is the most informative number in the log).

---

7. Known failure modes to watch for (spoilers, deliberately)

Recorded from the reference build. A submission that trips one of these and then fixes it, with evidence in the log, scores better than one that never trips it but is quietly wrong.

1. Foreleg attached to the ribs (visible leg top treated as the shoulder).
2. Foreleg hinged at the elbow instead of the shoulder — anatomically closer, still wrong.
3. Grounded paw sliding forward = moonwalk; and the subtler variant where the limb extends early so the paw drags forward for the last quarter of the cycle.
4. Hind limb driven by foreleg numbers → rear paws never leave the path and skid.
5. Ear behind the skull → horn.
6. Collar sized like a bandage, or wrapping the chest instead of the neck + throat.
7. Night mode dimming the white belly until it merges with the trail.
8. Zoomies as a fast trot; scaleY squash shortening the tail.
9. Hop removing the wrong leg, or being invisible in reduced motion, or repeating every cycle.
10. Paws welded to shins (no wrist), and contact dots on the wrong phase.
11. Trusting getScreenCTM() numbers, or trusting a probe that contradicts the screenshot.