lovey/husky-bench/PLAN.md

# husky-bench — a recreate-the-website benchmark

A benchmark for coding LLMs that produces a **visually and physically judged** artifact, not a unit-
testable one. The task: rebuild `husky-walk`, an animated walking Siberian husky in a scrolling
park, as a single static web page. Anatomy, gait direction, layering and secondary motion are the
hard parts; they are also the parts a model gets plausibly, silently wrong.

Run label: `husky-walk-recreate`

---

## 1. What the model must produce

A directory containing exactly:

```
index.html      the whole scene as INLINE SVG (no <img>, no canvas, no WebGL, no build step)
styles.css      theming + all animation
app.js          state, counters, persistence, particle spawning
favicon.svg     day icon (explicit width AND height attributes)
favicon-night.svg
favicon.ico     binary fallback
README.md       user-facing description
tools/audit.js      headless regression guards (Playwright)
tools/poses.js      frozen-phase screenshots of every gait
tools/responsive.js viewport screenshots, day and night
```

Hard constraints:

- **No runtime dependencies, no build step.** `index.html` opened directly (`file://`) must work.
- Playwright is allowed **only** under `tools/` (`tools/package.json`, local `node_modules`).
- The only image files in the project are the three favicons.
- No screenshots or PNGs committed as scene assets.
- Runs offline. No fonts, CDNs, or network calls.

The benchmark is run by a human or a script that opens the page and reads screenshots. A submission
without working verification tooling is scored down even if it looks right, because it cannot be
re-checked later.

---

## 2. Required behaviour (the contract)

### 2.1 Scene and coordinate system

| Contract | Value |
|---|---|
| SVG viewBox | `0 0 900 460` — never change it |
| Scenery strips | each exactly **900 units** wide, cloned seamlessly with `<use href="#strip" x="900"/>` |
| Background motion | one keyframe per strip: `translateX(0 → -900px)`; speed is a dial, never a re-authored path |
| Ground / paw line | `y ≈ 340`; paws land **336–344** |
| Dog footprint | `x ≈ 300–620`, head centred near `(560, 180)` |
| Contact shadow | ellipse at `(426, 352)` |
| Lamppost | post at `x = 664`, glass centred `(667.5, 166)` |
| Parallax | four strips (far hills / mid trees / near bushes / dirt), four dials `--glide-far`, `--glide-mid`, `--glide-near`, `--glide-dirt`; **ratio between them is the depth cue and must be preserved when the pace changes** |

### 2.2 Controls

- **Four paces**: `stroll`, `walk`, `hop`, `zoomies` (in that order). Buttons and ← / → key
  cycling must both derive from one ordered list, so they can never drift apart.
- **Pause** — stops the gait but **not** the animal: head still looks around, ear still flicks.
- **Day / night** — themes the *entire page*, not just the SVG: `<html data-theme="day|night">` is
  the single source of truth for chrome and scene.
- **Reduced motion** — the scene renders as a correct still frame; an opt-in button (labelled
  `Start the walk`) is appended to the existing `.controls` container.
- **Counters** (distance walked, steps, elapsed, …) driven from the same pace table, and they must
  keep counting at a rate consistent with the stride frequency.

One dial per gait: `--cycle`. Every limb, bob, nod, wag and shadow derives from it. Stroll
`1.06s`, walk `0.78s`, hop `0.62s`, zoomies `0.38s`. When a pace changes speed, the four parallax
durations are divided by the **same ratio**.

### 2.3 Anatomy (profile view)

- The dog faces **right**, so the flank we see is his **right** side. `*-near-*` = right legs,
  `*-far-*` = left legs. Getting this backwards breaks the hop gag later.
- **One eye, one ear, one brow** visible. The near ear must be painted **after** the skull and the
  white face mask — an ear drawn behind the skull collapses into a horn.
- Ear: a broad-based triangle, base ≈42 units, height ≈53, leaning back ≈20°, pivoting at its base.
  Resize by scaling the whole group about that pivot, never by re-authoring points.
- **A foreleg is a chain hinged at the shoulder on the side of the body** — blade (scapula lying on
  the ribcage) → upper arm → **elbow** → forearm → wrist → paw. The elbow is the *second* joint,
  not the attachment point. Reference pivots, near leg: `498,214` → `482,252` → `485,340`; far leg
  `478,212` → `464,250` → `466,340`.
- The near foreleg is painted **last, over the head/neck/chest fur**. Overlap is the only depth cue
  a flat profile drawing gets; a leg painted under the neck coat has its shoulder sheared off.
- The scapula is a **filled plane on the body** (low-opacity fill), not a stroked outline or a
  hollow. A path with no matching fill rule renders **black**.
- Upper arm thicker than forearm: `stroke-width` 21 vs 14.5.
- The **hind limb is not a mirrored foreleg**: femur leans back, shank leans forward, so it needs
  its own fold numbers.
- Collar encircles **neck + throat** at the base of the neck, tilted ~17° forward, width ≈1/8 of
  skull width, ends flush with the body outline, front edge disappearing behind the skull.
- Underside is **one continuous white bib** from chest to belly (one visual piece from two paths
  that must overlap), deep enough to have area (~19 units of white above the underside), not a stripe.

### 2.4 Gait physics — the part most models fail

- **A paw that is on the ground must travel backward** (leftward, opposite the scroll). Foot lands
  *in front* of the shoulder, body passes over it, it leaves the ground *behind* him. Sign trap: in
  a y-down SVG/CSS space a **positive rotation swings a downward-pointing limb backward**, toward
  the tail. Flip the sign and the dog moonwalks — which looks finished and is wrong.
- Stance stops use `linear` timing (a planted foot cannot accelerate). Free-swing stops use
  `cubic-bezier` so the limb whips and the body dwells at the top of its arc — the dwell is what
  reads as suspension.
- Reference walk: strike at `0%` (limb protracted, elbow near straight), toe-off at `46%` (limb
  retracted, still straight), swing `46→100%` with the **elbow folded ~60°**, and the limb only
  straightening in the last few percent so the reach finishes *in the air* and the descent is last.
- The paw is a third animated part (`.pawgrp` hinged at the wrist): flexed reaching, flat loaded,
  extended at toe-off, tucked through the swing. If a rig has three animated leg parts, **all three
  must appear in every `animation-delay` rule** or a paw runs on a different phase from its shin.
- Four contact-shadow dots on the path, one per paw, each with the **exact** computed
  `animation-delay` of its own leg, and declared **outside** the dog group so they never inherit
  its bob.
- Tail length must be identical across gaits: reach the low streamed tail with **rotation only**.
  A bounding-box diagonal is not rotation invariant; measure base-to-tip distance through the
  transform.
- Secondary action: tail tip lags the tail, collar tag swings, ears flick, eyes blink, body absorbs
  the landing.

### 2.5 Gaits, in detail

- **stroll / walk** — four-beat walk, diagonal pairs offset by half a cycle, far pair offset ~4%
  from the near pair to break mechanical lockstep.
- **hop** — a *tripod*: the dog's **right foreleg** (the near one) detaches, arcs backwards, lands
  once on the path behind him and stays gone while hop is selected. One-shot (`fill-mode: both`,
  never repeating). The detached group must pass far enough left that it clearly reads as
  separated, and the remaining three limbs beat together while the whole dog springs (peak ≈ −27
  units, touchdown ≈ +2, paws still inside the path band). In **still / reduced-motion mode the
  missing leg must be hidden by CSS, not only by the animation** — a one-shot gag cannot be the
  only thing hiding a limb.
- **zoomies** — a **gallop, not a fast trot**: legs beat in *pairs* (near-front `0`, far-front
  `−0.1`, near-rear `−0.5`, far-rear `−0.6` of a cycle), knees fold in the air and limbs straighten
  at the ground beats, body bounce and shadow run at `calc(var(--cycle) * .5)` because full
  extension happens twice per stride. Tail streamed out low. Tongue blown back.

### 2.6 Extras that are part of the spec

- **Shedding**: at hop and zoomies only, clumps of undercoat come loose and tumble behind him
  (randomised anchor, size, spin, flight time, capped). Never while paused, never in a still scene.
- **Hearts** when the dog is happy: transform via custom properties (`--x`, `--y`, `--s`) inside the
  keyframes, because a CSS animation overrides an SVG `transform` attribute and the hearts then
  render off-canvas. All hearts must be inside the viewBox.
- **Night**: the animal's white underside stays **high-value white**. Do not dim the belly with the
  rest of the scene; lift/darken coat and trail instead. Reference: `--fur-white:#f4f7fc`,
  `--coat-dark:#333d52`, `--coat-mid:#48556d`, `--trail:#464e66`.
- **Persistence**: day/night remembered across reloads. Cookie `husky-sky` as primary, mirrored to
  `localStorage` because `file://` rejects cookies and private modes can throw. A synchronous
  `<script>` in `<head>` applies the theme **before first paint** (no white flash). Pace is
  deliberately *not* persisted.
- **Favicons**: real files, swapped day/night by JS. An SVG with only a `viewBox` rasterises blank,
  so `favicon.svg` needs explicit `width`/`height`.

---

## 3. Verification tooling (also graded)

Headless Chromium via Playwright, screenshots in a temp directory, nothing committed.

- `audit.js` — asserts, at minimum: no page errors; running-animation count; pace/sky/paused attrs;
  hearts inside the viewBox; favicon files fetch and decode; contact-dot delays equal their leg's
  delays in every pace; paused-but-alive (gait paused, head still animating); tail drop and tail
  length across gaits; foreleg pivot pins have not drifted; blade stays under the topline and on
  the body; paw envelope stays within the path band; leg taper ratio; white-bib area via
  `isPointInFill`; elbow sweep per pace; **grounded-paw drift** (sample the cycle, find the frames
  each paw is down at the path, sum x travel — must be negative); cookie/localStorage persistence
  after a reload in a real HTTP context; and a **second context with `reducedMotion: 'reduce'`**
  because the default context never exercises that class of bug.
- `poses.js` — every gait frozen at chosen phases. Frozen sample times are **coupled to `--cycle`**:
  if the cycle changes, the sample times change too.
- `responsive.js` — full-page day and night on desktop and at ~390 px.

Tooling traps that cost real time in the reference build:

- `page.screenshot({clip})` takes **page pixels**, not SVG user units. Convert with the scene's
  bounding rect and `rect.width / 900`.
- `getScreenCTM()` of a group under **nested CSS transforms double-counts ancestor rotations** and
  can report a paw 200 units underground. Measure `getBoundingClientRect()` and divide by the
  scene scale. If a probe disagrees with the screenshots, the probe is the bug.
- `document.querySelector('[data-speed="walk"]')` matches the **scene**, not the button — the click
  silently does nothing. Always `button[data-speed="…"]`.
- `str.replace()` in a fix-up script does not fail when the pattern is missing. Every scripted edit
  must assert its pattern is present, then re-read the audit numbers.
- `<use>`-cloned scenery cannot be themed by class rules that depend on ancestors; theme with
  custom properties inherited from the SVG root. `fill="url(#…)"` paints **black** on clones.
- SVG `transform` / `transform-origin` need `transform-box: view-box`.

---

## 4. Run protocol

1. Record start time. Work until `audit.js` reports `errors: none` and the screenshots look right.
2. Do not copy from the reference repo. The reference exists only to score against.
3. Self-check before submitting:
   - open `file:///…/index.html`, cycle all four paces, pause, switch to night, reload (night must
     survive and the pace must reset to walk), resize to a phone, and re-run with
     `prefers-reduced-motion` emulated;
   - confirm the dog reads as **walking forward**: on any frame where a paw is on the path, that paw
     is behind where it landed;
   - confirm hop removes the **right** foreleg and the still frame still reads as a tripod;
   - confirm the belly is one continuous white piece, day **and** night.
4. Fill in the metadata footer (below) with the run's real numbers, then stop.

---

## 5. Required footer

Every submission must render, at the bottom of the page, a block that always prints exactly these
five labels, filled in with the run's real values:

```
Time Taken:
Tokens Used:
Model:
Engine:
Thinking level:
```

- **Time Taken** — wall clock from run start to the final verification pass (e.g. `14m 20s`).
- **Tokens Used** — total tokens consumed by the session (prompt + completion), as reported by the
  agent harness. Write `unreported` if the harness exposes no counter.
- **Model** — the exact model id under test.
- **Engine** — serving stack/version (e.g. `llama.cpp 0.1.51`, `Anthropic API`, `vLLM 0.11`).
- **Thinking level** — the reasoning budget the run was given (e.g. `off`, `low`, `high`).

Rules: the footer is always visible, it is never hidden or removed by any state (day, night, paused,
reduced motion, any pace), and the values are honest — an empty or guessed field scores worse than
`unreported`. In pi, these come from `PI_MODEL`, `PI_PROVIDER`, `PI_REASONING_LEVEL`, the session
file, and the run's wall clock.

---

## 6. Scoring

Each row is pass / partial / fail, judged from screenshots plus `audit.js` output.

| # | Check | Weight |
|---|---|---|
| 1 | Loads from `file://`, no console errors, no runtime deps | 3 |
| 2 | **Gait direction** — grounded paws sweep backward; no moonwalk | 8 |
| 3 | Foreleg hinged at the **shoulder**, elbow visibly moves, upper arm thicker than forearm | 8 |
| 4 | Near foreleg painted over neck/chest; blade reads as a plane on the body | 4 |
| 5 | White bib continuous and deep, day and night; night keeps the belly white | 6 |
| 6 | Ear reads as an ear, in front of the skull, one of each facial feature | 4 |
| 7 | Zoomies is a gallop (pair beats, fold in air, straight at ground); tail length unchanged | 6 |
| 8 | Hop tripod: right foreleg gone, once only, tripod still legible in a still frame | 6 |
| 9 | Paw articulation, contact dots in phase, secondary action (tail tip, tag, blink, ear flick) | 5 |
| 10 | One `--cycle` dial drives everything; parallax ratios preserved across paces | 4 |
| 11 | Page-wide night theme, persisted before first paint, pace not persisted | 4 |
| 12 | Reduced-motion still mode + opt-in button | 3 |
| 13 | Pause stops the gait but not the animal | 2 |
| 14 | Tooling: audit + poses + responsive run clean and assert the contracts above | 6 |
| 15 | Footer present with all five fields filled honestly | 2 |
| — | Total | **75** |

Speed is recorded separately, not folded into quality: **time taken**, **tokens used**, **tool calls
made**, and **number of verification passes before the first clean audit** (how long the model took
to catch its own wrong answer is the most informative number in the log).

---

## 7. Known failure modes to watch for (spoilers, deliberately)

Recorded from the reference build. A submission that trips one of these and then *fixes* it, with
evidence in the log, scores better than one that never trips it but is quietly wrong.

1. Foreleg attached to the ribs (visible leg top treated as the shoulder).
2. Foreleg hinged at the **elbow** instead of the shoulder — anatomically closer, still wrong.
3. Grounded paw sliding **forward** = moonwalk; and the subtler variant where the limb extends
   early so the paw drags forward for the last quarter of the cycle.
4. Hind limb driven by foreleg numbers → rear paws never leave the path and skid.
5. Ear behind the skull → horn.
6. Collar sized like a bandage, or wrapping the chest instead of the neck + throat.
7. Night mode dimming the white belly until it merges with the trail.
8. Zoomies as a fast trot; `scaleY` squash shortening the tail.
9. Hop removing the wrong leg, or being invisible in reduced motion, or repeating every cycle.
10. Paws welded to shins (no wrist), and contact dots on the wrong phase.
11. Trusting `getScreenCTM()` numbers, or trusting a probe that contradicts the screenshot.