docs: compare-screen redesign spec, expert review, and data-foundation plan

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146VHeLAWjDVE2B5uU67jCB

docs/superpowers/specs/2026-07-11-compare-screen-redesign-design.md

@@ -0,0 +1,318 @@
+# Compare Screen Redesign — Audit & Design
+
+**Date:** 2026-07-11
+**Status:** Draft — awaiting review
+**Scope:** `/compare` page (nextjs-app), `/api/compare` endpoint (backend)
+
+## 1. Audit of the current screen
+
+The current compare page (`nextjs-app/components/ComparisonView.tsx`) is a
+single-metric analyst tool: a `<select>` with ~40 KS2/GCSE metrics, one
+line chart over time, and a year-by-year table — all for the one selected
+metric. Observed on production with 3 primary schools:
+
+**What works**
+
+- URL-shareable state (`?urns=…&metric=…`), native share sheet.
+- Phase tabs (primary/secondary) with sensible auto-detection.
+- Colour-coded school cards tied to chart series.
+- Metric descriptions from `/api/metrics` (single source of truth).
+
+**What doesn't**
+
+1. **Performance-only.** The database already holds Ofsted inspections,
+   admissions/oversubscription history, pupil characteristics (FSM/EAL),
+   SEN, deprivation (IDACI), finance, capacity, faith, gender, trust —
+   none of it reaches the compare screen. `/api/compare` returns only
+   `yearly_data` + minimal `school_info`, while `/api/schools/{urn}`
+   already returns all supplementary blocks.
+2. **One metric at a time.** A parent must know which of ~40 metrics
+   matters, select each in turn, and hold results in their head. There is
+   no side-by-side overview and no way to see two dimensions at once.
+3. **No benchmarks.** Numbers float without anchors: is 79% RWM good?
+   The DB has official national averages (`fact_ks2_national_averages`)
+   but the page never shows them.
+4. **Domain jargon untranslated.** "GPS Expected %", "Progress scores",
+   "RWM Combined" assume DfE literacy. The only plain-English help is one
+   note for progress scores.
+5. **Raw numbers, no judgement support.** 87.0% vs 92.0% vs 79.0% — the
+   page never says "all three are well above the England average of 62%",
+   which is the fact a parent actually needs.
+6. **Bugs/paper cuts observed:** the third school's series did not render
+   on the production chart despite table data (worth a separate fix);
+   the COVID gap (2018/19 → 2022/23) renders as equal spacing with no
+   annotation; table shows "87.0%" precision that implies false accuracy.
+
+## 2. Data inventory (available vs shown)
+
+| Domain | Source table | On detail page | On compare |
+|---|---|---|---|
+| KS2 attainment/progress | fact_ks2_performance | yes | **yes** (only thing shown) |
+| National averages | fact_ks2_national_averages | partial | no |
+| Ofsted (latest + subgrades + report-card fields) | fact_ofsted_inspection, dim_school | yes | no |
+| Admissions & oversubscription (multi-year) | fact_admissions | yes | no |
+| Pupil characteristics (FSM, EAL, gender split) | fact_pupil_characteristics | yes | no |
+| Context (SEN, disadvantaged, stability, absence) | fact_ks2_performance | via metric picker | buried in picker |
+| Deprivation (IDACI) | fact_deprivation | yes | no |
+| Finance (per-pupil spend) | fact_finance | yes | no |
+| School facts (capacity, faith, ages, trust, nursery, gender) | dim_school | yes | no |
+| Location/distance | dim_location | map | no |
+
+## 3. Design goals
+
+1. **Answer parent questions, in order:** Is it a good school (Ofsted)?
+   Do children do well there (academics vs England)? Will my child get a
+   place (admissions)? What is the school like (size, community, faith)?
+2. **Every number gets an anchor** — the England average, rendered as a
+   consistent visual tick, plus a plain-English chip
+   (Above / Close to / Below England average).
+3. **Plain English first, jargon on demand.** Labels are questions or
+   sentences ("Children reaching the expected standard in reading,
+   writing and maths"), codes/acronyms live in tooltips.
+4. **Scan whole-picture first, drill down second.** The single-metric
+   trend explorer survives, demoted to an "Explore trends" section at the
+   bottom rather than being the entire page.
+
+## 4. Proposed structure
+
+Columns = schools (max 4 visible on desktop, horizontal scroll beyond),
+rows = dimensions. Sticky compact school header keeps column identity
+while scrolling. Sections, in order:
+
+1. **At a glance** — verdict row per school: Ofsted badge, headline
+   attainment vs England (dot strip + chip), oversubscription chip,
+   size, distance (when a location is set).
+2. **Ofsted inspection** — must handle all three inspection regimes,
+   which will coexist in comparisons for years:
+   - **Legacy graded (pre-Sept 2024):** overall grade badge
+     (Outstanding/Good/Requires improvement/Inadequate). Subgrades,
+     where published, are rendered in the **same area-by-rating chip
+     list UX as report cards** (one row per judgement area, rating as
+     a chip) — one visual grammar for inspection detail across both
+     regimes. Where our dataset has no subgrades for an inspection,
+     say so honestly ("We don't hold area-by-area detail for this
+     inspection") and point to the school's Ofsted page — never claim
+     the inspection itself published no detail (graded inspections
+     always have subgrades; if it was ungraded, the grade is
+     carried forward and must be labelled as such).
+   - **Interim ungraded (Sept 2024 – Nov 2025):** parsed outcome
+     ("remains Good") shown as the effective grade, marked as such.
+   - **Renewed framework report card (from Nov 2025):** no overall
+     grade exists. Render the report card as an area-by-rating list
+     using Ofsted's 5-point scale (Exceptional / Strong standard /
+     Expected standard / Attention needed / Urgent improvement) across
+     the evaluation areas we model (`rc_inclusion`,
+     `rc_curriculum_teaching`, `rc_achievement`,
+     `rc_attendance_behaviour`, `rc_personal_development`,
+     `rc_leadership_governance`, `rc_early_years`, `rc_sixth_form`)
+     plus the separate safeguarding met/not-met flag. **At-a-glance
+     summary rule:** never an unlabelled colour strip — summarise by
+     counting areas per rating, best first ("5 areas Strong standard ·
+     3 areas Expected standard"), and always name any area rated
+     Attention needed or Urgent improvement explicitly (never fold
+     problems into a count), plus "Safeguarding not met" whenever that
+     flag is false. When everything is Expected standard or better,
+     add the reassurance line "No areas need attention".
+   When a comparison mixes regimes, show a one-line comparability note
+   ("Ofsted changed how it reports in Nov 2025 — a report card and an
+   older overall grade aren't directly comparable"). Never derive a
+   fake overall grade from report-card areas.
+3. **Academics (KS2)** — one dot-strip row per headline measure (RWM
+   expected, RWM higher, reading/writing/maths expected), each with the
+   England-average tick and per-school dots; copy must say "tests and
+   teacher assessments" (writing is TA, not a test). Progress scores
+   translated to Above/Average/Below chips (CI-based) — **but note KS2
+   progress measures ended with 2022/23** (no KS1 baseline afterwards)
+   and return only when the reception-baseline cohort reaches Y6
+   (~2027/28), so progress chips apply to historical years in the
+   trends explorer, not the headline view. Sparkline per school over
+   the full published period, with an honest gap for the cancelled
+   test years (2019/20–2020/21). Disadvantaged-pupils row under an
+   "Equity" subheading, always with cohort size shown and DfE's full
+   definition (FSM6 **or** looked-after/previously looked-after).
+4. **Getting a place** — oversubscription ratio as plain sentence
+   ("184 applications for 80 places"), first-preference success %, trend
+   vs last year, admissions policy.
+5. **Who goes there** — pupils on roll (vs capacity), boys/girls, FSM %,
+   EAL %, SEN support %, faith, ages, nursery, trust. *Post-v1:* IDACI
+   decile in words (needs a coverage check of `fact_deprivation` and
+   the neighbourhood-not-school caveat, §8.7).
+6. **Attendance** — *post-v1.* The KS2 test-day absence fields are the
+   only per-school absence data we hold; they're near-zero for most
+   schools and easy to misread as general attendance. Ship only if a
+   general-absence source lands.
+7. **Explore trends** (existing feature, collapsed) — metric picker +
+   multi-year line chart + table, with an added England-average
+   reference line and a COVID-gap annotation.
+
+**Metric exposure model (three tiers).** No measure from the current
+page is lost; they surface at three levels of prominence:
+- **Tier 1 — headline strips (always visible):** RWM expected,
+  reading/writing/maths expected, RWM higher standard.
+- **Tier 2 — "More measures" expansion inside Academics:** GPS and
+  science expected % (science labelled teacher-assessed), average
+  scaled scores (reading/maths/GPS, same dot-strip grammar showing
+  the 100–120 window of the 80–120 scale, widening below 100, with
+  the England tick) — one tap/click away, same visual language.
+  *Post-v1:* gender split and absence (see §4.6).
+- **Tier 3 — Explore trends:** the full grouped catalogue (the
+  current page's ~40 metrics, including equity and school-context
+  measures, and the GCSE set for secondary phase) drives the
+  year-by-year chart and table via the grouped metric picker.
+The tier assignment is a content decision per phase (secondary:
+Attainment 8, Progress 8 banding, grade 5+ English & maths as tier 1;
+EBacc and subject entries as tier 2).
+
+Finance (per-pupil spend) is deliberately deferred: low parent value,
+risk of misreading. Revisit later.
+
+**Mobile (design target — mobile first):** the desktop grid is the
+adaptation, not the other way round. On mobile the layout goes
+*measure-first*: each row is one measure with all schools listed under
+it (colour dot + short name + value + chip), so comparison never
+requires horizontal swiping between school cards. A sticky horizontal
+school-chip bar keeps identity and add/remove available while
+scrolling. Dot strips already read measure-first and carry over
+unchanged. The trend chart scrolls horizontally inside its container.
+
+## 5. Data strategy — existing dataset only
+
+Constraint (agreed 2026-07-11): use only data already in marts plus
+fields already present in the `raw` schema extracts we pull today.
+No new external sources.
+
+**Gaps in the mockup, resolved within this constraint:**
+
+| Mockup element | Resolution |
+|---|---|
+| England average for disadvantaged pupils | Compute from our own data: `stg_ees_ks2` already pivots the Disadvantaged breakdown per school; aggregate it (weighted by eligible pupils) into `fact_ks2_national_averages` or compute in the API. Label it "England average (state schools)". |
+| England context for FSM / EAL / SEN chips | Compute dataset-wide medians per phase, same pattern as `/api/national-averages` does for KS4. |
+| "Much larger than average" size label | Dataset median pupils-on-roll per phase. |
+| Ofsted link | We don't have deep links to the latest report, so always link to the school's Ofsted provider page, `https://reports.ofsted.gov.uk/provider/21/{urn}`, derived from URN (label it "the school's Ofsted page", not "the report"). |
+
+**Raw fields we already pull but don't store — promote to marts (one
+dbt/pipeline PR, no tap changes):**
+
+- `raw.ees_admissions`: 2nd/3rd preference applications and offers,
+  total-preference counts, cross-LA applications and offers → richer
+  "Getting a place" (e.g. "offers reached 2nd-choice families",
+  competition from outside the borough).
+- `raw.ees_ks2_attainment`: progress-measure confidence intervals and
+  "working towards" % → lets the Above/Average/Below progress chips be
+  statistically honest (band by CI overlap with 0, mirroring DfE
+  methodology) instead of thresholding the point estimate.
+- `raw.ees_ks4_performance` / `ees_ks4_info`: `progress8_banding`
+  (DfE's own plain-English "well above average … well below average"
+  label — exactly the chip we want for secondary), EBacc entry/APS,
+  grade-5+ English & maths, `attainment8_diffn`/`progress8_diffn`
+  (disadvantage gaps) → the secondary-phase version of the Academics
+  section.
+- `raw.ees_census`: young-carer % and the ethnicity breakdown →
+  optional "Who goes there" enrichment; hold for a later iteration
+  (presentation needs care), but the data requires no new extract.
+- `raw.ofsted_inspections` / tap-uk-ofsted: the `rc_*` report-card
+  columns exist in staging/marts but are stubbed `null` — the tap has a
+  TODO to map the report-card column names from the Ofsted MI file
+  (same monthly extract we already download; inspections from Nov 2025
+  onward carry them). This is the one promotion that needs a small tap
+  schema addition, and it's a prerequisite for the new-framework Ofsted
+  display above.
+
+Explicitly out (not in any current extract): school-level phonics,
+workforce/teacher data, per-school attendance beyond the KS2 test-day
+absence fields, Ofsted report-card documents themselves.
+
+## 6. API changes
+
+Extend `GET /api/compare` response per URN with the same supplementary
+blocks the detail endpoint already builds (`get_supplementary_data`):
+`ofsted`, `census`, `admissions` (+ `admissions_history`), `deprivation`,
+plus a top-level `national_averages` block for the latest year. Reuse the
+existing function; no new tables. Response stays backward-compatible
+(additive fields only). Add derived helper fields server-side or compute
+chips client-side from `national_averages` (client-side preferred — no
+schema churn).
+
+## 7. Accessibility & comprehension devices
+
+- Verdict chips are text + colour + position (never colour alone).
+- Every acronym has a tooltip using existing `MetricTooltip`.
+- "How to read this" one-liner at the top of each section.
+- Chart palette: coral `#e07256`, teal `#00949b`, purple `#8664c9`
+  (validated: lightness band, chroma, CVD separation, contrast — the
+  current `--chart-2/-4` tokens fail chroma/contrast checks and should
+  be nudged to these).
+- Numbers rounded to whole percents; England tick labelled on first use.
+
+## 8. Expert-review requirements
+
+An adversarial review by an education-data expert (full findings in
+`2026-07-11-compare-screen-expert-review.md`) was applied to the
+mockups on 2026-07-12. The following are binding requirements for
+implementation, beyond what the mockups can show:
+
+1. **Chart truthfulness:** KS2 tests were cancelled 2019/20–2020/21
+   only; 2021/22 results exist and must be loaded (school-level rows
+   and the 2015/16 national figure, 53%, are currently missing from
+   our dataset — pipeline task). Load official England averages for
+   GPS/science expected % and the scaled scores (reading/maths/GPS) —
+   the mockups' scaled-score ticks are indicative until then. Never
+   render missing years as if time were continuous.
+2. **Report-card summaries** count graded areas only — safeguarding is
+   a separate binary flag, never included in rating counts.
+3. **Admissions:** use the real preference-breakdown fields from
+   `raw.ees_admissions`; never derive "lower-preference offers" as
+   places − first-preference offers. Frame total applications as
+   "named on N forms" (any rank), lead with first-preference success,
+   and standardise at-a-glance chips on that one metric. Explain the
+   equal-preference rule; caveat offers vs final intake (waiting
+   lists/appeals); condition "distance decides" on the school's actual
+   oversubscription criteria where we have the admissions-policy field.
+4. **Ofsted:** overall grades ended September 2024 (report cards from
+   November 2025); the interim regime must be renderable. Distinguish
+   graded (s5) vs ungraded (s8) inspections and surface carried-forward
+   grades as such; "we don't hold the detail" is a statement about our
+   dataset, never about the inspection. Verify every scale/area label
+   against Ofsted's final published toolkit before launch (e.g. "Needs
+   attention" vs "Attention needed"; "Personal development and
+   well-being" vs the identically-named legacy judgement). Check
+   whether a school's latest inspection is merely its latest *graded*
+   one. Confirm Early years provision subgrades flow through the
+   pipeline for schools with nurseries.
+5. **Subgroup honesty:** disadvantaged-pupil percentages carry cohort
+   sizes and follow the DfE suppression rule (≤5 pupils suppressed);
+   state the rule verbatim in the footer.
+6. **Benchmark provenance:** official DfE figures and
+   dataset-computed benchmarks must be labelled distinctly and
+   consistently everywhere (a computed median is a "benchmark",
+   not an "England average").
+7. **Copy details:** "Latest Ofsted inspection" (not "Ofsted rating");
+   "EHC plans"; SEN-support benchmark ≈14%; high SEN share may
+   indicate resourced provision (say so neutrally); "at or above
+   capacity" rather than "full" (capacity data is often stale);
+   disambiguate Ofsted's "Expected standard" from the KS2 measure;
+   give official terms ("first preference") alongside plain English.
+   Writing has no "higher standard" — its TA outcome is "greater
+   depth (GDS)"; never list writing under a higher-standard group.
+   Science and writing are teacher-assessed and must be labelled as
+   such (no KS2 science test since 2009). House style for benchmark
+   provenance: official DfE figures say "England average"; computed
+   figures say "state-school average (computed from our dataset)" —
+   applied to every chip, tooltip, header note and section intro.
+   EAL uses the census wording: first language known or believed to
+   be other than English. If IDACI ships, caveat that it describes
+   pupils' home neighbourhoods, not the school.
+
+## 9. Rollout
+
+1. **PR 1 (backend):** extend `/api/compare` + tests.
+2. **PR 2 (frontend):** new compare layout behind the existing route;
+   e2e journey updated in the same PR (promotion gate).
+3. **Fix separately:** missing third series on the current chart.
+
+## 10. Open questions for review
+
+- Max schools: keep 10 in API but cap visible columns at 4 with scroll?
+- Should distance-from-home appear when the user searched by postcode
+  (data exists via `dim_location`)?
+- Keep finance out of v1? (Recommended: yes, out.)