Files
school_compare/docs/superpowers/plans/2026-07-13-compare-frontend-rebuild.md

288 lines
22 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Compare Screen Frontend Rebuild Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Rebuild `/compare` in the Next.js app to match the approved mockups — parent-first sections (At a glance / Ofsted / Academics / Getting a place / Who goes there / Explore trends), England-average anchoring with provenance-correct labels, mobile-first measure-first layout — consuming the enriched `/api/compare` payload from PR #34, with e2e journeys updated in the same PR (they are the promotion gate).
**Architecture:** `ComparisonView` becomes an assembly of section components fed by one enriched fetch. All comprehension rules from the two expert reviews live in a pure, jest-tested module (`lib/compareLogic.ts`) — components stay presentational. The mockups are committed at `docs/superpowers/specs/mockups/compare-desktop.html` and `compare-mobile.html`: **all user-facing copy (labels, tooltips, chips, footnote wording) is taken verbatim from them** — they carry two rounds of education-expert review; do not paraphrase.
**Tech Stack:** Next.js (app router, SSR page + client view), CSS modules, Chart.js (existing `ComparisonChart`), Jest (`npm test` in `nextjs-app/`), Playwright e2e (`e2e/`).
## Global Constraints
- **Never push to `main`.** Branch: `feat/compare-frontend-rebuild`.
- **Copy is expert-reviewed:** take it verbatim from the committed mockups. Binding rules (spec §8): Ofsted scale labels come from the API's `report_card[..].label` (never hardcode area labels beyond the mockups'); official DfE numbers say "England average", computed ones say "state-school average (computed from our dataset)"; the 2021/22 chart gap note says "DfE didn't publish school-level figures for 2021/22"; never derive an overall grade from report-card areas; safeguarding never counts as a graded area; "Latest Ofsted inspection", "EHC plans", "at or above capacity", "Over 1 in 4", "first choice (officially 'first preference')".
- **Mobile-first:** the measure-first stacked layout (mobile mockup) is the base CSS; the desktop label-column grid is the `min-width` enhancement.
- **URL contract unchanged:** `?urns=` (and `metric=` now scoped to Explore trends) keep working; share flow, `useComparison` basket, phase tabs, and `compare_viewed`/`compare_metric_changed` analytics events are preserved.
- **Do not run a local server** (CLAUDE.md); verification = jest + `tsc` + the e2e suite against staging after merge. e2e must pass on **staging data** — remember staging has partial history: assert against the *latest* year, never oldest.
- Existing `/api/compare` consumers elsewhere in the app (SchoolDetail links, toasts) must not break — the response is additive, and this PR only rewrites the compare page's own components.
- **Post-v1 (do not build):** IDACI, attendance section, gender-split/absence tier-2 measures, finance (spec §4).
---
### Task 0: Branch + design sources
- [ ] `git checkout main && git pull && git checkout -b feat/compare-frontend-rebuild`
- [ ] The mockups and this plan are already in the working tree (`docs/superpowers/specs/mockups/compare-{desktop,mobile}.html`) — commit them: `docs: compare mockups as frontend design source + rebuild plan`
---
### Task 1: Types for the enriched payload
**Files:**
- Modify: `nextjs-app/lib/types.ts` (extend `SchoolResult`, `ComparisonData`, `ComparisonResponse` — located around lines 293-314)
**Interfaces (produced for every later task):**
```ts
export interface ReportCardEntry { code: number; label: string; }
export interface OfstedBlock {
framework: string | null;
inspection_date: string | null;
inspection_type: string | null;
overall_effectiveness: number | null;
grade_source: 'graded' | 'ungraded_carried_forward' | null;
quality_of_education: number | null;
behaviour_attitudes: number | null;
personal_development: number | null;
leadership_management: number | null;
early_years_provision: number | null;
sixth_form_provision: number | null;
rc_safeguarding_met: boolean | null;
report_card: Record<string, ReportCardEntry>;
ofsted_page_url: string;
report_url: string | null;
}
export interface CensusBlock {
year: number | null; total_pupils: number | null;
female_pupils: number | null; male_pupils: number | null;
fsm_pct: number | null; eal_pct: number | null;
}
export interface AdmissionsRow {
year: number; school_phase: string | null;
places_offered: number | null; total_applications: number | null;
first_preference_applications: number | null; first_preference_offers: number | null;
first_preference_offer_pct: number | null; oversubscription_ratio: number | null;
oversubscribed: boolean | null;
total_offers: number | null; second_preference_offers: number | null;
third_preference_offers: number | null;
cross_la_applications: number | null; cross_la_offers: number | null;
}
export interface DeprivationBlock {
lsoa_code: string | null; idaci_score: number | null; idaci_decile: number | null;
}
export interface BenchmarkBlock {
eal_pct: number | null; sen_support_pct: number | null;
disadvantaged_pct: number | null; median_pupils: number | null;
disadvantaged_rwm_expected_pct?: number | null;
}
export interface Benchmarks {
source: string; year: number;
primary: BenchmarkBlock; secondary: BenchmarkBlock;
}
export interface NationalAverages {
year: number;
primary: Record<string, number>;
secondary: Record<string, number>;
by_year: Array<{ year: number; primary: Record<string, number>; secondary: Record<string, number> }>;
}
```
- [ ] **Step 1:** Add the interfaces above; extend `ComparisonData` with optional `ofsted?: OfstedBlock | null; census?: CensusBlock | null; admissions?: AdmissionsRow | null; admissions_history?: AdmissionsRow[]; deprivation?: DeprivationBlock | null;` and `ComparisonResponse` with `national_averages?: NationalAverages; benchmarks?: Benchmarks;` (optional so the UI degrades on an old backend). Extend `SchoolResult` with the ten new yearly columns (`reading_progress_lower_ci``maths_progress_upper_ci`, `writing_working_towards_pct`, `progress_8_banding: string | null`, `attainment_8_disadvantage_gap`, `progress_8_disadvantage_gap`).
- [ ] **Step 2:** `cd nextjs-app && npx tsc --noEmit` → clean. Commit: `feat(compare): types for enriched comparison payload`
---
### Task 2: `lib/compareLogic.ts` — the comprehension rules, jest-tested
**Files:**
- Create: `nextjs-app/lib/compareLogic.ts`
- Test: `nextjs-app/__tests__/lib/compareLogic.test.ts`
**Interfaces (produced):**
```ts
export type Verdict = 'above' | 'close' | 'below';
export function verdict(value: number, anchor: number, tolerance?: number): Verdict; // default tolerance 2pp
// Report-card summary per spec §4.2: count graded areas per label (best
// first), NAME any 'Needs attention'/'Urgent improvement' area, safeguarding
// separate, "No areas need attention" reassurance when applicable.
export interface ReportCardSummary {
counts: Array<{ label: string; count: number }>; // best grade first
problems: Array<{ areaLabel: string; label: string }>; // named, never counted-away
safeguarding: 'met' | 'not_met' | null;
allClear: boolean;
}
export function summariseReportCard(ofsted: OfstedBlock): ReportCardSummary;
// One display model for all three inspection regimes.
export type OfstedDisplay =
| { kind: 'none' }
| { kind: 'graded'; grade: number; gradeLabel: string; carriedForward: false }
| { kind: 'carried_forward'; grade: number; gradeLabel: string; carriedForward: true }
| { kind: 'report_card'; summary: ReportCardSummary };
export function ofstedDisplay(ofsted: OfstedBlock | null | undefined): OfstedDisplay;
export const OFSTED_LEGACY_GRADES: Record<number, string>; // 1 Outstanding, 2 Good, 3 Requires improvement, 4 Inadequate
// Human-readable area label from an rc_ key: 'rc_attendance_behaviour' →
// 'Attendance & behaviour' (mapping table copied from the mockups' area rows).
export function rcAreaLabel(key: string): string;
// Admissions, one consistent chip metric (first-preference success).
export interface AdmissionsSummary {
firstPrefPct: number | null;
chip: { tone: 'good' | 'warn' | 'neutral'; text: string } | null; // "97% of first choices offered" / "Over 1 in 4 first choices missed out" wording per mockups
interest: string | null; // "Named on 457 forms · 180 places"
}
export function summariseAdmissions(a: AdmissionsRow | null | undefined): AdmissionsSummary;
// CI-based progress band for historical years (null when no CI published).
export function progressBand(score: number | null, lower: number | null, upper: number | null):
'above' | 'average' | 'below' | null; // CI entirely >0 → above; entirely <0 → below; straddles → average
// Dot-strip geometry (used by the DotStrip component; pure for testing).
export interface StripPoint { pos: number; labelAbove: boolean; value: number; schoolIndex: number; }
export function stripPositions(values: Array<number | null>, min: number, max: number): StripPoint[];
// pos = (v-min)/(max-min)*100 clamped 0..100; labels within 4% of range of a
// lower neighbour flip above (the mockups' collision nudge).
```
- [ ] **Step 1: Failing tests** covering, at minimum:
- `summariseReportCard`: 4 Strong + 2 Expected + 1 Needs-attention + safeguarding met → counts `[Strong standard×4, Expected standard×2]`, `problems=[{areaLabel:'Attendance & behaviour', label:'Needs attention'}]`, `allClear=false`; safeguarding NEVER in counts; all-Expected+met → `allClear=true`; labels come from the input's `.label` (assert the function never invents "Attention needed").
- `ofstedDisplay`: report_card present → `kind:'report_card'` even if a legacy grade also exists; `grade_source:'ungraded_carried_forward'``carriedForward:true`; null → `'none'`.
- `summariseAdmissions`: 73% → warn chip text `Over 1 in 4 first choices missed out`; 97% → good chip `97% of first choices offered`; 100% → `All first choices offered`; interest string `Named on 342 forms · 120 places`; nulls → null chip.
- `progressBand`: (1.2, 0.4, 2.0)→above; (-1.2, -2.0, -0.4)→below; (0.3, -0.5, 1.1)→average; missing CI → null.
- `stripPositions`: 100120 domain maps 106→30; values 91 and 92 on 0100 → second label flips above; nulls skipped.
- `verdict`: 87 vs 62 → above; 61 vs 62 → close (within 2pp); 40 vs 62 → below.
- [ ] **Step 2:** `cd nextjs-app && npm test -- compareLogic` → FAIL. **Step 3:** implement. **Step 4:** pass + `tsc` clean. **Step 5:** Commit: `feat(compare): comprehension logic (report cards, admissions, verdicts, strips)`
---
### Task 3: `DotStrip` component
**Files:**
- Create: `nextjs-app/components/DotStrip.tsx`, `nextjs-app/components/DotStrip.module.css`
**Interfaces:**
```ts
export interface DotStripProps {
label: string;
values: Array<number | null>; // one per school, school order = chart colour order
anchor?: { value: number; label: string } | null; // e.g. {62, "England 62%"} — omit when benchmark absent
min?: number; max?: number; // default 0..100
unit?: string; // default '%'
tip?: string; // title tooltip on the label
note?: string; // e.g. "(teacher-assessed)" suffix handled by caller in label
}
```
- [ ] Render per the mockups' `.strip-row` anatomy: label row, 4px track, England tick + tick label, 16px dots coloured by `CHART_COLORS[index]` with white ring, value labels below (flipped above on collision via `stripPositions`). `role="img"` + `aria-label` enumerating anchor and each school's value (copy the aria pattern from the mockups). CSS module mirrors the mockup styles using the app's CSS variables (`--border-light`, `--text-muted`, etc.).
- [ ] Jest: render with `@testing-library/react` (already configured — see `__tests__/components/SecondarySchoolRow.test.tsx` for the harness pattern): asserts aria-label content, tick present when anchor given, absent otherwise.
- [ ] Commit: `feat(compare): DotStrip with England-average anchor`
---
### Task 4: Section components — At a glance, Ofsted, Getting a place, Who goes there
**Files:**
- Create: `nextjs-app/components/compare/CompareAtAGlance.tsx` (+ `.module.css`)
- Create: `nextjs-app/components/compare/CompareOfsted.tsx`
- Create: `nextjs-app/components/compare/CompareAdmissions.tsx`
- Create: `nextjs-app/components/compare/CompareCommunity.tsx`
- Create: `nextjs-app/components/compare/compareSections.module.css` (shared measure-first grid)
- Test: `nextjs-app/__tests__/components/CompareOfsted.test.tsx`
**Shared layout contract (all four):** props `{ schools: School[]; data: Record<string, ComparisonData>; benchmarks?: Benchmarks; nationalAverages?: NationalAverages }`. Base CSS is the mobile mockup's measure-first stack (`.measure` card → `.srow` per school with colour dot + short name + value + chip + note); at `min-width: 761px` it becomes the desktop mockup's grid (200px row-label column + one column per school). Section headers use the existing `.section-title` idiom; every section carries its mockup "how" line verbatim.
**Content per section = the mockups, row for row.** Structure/tone rules already encoded in Task 2's helpers:
- *At a glance*: Latest Ofsted inspection row (badge via `ofstedDisplay`; report-card case renders `ReportCardSummary` chips — counts best-first + named problem chips + safeguarding line); expected-standard row (big % + `verdict` chip vs `national_averages.primary.rwm_expected_pct`, small "England average N%"); Getting a place row (chip from `summariseAdmissions`, note = `interest`); Size row (pupils + "at or above capacity"/"N% full" from census/capacity, vs `benchmarks.*.median_pupils` for "larger/smaller than average" phrasing).
- *Ofsted*: the section's `how` paragraph (regime explanation + non-comparability + "Expected standard" disambiguation) verbatim from the desktop mockup; Result row; Inspected row (date + "4+ years ago" chip when >4y, computed from `inspection_date`); Judgement detail row — **one chip-list grammar for both regimes** (legacy subgrades via `OFSTED_LEGACY_GRADES`; report card via `report_card` labels; "We don't hold area-by-area detail for this inspection" when neither); Ofsted page row linking `ofsted_page_url` ("<Name>'s Ofsted page →").
- *Getting a place*: `how` paragraph (first preference/equal preference/offer-day caveats) verbatim; Interest row; first-choice success row with mini bar; "What this means" row (distance note: "check the school's admission criteria (for most non-faith primaries, distance decides)" only when oversubscribed).
- *Who goes there*: pupils-on-roll (census + capacity), girls/boys, FSM (chip vs `benchmarks` with "state-school average" wording), EAL, SEN (tooltip incl. "EHC plans" + specialist-provision note), faith, ages · nursery, run by (trust name or "<LA> council").
- [ ] **Step 1:** Failing jest test for `CompareOfsted` (the riskiest): given one graded school, one carried-forward, one report-card school → asserts the three Result cells ("Outstanding" badge; badge + carried-forward marker; "Report card" + no invented overall grade), the chip-list judgement rows, and the comparability note appearing only for the mixed case.
- [ ] **Step 2-4:** Implement all four sections; test passes; `tsc` clean; `npm test` full suite green.
- [ ] **Step 5:** Commit: `feat(compare): at-a-glance, Ofsted, admissions and community sections`
---
### Task 5: `CompareAcademics` — strips + More measures
**Files:**
- Create: `nextjs-app/components/compare/CompareAcademics.tsx`
- Test: extend `nextjs-app/__tests__/lib/compareLogic.test.ts` with the metric-extraction helper below
**Interfaces:**
- Add to `compareLogic.ts`: `latestValues(data, urns, metricKey) => Array<number|null>` (latest non-null yearly value per school) — tested.
- [ ] Tier 1 strips (always visible), each a `DotStrip` with the England anchor from `national_averages.primary`: RWM expected, Reading, Writing, Maths, "Working at a higher standard than expected" (tooltip: composition sentence from the mockups). Section `how` line: "tests and teacher assessments … writing is assessed by teachers, not tested" verbatim.
- [ ] Tier 2 `<details>` "More measures — grammar, punctuation & spelling, science, average scaled scores": GPS + Science (teacher-assessed, tooltip verbatim) with anchors from `national_averages` **when present, no tick + honest note when null**; scaled scores (reading/maths/GPS) on `min=100 max=120` with the mockups' window caption.
- [ ] Equity row: disadvantaged pupils' RWM per school + chip vs `benchmarks.primary.disadvantaged_rwm_expected_pct` with the "state-school average" wording and small-cohort tooltip verbatim.
- [ ] Secondary phase variant (when active phase is secondary): tier-1 rows are Attainment 8 (anchor `national_averages.secondary.attainment_8_score`), Progress 8 banding (chip showing `progress_8_banding` verbatim — DfE's own label), grade 5+ English & maths %; tier-2: EBacc entry/APS. Measure-first rows (no strips needed for banding).
- [ ] `npm test` + `tsc`; commit: `feat(compare): academics strips with England anchors and More measures`
---
### Task 6: Trends explorer — England line, gap-honest axis, series bug
**Files:**
- Modify: `nextjs-app/components/ComparisonChart.tsx`
- Create: `nextjs-app/components/compare/TrendsExplorer.tsx`
- Test: `nextjs-app/__tests__/components/ComparisonChart.test.tsx`
- [ ] **Step 1 (bug first): root-cause the missing third series** seen on production (3 schools in table, 2 lines on chart). Write a failing jest test: 3 schools whose `yearly_data` year values are floats (`202425.0`) vs the labels array — the suspect is the year-matching in `ComparisonChart.tsx:69` (`years.map(...)` built from school 1 only + strict equality against other schools' years). Fix so every school's series renders and years are the union of all schools' years, sorted.
- [ ] **Step 2:** Add optional `nationalByYear?: Record<number, number|null>` prop → dashed grey "England average" dataset (colour `--text-muted`, `borderDash:[5,4]`, no fill, `spanGaps:false`).
- [ ] **Step 3:** Gap honesty: x-axis category labels include 2019/20 and 2020/21 as empty slots (band label "tests cancelled 2019/202020/21" via a Chart.js annotation-free approach: two category ticks with all-null data and a subtitle note under the chart, copy verbatim: the chart footnote "DfE didn't publish school-level figures for 2021/22" appears when the metric is a KS2 measure and 2021/22 school values are null while the England value exists). `spanGaps:false` on school datasets so dataset gaps break lines.
- [ ] **Step 4:** `TrendsExplorer` wraps the grouped metric picker (existing optgroup structure and `metrics` from `/api/metrics`, existing analytics event) + the chart + the existing year-by-year table, inside a collapsed-by-default `<details>` ("Explore trends"). Progress metrics annotate cells with `progressBand` chips for years where CIs exist.
- [ ] Tests pass; commit: `feat(compare): trends explorer with England line; fix missing series`
---
### Task 7: Assemble the new `ComparisonView`
**Files:**
- Rewrite: `nextjs-app/components/ComparisonView.tsx` (+ its `.module.css`)
- Modify: `nextjs-app/app/compare/page.tsx` metadata description (mention Ofsted/admissions, not just KS2)
- [ ] Preserve intact: `useComparison` basket seeding/URL sync (lines 76-122 of the current file), share handler, phase tabs + auto-detection, `compare_viewed` analytics, empty states, `SchoolSearchModal`, max-4-visible column scroll. Replace the metric-picker/chart/table body with the section stack: sticky school chip bar (mockup `.school-bar`) → `CompareAtAGlance``CompareOfsted``CompareAcademics``CompareAdmissions``CompareCommunity``TrendsExplorer`. The page-level `metric` URL param now initialises `TrendsExplorer`'s picker only.
- [ ] Top-of-page subtitle + sources footnote verbatim from the mockups (minus the "Mockup" banner), including the suppression rule sentence and provenance sentence.
- [ ] `npm test` full suite + `tsc` clean. Commit: `feat(compare): parent-first compare screen assembly`
---
### Task 8: e2e journeys (the promotion gate)
**Files:**
- Modify: `e2e/tests/journeys.spec.ts` (the two compare tests, lines ~141-215; extend, don't delete coverage)
- [ ] Update 'comparing two schools shows both side by side': after loading `/compare?urns=…` assert the new section headings (`At a glance`, `Ofsted inspection`, `How children do academically`, `Getting a place`, `Who goes there`, `Explore trends`), both school names in the sticky bar, at least one England-average tick label (`text=/England \d+%/`), and one provenance string `state-school average` somewhere (benchmarks row). Data-invariant style — no exact numbers (staging data shifts; use latest-year values only).
- [ ] Update the mobile test: 390px viewport, assert measure-first stacking (a `.measure`-card contains all selected school names within one card) and that the trends chart container scrolls (`overflow-x`).
- [ ] Add a report-card presence-agnostic assertion: the Ofsted section renders either a grade badge or "Report card" without an overall grade — i.e. never both an overall-grade badge AND report-card chips for the same school.
- [ ] Run against staging from the host if reachable (`cd e2e && BASE_URL=https://stx.schoolcompare.co.uk npx playwright test -g "compar"`) — staging still runs the OLD UI until this PR merges, so expect failures locally; the authoritative run is the Stage pipeline post-merge. Still commit only after jest+tsc are green.
- [ ] Commit: `test(e2e): compare journeys for the parent-first redesign`
---
### Task 9: PR + post-merge verification
- [ ] Full gates: `cd nextjs-app && npm test && npx tsc --noEmit`.
- [ ] Push; open PR via Gitea API (credential-helper basic auth). PR body: before/after summary, link to mockups + spec §4/§8, the copy-verbatim rule, the fixed third-series bug, deploy note (needs PR #34's API on the same environment — merge order: #34 first), and that the e2e suite is the staging gate.
- [ ] Post-merge: watch the Stage pipeline — its e2e run against staging is the real verification. Then the human tests staging and promotes (two-stage model). Update memory: compare redesign shipped to staging.
---
## Out of scope
- IDACI / attendance / gender-absence / finance (post-v1, spec §4).
- Backend changes of any kind (PR #34 must merge first).
- Chart palette overhaul beyond the England-line addition (`CHART_COLORS` swap to the validated trio is a candidate follow-up, flagged not included — it affects every chart in the app).