The final review claimed DfE published school-level 2021/22 KS2 in Dec
2022. Re-verified: the GOV.UK announcement 'Primary school performance
tables: 2022' is CANCELLED ('will not be published in key stage 2
performance tables in academic year 2021/22'), and the EES 2021/22
release carries the same statement. No code change.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146VHeLAWjDVE2B5uU67jCB
New ees_ks4_national stream ingests the EES 'National characteristics
summary data' series (England, state-funded, all pupils). The old mart's
unweighted school means were 7-15 points off every headline measure and
produced an impossible national Progress 8 (-0.27). The API's computed
fallback is gone too: the footnote calls these figures official, so an
unbuilt mart now yields an empty series, never a stand-in.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146VHeLAWjDVE2B5uU67jCB
The FSM chip anchored against disadvantaged_pct (a different measure,
FSM6+CLA) whenever fsm_pct was null — which it always was, since the
performance df has no fsm_pct. New fact_census_benchmarks mart supplies
pupil-weighted FSM/EAL means per phase; the KS2-column medians that
produced a bogus 50% 'secondary disadvantaged' anchor are gone.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146VHeLAWjDVE2B5uU67jCB
The MI file's report-card grade columns belong to the latest FULL
inspection (col 'Inspection start date'), but inspection_date maps to the
legacy OEIF graded/ungraded dates — so report cards were being dated with
pre-Nov-2025 inspections. Also discover_csv_url() returned matches[0],
the oldest (2017) link on the GOV.UK page.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146VHeLAWjDVE2B5uU67jCB
The Navigation header is position:sticky top:0 (z-index 1000). The school
bar was also sticky top:0 (z-index 10), so when scrolled it pinned at the
same top:0 *behind* the header — on mobile 57 of its 72px were covered,
leaving only a sliver, so you couldn't see which schools were selected.
Offset the bar's sticky top to the header height (65px desktop, 57px
mobile — the Navigation breakpoint is also 640px) so it pins just below.
e2e guard asserts the bar's sticky offset is at least the header height
(verified it fails against the pre-fix build).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146VHeLAWjDVE2B5uU67jCB
The grid sections (At a glance, Ofsted, Getting a place, Who goes there)
collapsed generically on mobile — grey label pills, full names wrapping
to 3 lines, no dots — making the page ~2x the mockup's height and
'significantly different' from the mobile design.
Each measure is now wrapped in a <Measure> that is display:contents on
desktop (so the label + cells still flow into the shared aligned grid,
unchanged) and a white card on mobile with compact [dot][short name]
[value] rows — matching the mobile mockup. The sticky school bar becomes
scrollable short-name pills on mobile too. Adds a shortName() util.
Desktop layout is unchanged (display:contents dissolves the wrapper).
Validated the card mechanism and real content shapes (report-card cell,
badges, %+chip rows) via static previews at both widths.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146VHeLAWjDVE2B5uU67jCB
The mockup's Explore trends section is the measure picker + chart only —
no data table. Removes the table (and the now-unused progressBand / band
chip / formatMetricValue plumbing that only fed it).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146VHeLAWjDVE2B5uU67jCB
ComparisonChart runs Chart.js with maintainAspectRatio:false, so it sizes
to its container's height — which must be definite. TrendsExplorer gave
.chartBox a min-height, which doesn't resolve the chart wrapper's
height:100%, so Chart.js fell back to its ~150px default: a squashed
8.6:1 sliver that didn't match the mockups. Set a definite height (420px
desktop, 360px mobile where the chips row sits above the canvas).
Verified on staging by patching the live height: canvas went from
1287x150 to 1287x392 (desktop) / 284 (mobile) — proper ~3:1 proportions
matching the mockup, with the England dashed line, COVID/2021-22 gap and
table all reading correctly.
An e2e guard asserts the trends canvas is taller than 220px so the
squash can't regress.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146VHeLAWjDVE2B5uU67jCB
The compare page's phase tabs put all-through schools (which carry KS4
data) on the secondary tab, so comparing an all-through school with a
pure primary splits them across tabs and only the active tab renders its
link. The test picked the first two 'primary' search hits without
guaranteeing same phase, so it flaked whenever a search returned an
all-through school first (e.g. URN 137306). Now selects two pure-Primary
URNs via the API — deterministic and data-invariant.
Verified against staging: was a 15.6s timeout, now passes in ~1.8s.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146VHeLAWjDVE2B5uU67jCB
The job does ~3s of real work (typecheck 1.4s + jest 1.3s) but installs
452 MB / 460 packages every run. Two changes:
- Cache nextjs-app/node_modules keyed on the lockfile hash (OS + node
major pinned) and skip npm ci entirely on a hit — deps change rarely,
so most PR pushes now do zero install. On miss, npm ci runs with
--prefer-offline --no-audit --no-fund.
- Workflow-level concurrency with cancel-in-progress: a new commit (or an
empty re-trigger) aborts the previous run instead of stacking a second
full matrix.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146VHeLAWjDVE2B5uU67jCB
get_supplementary_data ran ~5 sequential DB round-trips per URN, so
/api/compare scaled at ~37ms/school (measured on staging: 1 school 155ms,
3 schools 220ms, 6 schools 340ms). get_supplementary_data_batch fetches
each table once with WHERE urn IN (...) and groups in Python, collapsing
5*N round-trips to a constant 5. get_supplementary_data is now a thin
wrapper so the detail endpoint is unchanged; the compare endpoint makes
one batched call. Each table degrades independently on failure.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146VHeLAWjDVE2B5uU67jCB
Refresh bug: on mount the basket is empty for a beat before it hydrates
from the URL. The fetch effect nulled comparisonData on that transient
empty urnKey, then the one-shot 'SSR covers it' skip suppressed the
refetch — leaving the page blank on reload. The effect is now gated on
isInitialized, never blanks on empty (the render already shows the empty
state when nothing is selected), and decides fetch-vs-skip by whether it
already holds each requested school's data (SSR or a prior fetch).
Perf: useComparison ran a useSWR('/api/compare') whose result nothing
consumed — dead weight that fired on every page (Navigation + Toast are
global) whenever the basket was non-empty, and duplicated ComparisonView's
own fetch on the compare page. Removed; the hook now exposes basket state
only.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146VHeLAWjDVE2B5uU67jCB
- Metric changes no longer refire /api/compare (the data is already
client-side; the picker is presentational) — the fetch effect depends
only on the URN set, with URL sync split into its own effect.
- The initial client fetch is skipped when the SSR payload already covers
the selected schools; national averages + benchmarks now arrive via SSR
props so nothing is lost by skipping.
- page.tsx fetches comparison and metrics in parallel.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146VHeLAWjDVE2B5uU67jCB
fact_ks4_national_averages is computed once at dbt build time (covered by
the EES DAG's stg_ees_ks4+ selector). _national_averages_payload now reads
both national-averages marts instead of scanning the performance dataframe
per year on every /api/compare request (~250ms saved per call). Fallback
for the deploy-before-DAG window computes the latest year only.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146VHeLAWjDVE2B5uU67jCB
Addresses the AI review findings on PR #33:
- severe: the workflow_dispatch sha input was interpolated directly into
the run script (shell injection with REGISTRY_TOKEN + prod webhook in
scope). It now reaches the shell only via env, is rejected if it
starts with '-', and is resolved locally with git rev-parse.
- minor: the resolved sha must be a 40-hex ancestor of origin/main —
non-main refs are refused explicitly instead of implicitly.
- minor: a prod-promotion concurrency group serialises promotions
(cancel-in-progress: false).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146VHeLAWjDVE2B5uU67jCB
The AI review gate caught that stg_ees_ks2's 7 new columns broke the
positional UNION ALL with stg_legacy_ks2 in int_ks2_with_lineage, and
that the lineage CTEs never emitted them (same class of bug fixed for
KS4 in 34a5de2). Legacy gets typed null placeholders at matching
positions; both lineage CTEs pass the columns through. 45/45 columns
verified name-identical in order across both union branches.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146VHeLAWjDVE2B5uU67jCB
Evidence trail for the rc_* mapping in the prior commit: real value_counts()
over the 7 MI report-card columns, confirming the 5-value grade vocabulary
and that 'Achievement'/'Safeguarding standards' match by exact string only
(no legacy OEIF column accidentally consumed).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146VHeLAWjDVE2B5uU67jCB
Wires the tap TODO in stg_ofsted_inspections.sql: maps the 7 confirmed
report-card MI columns (Safeguarding standards, Inclusion, Curriculum
and teaching, Achievement, Attendance and behaviour, Personal
development and wellbeing, Leadership and governance) into rc_*
fields, parsed via the new parse_report_card_grade macro against
real sampled grade values (Exceptional/Strong standard/Expected
standard/Needs attention/Urgent improvement). rc_safeguarding_met
becomes boolean from Met/Not met. rc_early_years/rc_sixth_form have
no MI column yet and are intentionally omitted from COLUMN_PRIORITY,
staying NULL.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146VHeLAWjDVE2B5uU67jCB
DfE never published school-level KS2 2021/22 data publicly (confirmed via
EES release notes and by walking the Compare School Performance download
wizard, which has no ks2 checkbox for 2021-2022, same as the COVID-cancelled
2020-2021 year). No archive exists to verify column headers against or
upload to the filebrowser; Task 6 is blocked at the source-data level.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146VHeLAWjDVE2B5uU67jCB
Widen the year filter in stg_ees_ks2_national.sql from >= 201617 to
>= 201516 so the England national-averages line no longer starts a
year late; the catalogue CSV has a real, comparable 201516 row (2015/16
was the first year of the current expected-standard tests, so it's the
correct floor).
GPS/science/scaled-score national columns confirmed present at source
with correct mapping; prod NULLs are stale raw data, backfilled by the
next extract run. No _KS2_NATIONAL_COL_MAP change accompanies this fix.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146VHeLAWjDVE2B5uU67jCB
int_ks4_with_lineage.sql unions stg_ees_ks4 and stg_legacy_ks4 via
`select *`, which PostgreSQL aligns positionally. stg_legacy_ks4 listed
sen_support_pct before sen_ehcp_pct while stg_ees_ks4 lists sen_ehcp_pct
before sen_support_pct, swapping the two values for legacy-sourced rows
in marts.fact_ks4_performance. Reordered stg_legacy_ks4's final select
to match.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146VHeLAWjDVE2B5uU67jCB
ReligiousCharacter 99 (~4k schools) and AdmissionsPolicy 9 (~5.6k) carry a
code with a blank name in the GIAS CSV; the generator skipped them so they
hit the Unknown(<code>) path — wrongly triggering the Faith-priority tag
and polluting filters. Blank-only codes now map to "" (byte-identical to
the old name pipeline); accepted_values lists extended to match the seed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Addresses AI-review findings: the annual IDACI DAG also rebuilds a mart
(fact_deprivation) and needs the reload; curl gets connect/max timeouts
so an unreachable backend fails fast instead of hanging the task.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The daily/monthly/annual DAG docstring promised an Invalidate Cache step
that never existed — after a marts rebuild the backend kept serving its
startup-cached (possibly empty) DataFrame until a container restart.
Add a POST /api/admin/reload task at the end of each pipeline DAG,
mirroring the sitemap DAG's admin-call pattern.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
str(ProgrammingError) embeds the full SQL, which contains every column
name — the substring check matched any error and could take the wrong
retry branch. Parse the missing column from exc.orig instead.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Closes the deploy window flagged by CI review — the backend now works
against both the old (name) and new (code) mart schemas.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Amber tag in listing rows (option A) and a slim notice strip under the
detail-page header (option E): proposed for closure, formal process not
necessarily started, check with the local authority before applying.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
These schools are still operating and publish results; they drop out
automatically when GIAS flips them to Closed since marts fully rebuild
each run.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Matches the phase derivation's guard against casing/whitespace variants in
raw GIAS data; an unmatched variant previously fell through silently to the
statutory-age fallback.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- data_loader.load_school_data_as_dataframe now catches a ProgrammingError
whose message mentions has_sixth_form (psycopg2 UndefinedColumn) and
retries with a NULL-AS-has_sixth_form query variant, so the API keeps
serving data (and the app.py column-fallback branch stays reachable)
even before the nightly pipeline has rebuilt marts.dim_school.
- utils.convert_to_native now handles numpy.bool_ so GET /api/schools/{urn}
doesn't 500 once has_sixth_form is a populated bool-dtype column.
- Update the now-stale comment on the app.py age-range fallback branch.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Schools without KS2/KS4 results (special post-16 institutions, sixth-form
centres, PRUs, new schools) come back from the marts LEFT JOIN with NaN in
every numeric column. school_info passed those raw pandas values straight
into JSONResponse, which renders with allow_nan=False, so the detail
endpoint 500d and the frontend turned that into a 404 on every such SEO
landing page.
Run school_info values through convert_to_native (the same treatment
yearly_data already gets), add backend unit tests plus a pytest step in PR
checks, and an e2e journey that finds a results-less school via the search
API and asserts its page renders.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The geolocation shortcut lived in the discovery strip below the results,
away from the search. Move it directly under the hero search input, paired
with the postcode hint, so the two ways to find nearby schools ("type a
postcode" / "use my location") read as one idea and are visible at first
glance.
- FilterBar gains optional onNearMe/geoState/geoError props and renders the
teal "Use my location" pill (with spinner + error) in hero mode; the
geolocation flow itself still lives in HomeView.
- Remove the now-duplicate near-me button and its dead CSS from the
discovery section.
- Refresh the search hint copy to pair with the button.
- e2e: assert the "use my location" shortcut renders in the hero.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The results-view map's fullscreen button called requestFullscreen(),
which iOS Safari doesn't implement (fullscreen is video-only there), so
tapping it did nothing on iPhones — the same gap already fixed for the
school hero map.
When the Fullscreen API is missing or its promise rejects, fall back to
a fixed-position overlay (.fsFallback, z-index 5000) driven by state,
locking body scroll while open. Leaflet re-measures on window resize, so
dispatch a resize when fullscreen toggles (the CSS overlay fires none) or
the map would fill only part of the screen. Native fullscreen is
unchanged.
New e2e journey deletes Element.requestFullscreen on a mobile viewport,
opens the results map fullscreen, and asserts the exit control appears
then releases; it fails against current production, reproducing the bug.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
On a location search the results header shows the view toggle and the
sort <select> side by side. The select sizes to its widest option
('Highest Reading, Writing & Maths %', ~273px), so on a phone its right
edge ran ~46px past the viewport and was clipped off-screen.
On mobile let the select flex into the remaining space with min-width:0
so its label truncates instead of overflowing, and keep the view toggle
from shrinking. Verified live at 390px: the select now sits fully within
the viewport.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The mobile chart legend stacked one school chip per line, so up to five
schools pushed the chart down and left the plot cramped. Switch the chip
row to a two-column grid; each chip fills its column and truncates its
name with an ellipsis (full names remain on the school cards and in the
tooltip). Five schools now take three rows instead of five, giving the
chart noticeably more height.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The test picked the first two /school/ links from a 'primary' search and
asserted exactly two mobile chips. But a 'primary' search can return
all-through schools (e.g. 'Hessle High School and Penshurst Primary')
that classify as secondary, so the two picks can split across phases —
the active phase then holds one school and the chips are correctly gated
out (they need ≥2 in the active phase), while the canvas still shows one
line. That's a test artefact, not a bug.
Pick three schools instead: across two phases the auto-selected majority
phase always holds ≥2, so the chip legend is guaranteed. Assert ≥2 chips
(the majority may be 2 or 3). Verified against staging.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
next.config.js rewrites() bakes its destination into the build
(routes-manifest.json), capturing FASTAPI_URL at build time. Because one
frontend image is promoted staging->prod, the baked backend host forced
every environment to name the backend service identically; staging names
it 'backend_stg', so the browser's /api/* calls proxied to the baked
'http://backend' and failed with getaddrinfo ENOTFOUND backend. (SSR was
unaffected because lib/api.ts reads FASTAPI_URL at runtime.)
Replace the rewrites with route handlers that read FASTAPI_URL per
request:
- app/api/[...path]/route.ts — transparent proxy for all methods, streams
the response, strips hop-by-hop headers, and returns 502 on upstream
failure instead of crashing.
- app/sitemap.xml/route.ts — proxies the backend sitemap (robots.ts points
crawlers here).
The same promoted image now adapts to whatever the backend is called in
each environment. Verified: production build succeeds with /api/[...path]
and /sitemap.xml as dynamic routes and an empty rewrites manifest.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Removes the 'What Parents Say' section and all supporting elements:
Frontend:
- Drop the OfstedParentView type, the parent_view field, the survey
section and the 'X% would recommend' callouts in the primary and
secondary detail views, the Parents nav item, and the parent-view CSS.
Backend:
- Remove the FactParentView model, its loading in data_loader, and
parent_view from the school-details API response.
- Bump SCHEMA_VERSION to 6 and add an idempotent drop step
(DROP TABLE IF EXISTS marts.fact_parent_view) to the CLI migration;
add scripts/sql/drop_fact_parent_view.sql to apply directly to the
dbt-owned marts DBs on staging and prod.
Pipeline:
- Delete the stg_parent_view + fact_parent_view dbt models and their
source/schema entries, the tap-uk-parent-view Meltano extractor, and
the monthly Parent View DAG; drop it from the Dockerfile and the
staging bootstrap docs.
The rest of dbt (which builds every mart the app reads) is untouched.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
