Compare commits

..
Author SHA1 Message Date
tudor 404ba95275 Merge branch 'main' into fix/gias-blank-name-codes
PR Checks / Frontend Typecheck + Tests (pull_request) Successful in 1m3s
PR Checks / Backend Smoke (pull_request) Successful in 7s
PR Checks / Build Backend (no push) (pull_request) Successful in 10s
PR Checks / Build Frontend (no push) (pull_request) Successful in 50s
PR Checks / Build Pipeline (no push) (pull_request) Successful in 10s
PR Checks / AI Code Review (Claude) (pull_request) Successful in 33s
2026-07-16 09:14:42 +00:00
tudor d98e88f0b4 Merge pull request 'fix: preserve literal 'NULL' strings for primary key columns' (#49) from feature/ingest-independent-schools into main
Stage (build -> staging -> E2E gate) / Build Backend (FastAPI) (push) Successful in 13s
Stage (build -> staging -> E2E gate) / Build Frontend (Next.js) (push) Successful in 50s
Stage (build -> staging -> E2E gate) / Build Pipeline (Meltano + dbt + Airflow) (push) Successful in 1m25s
Stage (build -> staging -> E2E gate) / Deploy to Staging (push) Successful in 0s
Stage (build -> staging -> E2E gate) / E2E Journeys against Staging (push) Failing after 44s
Reviewed-on: #49
2026-07-16 07:54:41 +00:00
Tudor 609bb923d9 fix: preserve literal 'NULL' strings for primary key columns
PR Checks / Frontend Typecheck + Tests (pull_request) Successful in 1m3s
PR Checks / Backend Smoke (pull_request) Successful in 7s
PR Checks / Build Backend (no push) (pull_request) Successful in 10s
PR Checks / Build Frontend (no push) (pull_request) Successful in 48s
PR Checks / Build Pipeline (no push) (pull_request) Successful in 47s
PR Checks / AI Code Review (Claude) (pull_request) Successful in 8s
2026-07-16 08:54:14 +01:00
tudor 4bfcd9ba9a Merge pull request 'fix: convert NaN/NULL to None and restore record properties structure in tap.py' (#48) from feature/ingest-independent-schools into main
Stage (build -> staging -> E2E gate) / Build Backend (FastAPI) (push) Successful in 14s
Stage (build -> staging -> E2E gate) / Build Frontend (Next.js) (push) Successful in 50s
Stage (build -> staging -> E2E gate) / Build Pipeline (Meltano + dbt + Airflow) (push) Successful in 1m22s
Stage (build -> staging -> E2E gate) / Deploy to Staging (push) Successful in 1s
Stage (build -> staging -> E2E gate) / E2E Journeys against Staging (push) Failing after 41s
Reviewed-on: #48
2026-07-16 07:40:28 +00:00
Tudor 95f10bf352 fix: convert NaN/NULL to None and restore record properties structure in tap.py
PR Checks / Frontend Typecheck + Tests (pull_request) Successful in 1m2s
PR Checks / Backend Smoke (pull_request) Successful in 7s
PR Checks / Build Backend (no push) (pull_request) Successful in 12s
PR Checks / Build Frontend (no push) (pull_request) Successful in 42s
PR Checks / Build Pipeline (no push) (pull_request) Successful in 47s
PR Checks / AI Code Review (Claude) (pull_request) Successful in 1m46s
2026-07-16 08:29:59 +01:00
tudor 674470ceb6 Merge pull request 'feat: ingest independent schools in Ofsted tap and dbt staging' (#47) from feature/ingest-independent-schools into main
Stage (build -> staging -> E2E gate) / Build Backend (FastAPI) (push) Successful in 14s
Stage (build -> staging -> E2E gate) / Build Frontend (Next.js) (push) Successful in 51s
Stage (build -> staging -> E2E gate) / Build Pipeline (Meltano + dbt + Airflow) (push) Successful in 1m25s
Stage (build -> staging -> E2E gate) / Deploy to Staging (push) Successful in 1s
Stage (build -> staging -> E2E gate) / E2E Journeys against Staging (push) Failing after 44s
Reviewed-on: #47
2026-07-15 22:26:39 +00:00
Tudor 8abff7a0a1 feat: ingest independent schools in Ofsted tap and dbt staging
PR Checks / Frontend Typecheck + Tests (pull_request) Successful in 1m6s
PR Checks / Backend Smoke (pull_request) Successful in 8s
PR Checks / Build Backend (no push) (pull_request) Successful in 10s
PR Checks / Build Frontend (no push) (pull_request) Successful in 50s
PR Checks / Build Pipeline (no push) (pull_request) Successful in 47s
PR Checks / AI Code Review (Claude) (pull_request) Failing after 10s
2026-07-15 23:21:40 +01:00
tudor 6f62c25f47 Merge pull request 'Pass phase state to compare sub-components to prevent phase metrics override by multi-phase schools' (#46) from fix/compare-expert-fixes into main
Stage (build -> staging -> E2E gate) / Build Backend (FastAPI) (push) Successful in 14s
Stage (build -> staging -> E2E gate) / Build Frontend (Next.js) (push) Successful in 51s
Stage (build -> staging -> E2E gate) / Build Pipeline (Meltano + dbt + Airflow) (push) Successful in 13s
Stage (build -> staging -> E2E gate) / Deploy to Staging (push) Successful in 1s
Stage (build -> staging -> E2E gate) / E2E Journeys against Staging (push) Failing after 49s
Reviewed-on: #46
2026-07-15 16:38:49 +00:00
Tudor e74d3882ce Pass phase state to compare sub-components to prevent phase metrics override by multi-phase schools
PR Checks / Frontend Typecheck + Tests (pull_request) Successful in 1m5s
PR Checks / Backend Smoke (pull_request) Successful in 7s
PR Checks / Build Backend (no push) (pull_request) Successful in 11s
PR Checks / Build Frontend (no push) (pull_request) Successful in 47s
PR Checks / Build Pipeline (no push) (pull_request) Successful in 10s
PR Checks / AI Code Review (Claude) (pull_request) Successful in 9s
2026-07-15 17:38:02 +01:00
tudor 3fb3db1cc4 Merge pull request 'Fix Ofsted transitional inspections, phase tab exclusions, and FSM benchmark comparison' (#45) from fix/compare-expert-fixes into main
Stage (build -> staging -> E2E gate) / Build Backend (FastAPI) (push) Successful in 22s
Stage (build -> staging -> E2E gate) / Build Frontend (Next.js) (push) Successful in 54s
Stage (build -> staging -> E2E gate) / Build Pipeline (Meltano + dbt + Airflow) (push) Successful in 14s
Stage (build -> staging -> E2E gate) / Deploy to Staging (push) Successful in 1s
Stage (build -> staging -> E2E gate) / E2E Journeys against Staging (push) Failing after 42s
Reviewed-on: #45
2026-07-15 16:30:00 +00:00
Tudor b4b0249a06 Fix Ofsted transitional inspections, phase tab exclusions, and FSM benchmark comparison
PR Checks / Frontend Typecheck + Tests (pull_request) Successful in 1m9s
PR Checks / Backend Smoke (pull_request) Successful in 8s
PR Checks / Build Backend (no push) (pull_request) Successful in 19s
PR Checks / Build Frontend (no push) (pull_request) Successful in 47s
PR Checks / Build Pipeline (no push) (pull_request) Successful in 10s
PR Checks / AI Code Review (Claude) (pull_request) Failing after 10s
2026-07-15 17:23:40 +01:00
tudor e39aef2935 Merge pull request 'fix(compare): sticky school bar hidden behind the site header' (#44) from fix/schoolbar-sticky-offset into main
Stage (build -> staging -> E2E gate) / Build Backend (FastAPI) (push) Successful in 14s
Stage (build -> staging -> E2E gate) / Build Frontend (Next.js) (push) Successful in 51s
Stage (build -> staging -> E2E gate) / Build Pipeline (Meltano + dbt + Airflow) (push) Successful in 13s
Stage (build -> staging -> E2E gate) / Deploy to Staging (push) Successful in 1s
Stage (build -> staging -> E2E gate) / E2E Journeys against Staging (push) Successful in 39s
Reviewed-on: #44
2026-07-15 12:02:29 +00:00
TudorandClaude Fable 5 fef83b3bf2 fix(compare): sticky school bar hidden behind the site header
PR Checks / Frontend Typecheck + Tests (pull_request) Successful in 1m7s
PR Checks / Backend Smoke (pull_request) Successful in 6s
PR Checks / Build Backend (no push) (pull_request) Successful in 10s
PR Checks / Build Frontend (no push) (pull_request) Successful in 47s
PR Checks / Build Pipeline (no push) (pull_request) Successful in 10s
PR Checks / AI Code Review (Claude) (pull_request) Successful in 2m3s
The Navigation header is position:sticky top:0 (z-index 1000). The school
bar was also sticky top:0 (z-index 10), so when scrolled it pinned at the
same top:0 *behind* the header — on mobile 57 of its 72px were covered,
leaving only a sliver, so you couldn't see which schools were selected.

Offset the bar's sticky top to the header height (65px desktop, 57px
mobile — the Navigation breakpoint is also 640px) so it pins just below.

e2e guard asserts the bar's sticky offset is at least the header height
(verified it fails against the pre-fix build).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146VHeLAWjDVE2B5uU67jCB
2026-07-15 12:32:36 +01:00
tudor cf458fe05c Merge pull request 'School name cutoff fix' (#43) from fix/compare-mockup-drift into main
Stage (build -> staging -> E2E gate) / Build Backend (FastAPI) (push) Successful in 13s
Stage (build -> staging -> E2E gate) / Build Frontend (Next.js) (push) Successful in 50s
Stage (build -> staging -> E2E gate) / Build Pipeline (Meltano + dbt + Airflow) (push) Successful in 13s
Stage (build -> staging -> E2E gate) / Deploy to Staging (push) Successful in 1s
Stage (build -> staging -> E2E gate) / E2E Journeys against Staging (push) Successful in 44s
Reviewed-on: #43
2026-07-15 08:01:47 +00:00
Tudor f579630fab School name cutoff fix
PR Checks / Frontend Typecheck + Tests (pull_request) Successful in 1m2s
PR Checks / Backend Smoke (pull_request) Successful in 6s
PR Checks / Build Backend (no push) (pull_request) Successful in 10s
PR Checks / Build Frontend (no push) (pull_request) Successful in 46s
PR Checks / Build Pipeline (no push) (pull_request) Successful in 9s
PR Checks / AI Code Review (Claude) (pull_request) Failing after 9s
2026-07-15 08:57:21 +01:00
tudor 19b41b6999 Merge pull request 'fix(compare): match mockups — remove trends table + mobile measure-first cards' (#42) from fix/compare-mockup-drift into main
Stage (build -> staging -> E2E gate) / Build Backend (FastAPI) (push) Successful in 13s
Stage (build -> staging -> E2E gate) / Build Frontend (Next.js) (push) Successful in 48s
Stage (build -> staging -> E2E gate) / Build Pipeline (Meltano + dbt + Airflow) (push) Successful in 12s
Stage (build -> staging -> E2E gate) / Deploy to Staging (push) Successful in 1s
Stage (build -> staging -> E2E gate) / E2E Journeys against Staging (push) Successful in 37s
Reviewed-on: #42
2026-07-15 07:17:08 +00:00
TudorandClaude Fable 5 66bc5523f6 fix(compare): mobile measure-first cards to match the mockup
PR Checks / Frontend Typecheck + Tests (pull_request) Successful in 1m2s
PR Checks / Backend Smoke (pull_request) Successful in 7s
PR Checks / Build Backend (no push) (pull_request) Successful in 11s
PR Checks / Build Frontend (no push) (pull_request) Successful in 41s
PR Checks / Build Pipeline (no push) (pull_request) Successful in 10s
PR Checks / AI Code Review (Claude) (pull_request) Successful in 2m57s
The grid sections (At a glance, Ofsted, Getting a place, Who goes there)
collapsed generically on mobile — grey label pills, full names wrapping
to 3 lines, no dots — making the page ~2x the mockup's height and
'significantly different' from the mobile design.

Each measure is now wrapped in a <Measure> that is display:contents on
desktop (so the label + cells still flow into the shared aligned grid,
unchanged) and a white card on mobile with compact [dot][short name]
[value] rows — matching the mobile mockup. The sticky school bar becomes
scrollable short-name pills on mobile too. Adds a shortName() util.

Desktop layout is unchanged (display:contents dissolves the wrapper).
Validated the card mechanism and real content shapes (report-card cell,
badges, %+chip rows) via static previews at both widths.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146VHeLAWjDVE2B5uU67jCB
2026-07-15 07:50:01 +01:00
TudorandClaude Fable 5 3cb72d0a0f fix(compare): remove the year-by-year data table from Explore trends
PR Checks / Frontend Typecheck + Tests (pull_request) Successful in 1m2s
PR Checks / Backend Smoke (pull_request) Successful in 7s
PR Checks / Build Backend (no push) (pull_request) Successful in 12s
PR Checks / Build Frontend (no push) (pull_request) Successful in 41s
PR Checks / Build Pipeline (no push) (pull_request) Successful in 9s
PR Checks / AI Code Review (Claude) (pull_request) Successful in 39s
The mockup's Explore trends section is the measure picker + chart only —
no data table. Removes the table (and the now-unused progressBand / band
chip / formatMetricValue plumbing that only fed it).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146VHeLAWjDVE2B5uU67jCB
2026-07-15 07:29:24 +01:00
tudor b89fa47ec5 Merge pull request 'fix(compare): trends chart was squashed to ~150px — give it a real height' (#41) from fix/trends-chart-height into main
Stage (build -> staging -> E2E gate) / Build Backend (FastAPI) (push) Successful in 13s
Stage (build -> staging -> E2E gate) / Build Frontend (Next.js) (push) Successful in 52s
Stage (build -> staging -> E2E gate) / Build Pipeline (Meltano + dbt + Airflow) (push) Successful in 13s
Stage (build -> staging -> E2E gate) / Deploy to Staging (push) Successful in 1s
Stage (build -> staging -> E2E gate) / E2E Journeys against Staging (push) Successful in 38s
Reviewed-on: #41
2026-07-14 22:51:04 +00:00
tudor 0c7ad0f309 Merge pull request 'test(e2e): fix flaky compare journey — select two same-phase schools' (#40) from fix/e2e-compare-samephase into main
Stage (build -> staging -> E2E gate) / Build Backend (FastAPI) (push) Successful in 13s
Stage (build -> staging -> E2E gate) / Build Frontend (Next.js) (push) Successful in 54s
Stage (build -> staging -> E2E gate) / Build Pipeline (Meltano + dbt + Airflow) (push) Successful in 12s
Stage (build -> staging -> E2E gate) / Deploy to Staging (push) Successful in 1s
Stage (build -> staging -> E2E gate) / E2E Journeys against Staging (push) Successful in 37s
Reviewed-on: #40
2026-07-14 22:30:18 +00:00
TudorandClaude Fable 5 e4565e9f15 fix(compare): give the trends chart a real height (was squashed to ~150px)
PR Checks / Frontend Typecheck + Tests (pull_request) Successful in 9m37s
PR Checks / Backend Smoke (pull_request) Successful in 7s
PR Checks / Build Backend (no push) (pull_request) Successful in 16s
PR Checks / Build Frontend (no push) (pull_request) Successful in 45s
PR Checks / Build Pipeline (no push) (pull_request) Successful in 10s
PR Checks / AI Code Review (Claude) (pull_request) Successful in 17s
ComparisonChart runs Chart.js with maintainAspectRatio:false, so it sizes
to its container's height — which must be definite. TrendsExplorer gave
.chartBox a min-height, which doesn't resolve the chart wrapper's
height:100%, so Chart.js fell back to its ~150px default: a squashed
8.6:1 sliver that didn't match the mockups. Set a definite height (420px
desktop, 360px mobile where the chips row sits above the canvas).

Verified on staging by patching the live height: canvas went from
1287x150 to 1287x392 (desktop) / 284 (mobile) — proper ~3:1 proportions
matching the mockup, with the England dashed line, COVID/2021-22 gap and
table all reading correctly.

An e2e guard asserts the trends canvas is taller than 220px so the
squash can't regress.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146VHeLAWjDVE2B5uU67jCB
2026-07-14 23:24:39 +01:00
TudorandClaude Fable 5 06e4898c30 test(e2e): pick two same-phase schools for the compare journey
PR Checks / Frontend Typecheck + Tests (pull_request) Successful in 9m41s
PR Checks / Backend Smoke (pull_request) Successful in 7s
PR Checks / Build Backend (no push) (pull_request) Successful in 16s
PR Checks / Build Frontend (no push) (pull_request) Successful in 45s
PR Checks / Build Pipeline (no push) (pull_request) Successful in 10s
PR Checks / AI Code Review (Claude) (pull_request) Successful in 47s
The compare page's phase tabs put all-through schools (which carry KS4
data) on the secondary tab, so comparing an all-through school with a
pure primary splits them across tabs and only the active tab renders its
link. The test picked the first two 'primary' search hits without
guaranteeing same phase, so it flaked whenever a search returned an
all-through school first (e.g. URN 137306). Now selects two pure-Primary
URNs via the API — deterministic and data-invariant.

Verified against staging: was a 15.6s timeout, now passes in ~1.8s.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146VHeLAWjDVE2B5uU67jCB
2026-07-14 23:12:23 +01:00
tudor a9611e21c3 Merge pull request 'perf(api): batch supplementary queries — /api/compare stops scaling per school' (#38) from perf/batch-supplementary into main
Stage (build -> staging -> E2E gate) / Build Backend (FastAPI) (push) Successful in 20s
Stage (build -> staging -> E2E gate) / Build Frontend (Next.js) (push) Successful in 51s
Stage (build -> staging -> E2E gate) / Build Pipeline (Meltano + dbt + Airflow) (push) Successful in 13s
Stage (build -> staging -> E2E gate) / Deploy to Staging (push) Successful in 1s
Stage (build -> staging -> E2E gate) / E2E Journeys against Staging (push) Failing after 1m8s
Reviewed-on: #38
2026-07-14 21:57:27 +00:00
tudor 0696518995 Merge pull request 'ci: speed up Frontend Typecheck + Tests (cache node_modules; cancel superseded runs)' (#39) from ci/frontend-checks-speedup into main
Stage (build -> staging -> E2E gate) / Build Backend (FastAPI) (push) Successful in 13s
Stage (build -> staging -> E2E gate) / Build Frontend (Next.js) (push) Successful in 53s
Stage (build -> staging -> E2E gate) / Build Pipeline (Meltano + dbt + Airflow) (push) Successful in 13s
Stage (build -> staging -> E2E gate) / Deploy to Staging (push) Successful in 1s
Stage (build -> staging -> E2E gate) / E2E Journeys against Staging (push) Failing after 1m10s
Reviewed-on: #39
2026-07-14 21:57:08 +00:00
TudorandClaude Fable 5 090d5f7bec ci: speed up Frontend Typecheck + Tests; cancel superseded PR runs
PR Checks / Frontend Typecheck + Tests (pull_request) Successful in 1m34s
PR Checks / Backend Smoke (pull_request) Successful in 7s
PR Checks / Build Backend (no push) (pull_request) Successful in 10s
PR Checks / Build Frontend (no push) (pull_request) Successful in 46s
PR Checks / Build Pipeline (no push) (pull_request) Successful in 9s
PR Checks / AI Code Review (Claude) (pull_request) Successful in 9s
The job does ~3s of real work (typecheck 1.4s + jest 1.3s) but installs
452 MB / 460 packages every run. Two changes:

- Cache nextjs-app/node_modules keyed on the lockfile hash (OS + node
  major pinned) and skip npm ci entirely on a hit — deps change rarely,
  so most PR pushes now do zero install. On miss, npm ci runs with
  --prefer-offline --no-audit --no-fund.
- Workflow-level concurrency with cancel-in-progress: a new commit (or an
  empty re-trigger) aborts the previous run instead of stacking a second
  full matrix.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146VHeLAWjDVE2B5uU67jCB
2026-07-14 22:54:32 +01:00
tudor abc03a0dd3 Merge pull request 'fix(compare): blank page on refresh + remove dead per-page comparison fetch' (#37) from fix/compare-refresh-and-fetch into main
Stage (build -> staging -> E2E gate) / Build Backend (FastAPI) (push) Successful in 13s
Stage (build -> staging -> E2E gate) / Build Frontend (Next.js) (push) Successful in 53s
Stage (build -> staging -> E2E gate) / Build Pipeline (Meltano + dbt + Airflow) (push) Successful in 13s
Stage (build -> staging -> E2E gate) / Deploy to Staging (push) Successful in 1s
Stage (build -> staging -> E2E gate) / E2E Journeys against Staging (push) Failing after 1m9s
Reviewed-on: #37
2026-07-14 21:39:46 +00:00
TudorandClaude Fable 5 43a2c4a6bc fix(compare): show SSR data on refresh; drop dead per-page comparison fetch
PR Checks / Frontend Typecheck + Tests (pull_request) Successful in 9m42s
PR Checks / Backend Smoke (pull_request) Successful in 7s
PR Checks / Build Backend (no push) (pull_request) Successful in 10s
PR Checks / Build Frontend (no push) (pull_request) Successful in 49s
PR Checks / Build Pipeline (no push) (pull_request) Successful in 10s
PR Checks / AI Code Review (Claude) (pull_request) Successful in 9s
Refresh bug: on mount the basket is empty for a beat before it hydrates
from the URL. The fetch effect nulled comparisonData on that transient
empty urnKey, then the one-shot 'SSR covers it' skip suppressed the
refetch — leaving the page blank on reload. The effect is now gated on
isInitialized, never blanks on empty (the render already shows the empty
state when nothing is selected), and decides fetch-vs-skip by whether it
already holds each requested school's data (SSR or a prior fetch).

Perf: useComparison ran a useSWR('/api/compare') whose result nothing
consumed — dead weight that fired on every page (Navigation + Toast are
global) whenever the basket was non-empty, and duplicated ComparisonView's
own fetch on the compare page. Removed; the hook now exposes basket state
only.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146VHeLAWjDVE2B5uU67jCB
2026-07-14 22:34:20 +01:00
Tudor 493ea39c29 Updating the number of schools
PR Checks / Frontend Typecheck + Tests (pull_request) Successful in 9m44s
PR Checks / Backend Smoke (pull_request) Successful in 7s
PR Checks / Build Backend (no push) (pull_request) Successful in 16s
PR Checks / Build Frontend (no push) (pull_request) Successful in 49s
PR Checks / Build Pipeline (no push) (pull_request) Successful in 38s
PR Checks / AI Code Review (Claude) (pull_request) Failing after 27s
2026-07-09 22:53:42 +01:00
28 changed files with 635 additions and 297 deletions
+20 -3
View File
@@ -5,6 +5,13 @@ on:
branches:
- main
# Cancel superseded runs: pushing a new commit to a PR (or an empty
# re-trigger) aborts the previous still-running checks instead of running
# a second full matrix alongside them.
concurrency:
group: pr-checks-${{ gitea.event.pull_request.number }}
cancel-in-progress: true
env:
REGISTRY: privaterepo.sitaru.org
BACKEND_IMAGE_NAME: ${{ gitea.repository }}-backend
@@ -23,12 +30,22 @@ jobs:
uses: actions/setup-node@v4
with:
node-version: 22
cache: npm
cache-dependency-path: nextjs-app/package-lock.json
# Cache the resolved node_modules (452 MB / 460 packages) keyed on the
# lockfile. On a hit — the common case, since deps change rarely — the
# whole `npm ci` step is skipped, not just its download phase. The key
# pins OS + node major so we never restore incompatible native binaries.
- name: Cache node_modules
id: node-modules-cache
uses: actions/cache@v4
with:
path: nextjs-app/node_modules
key: nextjs-node-modules-${{ runner.os }}-node22-${{ hashFiles('nextjs-app/package-lock.json') }}
- name: Install dependencies
if: steps.node-modules-cache.outputs.cache-hit != 'true'
working-directory: nextjs-app
run: npm ci
run: npm ci --prefer-offline --no-audit --no-fund
- name: Typecheck
working-directory: nextjs-app
+1
View File
@@ -577,6 +577,7 @@ def compute_benchmarks(df: pd.DataFrame) -> dict:
"eal_pct": _median(sub, "eal_pct"),
"sen_support_pct": _median(sub, "sen_support_pct"),
"disadvantaged_pct": _median(sub, "disadvantaged_pct"),
"fsm_pct": _median(sub, "fsm_pct"),
"median_pupils": median_pupils,
}
if with_disadvantaged:
+12 -9
View File
@@ -17,33 +17,33 @@ def _df():
# weighted = (40*100 + 60*300) / 400 = 55.0 ; unweighted mean = 50.0
dict(year=LATEST, attainment_8_score=np.nan, eligible_pupils=100,
rwm_expected_disadvantaged_pct=40.0, eal_pct=10.0,
sen_support_pct=10.0, disadvantaged_pct=20.0, total_pupils=200),
sen_support_pct=10.0, disadvantaged_pct=20.0, fsm_pct=15.0, total_pupils=200),
dict(year=LATEST, attainment_8_score=np.nan, eligible_pupils=300,
rwm_expected_disadvantaged_pct=60.0, eal_pct=20.0,
sen_support_pct=14.0, disadvantaged_pct=24.0, total_pupils=280),
sen_support_pct=14.0, disadvantaged_pct=24.0, fsm_pct=17.0, total_pupils=280),
dict(year=LATEST, attainment_8_score=np.nan, eligible_pupils=np.nan,
rwm_expected_disadvantaged_pct=99.0, eal_pct=30.0,
sen_support_pct=18.0, disadvantaged_pct=30.0, total_pupils=300),
sen_support_pct=18.0, disadvantaged_pct=30.0, fsm_pct=19.0, total_pupils=300),
dict(year=LATEST, attainment_8_score=np.nan, eligible_pupils=50,
rwm_expected_disadvantaged_pct=np.nan, eal_pct=np.nan,
sen_support_pct=np.nan, disadvantaged_pct=np.nan, total_pupils=np.nan),
sen_support_pct=np.nan, disadvantaged_pct=np.nan, fsm_pct=np.nan, total_pupils=np.nan),
dict(year=LATEST, attainment_8_score=np.nan, eligible_pupils=40,
rwm_expected_disadvantaged_pct=np.nan, eal_pct=40.0,
sen_support_pct=20.0, disadvantaged_pct=40.0, total_pupils=350),
sen_support_pct=20.0, disadvantaged_pct=40.0, fsm_pct=21.0, total_pupils=350),
dict(year=LATEST, attainment_8_score=np.nan, eligible_pupils=60,
rwm_expected_disadvantaged_pct=np.nan, eal_pct=50.0,
sen_support_pct=22.0, disadvantaged_pct=44.0, total_pupils=400),
sen_support_pct=22.0, disadvantaged_pct=44.0, fsm_pct=23.0, total_pupils=400),
# Two secondary schools (attainment_8 non-null)
dict(year=LATEST, attainment_8_score=45.0, eligible_pupils=180,
rwm_expected_disadvantaged_pct=np.nan, eal_pct=15.0,
sen_support_pct=12.0, disadvantaged_pct=22.0, total_pupils=1000),
sen_support_pct=12.0, disadvantaged_pct=22.0, fsm_pct=12.0, total_pupils=1000),
dict(year=LATEST, attainment_8_score=50.0, eligible_pupils=200,
rwm_expected_disadvantaged_pct=np.nan, eal_pct=25.0,
sen_support_pct=16.0, disadvantaged_pct=26.0, total_pupils=1200),
sen_support_pct=16.0, disadvantaged_pct=26.0, fsm_pct=14.0, total_pupils=1200),
# An older-year primary row that must NOT influence anything
dict(year=202324, attainment_8_score=np.nan, eligible_pupils=500,
rwm_expected_disadvantaged_pct=1.0, eal_pct=99.0,
sen_support_pct=99.0, disadvantaged_pct=99.0, total_pupils=9999),
sen_support_pct=99.0, disadvantaged_pct=99.0, fsm_pct=99.0, total_pupils=9999),
]
return pd.DataFrame(rows)
@@ -59,6 +59,8 @@ def test_medians_ignore_nan_and_older_years():
assert b["year"] == LATEST
# eal medians over [10,20,30,40,50] = 30
assert b["primary"]["eal_pct"] == 30.0
# fsm medians over [15,17,19,21,23] = 19
assert b["primary"]["fsm_pct"] == 19.0
# median pupils over [200,280,300,350,400] = 300
assert b["primary"]["median_pupils"] == 300
@@ -66,6 +68,7 @@ def test_medians_ignore_nan_and_older_years():
def test_secondary_block_has_no_disadvantaged_rwm():
b = compute_benchmarks(_df())
assert "disadvantaged_rwm_expected_pct" not in b["secondary"]
assert b["secondary"]["fsm_pct"] == 13.0
assert b["secondary"]["median_pupils"] == 1100
@@ -89,7 +89,7 @@ These are strengths the fixes below must not regress:
- Uplift: **home 46% exit rate — decrease, moderate** and **share of sessions reaching a school page — increase, moderate** (assists the majority entry path at its first interaction).
- **P1.7 — The mobile hero omits the value proposition entirely** *(J1-F6)*
- Evidence: desktop shows the "UPDATED WITH 2026/2027 ADMISSIONS RESULTS" trust badge and the "24,000+ schools… side by side, in one place" subheading; mobile renders only the poetic H1 ("Every school in England, *compared.*") and a bare search box (`j1-home-desktop-fold.png` vs `j1-home-mobile-fold.png`).
- Evidence: desktop shows the "UPDATED WITH 2026/2027 ADMISSIONS RESULTS" trust badge and the "27,000+ schools… side by side, in one place" subheading; mobile renders only the poetic H1 ("Every school in England, *compared.*") and a bare search box (`j1-home-desktop-fold.png` vs `j1-home-mobile-fold.png`).
- Criterion: mobile content parity; Nielsen #1 — first-visit orientation ("what is this, why trust it") absent on the primary viewport.
- Argument: 63% of entries land here and 56% of traffic is mobile; a first-time visitor gets no statement of coverage, data source, or freshness above the fold. Weak value proposition at first glance is a classic bounce driver and plausibly a material slice of the 46% exit rate.
- Recommendation: restore a compact version of the badge + one-line value prop under the mobile H1 (one text block; the fold has room above the deadline rail).
+45 -12
View File
@@ -19,6 +19,27 @@ function schoolLinks(page: Page) {
return page.locator('a[href^="/school/"]');
}
/**
* Two URNs guaranteed to be pure-primary (same phase). The compare page's
* phase tabs split all-through schools (which carry KS4 data) onto the
* secondary tab, so picking two arbitrary "primary" search hits can land
* them on different tabs where only the active one renders. Selecting via
* the API by exact phase keeps both on the same tab. Data-invariant: uses
* whatever primaries the environment holds.
*/
async function twoPrimaryUrns(page: Page): Promise<[string, string]> {
const res = await page.request.get('/api/schools?search=primary&per_page=50');
expect(res.ok()).toBeTruthy();
const body = await res.json();
const urns: string[] = (body.schools ?? [])
.filter((s: { phase?: string; rwm_expected_pct?: number | null }) =>
s.phase === 'Primary' && s.rwm_expected_pct != null,
)
.map((s: { urn: number }) => String(s.urn));
expect(urns.length).toBeGreaterThanOrEqual(2);
return [urns[0], urns[1]];
}
test('home page loads with hero search', async ({ page }) => {
await page.goto('/');
await expect(page.locator('h1').first()).toBeVisible();
@@ -139,19 +160,13 @@ test('results map fullscreen falls back to an overlay on iOS', async ({ page })
});
test('comparing two schools shows the parent-first sections side by side', async ({ page }) => {
// Collect two school URNs from search results, then load the share URL
await searchByName(page, 'primary');
await expect(schoolLinks(page).first()).toBeVisible({ timeout: 15_000 });
const hrefs = await schoolLinks(page).evaluateAll((links) =>
links.map((l) => (l as HTMLAnchorElement).getAttribute('href') || '')
);
const urns = [...new Set(hrefs.map((h) => h.match(/\/school\/(\d+)/)?.[1]).filter(Boolean))];
expect(urns.length).toBeGreaterThanOrEqual(2);
// Two same-phase (pure primary) schools so both stay on one tab.
const [urn0, urn1] = await twoPrimaryUrns(page);
await page.goto(`/compare?urns=${urns[0]},${urns[1]}`);
await page.goto(`/compare?urns=${urn0},${urn1}`);
// Both schools' detail links should render in the comparison view
await expect(page.locator(`a[href*="${urns[0]}"]`).first()).toBeVisible({ timeout: 15_000 });
await expect(page.locator(`a[href*="${urns[1]}"]`).first()).toBeVisible();
await expect(page.locator(`a[href*="${urn0}"]`).first()).toBeVisible({ timeout: 15_000 });
await expect(page.locator(`a[href*="${urn1}"]`).first()).toBeVisible();
// The parent-first sections render in order (data-invariant: headings only)
for (const heading of [
@@ -212,8 +227,26 @@ test('compare chart on mobile shows school chips with tap-to-focus', async ({ pa
);
expect(bodyOverflowsX).toBe(false);
// The sticky school bar must pin *below* the sticky site header, not at
// top:0 where the header covers it and the selected schools are hidden.
// Assert the sticky offset directly (robust — no scroll timing needed).
const barTop = await page
.locator('[class*="schoolBar"]')
.first()
.evaluate((el) => parseFloat(getComputedStyle(el).top));
const headerHeight = await page
.locator('[class*="header"]')
.first()
.evaluate((el) => el.getBoundingClientRect().height);
expect(barTop).toBeGreaterThanOrEqual(headerHeight - 1);
// The trends chart still renders (inside the Explore trends section)…
await expect(page.locator('canvas:visible').first()).toBeVisible({ timeout: 15_000 });
const chartCanvas = page.locator('canvas:visible').first();
await expect(chartCanvas).toBeVisible({ timeout: 15_000 });
// …at a real height, not the squashed ~150px Chart.js fallback that
// appears when the container lacks a definite height.
const chartBox = await chartCanvas.boundingBox();
expect(chartBox && chartBox.height).toBeGreaterThan(220);
// …with the mobile chart legend chips and tap-to-focus behaviour intact.
const chipGroup = page.getByRole('group', { name: /highlight a school/i });
@@ -95,4 +95,12 @@ describe('CompareOfsted', () => {
expect(links).toHaveLength(3);
expect(links[0]).toHaveAttribute('href', 'https://reports.ofsted.gov.uk/provider/21/1');
});
it('renders a per-measure mobile tag with the short school name', () => {
render(<CompareOfsted schools={schools} data={data} />);
// Each measure repeats the schools, so the short name ("Graded" from
// "Graded School") appears once per measure (4) via the cell tag.
expect(screen.getAllByText('Graded').length).toBe(4);
expect(screen.getAllByText('Card').length).toBe(4);
});
});
@@ -0,0 +1,82 @@
/**
* Regression: on refresh, the compare page must show the SSR-rendered data.
*
* The basket hydrates from the URL a beat after mount (selectedSchools is
* empty for the first render), so the fetch effect must not blank the
* SSR payload during that window — and must not refetch data the server
* already provided.
*/
import { render, screen, waitFor } from '@testing-library/react';
import { ComparisonView } from '@/components/ComparisonView';
import { ComparisonProvider } from '@/context/ComparisonProvider';
import type { ComparisonData, School } from '@/lib/types';
const fetchComparison = jest.fn();
jest.mock('@/lib/api', () => ({
fetchComparison: (...args: unknown[]) => fetchComparison(...args),
}));
jest.mock('@/lib/analytics', () => ({ track: jest.fn() }));
function school(urn: number, name: string): School {
return {
urn,
school_name: name,
local_authority: 'Testshire',
school_type: 'Community school',
rwm_expected_pct: 80,
phase: 'Primary',
} as School;
}
function data(urn: number, name: string): ComparisonData {
return {
school_info: school(urn, name),
yearly_data: [{ year: 202425, rwm_expected_pct: 80 }] as ComparisonData['yearly_data'],
ofsted: null,
census: null,
admissions: null,
admissions_history: [],
deprivation: null,
};
}
const INITIAL_DATA = {
'100': data(100, 'Alpha Primary'),
'200': data(200, 'Beta Primary'),
};
beforeEach(() => {
fetchComparison.mockReset();
});
test('renders SSR data on refresh without wiping it or refetching', async () => {
render(
<ComparisonProvider>
<ComparisonView
initialData={INITIAL_DATA}
initialNationalAverages={{
year: 202425,
primary: { rwm_expected_pct: 62 },
secondary: {},
by_year: [],
}}
initialBenchmarks={undefined}
initialUrns={[100, 200]}
metrics={[]}
selectedMetric="rwm_expected_pct"
/>
</ComparisonProvider>,
);
// Both SSR-provided schools appear (data was not blanked during hydration)
await waitFor(() => {
expect(screen.getAllByText('Alpha Primary').length).toBeGreaterThan(0);
});
expect(screen.getAllByText('Beta Primary').length).toBeGreaterThan(0);
expect(screen.getByRole('heading', { name: 'At a glance' })).toBeInTheDocument();
// …and the client never refetched data the server already rendered.
expect(fetchComparison).not.toHaveBeenCalled();
});
@@ -126,6 +126,13 @@ describe('ofstedDisplay', () => {
expect(ofstedDisplay(ofsted({})).kind).toBe('none');
});
it('identifies transitional inspections without overall grades', () => {
const transitional = ofstedDisplay(
ofsted({ overall_effectiveness: null, inspection_date: '2024-11-05' }),
);
expect(transitional.kind).toBe('transitional');
});
it('uses the four legacy grade words', () => {
expect(OFSTED_LEGACY_GRADES).toEqual({
1: 'Outstanding',
+15
View File
@@ -10,6 +10,7 @@ import {
debounce,
buildOfstedListBadge,
metricKind,
shortName,
computeYBounds,
} from '@/lib/utils';
@@ -223,3 +224,17 @@ describe('isProposedToClose', () => {
expect(isProposedToClose({})).toBe(false);
});
});
describe('shortName', () => {
it('drops the trailing establishment-type words', () => {
expect(shortName('Barclay Primary School')).toBe('Barclay');
expect(shortName('Elmhurst Primary School')).toBe('Elmhurst');
expect(shortName("St Mary's Catholic Primary School")).toBe("St Mary's");
expect(shortName('Riverside Community Junior School')).toBe('Riverside');
});
it('keeps a name that carries no type suffix, capping very long ones', () => {
expect(shortName('Beaver Road')).toBe('Beaver Road');
expect(shortName('A'.repeat(30), 10)).toBe('AAAAAAAAA…');
});
});
@@ -80,10 +80,12 @@
}
/* Sticky school bar — column identity while scrolling; horizontal scroll on
narrow screens */
narrow screens. Offset by the sticky site header's height (Navigation is
position: sticky, top: 0) so this bar pins just below it instead of
sliding underneath and being hidden. Header ≈ 65px desktop / 57px mobile. */
.schoolBar {
position: sticky;
top: 0;
top: 65px;
z-index: 10;
background: var(--bg-primary, #faf7f2);
display: flex;
@@ -132,6 +134,11 @@
color: var(--accent-coral-dark, #b04a2e);
}
/* Full name on desktop, short name on the compact mobile pills. */
.chipNameShort {
display: none;
}
.chipMeta {
display: block;
font-size: 0.78rem;
@@ -163,3 +170,45 @@
padding-top: 1rem;
max-width: 75ch;
}
/* Mobile: the sticky school bar becomes compact, horizontally-scrollable
pills with short names (matching the mobile mockup) instead of full-width
cards whose names wrap to several lines. */
@media (max-width: 640px) {
/* The mobile Navigation header is shorter (≈57px). */
.schoolBar {
top: 57px;
}
.schoolChip {
flex: 0 0 auto;
min-width: 0;
border-top-width: 2px;
border-radius: 999px;
padding: 0.35rem 0.7rem;
box-shadow: none;
}
.chipName {
font-size: 0.85rem;
white-space: nowrap;
}
.chipNameFull {
display: none;
}
.chipNameShort {
display: inline;
}
.chipMeta {
display: none;
}
.chipRemove {
width: 18px;
height: 18px;
font-size: 0.75rem;
}
}
+41 -32
View File
@@ -28,7 +28,7 @@ import type {
NationalAverages,
School,
} from '@/lib/types';
import { CHART_COLORS, schoolUrl } from '@/lib/utils';
import { CHART_COLORS, schoolUrl, shortName } from '@/lib/utils';
import { fetchComparison } from '@/lib/api';
import { track } from '@/lib/analytics';
import styles from './ComparisonView.module.css';
@@ -107,24 +107,26 @@ export function ComparisonView({
router.replace(newUrl, { scroll: false });
}, [urnKey, selectedMetric, pathname, searchParams, router]);
// Fetch only when the school set changes. The very first run is skipped
// when the SSR payload already covers the current set — no double-fetch
// of data the server just rendered.
const firstFetchRef = useRef(true);
useEffect(() => {
if (!urnKey) {
setComparisonData(null);
setNationalAverages(undefined);
setBenchmarks(undefined);
return;
}
// Fetch when the school set changes, but only for schools we don't already
// have data for. This skips the refetch of SSR-rendered data on load AND
// avoids a network call when a school is merely removed. A ref holds the
// latest data so the effect can read it without re-running on every fetch.
//
// Correctness note: we must NOT null the data on a transient empty urnKey.
// On mount the basket is empty for a beat before it hydrates from the URL,
// and blanking here (then skipping the refetch because SSR "covers" the set)
// was leaving the page empty on refresh. The render already shows the empty
// state whenever `selectedSchools` is empty, so stale data for deselected
// schools is harmless — it's simply unused.
const comparisonDataRef = useRef(comparisonData);
comparisonDataRef.current = comparisonData;
if (firstFetchRef.current) {
firstFetchRef.current = false;
const ssrUrns = new Set(Object.keys(initialData ?? {}));
const covered = urnKey.split(',').every((urn) => ssrUrns.has(urn));
if (covered && ssrUrns.size > 0) return;
}
useEffect(() => {
if (!isInitialized || !urnKey) return;
const have = comparisonDataRef.current ?? {};
const covered = urnKey.split(',').every((urn) => have[urn] != null);
if (covered) return;
fetchComparison(urnKey, { cache: 'no-store' })
.then((data) => {
@@ -138,22 +140,25 @@ export function ComparisonView({
// destroy a working comparison the user is looking at.
console.error('Failed to fetch comparison:', err);
});
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [urnKey]);
}, [urnKey, isInitialized]);
// Classify schools by phase using comparison data
const classifySchool = (school: School): 'primary' | 'secondary' => {
const primarySchools = selectedSchools.filter((school) => {
const info = comparisonData?.[school.urn]?.school_info;
if (info?.attainment_8_score != null) return 'secondary';
if (info?.rwm_expected_pct != null) return 'primary';
// Fallback: check yearly data
const yearlyData = comparisonData?.[school.urn]?.yearly_data;
if (yearlyData?.some((d) => d.attainment_8_score != null)) return 'secondary';
return 'primary';
};
const hasPrimaryData =
info?.rwm_expected_pct != null ||
comparisonData?.[school.urn]?.yearly_data?.some((d) => d.rwm_expected_pct != null);
if (hasPrimaryData) return true;
return school.phase?.toLowerCase().includes('primary') || false;
});
const primarySchools = selectedSchools.filter((s) => classifySchool(s) === 'primary');
const secondarySchools = selectedSchools.filter((s) => classifySchool(s) === 'secondary');
const secondarySchools = selectedSchools.filter((school) => {
const info = comparisonData?.[school.urn]?.school_info;
const hasSecondaryData =
info?.attainment_8_score != null ||
comparisonData?.[school.urn]?.yearly_data?.some((d) => d.attainment_8_score != null);
if (hasSecondaryData) return true;
return school.phase?.toLowerCase().includes('secondary') || false;
});
// Auto-select tab with more schools and sync the metric to match the phase.
useEffect(() => {
@@ -348,7 +353,8 @@ export function ComparisonView({
/>
<span className={styles.chipText}>
<a className={styles.chipName} href={schoolUrl(school.urn, school.school_name)}>
{school.school_name}
<span className={styles.chipNameFull}>{school.school_name}</span>
<span className={styles.chipNameShort}>{shortName(school.school_name)}</span>
</a>
<span className={styles.chipMeta}>
{[school.local_authority, school.school_type].filter(Boolean).join(' · ')}
@@ -373,6 +379,7 @@ export function ComparisonView({
data={activeComparisonData}
nationalAverages={nationalAverages}
benchmarks={benchmarks}
isSecondary={!isPrimary}
/>
<CompareOfsted schools={activeSchools} data={activeComparisonData} />
<CompareAcademics
@@ -380,12 +387,14 @@ export function ComparisonView({
data={activeComparisonData}
nationalAverages={nationalAverages}
benchmarks={benchmarks}
isSecondary={!isPrimary}
/>
<CompareAdmissions schools={activeSchools} data={activeComparisonData} />
<CompareCommunity
schools={activeSchools}
data={activeComparisonData}
benchmarks={benchmarks}
isSecondary={!isPrimary}
/>
<TrendsExplorer
schools={activeSchools}
+2 -2
View File
@@ -271,10 +271,10 @@ export function HomeView({ initialSchools, filters, totalSchools, howItWorks, ed
freshness, standing in for the hidden eyebrow too) on phones,
where every line above the fold costs. */}
<span className={styles.heroDescriptionFull}>
<strong>24,000+ primary and secondary schools</strong> with Key Stage 2 SATs, GCSE results, Ofsted grades, progress scores and admissions data side by side, in one place.
<strong>27,000+ primary and secondary schools</strong> with Key Stage 2 SATs, GCSE results, Ofsted grades, progress scores and admissions data side by side, in one place.
</span>
<span className={styles.heroDescriptionCompact}>
<strong>24,000+ English schools</strong> SATs, GCSEs, Ofsted &amp; admissions, side by side. Updated for 2026/27.
<strong>27,000+ English schools</strong> SATs, GCSEs, Ofsted &amp; admissions, side by side. Updated for 2026/27.
</span>
</p>
</div>
@@ -125,15 +125,17 @@ export function CompareAcademics({
data,
nationalAverages,
benchmarks,
isSecondary: propIsSecondary,
}: {
schools: School[];
data: Record<string, ComparisonData>;
nationalAverages?: NationalAverages;
benchmarks?: Benchmarks;
isSecondary?: boolean;
}) {
const urns = schools.map((school) => school.urn);
const schoolNames = schools.map((school) => school.school_name);
const isSecondary = schools.some(
const isSecondary = propIsSecondary !== undefined ? propIsSecondary : schools.some(
(school) => data[String(school.urn)]?.school_info?.attainment_8_score != null,
);
@@ -10,7 +10,7 @@
import { summariseAdmissions } from '@/lib/compareLogic';
import type { ComparisonData, School } from '@/lib/types';
import { CHART_COLORS } from '@/lib/utils';
import { Cell, Chip, RowLabel, Section, SectionGrid, sectionStyles as s } from './sectionShared';
import { Cell, Chip, Measure, Section, SectionGrid, sectionStyles as s } from './sectionShared';
export function CompareAdmissions({
schools,
@@ -49,9 +49,10 @@ export function CompareAdmissions({
}
>
<SectionGrid schools={schools}>
<RowLabel tip="How many application forms named the school at any preference rank — not the number of families competing head-to-head for a place.">
Interest in the school
</RowLabel>
<Measure
tip="How many application forms named the school at any preference rank — not the number of families competing head-to-head for a place."
label="Interest in the school"
>
{schools.map((school, i) => {
const a = rows[i];
return (
@@ -68,7 +69,9 @@ export function CompareAdmissions({
);
})}
<RowLabel>First-choice families offered a place</RowLabel>
</Measure>
<Measure label="First-choice families offered a place">
{schools.map((school, i) => {
const summary = summariseAdmissions(rows[i]);
return (
@@ -95,7 +98,9 @@ export function CompareAdmissions({
);
})}
<RowLabel>What this means</RowLabel>
</Measure>
<Measure label="What this means">
{schools.map((school, i) => {
const a = rows[i];
const summary = summariseAdmissions(a);
@@ -118,6 +123,7 @@ export function CompareAdmissions({
</Cell>
);
})}
</Measure>
</SectionGrid>
</Section>
);
@@ -15,7 +15,7 @@ import {
type ReportCardSummary,
} from '@/lib/compareLogic';
import type { Benchmarks, ComparisonData, NationalAverages, School } from '@/lib/types';
import { Cell, Chip, RowLabel, Section, SectionGrid, sectionStyles as s } from './sectionShared';
import { Cell, Chip, Measure, Section, SectionGrid, sectionStyles as s } from './sectionShared';
function ReportCardChips({ summary }: { summary: ReportCardSummary }) {
return (
@@ -47,14 +47,16 @@ export function CompareAtAGlance({
data,
nationalAverages,
benchmarks,
isSecondary: propIsSecondary,
}: {
schools: School[];
data: Record<string, ComparisonData>;
nationalAverages?: NationalAverages;
benchmarks?: Benchmarks;
isSecondary?: boolean;
}) {
const urns = schools.map((school) => school.urn);
const isSecondary = schools.some(
const isSecondary = propIsSecondary !== undefined ? propIsSecondary : schools.some(
(school) => data[String(school.urn)]?.school_info?.attainment_8_score != null,
);
const headlineKey = isSecondary ? 'attainment_8_score' : 'rwm_expected_pct';
@@ -69,7 +71,7 @@ export function CompareAtAGlance({
return (
<Section title="At a glance" how="The short version — each row below is explained in its own section further down.">
<SectionGrid schools={schools}>
<RowLabel>Latest Ofsted inspection</RowLabel>
<Measure label="Latest Ofsted inspection">
{schools.map((school, i) => {
const display = ofstedDisplay(data[String(school.urn)]?.ofsted);
return (
@@ -83,20 +85,28 @@ export function CompareAtAGlance({
{display.carriedForward && <span className={s.small}>Grade carried forward</span>}
</>
)}
{display.kind === 'transitional' && (
<>
<span className={s.badge} style={{ backgroundColor: '#e2e8f0', color: '#475569' }}>
No overall grade
</span>
<span className={s.small}>Sub-judgements only</span>
</>
)}
{display.kind === 'none' && <span className={s.small}>No inspection in our dataset</span>}
</Cell>
);
})}
</Measure>
<RowLabel
<Measure
tip={
isSecondary
? 'Average Attainment 8 score across GCSE subjects (latest year).'
: '% of Year 6 pupils reaching the expected standard in reading, writing and maths (latest year).'
}
label={isSecondary ? 'Attainment 8 score' : 'Children reaching the expected standard'}
>
{isSecondary ? 'Attainment 8 score' : 'Children reaching the expected standard'}
</RowLabel>
{schools.map((school, i) => {
const value = headlineValues[i];
return (
@@ -131,8 +141,9 @@ export function CompareAtAGlance({
</Cell>
);
})}
</Measure>
<RowLabel>Getting a place</RowLabel>
<Measure label="Getting a place">
{schools.map((school, i) => {
const summary = summariseAdmissions(data[String(school.urn)]?.admissions);
return (
@@ -148,8 +159,9 @@ export function CompareAtAGlance({
</Cell>
);
})}
</Measure>
<RowLabel>Size</RowLabel>
<Measure label="Size">
{schools.map((school, i) => {
const census = data[String(school.urn)]?.census;
const pupils = census?.total_pupils ?? school.total_pupils ?? null;
@@ -174,6 +186,7 @@ export function CompareAtAGlance({
</Cell>
);
})}
</Measure>
</SectionGrid>
</Section>
);
@@ -9,7 +9,7 @@
import { verdict } from '@/lib/compareLogic';
import type { Benchmarks, ComparisonData, School } from '@/lib/types';
import { Cell, Chip, RowLabel, Section, SectionGrid, sectionStyles as s } from './sectionShared';
import { Cell, Chip, Measure, Section, SectionGrid, sectionStyles as s } from './sectionShared';
function pctSplit(part: number | null | undefined, total: number | null | undefined): string | null {
if (part == null || total == null || total === 0) return null;
@@ -20,24 +20,27 @@ export function CompareCommunity({
schools,
data,
benchmarks,
isSecondary: propIsSecondary,
}: {
schools: School[];
data: Record<string, ComparisonData>;
benchmarks?: Benchmarks;
isSecondary?: boolean;
}) {
const isSecondary = schools.some(
const isSecondary = propIsSecondary !== undefined ? propIsSecondary : schools.some(
(school) => data[String(school.urn)]?.school_info?.attainment_8_score != null,
);
const bench = isSecondary ? benchmarks?.secondary : benchmarks?.primary;
const fsmChip = (value: number | null) => {
if (value == null || bench?.disadvantaged_pct == null) return null;
const v = verdict(value, bench.disadvantaged_pct, 3);
const anchor = bench?.fsm_pct ?? bench?.disadvantaged_pct ?? null;
if (value == null || anchor == null) return null;
const v = verdict(value, anchor, 3);
return (
<Chip tone="neutral">
{v === 'above' && 'Above the state-school average'}
{v === 'close' && 'About the state-school average'}
{v === 'below' && 'Below the state-school average'}
{v === 'above' && `Above the state-school average (${Math.round(anchor)}%)`}
{v === 'close' && `About the state-school average (${Math.round(anchor)}%)`}
{v === 'below' && `Below the state-school average (${Math.round(anchor)}%)`}
</Chip>
);
};
@@ -48,7 +51,7 @@ export function CompareCommunity({
how="The school's community, from the latest school census. State-school averages are computed from our dataset and shown for context — there's no “right” number here."
>
<SectionGrid schools={schools}>
<RowLabel>Pupils on roll</RowLabel>
<Measure label="Pupils on roll">
{schools.map((school, i) => {
const info = data[String(school.urn)]?.school_info as (School & { gias_total_pupils?: number | null; capacity?: number | null }) | undefined;
const census = data[String(school.urn)]?.census;
@@ -74,8 +77,9 @@ export function CompareCommunity({
</Cell>
);
})}
</Measure>
<RowLabel>Girls / boys</RowLabel>
<Measure label="Girls / boys">
{schools.map((school, i) => {
const census = data[String(school.urn)]?.census;
const girls = pctSplit(census?.female_pupils, census?.total_pupils);
@@ -86,10 +90,12 @@ export function CompareCommunity({
</Cell>
);
})}
</Measure>
<RowLabel tip="% of pupils eligible for free school meals — a common measure of how many pupils come from lower-income families. Benchmark computed across state schools in our dataset.">
Free school meals
</RowLabel>
<Measure
tip="% of pupils eligible for free school meals — a common measure of how many pupils come from lower-income families. Benchmark computed across state schools in our dataset."
label="Free school meals"
>
{schools.map((school, i) => {
const fsm = data[String(school.urn)]?.census?.fsm_pct ?? null;
return (
@@ -104,10 +110,12 @@ export function CompareCommunity({
</Cell>
);
})}
</Measure>
<RowLabel tip="% of pupils whose first language is known or believed to be other than English. State-school average computed from our dataset.">
English as an additional language
</RowLabel>
<Measure
tip="% of pupils whose first language is known or believed to be other than English. State-school average computed from our dataset."
label="English as an additional language"
>
{schools.map((school, i) => {
const eal = data[String(school.urn)]?.census?.eal_pct ?? null;
return (
@@ -116,10 +124,12 @@ export function CompareCommunity({
</Cell>
);
})}
</Measure>
<RowLabel tip="% of pupils receiving SEN support (not including EHC plans). A high figure can mean the school hosts specialist provision — often a strength, not a warning sign. State-school average computed from our dataset.">
Extra learning support (SEN)
</RowLabel>
<Measure
tip="% of pupils receiving SEN support (not including EHC plans). A high figure can mean the school hosts specialist provision — often a strength, not a warning sign. State-school average computed from our dataset."
label="Extra learning support (SEN)"
>
{schools.map((school, i) => {
const rows = data[String(school.urn)]?.yearly_data ?? [];
let sen: number | null = null;
@@ -143,8 +153,9 @@ export function CompareCommunity({
</Cell>
);
})}
</Measure>
<RowLabel>Faith character</RowLabel>
<Measure label="Faith character">
{schools.map((school, i) => {
const info = data[String(school.urn)]?.school_info;
const faith = info?.religious_denomination;
@@ -155,8 +166,9 @@ export function CompareCommunity({
</Cell>
);
})}
</Measure>
<RowLabel>Ages</RowLabel>
<Measure label="Ages">
{schools.map((school, i) => {
const info = data[String(school.urn)]?.school_info;
return (
@@ -165,8 +177,9 @@ export function CompareCommunity({
</Cell>
);
})}
</Measure>
<RowLabel>Run by</RowLabel>
<Measure label="Run by">
{schools.map((school, i) => {
const info = data[String(school.urn)]?.school_info;
const trust = info?.trust_name;
@@ -177,6 +190,7 @@ export function CompareCommunity({
</Cell>
);
})}
</Measure>
</SectionGrid>
</Section>
);
@@ -13,7 +13,7 @@ import {
type OfstedDisplay,
} from '@/lib/compareLogic';
import type { ComparisonData, OfstedInspection, School } from '@/lib/types';
import { Cell, Chip, RowLabel, Section, SectionGrid, sectionStyles as s } from './sectionShared';
import { Cell, Chip, Measure, Section, SectionGrid, sectionStyles as s } from './sectionShared';
const GRADE_TONE: Record<number, 'good' | 'warn' | 'bad'> = {
1: 'good',
@@ -51,6 +51,18 @@ function ResultCell({ display }: { display: OfstedDisplay }) {
</>
);
}
if (display.kind === 'transitional') {
return (
<>
<span className={s.badge} style={{ backgroundColor: '#e2e8f0', color: '#475569' }}>
No overall grade
</span>
<span className={s.small}>
Inspected under transitional framework (sub-judgements only)
</span>
</>
);
}
return (
<>
<span className={`${s.badge} ${display.grade <= 2 ? s.badgeGood : display.grade === 3 ? s.badgeWarn : s.badgeBad}`}>
@@ -157,14 +169,15 @@ export function CompareOfsted({
}
>
<SectionGrid schools={schools}>
<RowLabel>Result</RowLabel>
<Measure label="Result">
{schools.map((school, i) => (
<Cell key={school.urn} school={school} index={i}>
<ResultCell display={displays[i]} />
</Cell>
))}
</Measure>
<RowLabel>Inspected</RowLabel>
<Measure label="Inspected">
{schools.map((school, i) => {
const ofsted = data[String(school.urn)]?.ofsted;
const age = yearsSince(ofsted?.inspection_date ?? null);
@@ -176,9 +189,12 @@ export function CompareOfsted({
);
})}
<RowLabel tip="Older-style inspections: one rating per judgement area, where published. New-style inspections: the full report card, one rating per area of school life.">
Judgement detail
</RowLabel>
</Measure>
<Measure
tip="Older-style inspections: one rating per judgement area, where published. New-style inspections: the full report card, one rating per area of school life."
label="Judgement detail"
>
{schools.map((school, i) => {
const ofsted = data[String(school.urn)]?.ofsted;
return (
@@ -196,9 +212,12 @@ export function CompareOfsted({
);
})}
<RowLabel tip="Links to the school's page on ofsted.gov.uk, where all its inspection reports are listed.">
Ofsted page
</RowLabel>
</Measure>
<Measure
tip="Links to the school's page on ofsted.gov.uk, where all its inspection reports are listed."
label="Ofsted page"
>
{schools.map((school, i) => {
const url =
data[String(school.urn)]?.ofsted?.ofsted_page_url ??
@@ -211,6 +230,7 @@ export function CompareOfsted({
</Cell>
);
})}
</Measure>
</SectionGrid>
</Section>
);
@@ -60,37 +60,18 @@
margin: 0 0 1rem;
}
/* ComparisonChart runs Chart.js with maintainAspectRatio:false, so it fills
its container's height — which must be *definite*. A min-height alone does
not resolve the chart wrapper's height:100%, leaving Chart.js to fall back
to its ~150px default (a squashed sliver). Give it a real height. */
.chartBox {
min-height: 320px;
height: 420px;
}
.tableWrapper {
overflow-x: auto;
margin-top: 1.5rem;
}
.table {
width: 100%;
border-collapse: collapse;
font-size: 0.9rem;
}
.table th,
.table td {
text-align: left;
padding: 0.6rem 0.75rem;
border-bottom: 1px solid var(--border-light);
}
.table th {
background: var(--bg-secondary);
font-size: 0.8rem;
text-transform: uppercase;
letter-spacing: 0.03em;
color: var(--text-secondary);
}
.yearCell {
font-weight: 600;
white-space: nowrap;
@media (max-width: 640px) {
/* Taller on mobile: the mobile-only school chips sit above the canvas and
wrap to two rows for 3+ schools, so the plot keeps a usable height. */
.chartBox {
height: 360px;
}
}
@@ -1,19 +1,17 @@
/**
* Explore trends — the full grouped metric catalogue (nothing from the old
* compare page is lost; spec §4's tier 3) driving the year-by-year chart
* with its England reference line, plus the year-by-year table. Progress
* metrics carry CI-based bands for the years DfE published them.
* compare page is lost; spec §4's tier 3) driving the year-by-year chart with
* its England reference line. Matches the mockup: a measure picker and the
* chart only (no data table).
*/
'use client';
import dynamic from 'next/dynamic';
import { progressBand } from '@/lib/compareLogic';
import type { ComparisonData, MetricDefinition, NationalAverages, School } from '@/lib/types';
import { formatAcademicYear, formatMetricValue, metricKind } from '@/lib/utils';
import { track } from '@/lib/analytics';
import { Chip, Section, sectionStyles as s } from './sectionShared';
import { Section } from './sectionShared';
import styles from './TrendsExplorer.module.css';
const ComparisonChart = dynamic(
@@ -40,14 +38,6 @@ const SECONDARY_OPTGROUPS: { label: string; category: string }[] = [
export const PRIMARY_CATEGORIES = PRIMARY_OPTGROUPS.map((g) => g.category);
export const SECONDARY_CATEGORIES = SECONDARY_OPTGROUPS.map((g) => g.category);
const PROGRESS_CI: Record<string, [string, string]> = {
reading_progress: ['reading_progress_lower_ci', 'reading_progress_upper_ci'],
writing_progress: ['writing_progress_lower_ci', 'writing_progress_upper_ci'],
maths_progress: ['maths_progress_lower_ci', 'maths_progress_upper_ci'],
};
const BAND_LABEL = { above: 'Above average', average: 'Average', below: 'Below average' } as const;
export function TrendsExplorer({
schools,
data,
@@ -78,21 +68,11 @@ export function TrendsExplorer({
nationalByYear[entry.year] = block?.[metric] ?? null;
}
const years = [
...new Set(
schools.flatMap(
(school) => data[String(school.urn)]?.yearly_data.map((d) => Math.trunc(d.year)) ?? [],
),
),
].sort((a, b) => a - b);
const handleMetricChange = (next: string) => {
track('compare_metric_changed', { metric: next, phase: isPrimaryPhase ? 'primary' : 'secondary' });
onMetricChange(next);
};
const ciKeys = PROGRESS_CI[metric];
return (
<Section
title="Explore trends"
@@ -128,8 +108,7 @@ export function TrendsExplorer({
{metric.includes('progress') && (
<p className={styles.progressNote}>
Progress scores measure pupils&apos; progress from KS1 to KS2. A score of 0 equals the
national average. DfE stopped publishing KS2 progress after 2022/23 (no KS1 baseline);
bands use DfE&apos;s confidence intervals, not the raw score alone.
national average. DfE stopped publishing KS2 progress after 2022/23 (no KS1 baseline).
</p>
)}
@@ -142,52 +121,6 @@ export function TrendsExplorer({
nationalByYear={nationalByYear}
/>
</div>
{years.length > 0 && (
<div className={styles.tableWrapper}>
<table className={styles.table}>
<thead>
<tr>
<th>Year</th>
{schools.map((school) => (
<th key={school.urn}>{school.school_name}</th>
))}
</tr>
</thead>
<tbody>
{years.map((year) => (
<tr key={year}>
<td className={styles.yearCell}>{formatAcademicYear(year)}</td>
{schools.map((school) => {
const row = data[String(school.urn)]?.yearly_data.find(
(d) => Math.trunc(d.year) === year,
) as (Record<string, unknown> & { year: number }) | undefined;
const value = row?.[metric];
if (typeof value !== 'number') return <td key={school.urn}></td>;
const band = ciKeys
? progressBand(
value,
(row?.[ciKeys[0]] as number | null) ?? null,
(row?.[ciKeys[1]] as number | null) ?? null,
)
: null;
return (
<td key={school.urn}>
{formatMetricValue(value, metricKind(metric))}{' '}
{band && (
<Chip tone={band === 'above' ? 'good' : band === 'below' ? 'warn' : 'neutral'}>
{BAND_LABEL[band]}
</Chip>
)}
</td>
);
})}
</tr>
))}
</tbody>
</table>
</div>
)}
</div>
</details>
</Section>
@@ -31,43 +31,73 @@
margin-top: 1.25rem;
}
/* Mobile base: each measure is a card; each cell is a school row led by a
colour dot + short name. `display: contents` at ≥761px dissolves the card
back into the shared grid. */
.measure {
background: var(--bg-card);
border: 1px solid var(--border-light);
border-radius: 12px;
box-shadow: var(--shadow-soft);
padding: 0.75rem 0.85rem;
margin-bottom: 0.6rem;
}
.rowLabel {
font-size: 0.85rem;
font-weight: 600;
color: var(--text-secondary);
color: var(--text-primary);
display: flex;
align-items: center;
gap: 0.35rem;
background: var(--bg-secondary);
border-radius: 6px;
padding: 0.4rem 0.6rem;
margin-top: 0.8rem;
padding: 0 0 0.1rem;
}
.cell {
padding: 0.4rem 0.6rem;
display: flex;
align-items: baseline;
gap: 0.35rem 0.5rem;
flex-wrap: wrap;
padding: 0.5rem 0;
border-top: 1px solid var(--border-light);
margin-top: 0.5rem;
font-size: 0.95rem;
}
.cell::before {
content: attr(data-school);
display: block;
font-size: 0.72rem;
/* The school name gets its own full-width line above the value — real
school names are long and varied, so a fixed-width name column truncated
them ("Our Lady Queen of H…") or crowded the value. */
.cellTag {
display: inline-flex;
align-items: center;
gap: 0.4rem;
flex-basis: 100%;
font-size: 0.8rem;
font-weight: 600;
color: var(--sc, var(--text-muted));
color: var(--sc, var(--text-secondary));
margin-bottom: 0.15rem;
}
.cellDot {
width: 9px;
height: 9px;
border-radius: 50%;
background: var(--dot, var(--text-muted));
flex: none;
}
.big {
font-size: 1.35rem;
font-size: 1.05rem;
font-weight: 700;
font-variant-numeric: tabular-nums;
}
.small {
display: block;
flex-basis: 100%;
font-size: 0.8rem;
color: var(--text-muted);
margin-top: 0.1rem;
margin-top: 0;
}
.chip {
@@ -159,6 +189,7 @@
display: flex;
gap: 0.3rem;
flex-wrap: wrap;
flex-basis: 100%;
margin-top: 0.3rem;
}
@@ -197,20 +228,37 @@
gap: 0 0.75rem;
}
/* Dissolve the per-measure card so its label + cells become grid items of
.grid, keeping columns aligned across every measure. */
.measure {
display: contents;
}
.cellTag {
display: none;
}
.rowLabel {
background: none;
border-radius: 0;
margin-top: 0;
color: var(--text-secondary);
padding: 0.85rem 0.5rem 0.85rem 0;
border-bottom: 1px solid var(--border-light);
}
.cell {
display: block;
padding: 0.85rem 0.25rem;
border-top: none;
border-bottom: 1px solid var(--border-light);
margin-top: 0;
}
.cell::before {
content: none;
.big {
font-size: 1.35rem;
}
.small {
flex-basis: auto;
padding-left: 0;
margin-top: 0.1rem;
}
}
@@ -10,7 +10,7 @@
import type { CSSProperties, ReactNode } from 'react';
import type { School } from '@/lib/types';
import { CHART_TEXT_COLORS } from '@/lib/utils';
import { CHART_COLORS, CHART_TEXT_COLORS, shortName } from '@/lib/utils';
import styles from './compareSections.module.css';
export function Section({
@@ -61,6 +61,29 @@ export function RowLabel({ children, tip }: { children: ReactNode; tip?: string
);
}
/**
* One measure = its row label plus a cell per school. `display: contents` on
* desktop (see CSS) makes these flow into the section grid as if this wrapper
* weren't here, keeping columns aligned across measures; on mobile the wrapper
* becomes a card so each measure reads as its own block.
*/
export function Measure({
label,
tip,
children,
}: {
label: ReactNode;
tip?: string;
children: ReactNode;
}) {
return (
<div className={styles.measure}>
<RowLabel tip={tip}>{label}</RowLabel>
{children}
</div>
);
}
export function Cell({
school,
index,
@@ -73,9 +96,19 @@ export function Cell({
return (
<div
className={styles.cell}
data-school={school.school_name}
style={{ '--sc': CHART_TEXT_COLORS[index % CHART_TEXT_COLORS.length] } as CSSProperties}
style={
{
'--sc': CHART_TEXT_COLORS[index % CHART_TEXT_COLORS.length],
'--dot': CHART_COLORS[index % CHART_COLORS.length],
} as CSSProperties
}
>
{/* Mobile-only per-school tag (dot + short name); hidden on desktop,
where the column header identifies the school. */}
<span className={styles.cellTag}>
<span className={styles.cellDot} aria-hidden="true" />
{shortName(school.school_name)}
</span>
{children}
</div>
);
+9 -41
View File
@@ -1,50 +1,18 @@
/**
* Custom hook for managing school comparison state
* Uses shared context for real-time updates across components
* Custom hook for managing school comparison state.
*
* This hook is mounted on every page via the global Navigation and
* ComparisonToast, so it must stay cheap — it exposes basket state only.
* The compare page fetches `/api/compare` itself (ComparisonView); nothing
* ever read the comparison payload from here, so the previous per-page SWR
* fetch (which fired on every page whenever the basket was non-empty) was
* dead weight and has been removed.
*/
'use client';
import useSWR from 'swr';
import { fetcher } from '@/lib/api';
import { useComparisonContext } from '@/context/ComparisonContext';
import type { ComparisonResponse } from '@/lib/types';
export function useComparison() {
const {
selectedSchools,
addSchool,
removeSchool,
replaceSchools,
clearAll,
isSelected,
canAddMore,
isInitialized,
} = useComparisonContext();
// Fetch comparison data for selected schools
const urns = selectedSchools.map((s) => s.urn).join(',');
const { data, error, isLoading, mutate } = useSWR<ComparisonResponse>(
selectedSchools.length > 0 ? `/compare?urns=${urns}` : null,
fetcher,
{
revalidateOnFocus: false,
dedupingInterval: 10000,
}
);
return {
selectedSchools,
comparisonData: data?.comparison,
isLoading,
error,
addSchool,
removeSchool,
replaceSchools,
clearAll,
isSelected,
canAddMore,
isInitialized,
mutate,
};
return useComparisonContext();
}
+7 -1
View File
@@ -102,6 +102,7 @@ export type OfstedDisplay =
| { kind: 'none' }
| { kind: 'graded'; grade: number; gradeLabel: string; carriedForward: false }
| { kind: 'carried_forward'; grade: number; gradeLabel: string; carriedForward: true }
| { kind: 'transitional' }
| { kind: 'report_card'; summary: ReportCardSummary };
export function ofstedDisplay(
@@ -117,7 +118,12 @@ export function ofstedDisplay(
const grade = ofsted.overall_effectiveness;
const gradeLabel = grade != null ? OFSTED_LEGACY_GRADES[grade] : undefined;
if (grade == null || gradeLabel === undefined) return { kind: 'none' };
if (grade == null || gradeLabel === undefined) {
if (ofsted.inspection_date) {
return { kind: 'transitional' };
}
return { kind: 'none' };
}
if (ofsted.grade_source === 'ungraded_carried_forward') {
return { kind: 'carried_forward', grade, gradeLabel, carriedForward: true };
+1
View File
@@ -357,6 +357,7 @@ export interface BenchmarkBlock {
eal_pct: number | null;
sen_support_pct: number | null;
disadvantaged_pct: number | null;
fsm_pct?: number | null;
median_pupils: number | null;
/** Primary only — weighted by cohort size. */
disadvantaged_rwm_expected_pct?: number | null;
+18
View File
@@ -59,6 +59,24 @@ export function truncate(text: string, maxLength: number): string {
return text.slice(0, maxLength).trim() + '...';
}
/**
* A compact school label for tight spaces (mobile compare rows, chip bars):
* drop the trailing establishment-type words so "Barclay Primary School" →
* "Barclay", "St Mary's Catholic Primary School" → "St Mary's". Falls back to
* a length-capped truncation for names that don't carry a type suffix.
*/
export function shortName(name: string, maxLength = 32): string {
let s = name
.replace(
/\s+(primary|junior|infant|nursery|community|foundation|catholic|academy|school|college)\b.*$/i,
'',
)
.trim();
if (!s) s = name;
if (s.length > maxLength) s = s.slice(0, maxLength - 1).trim() + '…';
return s;
}
/**
* Format a school's age range for display, e.g. "3-11" → "Ages 311".
* Display-only — leaves the raw `age_range` field (used for sixth-form
+3
View File
@@ -49,6 +49,9 @@ plugins:
- name: mi_url
kind: string
description: Ofsted Management Information download URL
- name: independent_mi_url
kind: string
description: Ofsted Independent Schools Management Information download URL
- name: tap-uk-fbit
namespace: uk_fbit
@@ -2,6 +2,7 @@
from __future__ import annotations
from datetime import datetime
import io
import re
@@ -14,20 +15,28 @@ GOV_UK_PAGE = (
"monthly-management-information-ofsteds-school-inspections-outcomes"
)
INDEPENDENT_GOV_UK_PAGE = (
"https://www.gov.uk/government/statistical-data-sets/"
"non-association-independent-schools-inspections-and-outcomes-management-information"
)
# Column name → internal field, in priority order (first match wins).
# Handles both current and older file formats.
COLUMN_PRIORITY = {
"urn": ["URN", "Urn", "urn"],
"inspection_date": [
"Inspection start date of latest OEIF graded inspection",
"Inspection start date of latest OEIF standard inspection",
"Inspection start date",
"Inspection date",
],
"inspection_type": [
"Inspection type of latest OEIF graded inspection",
"Inspection type of latest OEIF standard inspection",
"Inspection type",
],
"event_type_grouping": [
"Event type grouping of latest OEIF standard inspection",
"Event type grouping",
"Inspection type grouping",
],
@@ -52,10 +61,12 @@ COLUMN_PRIORITY = {
"Effectiveness of leadership and management",
],
"early_years_provision": [
"Latest OEIF early years provision (where applicable)",
"Latest OEIF early years provision",
"Early years provision (where applicable)",
],
"sixth_form_provision": [
"Latest OEIF sixth form provision (where applicable)",
"Latest OEIF sixth form provision",
"Sixth form provision (where applicable)",
],
@@ -68,12 +79,7 @@ COLUMN_PRIORITY = {
"ungraded_inspection_date": [
"Date of latest ungraded inspection",
],
# Report Card fields (post-Nov 2025 framework). Confirmed verbatim MI
# headers per diagnose_compare_gaps.py's Task 1(c) findings. No MI column
# currently exists for early-years or sixth-form report-card grades, so
# those two fields are deliberately omitted here (see schema below) --
# they stay absent from every record, same as the existing `report_url`
# pattern for fields with no COLUMN_PRIORITY entry.
# Report Card fields (post-Nov 2025 framework).
"rc_safeguarding_met": ["Safeguarding standards"],
"rc_inclusion": ["Inclusion"],
"rc_curriculum_teaching": ["Curriculum and teaching"],
@@ -81,6 +87,13 @@ COLUMN_PRIORITY = {
"rc_attendance_behaviour": ["Attendance and behaviour"],
"rc_personal_development": ["Personal development and wellbeing"],
"rc_leadership_governance": ["Leadership and governance"],
"rc_early_years": ["Early years (where applicable)"],
"rc_sixth_form": ["Post-16 provision (where applicable)"],
"report_url": [
"Web Link (opens in new window)",
"Web link to Ofsted provider page",
"Web link",
],
}
@@ -103,6 +116,51 @@ def discover_csv_url() -> str | None:
return matches[0] if matches else None
def discover_independent_csv_url() -> str | None:
"""Scrape GOV.UK page to find the latest independent schools MI CSV download link."""
resp = requests.get(INDEPENDENT_GOV_UK_PAGE, timeout=30)
resp.raise_for_status()
# Look for CSV attachment links
csv_links = re.findall(
r'href="(https://assets\.publishing\.service\.gov\.uk/[^"]+\.csv)"',
resp.text,
)
if not csv_links:
# Fall back to ODS
csv_links = re.findall(
r'href="(https://assets\.publishing\.service\.gov\.uk/[^"]+\.ods)"',
resp.text,
)
months = {
'january': 1, 'february': 2, 'march': 3, 'april': 4, 'may': 5, 'june': 6,
'july': 7, 'august': 8, 'september': 9, 'october': 10, 'november': 11, 'december': 12
}
parsed_links = []
for link in csv_links:
normalized_link = link.lower().replace('-', '_')
if 'most_recent' not in normalized_link:
continue
match = re.search(r'as_at_(\d{1,2})_([a-z]+)_(\d{4})', normalized_link)
if match:
day, month_str, year = match.groups()
month = months.get(month_str)
if month:
try:
dt = datetime(int(year), month, int(day))
parsed_links.append((dt, link))
except ValueError:
continue
parsed_links.sort(reverse=True)
if parsed_links:
return parsed_links[0][1]
return csv_links[0] if csv_links else None
class OfstedInspectionsStream(Stream):
"""Stream: Ofsted inspection records."""
@@ -131,8 +189,6 @@ class OfstedInspectionsStream(Stream):
th.Property("rc_attendance_behaviour", th.StringType),
th.Property("rc_personal_development", th.StringType),
th.Property("rc_leadership_governance", th.StringType),
# No MI column exists for these yet; declared for forward
# compatibility with the mart schema, always emitted as absent/NULL.
th.Property("rc_early_years", th.StringType),
th.Property("rc_sixth_form", th.StringType),
th.Property("report_url", th.StringType),
@@ -148,15 +204,8 @@ class OfstedInspectionsStream(Stream):
break
return mapping
def get_records(self, context):
import pandas as pd
url = self.config.get("mi_url") or discover_csv_url()
if not url:
self.logger.error("Could not discover Ofsted MI download URL")
return
self.logger.info("Downloading Ofsted MI: %s", url)
def _fetch_and_parse_url(self, url: str, pd) -> list[dict]:
"""Download file and parse records."""
resp = requests.get(url, timeout=120)
resp.raise_for_status()
@@ -172,8 +221,6 @@ class OfstedInspectionsStream(Stream):
lines = text.split("\n")
header_idx = 0
for i, line in enumerate(lines[:20]):
# Match lines where URN appears as a CSV field (start or after comma),
# not as a substring of words like "turn" or "return".
if re.search(r'(?:^|,)\s*URN\s*(?:,|$)', line):
header_idx = i
break
@@ -191,16 +238,38 @@ class OfstedInspectionsStream(Stream):
for _, row in df.iterrows():
record = {}
for field, col in col_map.items():
record[field] = row.get(col, None)
val = row.get(col, None)
if pd.isna(val):
val = None
record[field] = val
# Cast URN
try:
record["urn"] = int(record["urn"])
record["urn"] = int(record.get("urn"))
except (ValueError, KeyError, TypeError):
continue
yield record
def get_records(self, context):
import pandas as pd
# 1. State-funded schools
state_url = self.config.get("mi_url") or discover_csv_url()
if state_url:
self.logger.info("Downloading Ofsted state-funded MI: %s", state_url)
yield from self._fetch_and_parse_url(state_url, pd)
else:
self.logger.error("Could not discover Ofsted state-funded MI download URL")
# 2. Independent schools
ind_url = self.config.get("independent_mi_url") or discover_independent_csv_url()
if ind_url:
self.logger.info("Downloading Ofsted independent MI: %s", ind_url)
yield from self._fetch_and_parse_url(ind_url, pd)
else:
self.logger.error("Could not discover Ofsted independent MI download URL")
class TapUKOfsted(Tap):
"""Singer tap for UK Ofsted Management Information."""
@@ -209,6 +278,7 @@ class TapUKOfsted(Tap):
config_jsonschema = th.PropertiesList(
th.Property("mi_url", th.StringType, description="Direct URL to Ofsted MI file"),
th.Property("independent_mi_url", th.StringType, description="Direct URL to Ofsted Independent Schools MI file"),
).to_dict()
def discover_streams(self):
@@ -46,12 +46,10 @@ renamed as (
{{ parse_report_card_grade('rc_attendance_behaviour') }}::integer as rc_attendance_behaviour,
{{ parse_report_card_grade('rc_personal_development') }}::integer as rc_personal_development,
{{ parse_report_card_grade('rc_leadership_governance') }}::integer as rc_leadership_governance,
-- No MI column exists for these yet (see tap.py); the tap never
-- emits rc_early_years/rc_sixth_form, so these stay NULL.
null::integer as rc_early_years,
null::integer as rc_sixth_form,
{{ parse_report_card_grade('rc_early_years') }}::integer as rc_early_years,
{{ parse_report_card_grade('rc_sixth_form') }}::integer as rc_sixth_form,
report_url
nullif(trim(report_url), 'NULL') as report_url
from source
where urn is not null
and (