The FSM chip anchored against disadvantaged_pct (a different measure,
FSM6+CLA) whenever fsm_pct was null — which it always was, since the
performance df has no fsm_pct. New fact_census_benchmarks mart supplies
pupil-weighted FSM/EAL means per phase; the KS2-column medians that
produced a bogus 50% 'secondary disadvantaged' anchor are gone.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146VHeLAWjDVE2B5uU67jCB
get_supplementary_data ran ~5 sequential DB round-trips per URN, so
/api/compare scaled at ~37ms/school (measured on staging: 1 school 155ms,
3 schools 220ms, 6 schools 340ms). get_supplementary_data_batch fetches
each table once with WHERE urn IN (...) and groups in Python, collapsing
5*N round-trips to a constant 5. get_supplementary_data is now a thin
wrapper so the detail endpoint is unchanged; the compare endpoint makes
one batched call. Each table degrades independently on failure.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146VHeLAWjDVE2B5uU67jCB
fact_ks4_national_averages is computed once at dbt build time (covered by
the EES DAG's stg_ees_ks4+ selector). _national_averages_payload now reads
both national-averages marts instead of scanning the performance dataframe
per year on every /api/compare request (~250ms saved per call). Fallback
for the deploy-before-DAG window computes the latest year only.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146VHeLAWjDVE2B5uU67jCB
ReligiousCharacter 99 (~4k schools) and AdmissionsPolicy 9 (~5.6k) carry a
code with a blank name in the GIAS CSV; the generator skipped them so they
hit the Unknown(<code>) path — wrongly triggering the Faith-priority tag
and polluting filters. Blank-only codes now map to "" (byte-identical to
the old name pipeline); accepted_values lists extended to match the seed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
str(ProgrammingError) embeds the full SQL, which contains every column
name — the substring check matched any error and could take the wrong
retry branch. Parse the missing column from exc.orig instead.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Closes the deploy window flagged by CI review — the backend now works
against both the old (name) and new (code) mart schemas.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- data_loader.load_school_data_as_dataframe now catches a ProgrammingError
whose message mentions has_sixth_form (psycopg2 UndefinedColumn) and
retries with a NULL-AS-has_sixth_form query variant, so the API keeps
serving data (and the app.py column-fallback branch stays reachable)
even before the nightly pipeline has rebuilt marts.dim_school.
- utils.convert_to_native now handles numpy.bool_ so GET /api/schools/{urn}
doesn't 500 once has_sixth_form is a populated bool-dtype column.
- Update the now-stale comment on the app.py age-range fallback branch.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Schools without KS2/KS4 results (special post-16 institutions, sixth-form
centres, PRUs, new schools) come back from the marts LEFT JOIN with NaN in
every numeric column. school_info passed those raw pandas values straight
into JSONResponse, which renders with allow_nan=False, so the detail
endpoint 500d and the frontend turned that into a 404 on every such SEO
landing page.
Run school_info values through convert_to_native (the same treatment
yearly_data already gets), add backend unit tests plus a pytest step in PR
checks, and an e2e journey that finds a results-less school via the search
API and asserts its page renders.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
tudor