Wires the tap TODO in stg_ofsted_inspections.sql: maps the 7 confirmed
report-card MI columns (Safeguarding standards, Inclusion, Curriculum
and teaching, Achievement, Attendance and behaviour, Personal
development and wellbeing, Leadership and governance) into rc_*
fields, parsed via the new parse_report_card_grade macro against
real sampled grade values (Exceptional/Strong standard/Expected
standard/Needs attention/Urgent improvement). rc_safeguarding_met
becomes boolean from Met/Not met. rc_early_years/rc_sixth_form have
no MI column yet and are intentionally omitted from COLUMN_PRIORITY,
staying NULL.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146VHeLAWjDVE2B5uU67jCB
DfE never published school-level KS2 2021/22 data publicly (confirmed via
EES release notes and by walking the Compare School Performance download
wizard, which has no ks2 checkbox for 2021-2022, same as the COVID-cancelled
2020-2021 year). No archive exists to verify column headers against or
upload to the filebrowser; Task 6 is blocked at the source-data level.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146VHeLAWjDVE2B5uU67jCB
Widen the year filter in stg_ees_ks2_national.sql from >= 201617 to
>= 201516 so the England national-averages line no longer starts a
year late; the catalogue CSV has a real, comparable 201516 row (2015/16
was the first year of the current expected-standard tests, so it's the
correct floor).
GPS/science/scaled-score national columns confirmed present at source
with correct mapping; prod NULLs are stale raw data, backfilled by the
next extract run. No _KS2_NATIONAL_COL_MAP change accompanies this fix.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146VHeLAWjDVE2B5uU67jCB
int_ks4_with_lineage.sql unions stg_ees_ks4 and stg_legacy_ks4 via
`select *`, which PostgreSQL aligns positionally. stg_legacy_ks4 listed
sen_support_pct before sen_ehcp_pct while stg_ees_ks4 lists sen_ehcp_pct
before sen_support_pct, swapping the two values for legacy-sourced rows
in marts.fact_ks4_performance. Reordered stg_legacy_ks4's final select
to match.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146VHeLAWjDVE2B5uU67jCB
ReligiousCharacter 99 (~4k schools) and AdmissionsPolicy 9 (~5.6k) carry a
code with a blank name in the GIAS CSV; the generator skipped them so they
hit the Unknown(<code>) path — wrongly triggering the Faith-priority tag
and polluting filters. Blank-only codes now map to "" (byte-identical to
the old name pipeline); accepted_values lists extended to match the seed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Addresses AI-review findings: the annual IDACI DAG also rebuilds a mart
(fact_deprivation) and needs the reload; curl gets connect/max timeouts
so an unreachable backend fails fast instead of hanging the task.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The daily/monthly/annual DAG docstring promised an Invalidate Cache step
that never existed — after a marts rebuild the backend kept serving its
startup-cached (possibly empty) DataFrame until a container restart.
Add a POST /api/admin/reload task at the end of each pipeline DAG,
mirroring the sitemap DAG's admin-call pattern.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
str(ProgrammingError) embeds the full SQL, which contains every column
name — the substring check matched any error and could take the wrong
retry branch. Parse the missing column from exc.orig instead.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Closes the deploy window flagged by CI review — the backend now works
against both the old (name) and new (code) mart schemas.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Amber tag in listing rows (option A) and a slim notice strip under the
detail-page header (option E): proposed for closure, formal process not
necessarily started, check with the local authority before applying.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
These schools are still operating and publish results; they drop out
automatically when GIAS flips them to Closed since marts fully rebuild
each run.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Matches the phase derivation's guard against casing/whitespace variants in
raw GIAS data; an unmatched variant previously fell through silently to the
statutory-age fallback.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- data_loader.load_school_data_as_dataframe now catches a ProgrammingError
whose message mentions has_sixth_form (psycopg2 UndefinedColumn) and
retries with a NULL-AS-has_sixth_form query variant, so the API keeps
serving data (and the app.py column-fallback branch stays reachable)
even before the nightly pipeline has rebuilt marts.dim_school.
- utils.convert_to_native now handles numpy.bool_ so GET /api/schools/{urn}
doesn't 500 once has_sixth_form is a populated bool-dtype column.
- Update the now-stale comment on the app.py age-range fallback branch.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Schools without KS2/KS4 results (special post-16 institutions, sixth-form
centres, PRUs, new schools) come back from the marts LEFT JOIN with NaN in
every numeric column. school_info passed those raw pandas values straight
into JSONResponse, which renders with allow_nan=False, so the detail
endpoint 500d and the frontend turned that into a 404 on every such SEO
landing page.
Run school_info values through convert_to_native (the same treatment
yearly_data already gets), add backend unit tests plus a pytest step in PR
checks, and an e2e journey that finds a results-less school via the search
API and asserts its page renders.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The geolocation shortcut lived in the discovery strip below the results,
away from the search. Move it directly under the hero search input, paired
with the postcode hint, so the two ways to find nearby schools ("type a
postcode" / "use my location") read as one idea and are visible at first
glance.
- FilterBar gains optional onNearMe/geoState/geoError props and renders the
teal "Use my location" pill (with spinner + error) in hero mode; the
geolocation flow itself still lives in HomeView.
- Remove the now-duplicate near-me button and its dead CSS from the
discovery section.
- Refresh the search hint copy to pair with the button.
- e2e: assert the "use my location" shortcut renders in the hero.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
tudor