Root cause: the UNION ALL query in data_loader.py produced two rows per
all-through school per year (one KS2, one KS4), with drop_duplicates()
silently discarding the KS4 row. Fixes:
- New dbt mart `fact_performance`: FULL OUTER JOIN of fact_ks2_performance
and fact_ks4_performance on (urn, year). One row per school per year.
All-through schools have both KS2 and KS4 columns populated.
- data_loader.py: replace 175-line UNION ALL with a simple JOIN to
fact_performance. No more duplicate rows or drop_duplicates needed.
- sync_typesense.py: single LATERAL JOIN to fact_performance instead of
two separate KS2/KS4 joins.
- app.py: remove drop_duplicates (no longer needed); add PHASE_GROUPS
constant so all-through/middle schools appear in primary and secondary
filter results (were previously invisible to both); scope result_filters
gender/admissions_policies to secondary schools only.
- HomeView.tsx: isSecondaryView is now majority-based (not "any secondary")
and isMixedView shows both sort option sets for mixed result sets.
- school/[slug]/page.tsx: all-through schools route to SchoolDetailView
(renders both SATs + GCSE sections) instead of SecondarySchoolDetailView
(KS4-only). Dedicated SEO metadata for all-through schools.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- URLs now /school/138267-school-name instead of /school/138267
- Bare URN URLs redirect to canonical slug (backward compat)
- Remove overflow-x:hidden that broke sticky tab nav on secondary pages
- ComparisonToast starts collapsed — user must click to open
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Schools with phases like "All-through" or null phase but with GCSE data
were falling through to the primary SchoolDetailView, rendering only
partial content. Now checks yearly_data for attainment_8_score as well.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds a full data integration pipeline for enriching school profiles with
supplementary data from Ofsted, GIAS, EES, IDACI, and FBIT.
Backend:
- Bump SCHEMA_VERSION to 3; add 8 new DB tables (ofsted_inspections,
ofsted_parent_view, school_census, admissions, sen_detail, phonics,
school_deprivation, school_finance) plus GIAS columns on schools
- Expose all supplementary data via GET /api/schools/{urn}
- Enrich school list responses with ofsted_grade + ofsted_date
Integrator (new service):
- FastAPI HTTP microservice; Kestra calls POST /run/{source}
- 9 source modules: ofsted, gias, parent_view, census, admissions,
sen_detail, phonics, idaci, finance
- 9 Kestra flow YAMLs with scheduled triggers and 3× retry
Frontend:
- SchoolRow: colour-coded Ofsted badge (Outstanding/Good/RI/Inadequate)
- SchoolDetailView: 7 new sections — Ofsted sub-judgements, Parent View
survey bars, Admissions, Pupils & Inclusion / SEN, Phonics, Deprivation
Context, Finances
- types.ts: 8 new interfaces + extended School/SchoolDetailsResponse
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Migrate from vanilla JavaScript SPA to Next.js 16 with App Router
- Add server-side rendering for all pages (Home, Compare, Rankings)
- Create individual school pages with dynamic routing (/school/[urn])
- Implement Chart.js and Leaflet map integrations
- Add comprehensive SEO with sitemap, robots.txt, and JSON-LD
- Set up Docker multi-service architecture (PostgreSQL, FastAPI, Next.js)
- Update CI/CD pipeline to build both backend and frontend images
- Fix Dockerfile to include devDependencies for TypeScript compilation
- Add Jest testing configuration
- Implement performance optimizations (code splitting, caching)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
