feat(data): integrate 9 UK government data sources via Kestra
Adds a full data integration pipeline for enriching school profiles with
supplementary data from Ofsted, GIAS, EES, IDACI, and FBIT.
Backend:
- Bump SCHEMA_VERSION to 3; add 8 new DB tables (ofsted_inspections,
ofsted_parent_view, school_census, admissions, sen_detail, phonics,
school_deprivation, school_finance) plus GIAS columns on schools
- Expose all supplementary data via GET /api/schools/{urn}
- Enrich school list responses with ofsted_grade + ofsted_date
Integrator (new service):
- FastAPI HTTP microservice; Kestra calls POST /run/{source}
- 9 source modules: ofsted, gias, parent_view, census, admissions,
sen_detail, phonics, idaci, finance
- 9 Kestra flow YAMLs with scheduled triggers and 3× retry
Frontend:
- SchoolRow: colour-coded Ofsted badge (Outstanding/Good/RI/Inadequate)
- SchoolDetailView: 7 new sections — Ofsted sub-judgements, Parent View
survey bars, Admissions, Pupils & Inclusion / SEN, Phonics, Deprivation
Context, Finances
- types.ts: 8 new interfaces + extended School/SchoolDetailsResponse
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -24,6 +24,7 @@ from .data_loader import (
|
||||
clear_cache,
|
||||
load_school_data,
|
||||
geocode_single_postcode,
|
||||
get_supplementary_data,
|
||||
)
|
||||
from .data_loader import get_data_info as get_db_info
|
||||
from .database import check_and_migrate_if_needed
|
||||
@@ -384,6 +385,16 @@ async def get_school_details(request: Request, urn: int):
|
||||
# Get latest info for the school
|
||||
latest = school_data.iloc[-1]
|
||||
|
||||
# Fetch supplementary data (Ofsted, Parent View, admissions, etc.)
|
||||
from .database import SessionLocal
|
||||
supplementary = {}
|
||||
try:
|
||||
db = SessionLocal()
|
||||
supplementary = get_supplementary_data(db, urn)
|
||||
db.close()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
return {
|
||||
"school_info": {
|
||||
"urn": urn,
|
||||
@@ -396,8 +407,23 @@ async def get_school_details(request: Request, urn: int):
|
||||
"latitude": latest.get("latitude"),
|
||||
"longitude": latest.get("longitude"),
|
||||
"phase": "Primary",
|
||||
# GIAS fields
|
||||
"website": latest.get("website"),
|
||||
"headteacher_name": latest.get("headteacher_name"),
|
||||
"capacity": latest.get("capacity"),
|
||||
"trust_name": latest.get("trust_name"),
|
||||
"gender": latest.get("gender"),
|
||||
},
|
||||
"yearly_data": clean_for_json(school_data),
|
||||
# Supplementary data (null if not yet populated by Kestra)
|
||||
"ofsted": supplementary.get("ofsted"),
|
||||
"parent_view": supplementary.get("parent_view"),
|
||||
"census": supplementary.get("census"),
|
||||
"admissions": supplementary.get("admissions"),
|
||||
"sen_detail": supplementary.get("sen_detail"),
|
||||
"phonics": supplementary.get("phonics"),
|
||||
"deprivation": supplementary.get("deprivation"),
|
||||
"finance": supplementary.get("finance"),
|
||||
}
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user