fix(api): survive missing has_sixth_form column and numpy bool serialization

- data_loader.load_school_data_as_dataframe now catches a ProgrammingError
  whose message mentions has_sixth_form (psycopg2 UndefinedColumn) and
  retries with a NULL-AS-has_sixth_form query variant, so the API keeps
  serving data (and the app.py column-fallback branch stays reachable)
  even before the nightly pipeline has rebuilt marts.dim_school.
- utils.convert_to_native now handles numpy.bool_ so GET /api/schools/{urn}
  doesn't 500 once has_sixth_form is a populated bool-dtype column.
- Update the now-stale comment on the app.py age-range fallback branch.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Tudor
2026-07-07 13:33:25 +01:00
co-authored by Claude Fable 5
parent 3fcb1340d4
commit a524cdc591
4 changed files with 115 additions and 1 deletions
+4 -1
View File
@@ -421,7 +421,10 @@ async def get_schools(
if has_sixth_form in ("yes", "no"):
if "has_sixth_form" in df_latest.columns:
flag = df_latest["has_sixth_form"].eq(True)
else: # DB predates the pipeline re-run — fall back to age range
else: # Defensive fallback only — data_loader now always synthesizes
# has_sixth_form as NULL when the DB predates the pipeline re-run,
# so this branch shouldn't normally trigger. Falls back to age
# range if the column is somehow absent anyway.
flag = df_latest["age_range"].str.contains("18", na=False)
df_latest = df_latest[flag if has_sixth_form == "yes" else ~flag]