Wires the tap TODO in stg_ofsted_inspections.sql: maps the 7 confirmed
report-card MI columns (Safeguarding standards, Inclusion, Curriculum
and teaching, Achievement, Attendance and behaviour, Personal
development and wellbeing, Leadership and governance) into rc_*
fields, parsed via the new parse_report_card_grade macro against
real sampled grade values (Exceptional/Strong standard/Expected
standard/Needs attention/Urgent improvement). rc_safeguarding_met
becomes boolean from Met/Not met. rc_early_years/rc_sixth_form have
no MI column yet and are intentionally omitted from COLUMN_PRIORITY,
staying NULL.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146VHeLAWjDVE2B5uU67jCB
Ungraded (Section 8) inspections don't assign a fresh grade — the export
only gives free text like "School remains Good". Parse that text into a
grade (remains Outstanding -> 1, remains Good -> 2, else null) and use it
as a last-resort fallback when no graded overall effectiveness exists.
Also retain schools that have only an ungraded inspection (no graded date)
by coalescing the inspection date, so ~8.5k previously-dropped schools now
carry a grade.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The preamble row in Ofsted CSVs contains 'turn off all filters' which
matched 'urn' in line.lower(), so header_idx was set to 0 instead of
the real header row. Use a regex that matches URN only as a CSV field.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Port critical patterns from the working integrator into Singer taps:
- GIAS: add 404 fallback to yesterday's date, increase timeout to 300s,
use latin-1 encoding, use dated URL for links (static URL returns 500)
- FBIT: add GIAS date fallback, increase timeout, fix encoding to latin-1
- IDACI: use dated GIAS URL with fallback instead of undated static URL,
fix encoding to latin-1, increase timeout to 300s
- Ofsted: try utf-8-sig then fall back to latin-1 encoding
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
tudorandClaude Sonnet 4.6