fix(pipeline): read the latest Ofsted visit from the dates

int_ofsted_latest treated any report card as the latest visit, relying on report cards postdating legacy inspections. That holds today (0 of 2,451 report-card schools in the 31 Aug 2026 MI) but is now not assumed: the latest visit is the newest of the three dates. A report card still leaves no legacy grade in force, which is the spec's rule rather than an inference. New unit test newer_legacy_visit_after_a_report_card failed RED on the old logic. Review on #183.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
TudorandClaude Opus 5.5 committed 2026-10-05 16:39:24 +01:00
1 parent dfa4928641
commit 93f17211b1
3 files changed
+26 -9

No files matched your search

@@ -7,12 +7,11 @@
with inspections as ( with inspections as (
select select
*, *,
-- Report cards began in Nov 2025, after the last legacy inspections, -- The newest of the three inspections, read from the dates. (Report
-- so a school's report card is always its latest visit. -- cards began in Nov 2025, after the last legacy inspections, so today
coalesce( -- a report card is always the latest; nothing below relies on that.)
rc_inspection_date, greatest(rc_inspection_date, graded_inspection_date, ungraded_inspection_date)
greatest(graded_inspection_date, ungraded_inspection_date) as latest_visit_date
) as latest_visit_date
from {{ ref('stg_ofsted_inspections') }} from {{ ref('stg_ofsted_inspections') }}
), ),
@@ -34,8 +33,9 @@ ranked as (
latest as ( latest as (
select select
*, *,
-- Same-day ties resolve report card, then graded, then ungraded.
case case
when rc_inspection_date is not null then 'report_card' when rc_inspection_date = latest_visit_date then 'report_card'
when graded_inspection_date = latest_visit_date then 'graded' when graded_inspection_date = latest_visit_date then 'graded'
else 'ungraded' else 'ungraded'
end as latest_visit_kind end as latest_visit_kind
@@ -46,11 +46,14 @@ latest as (
graded as ( graded as (
select select
*, *,
-- The grade still in force. A graded inspection's own overall grade -- The grade still in force. A report card replaced overall grades, so
-- none survives it. Otherwise: a graded inspection's own overall grade
-- (1-4; "Not judged" and the sentinel 9 are no grade); else an -- (1-4; "Not judged" and the sentinel 9 are no grade); else an
-- ungraded visit's "School remains X"; else, after an ungraded visit -- ungraded visit's "School remains X"; else, after an ungraded visit
-- that names no grade, the graded inspection's grade. -- that names no grade, the graded inspection's grade.
case case
when rc_inspection_date is not null
then null
when latest_visit_kind = 'graded' and overall_effectiveness between 1 and 4 when latest_visit_kind = 'graded' and overall_effectiveness between 1 and 4
then 'graded_latest' then 'graded_latest'
when latest_visit_kind = 'ungraded' and ungraded_grade is not null when latest_visit_kind = 'ungraded' and ungraded_grade is not null
@@ -97,3 +97,17 @@ unit_tests:
expect: expect:
rows: rows:
- {urn: 3, latest_visit_kind: graded, current_grade: null} - {urn: 3, latest_visit_kind: graded, current_grade: null}
- name: newer_legacy_visit_after_a_report_card
description: >
Not in Ofsted's data today (0 of 2,451 report-card schools in the 31 Aug
2026 MI), but the latest visit is read from the dates, not assumed. The
report card still leaves no legacy grade in force.
model: int_ofsted_latest
given:
- input: ref('stg_ofsted_inspections')
rows:
- {urn: 4, ungraded_inspection_date: '2026-03-02', ungraded_grade: 2, ungraded_outcome: 'School remains Good', rc_inspection_date: '2025-12-01', rc_inclusion: 3}
expect:
rows:
- {urn: 4, latest_visit_date: '2026-03-02', latest_visit_kind: ungraded, latest_visit_outcome: 'School remains Good', current_grade: null, current_grade_date: null, current_grade_basis: null}
@@ -7,4 +7,4 @@ from {{ ref('fact_ofsted_latest') }}
where current_grade_date > latest_visit_date where current_grade_date > latest_visit_date
or (current_grade is null) <> (current_grade_date is null) or (current_grade is null) <> (current_grade_date is null)
or (current_grade is null) <> (current_grade_basis is null) or (current_grade is null) <> (current_grade_basis is null)
or (latest_visit_kind = 'report_card' and current_grade is not null) or (rc_inspection_date is not null and current_grade is not null)