school_compare/docs/superpowers/specs/2026-10-05-ofsted-current-status-design.md

Ofsted Current Status — Design

Date: 2026-10-05 Status: approved design, not yet implemented Scope: pipeline/transform Ofsted models, backend/data_loader.py, list and detail API Ofsted fields, search and map badges, school-page Ofsted section, compare Ofsted rows, Typesense rating, sitemap Fixes: audit findings C1, M1 and (as a side effect) M2 and part of H3, from the 3 Oct 2026 accuracy audit

Goal

Never show an Ofsted grade under a date it was not awarded or confirmed on, and always date "Inspected" by the school's latest visit.

The problem

Ofsted's management information gives each school at most three inspections: the latest graded inspection (date G, an overall grade or "Not judged", area grades), the latest ungraded inspection (date U, an outcome sentence) and the latest report card (date RC).

The site derives a grade and a date from these with two independent rules:

• grade = the graded inspection's overall grade, or else the grade parsed from the ungraded outcome ("School remains Good" → 2);
• date = G, or else U.

The two rules can pick different inspections. Every inspection from September 2024 to November 2025 was graded with "Not judged" overall, so the grade falls back to an older ungraded visit while the date stays the new one.

• C1. Rabbsfarm Primary School (102408) shows "Good · 2025". Ofsted's 17 June 2025 inspection gave no overall grade and rated quality of education, behaviour and leadership Requires Improvement. The "Good" comes from an ungraded visit on 6 February 2020. Site-wide, 932 badges pair a carried-forward grade with a newer inspection's year, and 147 of them say Good or Outstanding while that inspection rated an area Requires Improvement or Inadequate. Acre Wood Academy (151783) reads "Good · 2024" though the October 2024 inspection rated all four areas Inadequate.
• M1. When a newer ungraded visit exists, the page shows the older graded date. Washwood Heath Academy (139888) reads "Inspected 3 Mar 2020 · 4+ years ago"; Ofsted visited on 21 May 2025. 667 schools.

Ofsted's own provider page for Rabbsfarm leads with the 2025 area judgements and "From September 2024, Ofsted no longer makes an overall effectiveness judgement". It shows no overall grade.

The rule is also implemented four times: dim_school.ofsted_grade (feeds Typesense), the list SQL in data_loader.py, _ofsted_block, and two separate "latest row" picks over marts.fact_ofsted_inspection by inspection_date, which tie arbitrarily on duplicate monthly rows.

Non-goals

• Predecessor inspections (audit M10): a grade Ofsted attributes to a previous URN stays unlabelled.
• Post-16 and ISI-inspected schools (H4) and the "Not yet inspected" label.
• The compare page's broken Ofsted link (M4).
• Report-card display, which is unchanged.

The rule

Computed once per URN in int_ofsted_latest.

Latest visit: the newest of RC, G and U.

• latest_visit_date
• latest_visit_kind: report_card, graded or ungraded
• latest_visit_outcome: the ungraded outcome text when the kind is ungraded, otherwise null

Current grade: the overall grade still in force, if any.

Situation	current_grade	current_grade_date	current_grade_basis
A report card exists	null	null	null
Latest is graded, overall 1–4	that grade	G	graded
Latest is graded, "Not judged"	null	null	null
Latest is ungraded, outcome "School remains X…" (any qualifier)	X	U	confirmed
Latest is ungraded, any other outcome ("Standards maintained", "Improved significantly", "Some aspects not as strong")	the graded inspection's overall grade if it is 1–4, else null	G when a grade is kept	graded when a grade is kept
No inspection	null	null	null

Report cards began in November 2025, after the last legacy graded and ungraded inspections, so when a school has a report card it is always the latest visit, and no legacy grade stays in force beside it. Ties between G, U and RC on the same date resolve in the order report card, graded, ungraded.

Invariants: current_grade is null or 1–4; current_grade_date <= latest_visit_date; current_grade_basis is null exactly when current_grade is null.

Expected results (Ofsted MI as at 31 Aug 2026)

URN	School	Ofsted data	Latest visit	Current grade
102408	Rabbsfarm Primary School	G 17 Jun 2025 Not judged; U 6 Feb 2020 remains Good	graded, 17 Jun 2025	none
151783	Acre Wood Academy	G 1 Oct 2024 Not judged; U 14 Mar 2023 remains Good (Concerns)	graded, 1 Oct 2024	none
139888	Washwood Heath Academy	G 3 Mar 2020 Good; U 21 May 2025 Standards maintained	ungraded, 21 May 2025, "Standards maintained"	Good, 3 Mar 2020, graded
104762	Robins Lane Community Primary	G 7 Jan 2020 Good; U 18 Jul 2024 School remains Good	ungraded, 18 Jul 2024	Good, 18 Jul 2024, confirmed
100094	Royal Free Hospital Children's School	G 9 Oct 2019 Outstanding; U 5 Feb 2025 Some aspects not as strong	ungraded, 5 Feb 2025	Outstanding, 9 Oct 2019, graded
136454	Oakgrove School	U 13 Nov 2024 Standards maintained only	ungraded, 13 Nov 2024	none
137086	Bishop Stopford School	G 1 Apr 2025 Not judged	graded, 1 Apr 2025	none
110048	The Willink School	U 5 Oct 2023 remains Good; RC 6 May 2026	report card, 6 May 2026	none (report card shown)
149612	St Michael's Catholic School	RC 10 Feb 2026 only	report card, 10 Feb 2026	none (report card shown)

What each page shows

Search and map badge (buildOfstedListBadge), first match wins:

1. Report card: "Report Card · RC year" (unchanged)
2. Current grade: "Grade · year of current_grade_date"
3. Latest visit: "Inspected · year of latest_visit_date"
4. "Not yet inspected" (unchanged)

School page (OfstedSection, both phases):

• Title date: "Inspected latest visit date".
• Headline: the report card; or the current grade with a source line ("Graded inspection, 6 July 2016" or "Confirmed at an ungraded inspection, 14 March 2023"); or "No overall grade" with "Ofsted stopped giving overall grades in September 2024".
• "Latest visit" line when the latest visit is not the grade's source, e.g. "Ungraded inspection, 13 Nov 2024: Standards maintained".
• The area grid shows the graded inspection's judgements through ofstedLegacyAreas(), dated by that inspection when it is not the latest visit. The primary and secondary no-grade branches merge into one; the secondary branch's four hard-coded areas (audit M2) go with it.

Compare: ofstedDisplay returns report_card, graded, confirmed, no_overall_grade or none. The "Latest Ofsted inspection", "Result" and "Inspected" rows use the same fields as the school page.

Delivery

Two pull requests. The mart columns exist before anything reads them, so neither needs compatibility code.

PR 1: pipeline (additive)

• stg_ofsted_inspections: keep graded_inspection_date, ungraded_inspection_date and rc_inspection_date as separate typed columns, with the report-card date's existing guard. Keep inspection_date (graded, else ungraded) for the current backend. Keep a row when any of the three dates is present, so report-card-only schools are no longer dropped (part of H3: 123 schools).
• int_ofsted_latest: pick one row per URN by latest_visit_date descending, then rc_inspection_date, ungraded_inspection_date and graded_inspection_date descending (nulls last). A duplicate monthly row that carries a newer report card therefore always wins. Add the five status columns.
• New mart marts.fact_ofsted_latest: one row per URN from int_ofsted_latest with every column the pages need (status, area grades, report-card grades, ungraded outcome, report URL). It does not join dim_school, so only the monthly Ofsted DAG builds it.
• dim_school is not changed in PR 1: the daily DAG does not rebuild int_ofsted_latest, and reading a column that the monthly DAG has not yet built would fail the daily run.
• Visible effect: report-card-only schools gain their report card, because the backend's existing reads of fact_ofsted_inspection now see their rows. Nothing else changes.

PR 2: backend and UI (after the Ofsted DAG has run on PR 1)

• data_loader.py: the list query and the batch query read marts.fact_ofsted_latest instead of picking the latest row of fact_ofsted_inspection. _ofsted_block reads the status columns and loses its fallback to ungraded_grade.
• List rows: ofsted_grade becomes current_grade; ofsted_date becomes latest_visit_date; new ofsted_grade_date. ofsted_rc_date stays.
• ofsted block: overall_effectiveness and inspection_date are the graded inspection's own result and date (they label the area grid); new current_grade {grade, date, basis} (or null) and latest_visit {date, kind, outcome}; grade_source is removed. Report-card fields are unchanged.
• dim_school.ofsted_grade becomes current_grade (Typesense's rating follows at the next sync); ofsted_date becomes latest_visit_date.
• Sitemap: lastmod from latest_visit_date; _PUBLISHABLE_FIELDS also counts a latest visit, so schools that lose a carried grade keep their sitemap entry.
• Front end: lib/types.ts, buildOfstedListBadge, OfstedSection, PrimarySchoolSections, SecondarySchoolSections, compareLogic.ofstedDisplay, CompareAtAGlance, CompareOfsted. Place-page counts need no change.
• Delete buildOfstedHeroChip and buildSchoolSummary in their own commit: nothing renders them and they encode the old rule.

Testing

PR 1

• dbt unit tests on int_ofsted_latest, one per table row above plus "report-card only" and "duplicate rows, newer report card wins".
• Schema tests on fact_ofsted_latest: unique, not-null urn; accepted values for latest_visit_kind and current_grade_basis; current_grade null or 1–4; current_grade_date <= latest_visit_date.
• Run locally against a throwaway Postgres from pgserver (no Docker here). If that fails, they still run in the Ofsted DAG's dbt build, which fails on any broken case.
• pipeline/tests/test_dag_selectors.py (PR #181) keeps passing.

PR 2

• pytest: contract test for the list and ofsted fields (style of test_school_page_flag_fields.py); _ofsted_block from a fact_ofsted_latest row; sitemap publishability.
• Jest: a badge case per table row; OfstedSection for graded, confirmed, no grade, the latest-visit line and a sixth-form area; ofstedDisplay kinds. Rewrite tests that assert carried_forward.
• E2E (same PR): Rabbsfarm's search row says "Inspected · 2025" and its page says "No overall grade" with quality of education Requires Improvement; a confirmed school says "Confirmed at an ungraded inspection". The existing report-card journey stays.

Rollout and verification

1. Merge PR 1. On staging, run school_data_monthly_ofsted, then:

   -- one row per school
   select count(*) = count(distinct urn) from marts.fact_ofsted_latest;
   -- the examples above
   select urn, latest_visit_date, latest_visit_kind, latest_visit_outcome,
          current_grade, current_grade_date, current_grade_basis
   from marts.fact_ofsted_latest
   where urn in (102408, 151783, 139888, 104762, 100094, 136454, 137086, 110048, 149612);
   -- C1: a grade in force although the latest inspection gave none (expect 0)
   select count(*) from marts.fact_ofsted_latest
   where current_grade is not null and latest_visit_kind = 'graded'
     and overall_effectiveness is null;
   -- invariant (expect 0)
   select count(*) from marts.fact_ofsted_latest where current_grade_date > latest_visit_date;
   

   Check through the API that St Michael's Catholic School (149612) shows its report card.

2. Promote PR 1 to production; run the Ofsted DAG there; repeat the checks.

3. Merge PR 2. Let the daily DAG run (or trigger it) so dim_school and Typesense pick up the change; run the E2E journeys; re-run the audit's C1, M1 and M2 checks against staging: expect 0.

4. Promote PR 2; repeat the audit checks on production.

Expected visible change

About 932 schools change from a grade badge dated by a no-grade inspection ("Good · 2025") to "Inspected · 2025". Counts of Good and Outstanding schools on place pages fall by the same schools, and Typesense's rating changes for them. Dates beside a grade can move earlier (to the inspection that awarded it); "Inspected" dates move later (to the latest visit).

Risks

• Ofsted changes its MI columns most months. Unknown grade text parses to null (safe_numeric), which degrades to "Inspected · year", never to a wrong grade.
• PR 2 depends on the Ofsted DAG having run on the target environment after PR
  1. If PR 2 is promoted first, the backend reads a missing table: promote in order.