school_compare/docs/superpowers/specs/2026-10-05-ofsted-current-status-design.md

# Ofsted Current Status — Design

**Date:** 2026-10-05
**Status:** approved design, not yet implemented
**Scope:** `pipeline/transform` Ofsted models, `backend/data_loader.py`, list and
detail API Ofsted fields, search and map badges, school-page Ofsted section,
compare Ofsted rows, Typesense rating, sitemap
**Fixes:** audit findings C1, M1 and (as a side effect) M2 and part of H3,
from the 3 Oct 2026 accuracy audit

## Goal

Never show an Ofsted grade under a date it was not awarded or confirmed on, and
always date "Inspected" by the school's latest visit.

## The problem

Ofsted's management information gives each school at most three inspections:
the latest graded inspection (date G, an overall grade or "Not judged", area
grades), the latest ungraded inspection (date U, an outcome sentence) and the
latest report card (date RC).

The site derives a grade and a date from these with two independent rules:

- grade = the graded inspection's overall grade, or else the grade parsed from
  the ungraded outcome ("School remains Good" → 2);
- date = G, or else U.

The two rules can pick different inspections. Every inspection from
September 2024 to November 2025 was graded with "Not judged" overall, so the
grade falls back to an older ungraded visit while the date stays the new one.

- **C1.** Rabbsfarm Primary School (102408) shows "Good · 2025". Ofsted's
  17 June 2025 inspection gave no overall grade and rated quality of education,
  behaviour and leadership Requires Improvement. The "Good" comes from an
  ungraded visit on 6 February 2020. Site-wide, 932 badges pair a
  carried-forward grade with a newer inspection's year, and 147 of them say Good
  or Outstanding while that inspection rated an area Requires Improvement or
  Inadequate. Acre Wood Academy (151783) reads "Good · 2024" though the
  October 2024 inspection rated all four areas Inadequate.
- **M1.** When a newer ungraded visit exists, the page shows the older graded
  date. Washwood Heath Academy (139888) reads "Inspected 3 Mar 2020 · 4+ years
  ago"; Ofsted visited on 21 May 2025. 667 schools.

Ofsted's own provider page for Rabbsfarm leads with the 2025 area judgements and
"From September 2024, Ofsted no longer makes an overall effectiveness
judgement". It shows no overall grade.

The rule is also implemented four times: `dim_school.ofsted_grade` (feeds
Typesense), the list SQL in `data_loader.py`, `_ofsted_block`, and two separate
"latest row" picks over `marts.fact_ofsted_inspection` by `inspection_date`,
which tie arbitrarily on duplicate monthly rows.

## Non-goals

- Predecessor inspections (audit M10): a grade Ofsted attributes to a previous
  URN stays unlabelled.
- Post-16 and ISI-inspected schools (H4) and the "Not yet inspected" label.
- The compare page's broken Ofsted link (M4).
- Report-card display, which is unchanged.

## The rule

Computed once per URN in `int_ofsted_latest`.

**Latest visit:** the newest of RC, G and U.

- `latest_visit_date`
- `latest_visit_kind`: `report_card`, `graded` or `ungraded`
- `latest_visit_outcome`: the ungraded outcome text when the kind is `ungraded`,
  otherwise null

**Current grade:** the overall grade still in force, if any.

| Situation | `current_grade` | `current_grade_date` | `current_grade_basis` |
|---|---|---|---|
| A report card exists | null | null | null |
| Latest is graded, overall 1–4 | that grade | G | `graded` |
| Latest is graded, "Not judged" | null | null | null |
| Latest is ungraded, outcome "School remains X…" (any qualifier) | X | U | `confirmed` |
| Latest is ungraded, any other outcome ("Standards maintained", "Improved significantly", "Some aspects not as strong") | the graded inspection's overall grade if it is 1–4, else null | G when a grade is kept | `graded` when a grade is kept |
| No inspection | null | null | null |

Report cards began in November 2025, after the last legacy graded and ungraded
inspections, so when a school has a report card it is always the latest visit,
and no legacy grade stays in force beside it. Ties between G, U and RC on the
same date resolve in the order report card, graded, ungraded.

Invariants: `current_grade` is null or 1–4; `current_grade_date <=
latest_visit_date`; `current_grade_basis` is null exactly when `current_grade`
is null.

### Expected results (Ofsted MI as at 31 Aug 2026)

| URN | School | Ofsted data | Latest visit | Current grade |
|---|---|---|---|---|
| 102408 | Rabbsfarm Primary School | G 17 Jun 2025 Not judged; U 6 Feb 2020 remains Good | graded, 17 Jun 2025 | none |
| 151783 | Acre Wood Academy | G 1 Oct 2024 Not judged; U 14 Mar 2023 remains Good (Concerns) | graded, 1 Oct 2024 | none |
| 139888 | Washwood Heath Academy | G 3 Mar 2020 Good; U 21 May 2025 Standards maintained | ungraded, 21 May 2025, "Standards maintained" | Good, 3 Mar 2020, graded |
| 104762 | Robins Lane Community Primary | G 7 Jan 2020 Good; U 18 Jul 2024 School remains Good | ungraded, 18 Jul 2024 | Good, 18 Jul 2024, confirmed |
| 100094 | Royal Free Hospital Children's School | G 9 Oct 2019 Outstanding; U 5 Feb 2025 Some aspects not as strong | ungraded, 5 Feb 2025 | Outstanding, 9 Oct 2019, graded |
| 136454 | Oakgrove School | U 13 Nov 2024 Standards maintained only | ungraded, 13 Nov 2024 | none |
| 137086 | Bishop Stopford School | G 1 Apr 2025 Not judged | graded, 1 Apr 2025 | none |
| 110048 | The Willink School | U 5 Oct 2023 remains Good; RC 6 May 2026 | report card, 6 May 2026 | none (report card shown) |
| 149612 | St Michael's Catholic School | RC 10 Feb 2026 only | report card, 10 Feb 2026 | none (report card shown) |

## What each page shows

**Search and map badge** (`buildOfstedListBadge`), first match wins:

1. Report card: "Report Card · *RC year*" (unchanged)
2. Current grade: "*Grade* · *year of `current_grade_date`*"
3. Latest visit: "Inspected · *year of `latest_visit_date`*"
4. "Not yet inspected" (unchanged)

**School page** (`OfstedSection`, both phases):

- Title date: "Inspected *latest visit date*".
- Headline: the report card; or the current grade with a source line
  ("Graded inspection, 6 July 2016" or "Confirmed at an ungraded inspection,
  14 March 2023"); or "No overall grade" with "Ofsted stopped giving overall
  grades in September 2024".
- "Latest visit" line when the latest visit is not the grade's source, e.g.
  "Ungraded inspection, 13 Nov 2024: Standards maintained".
- The area grid shows the graded inspection's judgements through
  `ofstedLegacyAreas()`, dated by that inspection when it is not the latest
  visit. The primary and secondary no-grade branches merge into one; the
  secondary branch's four hard-coded areas (audit M2) go with it.

**Compare:** `ofstedDisplay` returns `report_card`, `graded`, `confirmed`,
`no_overall_grade` or `none`. The "Latest Ofsted inspection", "Result" and
"Inspected" rows use the same fields as the school page.

## Delivery

Two pull requests. The mart columns exist before anything reads them, so
neither needs compatibility code.

### PR 1: pipeline (additive)

- `stg_ofsted_inspections`: keep `graded_inspection_date`,
  `ungraded_inspection_date` and `rc_inspection_date` as separate typed
  columns, with the report-card date's existing guard. Keep `inspection_date`
  (graded, else ungraded) for the current backend. Keep a row when any of the
  three dates is present, so report-card-only schools are no longer dropped
  (part of H3: 123 schools).
- `int_ofsted_latest`: pick one row per URN by `latest_visit_date` descending,
  then `rc_inspection_date`, `ungraded_inspection_date` and
  `graded_inspection_date` descending (nulls last). A duplicate monthly row that
  carries a newer report card therefore always wins. Add the five status
  columns.
- New mart `marts.fact_ofsted_latest`: one row per URN from `int_ofsted_latest`
  with every column the pages need (status, area grades, report-card grades,
  ungraded outcome, report URL). It does not join `dim_school`, so only the
  monthly Ofsted DAG builds it.
- `dim_school` is not changed in PR 1: the daily DAG does not rebuild
  `int_ofsted_latest`, and reading a column that the monthly DAG has not yet
  built would fail the daily run.
- Visible effect: report-card-only schools gain their report card, because the
  backend's existing reads of `fact_ofsted_inspection` now see their rows.
  Nothing else changes.

### PR 2: backend and UI (after the Ofsted DAG has run on PR 1)

- `data_loader.py`: the list query and the batch query read
  `marts.fact_ofsted_latest` instead of picking the latest row of
  `fact_ofsted_inspection`. `_ofsted_block` reads the status columns and loses
  its fallback to `ungraded_grade`.
- List rows: `ofsted_grade` becomes `current_grade`; `ofsted_date` becomes
  `latest_visit_date`; new `ofsted_grade_date`. `ofsted_rc_date` stays.
- `ofsted` block: `overall_effectiveness` and `inspection_date` are the graded
  inspection's own result and date (they label the area grid); new
  `current_grade` `{grade, date, basis}` (or null) and `latest_visit`
  `{date, kind, outcome}`; `grade_source` is removed. Report-card fields are
  unchanged.
- `dim_school.ofsted_grade` becomes `current_grade` (Typesense's rating follows
  at the next sync); `ofsted_date` becomes `latest_visit_date`.
- Sitemap: `lastmod` from `latest_visit_date`; `_PUBLISHABLE_FIELDS` also counts
  a latest visit, so schools that lose a carried grade keep their sitemap entry.
- Front end: `lib/types.ts`, `buildOfstedListBadge`, `OfstedSection`,
  `PrimarySchoolSections`, `SecondarySchoolSections`, `compareLogic.ofstedDisplay`,
  `CompareAtAGlance`, `CompareOfsted`. Place-page counts need no change.
- Delete `buildOfstedHeroChip` and `buildSchoolSummary` in their own commit:
  nothing renders them and they encode the old rule.

## Testing

**PR 1**

- dbt unit tests on `int_ofsted_latest`, one per table row above plus
  "report-card only" and "duplicate rows, newer report card wins".
- Schema tests on `fact_ofsted_latest`: unique, not-null `urn`; accepted values
  for `latest_visit_kind` and `current_grade_basis`; `current_grade` null or
  1–4; `current_grade_date <= latest_visit_date`.
- Run locally against a throwaway Postgres from `pgserver` (no Docker here). If
  that fails, they still run in the Ofsted DAG's `dbt build`, which fails on any
  broken case.
- `pipeline/tests/test_dag_selectors.py` (PR #181) keeps passing.

**PR 2**

- pytest: contract test for the list and `ofsted` fields (style of
  `test_school_page_flag_fields.py`); `_ofsted_block` from a
  `fact_ofsted_latest` row; sitemap publishability.
- Jest: a badge case per table row; `OfstedSection` for graded, confirmed, no
  grade, the latest-visit line and a sixth-form area; `ofstedDisplay` kinds.
  Rewrite tests that assert `carried_forward`.
- E2E (same PR): Rabbsfarm's search row says "Inspected · 2025" and its page
  says "No overall grade" with quality of education Requires Improvement; a
  confirmed school says "Confirmed at an ungraded inspection". The existing
  report-card journey stays.

## Rollout and verification

1. Merge PR 1. On staging, run `school_data_monthly_ofsted`, then:

   ```sql
   -- one row per school
   select count(*) = count(distinct urn) from marts.fact_ofsted_latest;
   -- the examples above
   select urn, latest_visit_date, latest_visit_kind, latest_visit_outcome,
          current_grade, current_grade_date, current_grade_basis
   from marts.fact_ofsted_latest
   where urn in (102408, 151783, 139888, 104762, 100094, 136454, 137086, 110048, 149612);
   -- C1: a grade in force although the latest inspection gave none (expect 0)
   select count(*) from marts.fact_ofsted_latest
   where current_grade is not null and latest_visit_kind = 'graded'
     and overall_effectiveness is null;
   -- invariant (expect 0)
   select count(*) from marts.fact_ofsted_latest where current_grade_date > latest_visit_date;
   ```

   Check through the API that St Michael's Catholic School (149612) shows its
   report card.
2. Promote PR 1 to production; run the Ofsted DAG there; repeat the checks.
3. Merge PR 2. Let the daily DAG run (or trigger it) so `dim_school` and
   Typesense pick up the change; run the E2E journeys; re-run the audit's C1,
   M1 and M2 checks against staging: expect 0.
4. Promote PR 2; repeat the audit checks on production.

## Expected visible change

About 932 schools change from a grade badge dated by a no-grade inspection
("Good · 2025") to "Inspected · 2025". Counts of Good and Outstanding schools
on place pages fall by the same schools, and Typesense's rating changes for
them. Dates beside a grade can move earlier (to the inspection that awarded
it); "Inspected" dates move later (to the latest visit).

## Risks

- Ofsted changes its MI columns most months. Unknown grade text parses to null
  (`safe_numeric`), which degrades to "Inspected · year", never to a wrong grade.
- PR 2 depends on the Ofsted DAG having run on the target environment after PR
  1. If PR 2 is promoted first, the backend reads a missing table: promote in
  order.