Audit findings C1 and M1 (3 Oct 2026 accuracy audit). The site carries a grade forward from an older ungraded visit and shows it under a newer inspection's date. Rabbsfarm Primary (102408) reads "Good · 2025", although Ofsted's 17 June 2025 inspection gave no overall grade and rated three areas Requires Improvement. Site-wide that's 932 badges, 147 of them contradicted by the inspection they're dated with. "Inspected" dates also show the last graded inspection rather than the latest visit (667 schools).
This is PR 1 of 2 (pipeline, additive). PR 2 switches dim_school, the backend and the UI to the new columns. It must wait until the Ofsted DAG has run with this PR.
Change
stg_ofsted_inspections:
Keeps the graded, ungraded and report-card dates as separate columns.
Keeps report-card-only schools, which it used to drop (123 schools, part of audit H3).
inspection_date stays for current readers and now falls back to the report-card date, so fact_ofsted_inspection's not_null test still holds.
int_ofsted_latest now holds the rule, one row per school:
Grade still in force:current_grade, current_grade_date, current_grade_basis (graded/confirmed).
Row choice: duplicate monthly rows resolve to the newest visit, so a newer report card always wins.
dim_school's columns are unchanged.
New mart marts.fact_ofsted_latest: one row per URN, with schema tests and tests/assert_ofsted_current_grade_consistent.sql. Built by the monthly Ofsted DAG only.
Visible effect of this PR alone
Report-card-only schools (e.g. St Michael's Catholic School, 149612) gain their report card. Nothing else on the site changes until PR 2.
Verification
dbt unit tests (11) cover every row of the spec's rule table, plus three edge cases: duplicate rows, a same-day graded and ungraded visit, and the "9" grade code. They ran locally against a throwaway Postgres (pgserver) with dbt-postgres 1.10. I also built the Ofsted models there from the real 31 Aug 2026 MI rows of the spec's nine example schools. Result: 25/25 nodes pass, and fact_ofsted_latest matches the spec's expected-results table 9/9.
URN
Latest visit
Current grade
102408 Rabbsfarm
graded 2025-06-17
none
151783 Acre Wood
graded 2024-10-01
none
139888 Washwood Heath
ungraded 2025-05-21 "Standards maintained"
Good, 2020-03-03, graded
104762 Robins Lane
ungraded 2024-07-18 "School remains Good"
Good, 2024-07-18, confirmed
100094 Royal Free Hospital Children's
ungraded 2025-02-05 "Some aspects not as strong"
Outstanding, 2019-10-09, graded
136454 Oakgrove
ungraded 2024-11-13 "Standards maintained"
none
137086 Bishop Stopford
graded 2025-04-01
none
110048 Willink
report card 2026-05-06
none
149612 St Michael's
report card 2026-02-10
none
Other checks:
dbt ls: the daily DAG's selection does not include fact_ofsted_latest; the monthly Ofsted DAG's does.
## Why
Audit findings C1 and M1 (3 Oct 2026 accuracy audit). The site carries a grade forward from an older ungraded visit and shows it under a newer inspection's date. Rabbsfarm Primary (102408) reads **"Good · 2025"**, although Ofsted's 17 June 2025 inspection gave no overall grade and rated three areas Requires Improvement. Site-wide that's 932 badges, 147 of them contradicted by the inspection they're dated with. "Inspected" dates also show the last *graded* inspection rather than the latest visit (667 schools).
Design: `docs/superpowers/specs/2026-10-05-ofsted-current-status-design.md`. Plan: `docs/superpowers/plans/2026-10-05-ofsted-current-status.md`.
This is **PR 1 of 2 (pipeline, additive)**. PR 2 switches `dim_school`, the backend and the UI to the new columns. It must wait until the Ofsted DAG has run with this PR.
## Change
- **`stg_ofsted_inspections`**:
- Keeps the graded, ungraded and report-card dates as separate columns.
- Keeps report-card-only schools, which it used to drop (123 schools, part of audit H3).
- `inspection_date` stays for current readers and now falls back to the report-card date, so `fact_ofsted_inspection`'s `not_null` test still holds.
- **`int_ofsted_latest`** now holds the rule, one row per school:
- **Latest visit:** `latest_visit_date`, `latest_visit_kind` (`report_card`/`graded`/`ungraded`), `latest_visit_outcome`.
- **Grade still in force:** `current_grade`, `current_grade_date`, `current_grade_basis` (`graded`/`confirmed`).
- **Row choice:** duplicate monthly rows resolve to the newest visit, so a newer report card always wins.
- `dim_school`'s columns are unchanged.
- **New mart `marts.fact_ofsted_latest`:** one row per URN, with schema tests and `tests/assert_ofsted_current_grade_consistent.sql`. Built by the monthly Ofsted DAG only.
## Visible effect of this PR alone
Report-card-only schools (e.g. St Michael's Catholic School, 149612) gain their report card. Nothing else on the site changes until PR 2.
## Verification
dbt unit tests (11) cover every row of the spec's rule table, plus three edge cases: duplicate rows, a same-day graded and ungraded visit, and the "9" grade code. They ran locally against a throwaway Postgres (pgserver) with dbt-postgres 1.10. I also built the Ofsted models there from the real 31 Aug 2026 MI rows of the spec's nine example schools. Result: 25/25 nodes pass, and `fact_ofsted_latest` matches the spec's expected-results table 9/9.
| URN | Latest visit | Current grade |
|---|---|---|
| 102408 Rabbsfarm | graded 2025-06-17 | none |
| 151783 Acre Wood | graded 2024-10-01 | none |
| 139888 Washwood Heath | ungraded 2025-05-21 "Standards maintained" | Good, 2020-03-03, graded |
| 104762 Robins Lane | ungraded 2024-07-18 "School remains Good" | Good, 2024-07-18, confirmed |
| 100094 Royal Free Hospital Children's | ungraded 2025-02-05 "Some aspects not as strong" | Outstanding, 2019-10-09, graded |
| 136454 Oakgrove | ungraded 2024-11-13 "Standards maintained" | none |
| 137086 Bishop Stopford | graded 2025-04-01 | none |
| 110048 Willink | report card 2026-05-06 | none |
| 149612 St Michael's | report card 2026-02-10 | none |
Other checks:
- `dbt ls`: the daily DAG's selection does not include `fact_ofsted_latest`; the monthly Ofsted DAG's does.
- `python -m pytest backend/tests pipeline/tests scripts/ci/tests -q`: 328 passed.
## After merge
1. Run `school_data_monthly_ofsted` on staging.
2. Run on the staging database (expected results in comments):
```sql
select count(*) = count(distinct urn) from marts.fact_ofsted_latest; -- true
select urn, latest_visit_date, latest_visit_kind, latest_visit_outcome,
current_grade, current_grade_date, current_grade_basis
from marts.fact_ofsted_latest
where urn in (102408, 151783, 139888, 104762, 100094, 136454, 137086, 110048, 149612); -- the table above
select count(*) from marts.fact_ofsted_latest
where current_grade is not null and latest_visit_kind = 'graded'
and overall_effectiveness is null; -- 0
```
3. Do the same on production after promotion. PR 2 must not be merged before step 1, or promoted before step 3.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Design for audit findings C1 and M1. A grade carried forward from an older
ungraded visit is shown under a newer inspection's date ("Good · 2025" for
Rabbsfarm, whose 2025 inspection gave no grade), and "Inspected" dates show the
last graded inspection rather than the latest visit. The rule moves into
int_ofsted_latest and a new fact_ofsted_latest mart, shipped as a pipeline PR
and then a backend/UI PR.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
int_ofsted_latest needs to know which inspection came last. Report-card-only schools are no longer dropped (123 schools, audit H3); inspection_date stays for current readers and now falls back to the report-card date, so fact_ofsted_inspection's not_null test still holds.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
int_ofsted_latest now picks each school's latest visit (report card, graded or ungraded inspection) and the overall grade still in force, dated by the inspection that awarded or confirmed it. A 'School remains Good' from an older ungraded visit is no longer carried past a newer inspection that gave no grade (audit C1). Duplicate monthly rows resolve to the newest visit, so a newer report card always wins. dim_school's columns are unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The backend will read this instead of picking the latest row of fact_ofsted_inspection itself (twice, with arbitrary ties). Schema tests and assert_ofsted_current_grade_consistent pin its invariants. Built by the monthly Ofsted DAG (int_ofsted_latest+), not the daily one.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Pipeline-only change plus the spec and plan docs. It stops merging Ofsted's graded, ungraded and report-card dates in staging, and works out one current status per school in int_ofsted_latest. It also adds a new marts.fact_ofsted_latest mart, a dbt unit test for each rule row, and an invariant test. The change is additive: dim_school is untouched, the old inspection_date is kept, and the columns existing models read are preserved. I found nothing that would break production or corrupt data.
🟡 Minor
pipeline/transform/models/intermediate/int_ofsted_latest.sql: latest_visit_kind returns 'report_card' whenever rc_inspection_date is set, even if a graded or ungraded date is newer. That is correct only if report cards always postdate legacy inspections, which the comment states as an assumption. If Ofsted MI ever shows a newer legacy visit beside an older report card, current_grade would be suppressed and the wrong visit shown. The latest_visit_date coalesce has the same dependency. Comparing the dates directly, or adding an invariant test, would make it robust.
docs/superpowers/plans/2026-10-05-ofsted-current-status.md: The plan's Global Constraints and commit templates specify a Co-Authored-By: Claude Opus 5.5 trailer. This session's attribution reminder asks for Claude Sonnet 5.5, so commits made by following the plan literally would carry the wrong trailer.
pipeline/transform/models/staging/stg_ofsted_inspections.yml: The unit tests list only some of the source columns. They rely on dbt filling the missing columns with null from the existing relation, so they need the raw and staging relations to exist when they run. Fresh-environment dbt build ordering is untested. Verify this in the first staging Ofsted DAG run, as the plan already notes.
## 🤖 AI Code Review (Claude Code)
Pipeline-only change plus the spec and plan docs. It stops merging Ofsted's graded, ungraded and report-card dates in staging, and works out one current status per school in `int_ofsted_latest`. It also adds a new `marts.fact_ofsted_latest` mart, a dbt unit test for each rule row, and an invariant test. The change is additive: `dim_school` is untouched, the old `inspection_date` is kept, and the columns existing models read are preserved. I found nothing that would break production or corrupt data.
### 🟡 Minor
- **pipeline/transform/models/intermediate/int_ofsted_latest.sql**: `latest_visit_kind` returns 'report_card' whenever `rc_inspection_date` is set, even if a graded or ungraded date is newer. That is correct only if report cards always postdate legacy inspections, which the comment states as an assumption. If Ofsted MI ever shows a newer legacy visit beside an older report card, `current_grade` would be suppressed and the wrong visit shown. The `latest_visit_date` coalesce has the same dependency. Comparing the dates directly, or adding an invariant test, would make it robust.
- **docs/superpowers/plans/2026-10-05-ofsted-current-status.md**: The plan's Global Constraints and commit templates specify a `Co-Authored-By: Claude Opus 5.5` trailer. This session's attribution reminder asks for `Claude Sonnet 5.5`, so commits made by following the plan literally would carry the wrong trailer.
- **pipeline/transform/models/staging/stg_ofsted_inspections.yml**: The unit tests list only some of the source columns. They rely on dbt filling the missing columns with null from the existing relation, so they need the raw and staging relations to exist when they run. Fresh-environment `dbt build` ordering is untested. Verify this in the first staging Ofsted DAG run, as the plan already notes.
int_ofsted_latest treated any report card as the latest visit, relying on report cards postdating legacy inspections. That holds today (0 of 2,451 report-card schools in the 31 Aug 2026 MI) but is now not assumed: the latest visit is the newest of the three dates. A report card still leaves no legacy grade in force, which is the spec's rule rather than an inference. New unit test newer_legacy_visit_after_a_report_card failed RED on the old logic. Review on #183.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The plan hard-coded one model's Co-Authored-By trailer; an executor should use the one its own session specifies. The spec's report-card rule now matches int_ofsted_latest. Review on #183.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The diff adds a dbt rule that computes one current Ofsted status per school (int_ofsted_latest) and exposes it as the new marts.fact_ofsted_latest. It also keeps the graded, ungraded and report-card dates apart in staging, and adds unit tests, schema tests and an invariant test. The SQL is additive and consistent with the spec: dim_school is untouched, the old columns the backend and dim_school read are kept, and fact_ofsted_inspection.inspection_date stays non-null. I found no production-breaking, data-leaking or data-corrupting problems.
🟡 Minor
docs/superpowers/plans/2026-10-05-ofsted-current-status.md: The Task 3 SQL in the plan has drifted from the implemented model. The plan computes latest_visit_date as coalesce(rc_inspection_date, greatest(graded, ungraded)) and assigns latest_visit_kind and the report-card grade rule without the rc_inspection_date is not null then null branch. The shipped int_ofsted_latest.sql uses greatest(rc, graded, ungraded) and a same-day tie order. The plan's unit-test list also lacks the newer_legacy_visit_after_a_report_card case, which the plan's own SQL would fail. Anyone following the plan literally would get different behaviour, so update the plan to match the code.
## 🤖 AI Code Review (Claude Code)
The diff adds a dbt rule that computes one current Ofsted status per school (`int_ofsted_latest`) and exposes it as the new `marts.fact_ofsted_latest`. It also keeps the graded, ungraded and report-card dates apart in staging, and adds unit tests, schema tests and an invariant test. The SQL is additive and consistent with the spec: `dim_school` is untouched, the old columns the backend and `dim_school` read are kept, and `fact_ofsted_inspection.inspection_date` stays non-null. I found no production-breaking, data-leaking or data-corrupting problems.
### 🟡 Minor
- **docs/superpowers/plans/2026-10-05-ofsted-current-status.md**: The Task 3 SQL in the plan has drifted from the implemented model. The plan computes `latest_visit_date` as `coalesce(rc_inspection_date, greatest(graded, ungraded))` and assigns `latest_visit_kind` and the report-card grade rule without the `rc_inspection_date is not null then null` branch. The shipped `int_ofsted_latest.sql` uses `greatest(rc, graded, ungraded)` and a same-day tie order. The plan's unit-test list also lacks the `newer_legacy_visit_after_a_report_card` case, which the plan's own SQL would fail. Anyone following the plan literally would get different behaviour, so update the plan to match the code.
tudor
merged commit 3728a63275 into main2026-10-05 15:44:59 +00:00
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Why
Audit findings C1 and M1 (3 Oct 2026 accuracy audit). The site carries a grade forward from an older ungraded visit and shows it under a newer inspection's date. Rabbsfarm Primary (102408) reads "Good · 2025", although Ofsted's 17 June 2025 inspection gave no overall grade and rated three areas Requires Improvement. Site-wide that's 932 badges, 147 of them contradicted by the inspection they're dated with. "Inspected" dates also show the last graded inspection rather than the latest visit (667 schools).
Design:
docs/superpowers/specs/2026-10-05-ofsted-current-status-design.md. Plan:docs/superpowers/plans/2026-10-05-ofsted-current-status.md.This is PR 1 of 2 (pipeline, additive). PR 2 switches
dim_school, the backend and the UI to the new columns. It must wait until the Ofsted DAG has run with this PR.Change
stg_ofsted_inspections:inspection_datestays for current readers and now falls back to the report-card date, sofact_ofsted_inspection'snot_nulltest still holds.int_ofsted_latestnow holds the rule, one row per school:latest_visit_date,latest_visit_kind(report_card/graded/ungraded),latest_visit_outcome.current_grade,current_grade_date,current_grade_basis(graded/confirmed).dim_school's columns are unchanged.marts.fact_ofsted_latest: one row per URN, with schema tests andtests/assert_ofsted_current_grade_consistent.sql. Built by the monthly Ofsted DAG only.Visible effect of this PR alone
Report-card-only schools (e.g. St Michael's Catholic School, 149612) gain their report card. Nothing else on the site changes until PR 2.
Verification
dbt unit tests (11) cover every row of the spec's rule table, plus three edge cases: duplicate rows, a same-day graded and ungraded visit, and the "9" grade code. They ran locally against a throwaway Postgres (pgserver) with dbt-postgres 1.10. I also built the Ofsted models there from the real 31 Aug 2026 MI rows of the spec's nine example schools. Result: 25/25 nodes pass, and
fact_ofsted_latestmatches the spec's expected-results table 9/9.Other checks:
dbt ls: the daily DAG's selection does not includefact_ofsted_latest; the monthly Ofsted DAG's does.python -m pytest backend/tests pipeline/tests scripts/ci/tests -q: 328 passed.After merge
school_data_monthly_ofstedon staging.🤖 Generated with Claude Code
Design for audit findings C1 and M1. A grade carried forward from an older ungraded visit is shown under a newer inspection's date ("Good · 2025" for Rabbsfarm, whose 2025 inspection gave no grade), and "Inspected" dates show the last graded inspection rather than the latest visit. The rule moves into int_ofsted_latest and a new fact_ofsted_latest mart, shipped as a pipeline PR and then a backend/UI PR. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>🤖 AI Code Review (Claude Code)
Pipeline-only change plus the spec and plan docs. It stops merging Ofsted's graded, ungraded and report-card dates in staging, and works out one current status per school in
int_ofsted_latest. It also adds a newmarts.fact_ofsted_latestmart, a dbt unit test for each rule row, and an invariant test. The change is additive:dim_schoolis untouched, the oldinspection_dateis kept, and the columns existing models read are preserved. I found nothing that would break production or corrupt data.🟡 Minor
latest_visit_kindreturns 'report_card' wheneverrc_inspection_dateis set, even if a graded or ungraded date is newer. That is correct only if report cards always postdate legacy inspections, which the comment states as an assumption. If Ofsted MI ever shows a newer legacy visit beside an older report card,current_gradewould be suppressed and the wrong visit shown. Thelatest_visit_datecoalesce has the same dependency. Comparing the dates directly, or adding an invariant test, would make it robust.Co-Authored-By: Claude Opus 5.5trailer. This session's attribution reminder asks forClaude Sonnet 5.5, so commits made by following the plan literally would carry the wrong trailer.dbt buildordering is untested. Verify this in the first staging Ofsted DAG run, as the plan already notes.🤖 AI Code Review (Claude Code)
The diff adds a dbt rule that computes one current Ofsted status per school (
int_ofsted_latest) and exposes it as the newmarts.fact_ofsted_latest. It also keeps the graded, ungraded and report-card dates apart in staging, and adds unit tests, schema tests and an invariant test. The SQL is additive and consistent with the spec:dim_schoolis untouched, the old columns the backend anddim_schoolread are kept, andfact_ofsted_inspection.inspection_datestays non-null. I found no production-breaking, data-leaking or data-corrupting problems.🟡 Minor
latest_visit_dateascoalesce(rc_inspection_date, greatest(graded, ungraded))and assignslatest_visit_kindand the report-card grade rule without therc_inspection_date is not null then nullbranch. The shippedint_ofsted_latest.sqlusesgreatest(rc, graded, ungraded)and a same-day tie order. The plan's unit-test list also lacks thenewer_legacy_visit_after_a_report_cardcase, which the plan's own SQL would fail. Anyone following the plan literally would get different behaviour, so update the plan to match the code.