1594 lines
75 KiB
Markdown
1594 lines
75 KiB
Markdown
# Ofsted Current Status Implementation Plan
|
||
|
||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||
|
||
**Goal:** Compute one current Ofsted status per school in dbt and make every page read it, so no grade is shown under a date it was not awarded or confirmed on (C1), and "Inspected" always means the latest visit (M1).
|
||
|
||
**Architecture:** `stg_ofsted_inspections` stops merging the graded, ungraded and report-card dates; `int_ofsted_latest` applies the rule and a new `marts.fact_ofsted_latest` (one row per URN) exposes it. PR 1 ships the pipeline change only (additive). PR 2, after the Ofsted DAG has run, switches `dim_school`, the backend and the UI to the new columns.
|
||
|
||
**Tech Stack:** dbt-postgres ~1.10 (unit tests in YAML), FastAPI + pandas + SQLAlchemy, Next.js 15 / React / TypeScript, Jest + Testing Library, Playwright.
|
||
|
||
**Spec:** `docs/superpowers/specs/2026-10-05-ofsted-current-status-design.md`
|
||
|
||
## Global Constraints
|
||
|
||
- The rule table and expected results in the spec are the contract; dbt unit tests encode them row for row.
|
||
- PR 1 must not change `dim_school` and must keep `fact_ofsted_inspection.inspection_date` not null.
|
||
- PR 2 must not be merged before `school_data_monthly_ofsted` has run on staging with PR 1, nor promoted before it has run on production.
|
||
- No compatibility code in the backend for marts built before PR 1 (agreed in brainstorming).
|
||
- Copy follows the site's factual rule: print what Ofsted published; no derived summaries. Dates in prose use long months ("14 March 2023").
|
||
- Commits end with `Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>`; PR bodies end with the Claude Code line. Never push to `main`.
|
||
|
||
## Review Focus
|
||
|
||
1. A school with only an ungraded visit and an outcome that names no grade ("Standards maintained") must show "Inspected · year" and the outcome, never "Not rated" or a grade. Pinned in Task 3 (unit test `ungraded_only_standards_maintained`) and Task 9 (Jest).
|
||
2. A graded inspection and an ungraded visit on the same day must resolve to the graded one (spec tie order). Pinned in Task 3 (`same_day_graded_wins`).
|
||
3. Area grades with the MI sentinel 9 must never become a current grade. Pinned in Task 3 (`overall_sentinel_is_not_a_grade`).
|
||
4. A list row with a current grade but no `ofsted_grade_date` (should not happen, but nulls do) must still render a badge without a year rather than crash. Pinned in Task 8 (Jest).
|
||
5. Report-card schools must keep exactly today's display. Pinned in Task 3 (`report_card_wins`), Task 8 and the existing E2E report-card journey.
|
||
|
||
## File Map
|
||
|
||
**PR 1 — branch `fix/ofsted-current-status-pipeline`**
|
||
|
||
- Modify `pipeline/transform/models/staging/stg_ofsted_inspections.sql` — three separate dates, report-card-only rows kept, compatibility `inspection_date`.
|
||
- Create `pipeline/transform/models/staging/stg_ofsted_inspections.yml` — unit tests for the staging model.
|
||
- Modify `pipeline/transform/models/intermediate/int_ofsted_latest.sql` — the rule.
|
||
- Create `pipeline/transform/models/intermediate/int_ofsted_latest.yml` — unit tests, one per rule row.
|
||
- Create `pipeline/transform/models/marts/fact_ofsted_latest.sql` — the mart.
|
||
- Modify `pipeline/transform/models/marts/_marts_schema.yml` — schema tests for the mart.
|
||
- Create `pipeline/transform/tests/assert_ofsted_current_grade_consistent.sql` — invariants.
|
||
|
||
**PR 2 — branch `fix/ofsted-current-status-site` (from `main`)**
|
||
|
||
- Modify `pipeline/transform/models/marts/dim_school.sql` — `ofsted_grade`/`ofsted_date` from the status columns.
|
||
- Modify `backend/models.py` — `FactOfstedLatest`.
|
||
- Modify `backend/data_loader.py` — list query join, batch query, `_ofsted_block`.
|
||
- Modify `backend/schemas.py` — `ofsted_grade_date` in `SCHOOL_COLUMNS`.
|
||
- Modify `backend/app.py` — `_PUBLISHABLE_FIELDS`.
|
||
- Tests: `backend/tests/test_supplementary_enrichment.py`, `test_supplementary_batch.py`, `test_compare_enrichment.py`, new `backend/tests/test_ofsted_status_payload.py`.
|
||
- Modify `nextjs-app/lib/types.ts`, `nextjs-app/lib/utils.ts` (`buildOfstedListBadge`; delete `buildOfstedHeroChip`, `buildSchoolSummary`), `nextjs-app/lib/compareLogic.ts` (`ofstedDisplay`).
|
||
- Create `nextjs-app/lib/ofstedStatus.ts` — the two sentences both pages print.
|
||
- Modify `nextjs-app/components/school/OfstedSection.tsx`, `PrimarySchoolSections.tsx`, `SecondarySchoolSections.tsx`, `components/compare/CompareAtAGlance.tsx`, `components/compare/CompareOfsted.tsx`.
|
||
- Tests: `nextjs-app/__tests__/lib/utils.test.ts`, `__tests__/lib/compareLogic.test.ts`, new `__tests__/lib/ofstedStatus.test.ts`, new `__tests__/components/OfstedSection.test.tsx`, `__tests__/components/CompareOfsted.test.tsx`, `__tests__/support/schoolFixtures.ts`, `__tests__/components/schoolDetail.characterization.test.tsx`.
|
||
- Modify `e2e/tests/journeys.spec.ts` — Rabbsfarm and a confirmed-grade journey.
|
||
|
||
---
|
||
|
||
## PR 1 — pipeline
|
||
|
||
### Task 1: Local dbt harness
|
||
|
||
CI has no dbt. Unit tests run inside every `dbt build` on Airflow, but they must also pass locally before the PR. There is no Docker or Postgres on this machine; `pgserver` bundles Postgres binaries in a pip wheel.
|
||
|
||
**Files:** none in the repo. Work in `$SCRATCH/dbt_local` (the session scratchpad).
|
||
|
||
- [ ] **Step 1: Start a throwaway Postgres**
|
||
|
||
```bash
|
||
mkdir -p "$SCRATCH/dbt_local" && cd "$SCRATCH/dbt_local"
|
||
cat > start_pg.py <<'EOF'
|
||
import pgserver, sys, time
|
||
srv = pgserver.get_server(sys.argv[1], cleanup_mode=None)
|
||
print(srv.get_uri())
|
||
srv.psql("create database school_compare;") if "school_compare" not in srv.psql("select datname from pg_database;") else None
|
||
EOF
|
||
uv run --no-project --with pgserver python start_pg.py "$SCRATCH/dbt_local/pgdata"
|
||
```
|
||
|
||
Expected: a URI such as `postgresql://postgres:@/postgres?host=/…/pgdata`. Note the socket directory after `host=`.
|
||
|
||
- [ ] **Step 2: Create the raw source table with the tap's columns**
|
||
|
||
```bash
|
||
uv run --no-project --with pgserver python - <<'EOF'
|
||
import pgserver, os
|
||
srv = pgserver.get_server(os.environ["SCRATCH"] + "/dbt_local/pgdata", cleanup_mode=None)
|
||
cols = ["inspection_date","inspection_type","event_type_grouping","overall_effectiveness",
|
||
"quality_of_education","behaviour_and_attitudes","personal_development",
|
||
"effectiveness_of_leadership_and_management","early_years_provision","sixth_form_provision",
|
||
"ungraded_outcome","ungraded_inspection_date","rc_safeguarding_met","rc_inclusion",
|
||
"rc_curriculum_teaching","rc_achievement","rc_attendance_behaviour","rc_personal_development",
|
||
"rc_leadership_governance","rc_early_years","rc_sixth_form","rc_inspection_date","report_url"]
|
||
ddl = ", ".join(f'"{c}" text' for c in cols)
|
||
srv.psql(f'\\c school_compare\ncreate schema if not exists raw;\ndrop table if exists raw.ofsted_inspections;\ncreate table raw.ofsted_inspections (urn integer, {ddl});')
|
||
print("ok")
|
||
EOF
|
||
```
|
||
|
||
- [ ] **Step 3: Load the nine example schools from the 31 Aug 2026 MI**
|
||
|
||
Map MI columns to raw columns with the tap's own resolver so the fixture is exactly what production loads:
|
||
|
||
```bash
|
||
uv run --no-project --with ./pipeline/plugins/extractors/tap-uk-ofsted --with pgserver --with pandas python - <<'EOF'
|
||
import os, pandas as pd, pgserver
|
||
from tap_uk_ofsted.tap import OfstedInspectionsStream, TapUKOfsted
|
||
mi = pd.read_csv(os.environ["SCRATCH"] + "/audit/ofsted_mi_latest.csv", dtype=str, keep_default_na=False, encoding="cp1252")
|
||
stream = OfstedInspectionsStream(TapUKOfsted(config={}))
|
||
col_map = stream._resolve_columns(list(mi.columns))
|
||
urns = {"102408","151783","139888","104762","100094","136454","137086","110048","149612"}
|
||
rows = mi[mi["URN"].isin(urns)]
|
||
srv = pgserver.get_server(os.environ["SCRATCH"] + "/dbt_local/pgdata", cleanup_mode=None)
|
||
def q(v): return "NULL" if v is None else "'" + str(v).replace("'", "''") + "'"
|
||
stmts = []
|
||
for _, r in rows.iterrows():
|
||
rec = {f: r[c] for f, c in col_map.items() if f != "urn"}
|
||
names = ", ".join(f'"{k}"' for k in rec)
|
||
stmts.append(f'insert into raw.ofsted_inspections (urn, {names}) values ({int(r["URN"])}, {", ".join(q(v) for v in rec.values())});')
|
||
srv.psql("\\c school_compare\n" + "\n".join(stmts))
|
||
print(len(stmts), "rows")
|
||
EOF
|
||
```
|
||
|
||
Expected: `9 rows`.
|
||
|
||
- [ ] **Step 4: Run dbt against it with today's models (baseline)**
|
||
|
||
```bash
|
||
cd pipeline/transform
|
||
export PG_HOST=<socket dir from step 1> PG_PORT=5432 PG_USER=postgres PG_PASSWORD= PG_DATABASE=school_compare
|
||
uv run --no-project --with "dbt-postgres~=1.10.0" python -m dbt.cli.main run --profiles-dir . --target dev \
|
||
--target-path "$SCRATCH/dbt_local/target" --select stg_ofsted_inspections int_ofsted_latest fact_ofsted_inspection
|
||
```
|
||
|
||
Expected: 3 models OK. If `pgserver` cannot start on this machine, stop and report; the unit tests will then be proven only by the staging Ofsted DAG run, and the PR description must say so.
|
||
|
||
### Task 2: `stg_ofsted_inspections` keeps the three dates apart
|
||
|
||
**Files:**
|
||
- Modify: `pipeline/transform/models/staging/stg_ofsted_inspections.sql`
|
||
- Create: `pipeline/transform/models/staging/stg_ofsted_inspections.yml`
|
||
|
||
**Interfaces:**
|
||
- Produces columns used by Task 3: `graded_inspection_date date`, `ungraded_inspection_date date`, `rc_inspection_date date` (null unless the row carries report-card grades), `inspection_date date` (= coalesce of graded, ungraded, report card; never null), plus every existing column unchanged.
|
||
|
||
- [ ] **Step 1: Write the failing unit tests**
|
||
|
||
`pipeline/transform/models/staging/stg_ofsted_inspections.yml`:
|
||
|
||
```yaml
|
||
version: 2
|
||
|
||
unit_tests:
|
||
- name: stg_ofsted_keeps_the_three_dates_apart
|
||
description: The graded and ungraded dates must stay separate so the latest visit can be found.
|
||
model: stg_ofsted_inspections
|
||
given:
|
||
- input: source('raw', 'ofsted_inspections')
|
||
rows:
|
||
- {urn: 102408, inspection_date: '17/06/2025', ungraded_inspection_date: '06/02/2020', overall_effectiveness: 'Not judged', ungraded_outcome: 'School remains Good', rc_inspection_date: '17/06/2025', rc_inclusion: 'NULL'}
|
||
expect:
|
||
rows:
|
||
- {urn: 102408, graded_inspection_date: '2025-06-17', ungraded_inspection_date: '2020-02-06', rc_inspection_date: null, inspection_date: '2025-06-17', ungraded_grade: 2}
|
||
|
||
- name: stg_ofsted_keeps_report_card_only_rows
|
||
description: A school whose only inspection is a report card used to be dropped (audit H3).
|
||
model: stg_ofsted_inspections
|
||
given:
|
||
- input: source('raw', 'ofsted_inspections')
|
||
rows:
|
||
- {urn: 149612, inspection_date: 'NULL', ungraded_inspection_date: 'NULL', rc_inspection_date: '10/02/2026', rc_inclusion: 'Expected standard', rc_safeguarding_met: 'Met'}
|
||
- {urn: 1, inspection_date: 'NULL', ungraded_inspection_date: 'NULL', rc_inspection_date: 'NULL'}
|
||
expect:
|
||
rows:
|
||
- {urn: 149612, graded_inspection_date: null, ungraded_inspection_date: null, rc_inspection_date: '2026-02-10', inspection_date: '2026-02-10', rc_inclusion: 3}
|
||
```
|
||
|
||
- [ ] **Step 2: Run them to see them fail**
|
||
|
||
```bash
|
||
uv run --no-project --with "dbt-postgres~=1.10.0" python -m dbt.cli.main test --profiles-dir . --target dev \
|
||
--target-path "$SCRATCH/dbt_local/target" --select "stg_ofsted_inspections,test_type:unit"
|
||
```
|
||
|
||
Expected: FAIL (`graded_inspection_date` does not exist).
|
||
|
||
- [ ] **Step 3: Rewrite the model**
|
||
|
||
Replace the whole of `stg_ofsted_inspections.sql` with:
|
||
|
||
```sql
|
||
-- Staging model: Ofsted inspection records
|
||
-- Handles both OEIF (pre-Nov 2025) and Report Card (post-Nov 2025) frameworks.
|
||
--
|
||
-- Ofsted's MI carries up to three inspections per school: the latest graded
|
||
-- one, the latest ungraded one and the latest report card. Their dates stay
|
||
-- separate here so int_ofsted_latest can tell which came last and which one a
|
||
-- grade belongs to (docs/superpowers/specs/2026-10-05-ofsted-current-status-design.md).
|
||
|
||
with source as (
|
||
select * from {{ source('raw', 'ofsted_inspections') }}
|
||
),
|
||
|
||
renamed as (
|
||
select
|
||
cast(urn as integer) as urn,
|
||
to_date(nullif(trim(inspection_date), 'NULL'), 'DD/MM/YYYY') as graded_inspection_date,
|
||
to_date(nullif(trim(ungraded_inspection_date), 'NULL'), 'DD/MM/YYYY') as ungraded_inspection_date,
|
||
inspection_type,
|
||
event_type_grouping as framework,
|
||
|
||
-- OEIF grades (1-4 scale; 9 = not applicable)
|
||
{{ safe_numeric('overall_effectiveness') }}::integer as overall_effectiveness,
|
||
{{ safe_numeric('quality_of_education') }}::integer as quality_of_education,
|
||
{{ safe_numeric('behaviour_and_attitudes') }}::integer as behaviour_attitudes,
|
||
{{ safe_numeric('personal_development') }}::integer as personal_development,
|
||
{{ safe_numeric('effectiveness_of_leadership_and_management') }}::integer as leadership_management,
|
||
{{ safe_numeric('early_years_provision') }}::integer as early_years_provision,
|
||
{{ safe_numeric('sixth_form_provision') }}::integer as sixth_form_provision,
|
||
|
||
-- Ungraded (Section 8) inspection outcome — free text, plus the grade
|
||
-- it confirms ("School remains Good" → 2); null for outcomes that
|
||
-- name no grade.
|
||
nullif(trim(ungraded_outcome), 'NULL') as ungraded_outcome,
|
||
{{ parse_ungraded_outcome('ungraded_outcome') }}::integer as ungraded_grade,
|
||
|
||
-- Report Card fields (post-Nov 2025 framework), 5-point scale:
|
||
-- 1 Exceptional · 2 Strong standard · 3 Expected standard
|
||
-- · 4 Needs attention · 5 Urgent improvement
|
||
case lower(trim(nullif(rc_safeguarding_met, 'NULL')))
|
||
when 'met' then true
|
||
when 'not met' then false
|
||
end as rc_safeguarding_met,
|
||
{{ parse_report_card_grade('rc_inclusion') }}::integer as rc_inclusion,
|
||
{{ parse_report_card_grade('rc_curriculum_teaching') }}::integer as rc_curriculum_teaching,
|
||
{{ parse_report_card_grade('rc_achievement') }}::integer as rc_achievement,
|
||
{{ parse_report_card_grade('rc_attendance_behaviour') }}::integer as rc_attendance_behaviour,
|
||
{{ parse_report_card_grade('rc_personal_development') }}::integer as rc_personal_development,
|
||
{{ parse_report_card_grade('rc_leadership_governance') }}::integer as rc_leadership_governance,
|
||
{{ parse_report_card_grade('rc_early_years') }}::integer as rc_early_years,
|
||
{{ parse_report_card_grade('rc_sixth_form') }}::integer as rc_sixth_form,
|
||
|
||
-- Start date of the latest FULL inspection (the report-card
|
||
-- inspection in the renewed framework). Only kept when the row
|
||
-- carries report-card grades, because in legacy-format files this
|
||
-- column is the legacy inspection date.
|
||
to_date(nullif(trim(rc_inspection_date), 'NULL'), 'DD/MM/YYYY') as rc_inspection_date_raw,
|
||
|
||
nullif(trim(report_url), 'NULL') as report_url
|
||
from source
|
||
where urn is not null
|
||
),
|
||
|
||
dated as (
|
||
select
|
||
*,
|
||
case
|
||
when rc_safeguarding_met is not null
|
||
or rc_inclusion is not null
|
||
or rc_curriculum_teaching is not null
|
||
or rc_achievement is not null
|
||
or rc_attendance_behaviour is not null
|
||
or rc_personal_development is not null
|
||
or rc_leadership_governance is not null
|
||
then rc_inspection_date_raw
|
||
end as rc_inspection_date
|
||
from renamed
|
||
)
|
||
|
||
select
|
||
*,
|
||
-- For readers that predate the separate dates (fact_ofsted_inspection and
|
||
-- the backend until it reads fact_ofsted_latest). Never null below.
|
||
coalesce(graded_inspection_date, ungraded_inspection_date, rc_inspection_date) as inspection_date
|
||
from dated
|
||
where coalesce(graded_inspection_date, ungraded_inspection_date, rc_inspection_date) is not null
|
||
```
|
||
|
||
- [ ] **Step 4: Run the staging unit tests and the baseline models**
|
||
|
||
```bash
|
||
uv run --no-project --with "dbt-postgres~=1.10.0" python -m dbt.cli.main build --profiles-dir . --target dev \
|
||
--target-path "$SCRATCH/dbt_local/target" --select stg_ofsted_inspections int_ofsted_latest fact_ofsted_inspection
|
||
```
|
||
|
||
Expected: unit tests PASS; `fact_ofsted_inspection`'s `not_null` on `inspection_date` PASS; 9 rows in `staging.stg_ofsted_inspections` (St Michael's 149612 now present).
|
||
|
||
- [ ] **Step 5: Commit**
|
||
|
||
```bash
|
||
git add pipeline/transform/models/staging/stg_ofsted_inspections.sql pipeline/transform/models/staging/stg_ofsted_inspections.yml
|
||
git commit -m "feat(pipeline): keep Ofsted's graded, ungraded and report-card dates apart" -m "int_ofsted_latest needs to know which inspection came last. Report-card-only schools are no longer dropped; inspection_date stays for current readers and now falls back to the report-card date." -m "Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>"
|
||
```
|
||
|
||
### Task 3: `int_ofsted_latest` computes the current status
|
||
|
||
**Files:**
|
||
- Modify: `pipeline/transform/models/intermediate/int_ofsted_latest.sql`
|
||
- Create: `pipeline/transform/models/intermediate/int_ofsted_latest.yml`
|
||
|
||
**Interfaces:**
|
||
- Consumes: Task 2's columns.
|
||
- Produces (Task 4 and PR 2): `latest_visit_date date`, `latest_visit_kind text` (`report_card`|`graded`|`ungraded`), `latest_visit_outcome text`, `current_grade integer` (1–4 or null), `current_grade_date date`, `current_grade_basis text` (`graded`|`confirmed`), `graded_inspection_date`, `ungraded_inspection_date`, plus every column the old model selected.
|
||
|
||
- [ ] **Step 1: Write the failing unit tests** (one per spec rule row and Review Focus item)
|
||
|
||
`pipeline/transform/models/intermediate/int_ofsted_latest.yml`:
|
||
|
||
```yaml
|
||
version: 2
|
||
|
||
unit_tests:
|
||
- name: graded_not_judged_has_no_grade
|
||
description: Rabbsfarm (102408). The 2025 inspection gave no overall grade, so the 2020 "remains Good" is not carried forward (audit C1).
|
||
model: int_ofsted_latest
|
||
given:
|
||
- input: ref('stg_ofsted_inspections')
|
||
rows:
|
||
- {urn: 102408, graded_inspection_date: '2025-06-17', ungraded_inspection_date: '2020-02-06', overall_effectiveness: null, ungraded_grade: 2, ungraded_outcome: 'School remains Good'}
|
||
expect:
|
||
rows:
|
||
- {urn: 102408, latest_visit_date: '2025-06-17', latest_visit_kind: graded, latest_visit_outcome: null, current_grade: null, current_grade_date: null, current_grade_basis: null}
|
||
|
||
- name: graded_with_overall_grade
|
||
model: int_ofsted_latest
|
||
given:
|
||
- input: ref('stg_ofsted_inspections')
|
||
rows:
|
||
- {urn: 1, graded_inspection_date: '2019-06-01', overall_effectiveness: 2}
|
||
expect:
|
||
rows:
|
||
- {urn: 1, latest_visit_date: '2019-06-01', latest_visit_kind: graded, current_grade: 2, current_grade_date: '2019-06-01', current_grade_basis: graded}
|
||
|
||
- name: ungraded_remains_good_confirms_the_grade
|
||
description: Robins Lane (104762). Graded Good 2020, "School remains Good" July 2024 — Good, dated by the confirming visit.
|
||
model: int_ofsted_latest
|
||
given:
|
||
- input: ref('stg_ofsted_inspections')
|
||
rows:
|
||
- {urn: 104762, graded_inspection_date: '2020-01-07', ungraded_inspection_date: '2024-07-18', overall_effectiveness: 2, ungraded_grade: 2, ungraded_outcome: 'School remains Good'}
|
||
expect:
|
||
rows:
|
||
- {urn: 104762, latest_visit_date: '2024-07-18', latest_visit_kind: ungraded, latest_visit_outcome: 'School remains Good', current_grade: 2, current_grade_date: '2024-07-18', current_grade_basis: confirmed}
|
||
|
||
- name: post_2024_ungraded_keeps_graded_grade_with_its_own_date
|
||
description: Washwood Heath (139888). Graded Good 2020, "Standards maintained" May 2025 — Good, dated 2020; latest visit May 2025 (audit M1).
|
||
model: int_ofsted_latest
|
||
given:
|
||
- input: ref('stg_ofsted_inspections')
|
||
rows:
|
||
- {urn: 139888, graded_inspection_date: '2020-03-03', ungraded_inspection_date: '2025-05-21', overall_effectiveness: 2, ungraded_grade: null, ungraded_outcome: 'Standards maintained'}
|
||
expect:
|
||
rows:
|
||
- {urn: 139888, latest_visit_date: '2025-05-21', latest_visit_kind: ungraded, latest_visit_outcome: 'Standards maintained', current_grade: 2, current_grade_date: '2020-03-03', current_grade_basis: graded}
|
||
|
||
- name: ungraded_only_standards_maintained
|
||
description: Oakgrove (136454). Only an ungraded visit, outcome names no grade.
|
||
model: int_ofsted_latest
|
||
given:
|
||
- input: ref('stg_ofsted_inspections')
|
||
rows:
|
||
- {urn: 136454, ungraded_inspection_date: '2024-11-13', ungraded_grade: null, ungraded_outcome: 'Standards maintained'}
|
||
expect:
|
||
rows:
|
||
- {urn: 136454, latest_visit_date: '2024-11-13', latest_visit_kind: ungraded, latest_visit_outcome: 'Standards maintained', current_grade: null, current_grade_date: null, current_grade_basis: null}
|
||
|
||
- name: report_card_wins
|
||
description: The Willink School (110048). A report card is the latest visit and no legacy grade stays in force.
|
||
model: int_ofsted_latest
|
||
given:
|
||
- input: ref('stg_ofsted_inspections')
|
||
rows:
|
||
- {urn: 110048, ungraded_inspection_date: '2023-10-05', ungraded_grade: 2, ungraded_outcome: 'School remains Good', rc_inspection_date: '2026-05-06', rc_inclusion: 3}
|
||
expect:
|
||
rows:
|
||
- {urn: 110048, latest_visit_date: '2026-05-06', latest_visit_kind: report_card, latest_visit_outcome: null, current_grade: null, current_grade_date: null, current_grade_basis: null}
|
||
|
||
- name: duplicate_rows_newer_report_card_wins
|
||
description: Monthly loads can leave an older row beside a newer one for the same graded date; the row with the report card must win (audit H3).
|
||
model: int_ofsted_latest
|
||
given:
|
||
- input: ref('stg_ofsted_inspections')
|
||
rows:
|
||
- {urn: 138186, graded_inspection_date: '2023-06-13', overall_effectiveness: 3}
|
||
- {urn: 138186, graded_inspection_date: '2023-06-13', overall_effectiveness: 3, rc_inspection_date: '2026-06-02', rc_inclusion: 3}
|
||
expect:
|
||
rows:
|
||
- {urn: 138186, latest_visit_date: '2026-06-02', latest_visit_kind: report_card, current_grade: null}
|
||
|
||
- name: same_day_graded_wins
|
||
model: int_ofsted_latest
|
||
given:
|
||
- input: ref('stg_ofsted_inspections')
|
||
rows:
|
||
- {urn: 2, graded_inspection_date: '2024-03-01', ungraded_inspection_date: '2024-03-01', overall_effectiveness: 1, ungraded_grade: 2, ungraded_outcome: 'School remains Good'}
|
||
expect:
|
||
rows:
|
||
- {urn: 2, latest_visit_kind: graded, current_grade: 1, current_grade_basis: graded}
|
||
|
||
- name: overall_sentinel_is_not_a_grade
|
||
model: int_ofsted_latest
|
||
given:
|
||
- input: ref('stg_ofsted_inspections')
|
||
rows:
|
||
- {urn: 3, graded_inspection_date: '2018-05-01', overall_effectiveness: 9}
|
||
expect:
|
||
rows:
|
||
- {urn: 3, latest_visit_kind: graded, current_grade: null}
|
||
```
|
||
|
||
- [ ] **Step 2: Run them to see them fail**
|
||
|
||
```bash
|
||
uv run --no-project --with "dbt-postgres~=1.10.0" python -m dbt.cli.main test --profiles-dir . --target dev \
|
||
--target-path "$SCRATCH/dbt_local/target" --select "int_ofsted_latest,test_type:unit"
|
||
```
|
||
|
||
Expected: FAIL (`latest_visit_date` does not exist).
|
||
|
||
- [ ] **Step 3: Rewrite the model**
|
||
|
||
Replace the whole of `int_ofsted_latest.sql` with:
|
||
|
||
```sql
|
||
-- Intermediate model: the current Ofsted status per URN
|
||
-- One row per school: its latest visit (report card, graded or ungraded
|
||
-- inspection) and the overall grade still in force, if any. A grade is dated
|
||
-- by the inspection that awarded or confirmed it, never by a later visit.
|
||
-- Rule and examples: docs/superpowers/specs/2026-10-05-ofsted-current-status-design.md
|
||
|
||
with inspections as (
|
||
select
|
||
*,
|
||
-- Report cards began in Nov 2025, after the last legacy inspections,
|
||
-- so a school's report card is always its latest visit.
|
||
coalesce(
|
||
rc_inspection_date,
|
||
greatest(graded_inspection_date, ungraded_inspection_date)
|
||
) as latest_visit_date
|
||
from {{ ref('stg_ofsted_inspections') }}
|
||
),
|
||
|
||
ranked as (
|
||
select
|
||
*,
|
||
-- Monthly loads can leave several rows per school. The newest visit
|
||
-- wins; the tie-breaks keep the choice deterministic.
|
||
row_number() over (
|
||
partition by urn
|
||
order by latest_visit_date desc,
|
||
rc_inspection_date desc nulls last,
|
||
graded_inspection_date desc nulls last,
|
||
ungraded_inspection_date desc nulls last
|
||
) as rn
|
||
from inspections
|
||
),
|
||
|
||
latest as (
|
||
select
|
||
*,
|
||
case
|
||
when rc_inspection_date is not null then 'report_card'
|
||
when graded_inspection_date = latest_visit_date then 'graded'
|
||
else 'ungraded'
|
||
end as latest_visit_kind
|
||
from ranked
|
||
where rn = 1
|
||
),
|
||
|
||
graded as (
|
||
select
|
||
*,
|
||
-- The grade still in force. A graded inspection's own overall grade
|
||
-- (1-4; "Not judged" and the sentinel 9 are no grade); else an
|
||
-- ungraded visit's "School remains X"; else, after an ungraded visit
|
||
-- that names no grade, the graded inspection's grade.
|
||
case
|
||
when latest_visit_kind = 'graded' and overall_effectiveness between 1 and 4
|
||
then 'graded_latest'
|
||
when latest_visit_kind = 'ungraded' and ungraded_grade is not null
|
||
then 'confirmed'
|
||
when latest_visit_kind = 'ungraded' and overall_effectiveness between 1 and 4
|
||
then 'graded_earlier'
|
||
end as grade_case
|
||
from latest
|
||
)
|
||
|
||
select
|
||
urn,
|
||
latest_visit_date,
|
||
latest_visit_kind,
|
||
case when latest_visit_kind = 'ungraded' then ungraded_outcome end as latest_visit_outcome,
|
||
case grade_case
|
||
when 'confirmed' then ungraded_grade
|
||
when 'graded_latest' then overall_effectiveness
|
||
when 'graded_earlier' then overall_effectiveness
|
||
end as current_grade,
|
||
case grade_case
|
||
when 'confirmed' then ungraded_inspection_date
|
||
when 'graded_latest' then graded_inspection_date
|
||
when 'graded_earlier' then graded_inspection_date
|
||
end as current_grade_date,
|
||
case grade_case
|
||
when 'confirmed' then 'confirmed'
|
||
when 'graded_latest' then 'graded'
|
||
when 'graded_earlier' then 'graded'
|
||
end as current_grade_basis,
|
||
graded_inspection_date,
|
||
ungraded_inspection_date,
|
||
inspection_date,
|
||
inspection_type,
|
||
framework,
|
||
overall_effectiveness,
|
||
quality_of_education,
|
||
behaviour_attitudes,
|
||
personal_development,
|
||
leadership_management,
|
||
early_years_provision,
|
||
sixth_form_provision,
|
||
ungraded_outcome,
|
||
ungraded_grade,
|
||
rc_safeguarding_met,
|
||
rc_inclusion,
|
||
rc_curriculum_teaching,
|
||
rc_achievement,
|
||
rc_attendance_behaviour,
|
||
rc_personal_development,
|
||
rc_leadership_governance,
|
||
rc_early_years,
|
||
rc_sixth_form,
|
||
rc_inspection_date,
|
||
report_url
|
||
from graded
|
||
```
|
||
|
||
Note `dim_school` still reads `overall_effectiveness`, `ungraded_grade`, `inspection_date` and `framework` from this model; all are kept.
|
||
|
||
- [ ] **Step 4: Run the unit tests**
|
||
|
||
Same command as Step 2. Expected: 9 PASS.
|
||
|
||
- [ ] **Step 5: Commit** — `feat(pipeline): work out each school's current Ofsted status once`
|
||
|
||
### Task 4: `fact_ofsted_latest` mart, schema tests and invariants
|
||
|
||
**Files:**
|
||
- Create: `pipeline/transform/models/marts/fact_ofsted_latest.sql`
|
||
- Modify: `pipeline/transform/models/marts/_marts_schema.yml` (after `fact_ofsted_inspection`)
|
||
- Create: `pipeline/transform/tests/assert_ofsted_current_grade_consistent.sql`
|
||
|
||
**Interfaces:** Produces `marts.fact_ofsted_latest`, one row per URN, with every column `int_ofsted_latest` selects. PR 2's `FactOfstedLatest` ORM model mirrors it.
|
||
|
||
- [ ] **Step 1: Write the schema tests and the invariant test**
|
||
|
||
Append to `_marts_schema.yml` after the `fact_ofsted_inspection` entry:
|
||
|
||
```yaml
|
||
- name: fact_ofsted_latest
|
||
description: >
|
||
Current Ofsted status, one row per URN: the latest visit and the overall
|
||
grade still in force. See int_ofsted_latest for the rule.
|
||
columns:
|
||
- name: urn
|
||
tests: [not_null, unique]
|
||
- name: latest_visit_date
|
||
tests: [not_null]
|
||
- name: latest_visit_kind
|
||
tests:
|
||
- not_null
|
||
- accepted_values:
|
||
values: ['report_card', 'graded', 'ungraded']
|
||
- name: current_grade
|
||
tests:
|
||
- accepted_values:
|
||
values: [1, 2, 3, 4]
|
||
quote: false
|
||
- name: current_grade_basis
|
||
tests:
|
||
- accepted_values:
|
||
values: ['graded', 'confirmed']
|
||
```
|
||
|
||
`tests/assert_ofsted_current_grade_consistent.sql`:
|
||
|
||
```sql
|
||
-- A grade is dated by the inspection that awarded or confirmed it, never
|
||
-- after the latest visit, and has a date and a basis exactly when it exists.
|
||
-- A report card leaves no legacy grade in force.
|
||
|
||
select urn
|
||
from {{ ref('fact_ofsted_latest') }}
|
||
where current_grade_date > latest_visit_date
|
||
or (current_grade is null) <> (current_grade_date is null)
|
||
or (current_grade is null) <> (current_grade_basis is null)
|
||
or (latest_visit_kind = 'report_card' and current_grade is not null)
|
||
```
|
||
|
||
- [ ] **Step 2: Run them to see them fail** — `dbt test --select fact_ofsted_latest assert_ofsted_current_grade_consistent` → fails: model missing.
|
||
|
||
- [ ] **Step 3: Create the mart**
|
||
|
||
`fact_ofsted_latest.sql`:
|
||
|
||
```sql
|
||
-- Mart: current Ofsted status — one row per URN
|
||
-- The backend reads this instead of choosing the latest row of
|
||
-- fact_ofsted_inspection itself. The rule lives in int_ofsted_latest.
|
||
|
||
select
|
||
urn,
|
||
latest_visit_date,
|
||
latest_visit_kind,
|
||
latest_visit_outcome,
|
||
current_grade,
|
||
current_grade_date,
|
||
current_grade_basis,
|
||
graded_inspection_date,
|
||
ungraded_inspection_date,
|
||
rc_inspection_date,
|
||
inspection_type,
|
||
framework,
|
||
overall_effectiveness,
|
||
quality_of_education,
|
||
behaviour_attitudes,
|
||
personal_development,
|
||
leadership_management,
|
||
early_years_provision,
|
||
sixth_form_provision,
|
||
ungraded_outcome,
|
||
ungraded_grade,
|
||
rc_safeguarding_met,
|
||
rc_inclusion,
|
||
rc_curriculum_teaching,
|
||
rc_achievement,
|
||
rc_attendance_behaviour,
|
||
rc_personal_development,
|
||
rc_leadership_governance,
|
||
rc_early_years,
|
||
rc_sixth_form,
|
||
report_url
|
||
from {{ ref('int_ofsted_latest') }}
|
||
```
|
||
|
||
- [ ] **Step 4: Build the Ofsted models end to end on the nine real schools**
|
||
|
||
```bash
|
||
uv run --no-project --with "dbt-postgres~=1.10.0" python -m dbt.cli.main build --profiles-dir . --target dev \
|
||
--target-path "$SCRATCH/dbt_local/target" \
|
||
--select stg_ofsted_inspections int_ofsted_latest fact_ofsted_inspection fact_ofsted_latest assert_ofsted_current_grade_consistent
|
||
```
|
||
|
||
Then run the spec's verification SQL against the local database and compare row for row with the spec's "Expected results" table:
|
||
|
||
```sql
|
||
select urn, latest_visit_date, latest_visit_kind, latest_visit_outcome,
|
||
current_grade, current_grade_date, current_grade_basis
|
||
from marts.fact_ofsted_latest order by urn;
|
||
```
|
||
|
||
Expected (from the spec): 100094 ungraded 2025-02-05 "Some aspects not as strong" / 1, 2019-10-09, graded · 102408 graded 2025-06-17 / null · 104762 ungraded 2024-07-18 / 2, 2024-07-18, confirmed · 110048 report_card 2026-05-06 / null · 136454 ungraded 2024-11-13 / null · 137086 graded 2025-04-01 / null · 139888 ungraded 2025-05-21 / 2, 2020-03-03, graded · 149612 report_card 2026-02-10 / null · 151783 graded 2024-10-01 / null.
|
||
|
||
- [ ] **Step 5: Check the selector rule from PR #181 still holds**
|
||
|
||
`fact_ofsted_latest` refs only `int_ofsted_latest`, so the monthly Ofsted build (`int_ofsted_latest+`) builds it and the daily build does not. If PR #181 has merged, run `python -m pytest pipeline/tests/test_dag_selectors.py -q` on a branch that contains both; otherwise confirm with:
|
||
|
||
```bash
|
||
uv run --no-project --with "dbt-postgres~=1.10.0" python -m dbt.cli.main ls --profiles-dir . --target dev --resource-type model \
|
||
--select stg_gias_establishments+ stg_gias_links+ gias_code_names+ --exclude int_ks2_with_lineage+ int_ks4_with_lineage+ | grep -c fact_ofsted_latest
|
||
```
|
||
|
||
Expected: `0`.
|
||
|
||
- [ ] **Step 6: Commit** — `feat(pipeline): add fact_ofsted_latest, one current Ofsted status per school`
|
||
|
||
### Task 5: Open PR 1
|
||
|
||
- [ ] **Step 1:** `python -m pytest backend/tests pipeline/tests scripts/ci/tests -q` (CI's command, via `uv run --no-project --with-requirements requirements.txt --with pytest --with "httpx<0.28" --with pyyaml`). Expected: all pass (no Python changed).
|
||
- [ ] **Step 2:** `git status --short` — only the intended files; push; open the PR via the Gitea API (basic auth from `git credential fill`). The body states: what changes, the one visible effect (report-card-only schools), the local unit-test and nine-school results, and the post-merge steps — run `school_data_monthly_ofsted` on staging, then the spec's verification SQL — and that PR 2 depends on that run.
|
||
|
||
---
|
||
|
||
## PR 2 — backend and UI
|
||
|
||
Branch `fix/ofsted-current-status-site` from `origin/main`. It does not need PR 1's code to build or test (tests use fixtures), but must not merge until the staging Ofsted DAG has run with PR 1.
|
||
|
||
### Task 6: `dim_school` reads the status
|
||
|
||
**Files:** Modify `pipeline/transform/models/marts/dim_school.sql:74-84`.
|
||
|
||
- [ ] **Step 1: Change the Ofsted columns**
|
||
|
||
```sql
|
||
-- Latest Ofsted (populated after monthly Ofsted pipeline runs). The grade
|
||
-- still in force and the latest visit, from int_ofsted_latest — never a
|
||
-- grade carried past a newer inspection.
|
||
{% if ofsted_relation is not none %}
|
||
o.current_grade as ofsted_grade,
|
||
o.latest_visit_date as ofsted_date,
|
||
o.framework as ofsted_framework
|
||
{% else %}
|
||
```
|
||
|
||
- [ ] **Step 2: Verify locally** — with the Task 1 harness plus a GIAS fixture is heavy; instead compile and check the SQL: `dbt compile --select dim_school` and confirm the compiled query selects `o.current_grade`. The behaviour is covered by Task 3's unit tests (same columns) and by the Task 13 staging check (Typesense rating for Rabbsfarm empty).
|
||
- [ ] **Step 3: Commit** — `fix(pipeline): dim_school's Ofsted grade is the one still in force`
|
||
|
||
### Task 7: Backend reads `fact_ofsted_latest`
|
||
|
||
**Files:**
|
||
- Modify: `backend/models.py` (add `FactOfstedLatest` after `FactOfstedInspection`)
|
||
- Modify: `backend/data_loader.py:21`, `:254-257`, `:336-352`, `:734-787`, `:1100-1115`
|
||
- Modify: `backend/schemas.py:576-579`
|
||
- Modify: `backend/app.py:101`
|
||
- Test: `backend/tests/test_supplementary_enrichment.py`, `backend/tests/test_supplementary_batch.py`, `backend/tests/test_compare_enrichment.py`, create `backend/tests/test_ofsted_status_payload.py`
|
||
|
||
**Interfaces:**
|
||
- Consumes: `marts.fact_ofsted_latest` (Task 4 columns).
|
||
- Produces (Task 8 onward): list rows carry `ofsted_grade` (int|null), `ofsted_grade_date` (ISO date|null), `ofsted_date` (latest visit, ISO|null), `ofsted_framework`, `ofsted_rc_date`. The `ofsted` block carries `current_grade: {grade, date, basis} | None`, `latest_visit: {date, kind, outcome} | None`, `overall_effectiveness` (graded inspection's own, 1–4 or None), `inspection_date` (graded inspection's date), all existing area and report-card keys, `report_card`, `ofsted_page_url`; no `grade_source`.
|
||
|
||
- [ ] **Step 1: Write the failing tests**
|
||
|
||
Replace the row helper and the grade-source test in `test_supplementary_enrichment.py`:
|
||
|
||
```python
|
||
def _row(**kw):
|
||
base = dict(
|
||
framework="RC", inspection_type=None,
|
||
graded_inspection_date=None, ungraded_inspection_date=None, rc_inspection_date=None,
|
||
latest_visit_date=None, latest_visit_kind=None, latest_visit_outcome=None,
|
||
current_grade=None, current_grade_date=None, current_grade_basis=None,
|
||
overall_effectiveness=None, quality_of_education=None,
|
||
behaviour_attitudes=None, personal_development=None,
|
||
leadership_management=None, early_years_provision=None,
|
||
sixth_form_provision=None, ungraded_outcome=None, ungraded_grade=None,
|
||
rc_safeguarding_met=None, rc_inclusion=None, rc_curriculum_teaching=None,
|
||
rc_achievement=None, rc_attendance_behaviour=None,
|
||
rc_personal_development=None, rc_leadership_governance=None,
|
||
rc_early_years=None, rc_sixth_form=None, report_url=None,
|
||
)
|
||
base.update(kw)
|
||
return types.SimpleNamespace(**base)
|
||
|
||
|
||
def test_no_grade_is_carried_past_a_newer_inspection():
|
||
# Rabbsfarm (102408): the 2025 inspection gave no overall grade.
|
||
block = _ofsted_block(_row(
|
||
graded_inspection_date=date(2025, 6, 17), ungraded_inspection_date=date(2020, 2, 6),
|
||
latest_visit_date=date(2025, 6, 17), latest_visit_kind="graded",
|
||
ungraded_grade=2, ungraded_outcome="School remains Good", quality_of_education=3,
|
||
), urn=102408)
|
||
assert block["current_grade"] is None
|
||
assert block["overall_effectiveness"] is None
|
||
assert block["latest_visit"] == {"date": "2025-06-17", "kind": "graded", "outcome": None}
|
||
assert block["inspection_date"] == "2025-06-17"
|
||
assert "grade_source" not in block
|
||
|
||
|
||
def test_confirmed_grade_is_dated_by_the_confirming_visit():
|
||
block = _ofsted_block(_row(
|
||
graded_inspection_date=date(2020, 1, 7), ungraded_inspection_date=date(2024, 7, 18),
|
||
latest_visit_date=date(2024, 7, 18), latest_visit_kind="ungraded",
|
||
latest_visit_outcome="School remains Good", overall_effectiveness=2,
|
||
current_grade=2, current_grade_date=date(2024, 7, 18), current_grade_basis="confirmed",
|
||
), urn=104762)
|
||
assert block["current_grade"] == {"grade": 2, "date": "2024-07-18", "basis": "confirmed"}
|
||
assert block["overall_effectiveness"] == 2
|
||
assert block["inspection_date"] == "2020-01-07"
|
||
|
||
|
||
def test_overall_sentinel_is_not_served_as_a_grade():
|
||
assert _ofsted_block(_row(overall_effectiveness=9), urn=1)["overall_effectiveness"] is None
|
||
```
|
||
|
||
Delete `test_grade_source_graded_vs_carried_forward`. In `test_ofsted_block_carries_rc_inspection_date` and `test_ofsted_block_rc_inspection_date_none_when_absent`, replace `inspection_date=` with `graded_inspection_date=` and remove `ungraded_grade=2`. In `test_ofsted_block_keeps_existing_keys`, drop `"grade_source"` from the key list if present and add `"current_grade", "latest_visit"`.
|
||
|
||
In `test_supplementary_batch.py`, rename the table key `"FactOfstedInspection"` to `"FactOfstedLatest"` in both the rows dict and the expected `session.queries` list, and rewrite `_ofsted_row` (one row per URN now; the mart already chose):
|
||
|
||
```python
|
||
def _ofsted_row(urn, date, oe):
|
||
base = {f: None for f in (
|
||
"framework", "inspection_type", "quality_of_education", "behaviour_attitudes",
|
||
"personal_development", "leadership_management", "early_years_provision",
|
||
"sixth_form_provision", "ungraded_outcome", "ungraded_grade",
|
||
"ungraded_inspection_date", "rc_inspection_date", "latest_visit_outcome",
|
||
"rc_safeguarding_met", "rc_inclusion", "rc_curriculum_teaching", "rc_achievement",
|
||
"rc_attendance_behaviour", "rc_personal_development", "rc_leadership_governance",
|
||
"rc_early_years", "rc_sixth_form", "report_url",
|
||
)}
|
||
when = types.SimpleNamespace(isoformat=lambda: date)
|
||
base.update(urn=urn, graded_inspection_date=when, latest_visit_date=when,
|
||
latest_visit_kind="graded", overall_effectiveness=oe,
|
||
current_grade=oe, current_grade_date=when, current_grade_basis="graded")
|
||
return types.SimpleNamespace(**base)
|
||
```
|
||
|
||
and give URN 1 a single row (`_ofsted_row(1, "2023-01-01", 2)`); the "keep the latest" comment becomes "one row per URN".
|
||
|
||
In `test_compare_enrichment.py:15`, replace `"grade_source": "graded"` with `"current_grade": {"grade": 2, "date": "2023-01-01", "basis": "graded"}` and the assertion at `:89` with `assert school["ofsted"]["current_grade"]["basis"] == "graded"`.
|
||
|
||
Create `backend/tests/test_ofsted_status_payload.py`:
|
||
|
||
```python
|
||
"""The search badge reads ofsted_grade, ofsted_grade_date and ofsted_date from
|
||
list rows (nextjs-app/lib/utils.ts buildOfstedListBadge). A field the list
|
||
never sends would silently fall back to "Not yet inspected"."""
|
||
|
||
import numpy as np
|
||
import pandas as pd
|
||
import pytest
|
||
from fastapi.testclient import TestClient
|
||
|
||
from backend.schemas import SCHOOL_COLUMNS
|
||
|
||
|
||
def test_list_columns_include_the_status_fields():
|
||
for field in ("ofsted_grade", "ofsted_grade_date", "ofsted_date", "ofsted_rc_date"):
|
||
assert field in SCHOOL_COLUMNS
|
||
|
||
|
||
def _df():
|
||
return pd.DataFrame([{
|
||
"urn": 102408, "school_name": "Rabbsfarm Primary School", "phase": "Primary",
|
||
"school_type": "Community school", "local_authority": "Hillingdon",
|
||
"address": "Gordon Road, Yiewsley, UB7 8AH", "postcode": "UB7 8AH",
|
||
"latitude": 51.51, "longitude": -0.47, "year": 202425, "rwm_expected_pct": 58.0,
|
||
"total_pupils": 60, "gias_total_pupils": 616,
|
||
"ofsted_grade": np.nan, "ofsted_grade_date": None, "ofsted_date": "2025-06-17",
|
||
"ofsted_framework": "Schools - S5", "ofsted_rc_date": None,
|
||
}])
|
||
|
||
|
||
@pytest.fixture()
|
||
def client(monkeypatch):
|
||
from backend import app as app_module
|
||
monkeypatch.setattr(app_module, "load_school_data", _df)
|
||
monkeypatch.setattr(app_module, "load_latest_school_data", _df)
|
||
return TestClient(app_module.app)
|
||
|
||
|
||
def test_search_rows_carry_the_status_fields(client):
|
||
row = client.get("/api/schools?search=Rabbsfarm").json()["schools"][0]
|
||
assert row["ofsted_grade"] is None
|
||
assert row["ofsted_date"] == "2025-06-17"
|
||
assert "ofsted_grade_date" in row
|
||
|
||
|
||
def test_a_school_with_only_an_inspection_is_publishable():
|
||
from backend.app import _has_publishable_data
|
||
assert _has_publishable_data({"rwm_expected_pct": None, "attainment_8_score": None,
|
||
"ofsted_grade": None, "ofsted_date": "2025-06-17"})
|
||
```
|
||
|
||
If `/api/schools?search=` needs Typesense in tests, mirror the client fixture in `test_school_page_flag_fields.py` (it patches the same loaders and works without Typesense); adjust the query to whatever that file uses.
|
||
|
||
- [ ] **Step 2: Run them to see them fail**
|
||
|
||
`uv run --no-project --with-requirements requirements.txt --with pytest --with "httpx<0.28" --with pyyaml python -m pytest backend/tests/test_supplementary_enrichment.py backend/tests/test_supplementary_batch.py backend/tests/test_compare_enrichment.py backend/tests/test_ofsted_status_payload.py -q` → failures on `current_grade`, `FactOfstedLatest`, `ofsted_grade_date`.
|
||
|
||
- [ ] **Step 3: Implement**
|
||
|
||
`backend/models.py`, after `FactOfstedInspection`:
|
||
|
||
```python
|
||
class FactOfstedLatest(Base):
|
||
"""Current Ofsted status — one row per URN (pipeline: int_ofsted_latest).
|
||
|
||
`current_grade` is the overall grade still in force, dated by the
|
||
inspection that awarded or confirmed it; `latest_visit_*` is the school's
|
||
most recent inspection of any kind.
|
||
"""
|
||
__tablename__ = "fact_ofsted_latest"
|
||
__table_args__ = MARTS
|
||
|
||
urn = Column(Integer, primary_key=True)
|
||
latest_visit_date = Column(Date)
|
||
latest_visit_kind = Column(String(20))
|
||
latest_visit_outcome = Column(String(100))
|
||
current_grade = Column(Integer)
|
||
current_grade_date = Column(Date)
|
||
current_grade_basis = Column(String(20))
|
||
graded_inspection_date = Column(Date)
|
||
ungraded_inspection_date = Column(Date)
|
||
rc_inspection_date = Column(Date)
|
||
inspection_type = Column(String(100))
|
||
framework = Column(String(20))
|
||
overall_effectiveness = Column(Integer)
|
||
quality_of_education = Column(Integer)
|
||
behaviour_attitudes = Column(Integer)
|
||
personal_development = Column(Integer)
|
||
leadership_management = Column(Integer)
|
||
early_years_provision = Column(Integer)
|
||
sixth_form_provision = Column(Integer)
|
||
ungraded_outcome = Column(String(100))
|
||
ungraded_grade = Column(Integer)
|
||
rc_safeguarding_met = Column(Boolean)
|
||
rc_inclusion = Column(Integer)
|
||
rc_curriculum_teaching = Column(Integer)
|
||
rc_achievement = Column(Integer)
|
||
rc_attendance_behaviour = Column(Integer)
|
||
rc_personal_development = Column(Integer)
|
||
rc_leadership_governance = Column(Integer)
|
||
rc_early_years = Column(Integer)
|
||
rc_sixth_form = Column(Integer)
|
||
report_url = Column(Text)
|
||
```
|
||
|
||
`backend/data_loader.py`:
|
||
- import `FactOfstedLatest` alongside `FactOfstedInspection` (drop `FactOfstedInspection` from the import if nothing else uses it).
|
||
- In `_MAIN_QUERY` select list replace `foi.ofsted_grade, foi.ofsted_date, foi.ofsted_framework, foi.ofsted_rc_date,` with
|
||
|
||
```sql
|
||
foi.current_grade AS ofsted_grade,
|
||
foi.current_grade_date AS ofsted_grade_date,
|
||
foi.latest_visit_date AS ofsted_date,
|
||
foi.framework AS ofsted_framework,
|
||
foi.rc_inspection_date AS ofsted_rc_date,
|
||
```
|
||
|
||
- Replace the `LEFT JOIN (SELECT DISTINCT ON (urn) … FROM marts.fact_ofsted_inspection ORDER BY …) foi ON s.urn = foi.urn` block with `LEFT JOIN marts.fact_ofsted_latest foi ON s.urn = foi.urn`.
|
||
- Replace `_ofsted_block` with:
|
||
|
||
```python
|
||
def _iso(d):
|
||
return d.isoformat() if d else None
|
||
|
||
|
||
def _ofsted_block(o, urn: int) -> dict:
|
||
"""Serialize a fact_ofsted_latest row for API responses.
|
||
|
||
`current_grade` is the overall grade still in force, dated by the
|
||
inspection that awarded or confirmed it; `latest_visit` is the school's
|
||
most recent inspection of any kind. The rule lives in int_ofsted_latest
|
||
(docs/superpowers/specs/2026-10-05-ofsted-current-status-design.md).
|
||
`overall_effectiveness` and `inspection_date` describe the graded
|
||
inspection itself and label its area judgements. `report_card` holds the
|
||
renewed-framework (Nov 2025) area judgements; safeguarding is a separate
|
||
boolean and never appears among the graded areas.
|
||
"""
|
||
block = {
|
||
"framework": o.framework,
|
||
"inspection_date": _iso(o.graded_inspection_date),
|
||
"rc_inspection_date": _iso(o.rc_inspection_date),
|
||
"inspection_type": o.inspection_type,
|
||
"overall_effectiveness": o.overall_effectiveness if o.overall_effectiveness in (1, 2, 3, 4) else None,
|
||
"current_grade": (
|
||
{"grade": o.current_grade, "date": _iso(o.current_grade_date), "basis": o.current_grade_basis}
|
||
if o.current_grade is not None else None
|
||
),
|
||
"latest_visit": (
|
||
{"date": _iso(o.latest_visit_date), "kind": o.latest_visit_kind, "outcome": o.latest_visit_outcome}
|
||
if o.latest_visit_date else None
|
||
),
|
||
"quality_of_education": o.quality_of_education,
|
||
"behaviour_attitudes": o.behaviour_attitudes,
|
||
"personal_development": o.personal_development,
|
||
"leadership_management": o.leadership_management,
|
||
"early_years_provision": o.early_years_provision,
|
||
"sixth_form_provision": o.sixth_form_provision,
|
||
"previous_overall": None, # Not available in new schema
|
||
"rc_safeguarding_met": o.rc_safeguarding_met,
|
||
"rc_inclusion": o.rc_inclusion,
|
||
"rc_curriculum_teaching": o.rc_curriculum_teaching,
|
||
"rc_achievement": o.rc_achievement,
|
||
"rc_attendance_behaviour": o.rc_attendance_behaviour,
|
||
"rc_personal_development": o.rc_personal_development,
|
||
"rc_leadership_governance": o.rc_leadership_governance,
|
||
"rc_early_years": o.rc_early_years,
|
||
"rc_sixth_form": o.rc_sixth_form,
|
||
"report_url": o.report_url,
|
||
"ofsted_page_url": ofsted_page_url(urn),
|
||
}
|
||
block["report_card"] = report_card_labels(block)
|
||
return block
|
||
```
|
||
|
||
- In `get_supplementary_data_batch`, replace the `_ofsted` body with:
|
||
|
||
```python
|
||
# Ofsted — the mart already holds one current status per URN.
|
||
def _ofsted():
|
||
rows = (
|
||
db.query(FactOfstedLatest)
|
||
.filter(FactOfstedLatest.urn.in_(urns))
|
||
.all()
|
||
)
|
||
for o in rows:
|
||
result[o.urn]["ofsted"] = _ofsted_block(o, o.urn)
|
||
```
|
||
|
||
`backend/schemas.py`: add `"ofsted_grade_date",` after `"ofsted_grade",` in `SCHOOL_COLUMNS`.
|
||
|
||
`backend/app.py:101`: `_PUBLISHABLE_FIELDS = ("rwm_expected_pct", "attainment_8_score", "ofsted_grade", "ofsted_date")` and add to the comment above it: "An inspection with no overall grade still gives a page something to state."
|
||
|
||
Grep for any other reader of `grade_source` or `ungraded_carried_forward` in `backend/` and remove it.
|
||
|
||
- [ ] **Step 4: Run the backend suite**
|
||
|
||
`uv run --no-project --with-requirements requirements.txt --with pytest --with "httpx<0.28" --with pyyaml python -m pytest backend/tests -q` → all pass.
|
||
|
||
- [ ] **Step 5: Commit** — `fix(backend): serve the current Ofsted status from fact_ofsted_latest`
|
||
|
||
### Task 8: Types, badge and compare display
|
||
|
||
**Files:**
|
||
- Modify: `nextjs-app/lib/types.ts:78-84`, `:91-128`
|
||
- Modify: `nextjs-app/lib/utils.ts` (`buildOfstedListBadge`)
|
||
- Modify: `nextjs-app/lib/compareLogic.ts:100-131`
|
||
- Create: `nextjs-app/lib/ofstedStatus.ts`
|
||
- Test: `nextjs-app/__tests__/lib/utils.test.ts`, `__tests__/lib/compareLogic.test.ts`, create `__tests__/lib/ofstedStatus.test.ts`
|
||
|
||
**Interfaces:**
|
||
- Produces: `OfstedCurrentGrade`, `OfstedLatestVisit` types; `ofstedDisplay(ofsted)` returning `{kind:'none'} | {kind:'graded'|'confirmed'; grade; gradeLabel; gradeDate} | {kind:'no_overall_grade'} | {kind:'report_card'; summary}`; `gradeSourceLine(current)`, `latestVisitLine(visit)`, `showLatestVisitLine(ofsted)` from `lib/ofstedStatus.ts`.
|
||
|
||
- [ ] **Step 1: Write the failing tests**
|
||
|
||
`__tests__/lib/ofstedStatus.test.ts`:
|
||
|
||
```ts
|
||
import { gradeSourceLine, latestVisitLine, showLatestVisitLine } from '@/lib/ofstedStatus';
|
||
import type { OfstedInspection } from '@/lib/types';
|
||
|
||
const base = { report_card: {} } as unknown as OfstedInspection;
|
||
|
||
describe('gradeSourceLine', () => {
|
||
it('names the graded inspection and its date', () => {
|
||
expect(gradeSourceLine({ grade: 2, date: '2016-07-06', basis: 'graded' }))
|
||
.toBe('Graded inspection, 6 July 2016');
|
||
});
|
||
it('names the ungraded visit that confirmed it', () => {
|
||
expect(gradeSourceLine({ grade: 2, date: '2023-03-14', basis: 'confirmed' }))
|
||
.toBe('Confirmed at an ungraded inspection, 14 March 2023');
|
||
});
|
||
});
|
||
|
||
describe('latestVisitLine', () => {
|
||
it('prints the ungraded outcome', () => {
|
||
expect(latestVisitLine({ date: '2024-11-13', kind: 'ungraded', outcome: 'Standards maintained' }))
|
||
.toBe('Ungraded inspection, 13 November 2024: Standards maintained');
|
||
});
|
||
it('prints a graded visit without an outcome', () => {
|
||
expect(latestVisitLine({ date: '2025-06-17', kind: 'graded', outcome: null }))
|
||
.toBe('Graded inspection, 17 June 2025');
|
||
});
|
||
});
|
||
|
||
describe('showLatestVisitLine', () => {
|
||
it('shows a later visit than the one the grade came from', () => {
|
||
expect(showLatestVisitLine({ ...base,
|
||
current_grade: { grade: 2, date: '2020-03-03', basis: 'graded' },
|
||
latest_visit: { date: '2025-05-21', kind: 'ungraded', outcome: 'Standards maintained' } })).toBe(true);
|
||
});
|
||
it('hides it when the visit is the grade’s own source', () => {
|
||
expect(showLatestVisitLine({ ...base,
|
||
current_grade: { grade: 2, date: '2024-07-18', basis: 'confirmed' },
|
||
latest_visit: { date: '2024-07-18', kind: 'ungraded', outcome: 'School remains Good' } })).toBe(false);
|
||
});
|
||
it('shows an ungraded outcome when there is no grade', () => {
|
||
expect(showLatestVisitLine({ ...base, current_grade: null,
|
||
latest_visit: { date: '2024-11-13', kind: 'ungraded', outcome: 'Standards maintained' } })).toBe(true);
|
||
});
|
||
it('hides it for a graded visit with no grade (the title already dates it)', () => {
|
||
expect(showLatestVisitLine({ ...base, current_grade: null,
|
||
latest_visit: { date: '2025-06-17', kind: 'graded', outcome: null } })).toBe(false);
|
||
});
|
||
});
|
||
```
|
||
|
||
In `__tests__/lib/utils.test.ts`, replace the `buildOfstedListBadge` grade cases so the year comes from `ofsted_grade_date`, and add the C1 and Review Focus cases:
|
||
|
||
```ts
|
||
it('dates a grade by the inspection that awarded it, not the latest visit', () => {
|
||
const badge = buildOfstedListBadge({ ofsted_grade: 2, ofsted_grade_date: '2020-03-03', ofsted_date: '2025-05-21' });
|
||
expect(badge.label).toBe('Good · 2020');
|
||
});
|
||
|
||
it('shows Inspected for a latest inspection that gave no grade (Rabbsfarm)', () => {
|
||
const badge = buildOfstedListBadge({ ofsted_grade: null, ofsted_grade_date: null, ofsted_date: '2025-06-17' });
|
||
expect(badge.label).toBe('Inspected · 2025');
|
||
expect(badge.cssClass).toBe('ofstedInspected');
|
||
});
|
||
|
||
it('prints a grade without a year when its date is missing', () => {
|
||
expect(buildOfstedListBadge({ ofsted_grade: 2, ofsted_grade_date: null, ofsted_date: '2025-01-01' }).label).toBe('Good');
|
||
});
|
||
```
|
||
|
||
Update the existing "returns grade word + year…" cases to pass `ofsted_grade_date` instead of `ofsted_date`; delete the existing "returns grade word without year when date is missing" if it duplicates the new one.
|
||
|
||
In `__tests__/lib/compareLogic.test.ts` replace the `ofstedDisplay` block with:
|
||
|
||
```ts
|
||
describe('ofstedDisplay', () => {
|
||
it('returns none without an inspection', () => {
|
||
expect(ofstedDisplay(null).kind).toBe('none');
|
||
expect(ofstedDisplay(ofsted({})).kind).toBe('none');
|
||
});
|
||
it('distinguishes a graded grade from a confirmed one', () => {
|
||
expect(ofstedDisplay(ofsted({ current_grade: { grade: 1, date: '2019-10-09', basis: 'graded' },
|
||
latest_visit: { date: '2025-02-05', kind: 'ungraded', outcome: 'Some aspects not as strong' } })))
|
||
.toMatchObject({ kind: 'graded', gradeLabel: 'Outstanding', gradeDate: '2019-10-09' });
|
||
expect(ofstedDisplay(ofsted({ current_grade: { grade: 2, date: '2024-07-18', basis: 'confirmed' },
|
||
latest_visit: { date: '2024-07-18', kind: 'ungraded', outcome: 'School remains Good' } })))
|
||
.toMatchObject({ kind: 'confirmed', gradeLabel: 'Good' });
|
||
});
|
||
it('has no overall grade when the latest inspection gave none', () => {
|
||
expect(ofstedDisplay(ofsted({ current_grade: null, overall_effectiveness: null,
|
||
latest_visit: { date: '2025-06-17', kind: 'graded', outcome: null } })).kind).toBe('no_overall_grade');
|
||
});
|
||
});
|
||
```
|
||
|
||
(Keep the file's existing `ofsted()` helper and the report-card cases.)
|
||
|
||
- [ ] **Step 2: Run them to see them fail** — `cd nextjs-app && npx jest __tests__/lib -t "ofsted|Ofsted"` → fails (module and fields missing).
|
||
|
||
- [ ] **Step 3: Implement**
|
||
|
||
`lib/types.ts`: in `School`, after `ofsted_grade`, add
|
||
|
||
```ts
|
||
/** Date the grade was awarded or confirmed (null without a grade). */
|
||
ofsted_grade_date?: string | null;
|
||
```
|
||
|
||
and change the `ofsted_date` doc to `/** The school's latest inspection of any kind. */`. In `OfstedInspection`, replace `grade_source` with:
|
||
|
||
```ts
|
||
/** The overall grade still in force, dated by the inspection that awarded
|
||
* ("graded") or confirmed ("confirmed", an ungraded visit) it. Null when the
|
||
* latest inspection gave no overall grade, or for a report card. */
|
||
current_grade?: OfstedCurrentGrade | null;
|
||
/** The school's most recent inspection of any kind. */
|
||
latest_visit?: OfstedLatestVisit | null;
|
||
```
|
||
|
||
and add near `ReportCardEntry`:
|
||
|
||
```ts
|
||
export interface OfstedCurrentGrade {
|
||
grade: 1 | 2 | 3 | 4;
|
||
date: string | null;
|
||
basis: 'graded' | 'confirmed';
|
||
}
|
||
|
||
export interface OfstedLatestVisit {
|
||
date: string;
|
||
kind: 'report_card' | 'graded' | 'ungraded';
|
||
outcome: string | null;
|
||
}
|
||
```
|
||
|
||
Change the `overall_effectiveness` comment to "The graded inspection's own overall grade; never carried forward."
|
||
|
||
`lib/ofstedStatus.ts`:
|
||
|
||
```ts
|
||
/**
|
||
* The sentences the school page and the compare page print about where an
|
||
* Ofsted grade came from and what the latest visit was. One wording, two pages.
|
||
* Rule: docs/superpowers/specs/2026-10-05-ofsted-current-status-design.md
|
||
*/
|
||
|
||
import type { OfstedCurrentGrade, OfstedInspection, OfstedLatestVisit } from './types';
|
||
|
||
export function formatOfstedDate(iso: string | null | undefined): string {
|
||
if (!iso) return '';
|
||
const d = new Date(iso);
|
||
if (Number.isNaN(d.getTime())) return '';
|
||
return d.toLocaleDateString('en-GB', { day: 'numeric', month: 'long', year: 'numeric' });
|
||
}
|
||
|
||
const VISIT_KIND: Record<OfstedLatestVisit['kind'], string> = {
|
||
report_card: 'Report card inspection',
|
||
graded: 'Graded inspection',
|
||
ungraded: 'Ungraded inspection',
|
||
};
|
||
|
||
/** "Graded inspection, 6 July 2016" or "Confirmed at an ungraded inspection, 14 March 2023". */
|
||
export function gradeSourceLine(current: OfstedCurrentGrade): string {
|
||
const when = formatOfstedDate(current.date);
|
||
const what = current.basis === 'confirmed' ? 'Confirmed at an ungraded inspection' : 'Graded inspection';
|
||
return when ? `${what}, ${when}` : what;
|
||
}
|
||
|
||
/** "Ungraded inspection, 13 November 2024: Standards maintained". */
|
||
export function latestVisitLine(visit: OfstedLatestVisit): string {
|
||
const head = `${VISIT_KIND[visit.kind]}, ${formatOfstedDate(visit.date)}`;
|
||
return visit.outcome ? `${head}: ${visit.outcome}` : head;
|
||
}
|
||
|
||
/**
|
||
* Whether the latest visit needs its own line: when it is not where the grade
|
||
* came from, or when there is no grade but an ungraded outcome to report. A
|
||
* graded visit without a grade is already dated by the section title.
|
||
*/
|
||
export function showLatestVisitLine(ofsted: Pick<OfstedInspection, 'current_grade' | 'latest_visit'>): boolean {
|
||
const visit = ofsted.latest_visit;
|
||
if (!visit || visit.kind === 'report_card') return false;
|
||
const current = ofsted.current_grade;
|
||
if (current) return current.date !== visit.date;
|
||
return visit.kind === 'ungraded';
|
||
}
|
||
```
|
||
|
||
`lib/utils.ts` — `buildOfstedListBadge`: add `ofsted_grade_date?: string | null;` to the parameter type, update the doc comment's second bullet to "Current grade (ofsted_grade set): grade word + the year it was awarded or confirmed", and replace the body after the report-card branch with:
|
||
|
||
```ts
|
||
const yearOf = (iso?: string | null) => (iso ? new Date(iso).getFullYear() : null);
|
||
|
||
if (school.ofsted_grade) {
|
||
const labels: Record<number, string> = {
|
||
1: 'Outstanding',
|
||
2: 'Good',
|
||
3: 'Req. Improvement',
|
||
4: 'Inadequate',
|
||
};
|
||
const gradeYear = yearOf(school.ofsted_grade_date);
|
||
return {
|
||
label: `${labels[school.ofsted_grade]}${gradeYear ? ` · ${gradeYear}` : ''}`,
|
||
cssClass: `ofsted${school.ofsted_grade}`,
|
||
};
|
||
}
|
||
|
||
// An inspection is on record but no overall grade is in force: every
|
||
// inspection from Sept 2024 to Nov 2025, or an ungraded visit whose outcome
|
||
// names no grade. Dated by the latest visit.
|
||
if (school.ofsted_date != null || school.ofsted_framework != null) {
|
||
const visitYear = yearOf(school.ofsted_date);
|
||
return { label: `Inspected${visitYear ? ` · ${visitYear}` : ''}`, cssClass: 'ofstedInspected' };
|
||
}
|
||
```
|
||
|
||
`lib/compareLogic.ts` — replace `OfstedDisplay` and `ofstedDisplay` with:
|
||
|
||
```ts
|
||
export type OfstedDisplay =
|
||
| { kind: 'none' }
|
||
| { kind: 'graded' | 'confirmed'; grade: number; gradeLabel: string; gradeDate: string | null }
|
||
| { kind: 'no_overall_grade' }
|
||
| { kind: 'report_card'; summary: ReportCardSummary };
|
||
|
||
export function ofstedDisplay(
|
||
ofsted: OfstedInspection | null | undefined,
|
||
): OfstedDisplay {
|
||
if (!ofsted) return { kind: 'none' };
|
||
|
||
// A report card is the newest inspection format; when present it wins.
|
||
if (ofsted.report_card && Object.keys(ofsted.report_card).length > 0) {
|
||
return { kind: 'report_card', summary: summariseReportCard(ofsted) };
|
||
}
|
||
|
||
const current = ofsted.current_grade;
|
||
const gradeLabel = current ? OFSTED_LEGACY_GRADES[current.grade] : undefined;
|
||
if (current && gradeLabel !== undefined) {
|
||
return {
|
||
kind: current.basis === 'confirmed' ? 'confirmed' : 'graded',
|
||
grade: current.grade,
|
||
gradeLabel,
|
||
gradeDate: current.date,
|
||
};
|
||
}
|
||
return ofsted.latest_visit ? { kind: 'no_overall_grade' } : { kind: 'none' };
|
||
}
|
||
```
|
||
|
||
- [ ] **Step 4: Run** `npx jest __tests__/lib` and `npx tsc --noEmit`. Expected: the new tests pass; `tsc` reports errors only in the components Tasks 9–10 change (`carried_forward`, `transitional`, `carriedForward`) — fix those in Tasks 9 and 10, not by loosening types.
|
||
|
||
- [ ] **Step 5: Commit** — `fix(ofsted): badge and compare read the grade still in force`
|
||
|
||
### Task 9: School page Ofsted section
|
||
|
||
**Files:**
|
||
- Modify: `nextjs-app/components/school/OfstedSection.tsx`
|
||
- Modify: `nextjs-app/components/school/PrimarySchoolSections.tsx:57-80`, `SecondarySchoolSections.tsx:63-85`
|
||
- Test: create `nextjs-app/__tests__/components/OfstedSection.test.tsx`; update `__tests__/support/schoolFixtures.ts` (`grade_source` → `current_grade` + `latest_visit`) and `__tests__/components/schoolDetail.characterization.test.tsx` if its Ofsted assertions change.
|
||
|
||
**Interfaces:** Consumes `ofstedDisplay`, `gradeSourceLine`, `latestVisitLine`, `showLatestVisitLine`, `formatOfstedDate`, `ofstedLegacyAreas`. `OfstedSection` props become `{ ofsted, urn, isReportCard, ofstedInspectedDate, oeifAllSameGrade, oeifAreas }` (the `variant` prop is removed; its callers stop passing it).
|
||
|
||
- [ ] **Step 1: Write the failing tests**
|
||
|
||
`__tests__/components/OfstedSection.test.tsx`:
|
||
|
||
```tsx
|
||
import { render, screen } from '@testing-library/react';
|
||
import { OfstedSection } from '@/components/school/OfstedSection';
|
||
import { ofstedLegacyAreas } from '@/lib/utils';
|
||
import type { OfstedInspection } from '@/lib/types';
|
||
|
||
const empty = {
|
||
framework: null, inspection_date: null, inspection_type: null, overall_effectiveness: null,
|
||
quality_of_education: null, behaviour_attitudes: null, personal_development: null,
|
||
leadership_management: null, early_years_provision: null, sixth_form_provision: null,
|
||
previous_overall: null, rc_safeguarding_met: null, rc_inclusion: null, rc_curriculum_teaching: null,
|
||
rc_achievement: null, rc_attendance_behaviour: null, rc_personal_development: null,
|
||
rc_leadership_governance: null, rc_early_years: null, rc_sixth_form: null, report_url: null,
|
||
report_card: {},
|
||
} as OfstedInspection;
|
||
|
||
function renderSection(o: Partial<OfstedInspection>) {
|
||
const ofsted = { ...empty, ...o } as OfstedInspection;
|
||
render(
|
||
<OfstedSection
|
||
ofsted={ofsted}
|
||
urn={1}
|
||
isReportCard={false}
|
||
ofstedInspectedDate={ofsted.latest_visit?.date ?? null}
|
||
oeifAllSameGrade={false}
|
||
oeifAreas={ofstedLegacyAreas(ofsted)}
|
||
/>,
|
||
);
|
||
}
|
||
|
||
it('shows no overall grade, not an older one, after a no-grade inspection (Rabbsfarm)', () => {
|
||
renderSection({
|
||
inspection_date: '2025-06-17', quality_of_education: 3, behaviour_attitudes: 3,
|
||
personal_development: 2, leadership_management: 3, early_years_provision: 2,
|
||
current_grade: null, latest_visit: { date: '2025-06-17', kind: 'graded', outcome: null },
|
||
});
|
||
expect(screen.getByText(/Inspected 17 June 2025/)).toBeInTheDocument();
|
||
expect(screen.getByText('No overall grade')).toBeInTheDocument();
|
||
expect(screen.queryByText('Good', { selector: 'span' })).not.toBeInTheDocument();
|
||
expect(screen.queryByText('Not rated')).not.toBeInTheDocument();
|
||
expect(screen.getAllByText('Requires Improvement')).toHaveLength(3);
|
||
});
|
||
|
||
it('keeps the sixth-form judgement when there is no overall grade (audit M2)', () => {
|
||
renderSection({
|
||
inspection_date: '2025-04-01', quality_of_education: 1, behaviour_attitudes: 1,
|
||
personal_development: 1, leadership_management: 1, sixth_form_provision: 2,
|
||
current_grade: null, latest_visit: { date: '2025-04-01', kind: 'graded', outcome: null },
|
||
});
|
||
expect(screen.getByText(/sixth form/i)).toBeInTheDocument();
|
||
expect(screen.getByText('Good')).toBeInTheDocument();
|
||
});
|
||
|
||
it('dates a grade by its source and prints a later visit separately (Washwood Heath)', () => {
|
||
renderSection({
|
||
inspection_date: '2020-03-03', overall_effectiveness: 2, quality_of_education: 2,
|
||
current_grade: { grade: 2, date: '2020-03-03', basis: 'graded' },
|
||
latest_visit: { date: '2025-05-21', kind: 'ungraded', outcome: 'Standards maintained' },
|
||
});
|
||
expect(screen.getByText(/Inspected 21 May 2025/)).toBeInTheDocument();
|
||
expect(screen.getByText('Graded inspection, 3 March 2020')).toBeInTheDocument();
|
||
expect(screen.getByText(/Latest visit: Ungraded inspection, 21 May 2025: Standards maintained/)).toBeInTheDocument();
|
||
});
|
||
|
||
it('says a grade was confirmed at an ungraded visit (Robins Lane)', () => {
|
||
renderSection({
|
||
inspection_date: '2020-01-07', overall_effectiveness: 2,
|
||
current_grade: { grade: 2, date: '2024-07-18', basis: 'confirmed' },
|
||
latest_visit: { date: '2024-07-18', kind: 'ungraded', outcome: 'School remains Good' },
|
||
});
|
||
expect(screen.getByText('Confirmed at an ungraded inspection, 18 July 2024')).toBeInTheDocument();
|
||
expect(screen.queryByText(/Latest visit/)).not.toBeInTheDocument();
|
||
});
|
||
|
||
it('prints an ungraded outcome when there is no grade (Oakgrove)', () => {
|
||
renderSection({
|
||
current_grade: null,
|
||
latest_visit: { date: '2024-11-13', kind: 'ungraded', outcome: 'Standards maintained' },
|
||
});
|
||
expect(screen.getByText('No overall grade')).toBeInTheDocument();
|
||
expect(screen.getByText(/Ungraded inspection, 13 November 2024: Standards maintained/)).toBeInTheDocument();
|
||
});
|
||
```
|
||
|
||
- [ ] **Step 2: Run** `npx jest __tests__/components/OfstedSection.test.tsx` → fails.
|
||
|
||
- [ ] **Step 3: Implement**
|
||
|
||
Replace the non-report-card part of `OfstedSection` (everything after the report-card branch of the ternary, both the "Old OEIF layout" and the "Secondary only" branches) with one branch, and drop the `variant` prop:
|
||
|
||
```tsx
|
||
) : (
|
||
<>
|
||
{display.kind === 'graded' || display.kind === 'confirmed' ? (
|
||
<div className={styles.ofstedHeader}>
|
||
<span className={`${styles.ofstedGrade} ${styles[`ofstedGrade${display.grade}`]}`}>
|
||
{display.gradeLabel}
|
||
</span>
|
||
</div>
|
||
) : (
|
||
<div className={styles.ofstedHeader}>
|
||
<span className={styles.ofstedGrade}>No overall grade</span>
|
||
</div>
|
||
)}
|
||
<p className={styles.ofstedDisclaimer}>
|
||
{ofsted.current_grade
|
||
? gradeSourceLine(ofsted.current_grade)
|
||
: 'Ofsted stopped giving overall grades in September 2024.'}
|
||
</p>
|
||
{showLatestVisitLine(ofsted) && ofsted.latest_visit && (
|
||
<p className={styles.ofstedDisclaimer}>Latest visit: {latestVisitLine(ofsted.latest_visit)}</p>
|
||
)}
|
||
{oeifAllSameGrade && display.kind !== 'no_overall_grade' && display.kind !== 'none' ? (
|
||
<p className={styles.ofstedAllSame}>
|
||
Rated <strong>{display.gradeLabel}</strong> across all inspected areas: Quality of Teaching, Behaviour, Pupils' Development and Leadership.
|
||
</p>
|
||
) : oeifAreas.length > 0 ? (
|
||
<>
|
||
{ofsted.inspection_date && ofsted.inspection_date !== ofsted.latest_visit?.date && (
|
||
<p className={styles.ofstedDisclaimer}>
|
||
Area judgements from the graded inspection, {formatOfstedDate(ofsted.inspection_date)}.
|
||
</p>
|
||
)}
|
||
<div className={`${styles.metricsGrid} ${styles.gradeGrid}`}>
|
||
{oeifAreas.map(({ label, value }) => (
|
||
<div key={label} className={styles.metricCard}>
|
||
<div className={styles.metricLabel}>{label}</div>
|
||
<div className={`${styles.metricValue} ${styles[`ofstedGrade${value}`]}`}>
|
||
{OFSTED_LABELS[value]}
|
||
</div>
|
||
</div>
|
||
))}
|
||
</div>
|
||
</>
|
||
) : null}
|
||
</>
|
||
)}
|
||
```
|
||
|
||
with `const display = ofstedDisplay(ofsted);` at the top of the component and imports from `@/lib/compareLogic` and `@/lib/ofstedStatus`. Update the header comment: the primary/secondary fork is gone because both now follow the current-status rule. Keep the report-card branch and the title/link untouched.
|
||
|
||
In both section composers:
|
||
|
||
```tsx
|
||
const ofstedInspectedDate = isReportCard
|
||
? ofsted?.rc_inspection_date ?? null
|
||
: ofsted?.latest_visit?.date ?? null;
|
||
...
|
||
const oeifAllSameGrade =
|
||
!!ofsted &&
|
||
!isReportCard &&
|
||
ofsted.overall_effectiveness != null &&
|
||
oeifAreas.length >= 3 &&
|
||
oeifAreas.every((a) => a.value === ofsted.overall_effectiveness);
|
||
```
|
||
|
||
and remove `variant="primary"`/`variant="secondary"` from the `<OfstedSection>` calls.
|
||
|
||
Fixtures: in `schoolFixtures.ts` replace `grade_source: 'graded'` with `current_grade: { grade: <same as overall_effectiveness>, date: <same as inspection_date>, basis: 'graded' }, latest_visit: { date: <inspection_date>, kind: 'graded', outcome: null }`.
|
||
|
||
- [ ] **Step 4: Run** `npx jest` (whole suite) and `npx tsc --noEmit`. Fix characterization assertions only where the spec changes the copy (e.g. "Not rated" → "No overall grade"); any other change is a regression.
|
||
|
||
- [ ] **Step 5: Commit** — `fix(school): the Ofsted section shows the grade still in force and the latest visit`
|
||
|
||
### Task 10: Compare Ofsted rows
|
||
|
||
**Files:**
|
||
- Modify: `nextjs-app/components/compare/CompareAtAGlance.tsx:77-105`, `components/compare/CompareOfsted.tsx:42-77`, `:186-206`
|
||
- Test: `nextjs-app/__tests__/components/CompareOfsted.test.tsx`
|
||
|
||
- [ ] **Step 1: Write the failing tests** (append to `CompareOfsted.test.tsx`, using its existing render helper; if it has none, render `<CompareOfsted schools={[school]} data={{ [String(school.urn)]: { ofsted } }} />` with a minimal `School` fixture from `__tests__/support/schoolFixtures.ts`):
|
||
|
||
```tsx
|
||
it('dates "Inspected" by the latest visit, not the graded inspection (Washwood Heath)', () => {
|
||
renderCompare({
|
||
inspection_date: '2020-03-03', overall_effectiveness: 2,
|
||
current_grade: { grade: 2, date: '2020-03-03', basis: 'graded' },
|
||
latest_visit: { date: '2025-05-21', kind: 'ungraded', outcome: 'Standards maintained' },
|
||
});
|
||
expect(screen.getByText(/21 May 2025/)).toBeInTheDocument();
|
||
expect(screen.queryByText('4+ years ago')).not.toBeInTheDocument();
|
||
expect(screen.getByText('Graded inspection, 3 March 2020')).toBeInTheDocument();
|
||
});
|
||
|
||
it('shows no overall grade after a no-grade inspection', () => {
|
||
renderCompare({
|
||
inspection_date: '2025-06-17', quality_of_education: 3,
|
||
current_grade: null, latest_visit: { date: '2025-06-17', kind: 'graded', outcome: null },
|
||
});
|
||
expect(screen.getAllByText('No overall grade').length).toBeGreaterThan(0);
|
||
expect(screen.queryByText('Good')).not.toBeInTheDocument();
|
||
});
|
||
```
|
||
|
||
Remove tests that assert "Grade carried forward" or "transitional framework".
|
||
|
||
- [ ] **Step 2: Run** `npx jest __tests__/components/CompareOfsted.test.tsx` → fails.
|
||
|
||
- [ ] **Step 3: Implement**
|
||
|
||
`CompareOfsted.tsx` `ResultCell` — replace the `transitional` branch and the final branch with:
|
||
|
||
```tsx
|
||
if (display.kind === 'no_overall_grade') {
|
||
return (
|
||
<>
|
||
<span className={s.badge} style={{ backgroundColor: 'var(--bg-secondary)', color: 'var(--text-secondary)' }}>
|
||
No overall grade
|
||
</span>
|
||
<span className={s.small}>Ofsted stopped giving overall grades in September 2024</span>
|
||
</>
|
||
);
|
||
}
|
||
return (
|
||
<>
|
||
<span className={`${s.badge} ${display.grade <= 2 ? s.badgeGood : display.grade === 3 ? s.badgeWarn : s.badgeBad}`}>
|
||
{display.gradeLabel}
|
||
</span>
|
||
{current && <span className={s.small}>{gradeSourceLine(current)}</span>}
|
||
</>
|
||
);
|
||
```
|
||
|
||
passing `current={ofsted?.current_grade ?? null}` into `ResultCell` (add the prop: `{ display, current }: { display: OfstedDisplay; current: OfstedCurrentGrade | null }`).
|
||
|
||
"Inspected" measure: `const dateIso = ofsted?.latest_visit?.date ?? ofsted?.rc_inspection_date ?? null;` (delete the report-card special case and its comment, since `latest_visit` is the report card's own date when there is one) and render the visit kind under the date when present:
|
||
|
||
```tsx
|
||
{formatInspectionDate(dateIso)}{' '}
|
||
{age != null && age > 4 && <Chip tone="neutral">4+ years ago</Chip>}
|
||
{ofsted?.latest_visit?.outcome && <span className={s.small}>{ofsted.latest_visit.outcome}</span>}
|
||
```
|
||
|
||
Update the file's header comment ("three regimes (legacy graded, interim carried-forward, …)") to "legacy graded, no overall grade, renewed-framework report card".
|
||
|
||
`CompareAtAGlance.tsx` — replace the `graded || carried_forward` and `transitional` branches with:
|
||
|
||
```tsx
|
||
{(display.kind === 'graded' || display.kind === 'confirmed') && (
|
||
<>
|
||
<span className={`${s.badge} ${display.grade <= 2 ? s.badgeGood : display.grade === 3 ? s.badgeWarn : s.badgeBad}`}>
|
||
{display.gradeLabel}
|
||
</span>
|
||
{display.kind === 'confirmed' && <span className={s.small}>Confirmed at an ungraded inspection</span>}
|
||
</>
|
||
)}
|
||
{display.kind === 'no_overall_grade' && (
|
||
<span className={s.badge} style={{ backgroundColor: 'var(--bg-secondary)', color: 'var(--text-secondary)' }}>
|
||
No overall grade
|
||
</span>
|
||
)}
|
||
```
|
||
|
||
- [ ] **Step 4: Run** `npx jest` and `npx tsc --noEmit` → all pass.
|
||
|
||
- [ ] **Step 5: Commit** — `fix(compare): Ofsted rows show the grade still in force and the latest visit`
|
||
|
||
### Task 11: Remove the unused helpers that encode the old rule
|
||
|
||
**Files:** Modify `nextjs-app/lib/utils.ts` (delete `buildOfstedHeroChip`, `OfstedHeroChip`/`HeroTone` types if only it uses them, `buildSchoolSummary`, and `formatOfstedMonth` if nothing else uses it); delete their tests in `__tests__/lib/utils.test.ts`.
|
||
|
||
- [ ] **Step 1:** `grep -rn "buildOfstedHeroChip\|buildSchoolSummary\|formatOfstedMonth\|OfstedHeroChip\|HeroTone" nextjs-app/components nextjs-app/lib nextjs-app/app nextjs-app/__tests__` — confirm only `lib/utils.ts` and its test reference them.
|
||
- [ ] **Step 2:** Delete them and their tests.
|
||
- [ ] **Step 3:** `npx tsc --noEmit && npx jest` → pass.
|
||
- [ ] **Step 4: Commit** — `refactor(utils): drop the unused Ofsted hero chip and summary sentence`
|
||
|
||
### Task 12: E2E journeys
|
||
|
||
**Files:** Modify `e2e/tests/journeys.spec.ts` (add after the report-card journey).
|
||
|
||
- [ ] **Step 1: Write the journeys**
|
||
|
||
```ts
|
||
test('a school whose latest inspection gave no grade is not shown with an older grade', async ({ page }) => {
|
||
// Rabbsfarm (102408): Ofsted's 17 June 2025 inspection gave no overall
|
||
// grade and rated three areas Requires Improvement. The site used to carry
|
||
// a 2020 "remains Good" forward and print "Good · 2025".
|
||
const URN = 102408;
|
||
const res = await page.request.get(`/api/schools/${URN}`);
|
||
expect(res.ok()).toBeTruthy();
|
||
const ofsted = (await res.json()).ofsted;
|
||
expect(ofsted.current_grade).toBeNull();
|
||
expect(ofsted.latest_visit.date).toBe('2025-06-17');
|
||
|
||
await searchByName(page, 'Rabbsfarm Primary');
|
||
const row = page.locator(`a[href*="${URN}"]`).first();
|
||
await expect(row).toBeVisible({ timeout: 15_000 });
|
||
await expect(page.getByText('Inspected · 2025').first()).toBeVisible();
|
||
await expect(page.getByText(/Good · 2025/)).toHaveCount(0);
|
||
|
||
await page.goto(`/school/${URN}`);
|
||
const section = page.locator('#ofsted');
|
||
await expect(section.getByText('No overall grade')).toBeVisible();
|
||
await expect(section.getByText('Requires Improvement').first()).toBeVisible();
|
||
});
|
||
|
||
test('a grade confirmed at an ungraded visit says so and is dated by it', async ({ page }) => {
|
||
const URN = 104762; // Robins Lane: graded Good Jan 2020, "School remains Good" 18 July 2024
|
||
await page.goto(`/school/${URN}`);
|
||
const section = page.locator('#ofsted');
|
||
await expect(section).toBeVisible({ timeout: 15_000 });
|
||
await expect(section.getByText('Confirmed at an ungraded inspection, 18 July 2024')).toBeVisible();
|
||
});
|
||
```
|
||
|
||
- [ ] **Step 2:** These run against staging after merge (the gate is post-merge). Before opening the PR, run `npx playwright test --list` from the repo root to confirm they parse. If either school is missing on staging (`/api/schools/{urn}` 404), replace it with another URN from the same rule row using the audit's MI file.
|
||
- [ ] **Step 3: Commit** — `test(e2e): no grade is carried past a no-grade inspection`
|
||
|
||
### Task 13: Open PR 2
|
||
|
||
- [ ] **Step 1:** Full checks: `cd nextjs-app && npx tsc --noEmit && npm test`; backend/pipeline `pytest` as in Task 5; `dbt compile --select dim_school`.
|
||
- [ ] **Step 2:** `git status --short`; push; open the PR. The body leads with **"Do not merge until `school_data_monthly_ofsted` has run on staging with PR 1 — the backend reads `marts.fact_ofsted_latest` and has no fallback"**, then lists the visible change (about 932 badges, place-page counts, Typesense rating), the checks, and the post-merge verification: daily DAG run (for `dim_school` and Typesense), E2E, and re-running the audit's C1/M1/M2 checks (`scratchpad/audit/check_render.py` against staging) expecting 0.
|