Adds two sections to secondary school pages: After Year 11 (KS4 destination measures) and After the sixth form (16-18), replacing the "Post-16 destination data coming soon" placeholder that has stood in SecondaryAdmissionsSection since the exam-phase taxonomy work.
DfE suppresses individual cells with c, not whole cohorts, and the destination categories sum to the cohort — which is also published. So where exactly one category is suppressed, subtracting the published ones recovers it exactly. Verified against three real schools in the 2022/23 file:
School
URN
Withheld
Recovers to
North East Futures UTC
145900
School sixth form
3 pupils
Whitley Bay High School
108638
Further education
18 pupils
St Matthew's RC High School
148389
School sixth form
4 pupils
22% of mainstream secondaries have exactly one suppressed category in their disadvantaged group, so this is the normal case. Three rules follow, and they are executable rather than documentary — lib/destinations.ts plus four dbt tests:
R1 Never render a derived remainder. No number, and no bar at all for a group with any suppression: a segment sized by the residual can be read straight off the axis.
R2 Never aggregate across a suppression boundary. Compute a total from components only when all are published; render a DfE-published total only when it spans 0 or ≥2 suppressed cells.
R3 Where a category is withheld for disadvantaged pupils it is withheld for the other-pupils group too — the two partition the whole. Applied in the mart. DfE already does this in 493 of 498 cases; this closes the remaining 5.
safe_numeric is deliberately not used on destination columns. It maps every EES sentinel to NULL, which is right for attainment and wrong here: one state has to print "withheld" and the other has to print nothing.
Display
Question-led — three cards over one bar, with the cards acting as a lens on the bar rather than a summary beside it, so the grouping we chose is inspectable. The headline is deliberately not the sustained-destination rate, which sits between 92% and 97% for nearly every school in England; the mix is what varies.
The absence is hatched neutral, never a colour. "Activity not captured" covers independent schools and moving abroad, so a red segment would state something false — and the hatch doubles as the secondary encoding that rescues the neutral/blue pair, which separates at only ΔE 7.6 as flat fills. Every other adjacent pair clears ΔE 10.9 under protanopia.
Backend: 169 tests (batch-query guard grows from six tables to eight)
Tap: 10 tests
E2E: 5 new journeys, including one that computes the residual itself and asserts it appears nowhere on the page
Tap verified against the live API for 2022/23: 135,240 school records over 4,508 schools, exactly 30 each, zero duplicate keys, 31,382 c sentinels preserved. National reference reconciles: 151,912 disadvantaged + 441,823 other = 593,735.
Before merging
The marts are empty until the annual EES DAG runs. The E2E journeys fail loudly rather than skipping (following 4f01fbd), so they will be red on staging until you trigger Airflow.
Not in scope
Compare view and rankings (the long mart shape supports both), the ethnicity/sex/SEN breakdowns that travel in the same file, and primary schools — no KS2 destination measures publication exists.
Adds two sections to secondary school pages: **After Year 11** (KS4 destination measures) and **After the sixth form** (16-18), replacing the "Post-16 destination data coming soon" placeholder that has stood in `SecondaryAdmissionsSection` since the exam-phase taxonomy work.
Design: `docs/superpowers/specs/2026-08-28-destination-measures-design.md`
Plan: `docs/superpowers/plans/2026-08-28-destination-measures.md`
Mockup: https://claude.ai/code/artifact/5be149d6-252f-473c-9a4f-4c36b05161b0
## The finding that shapes the whole PR
DfE suppresses **individual cells** with `c`, not whole cohorts, and the destination categories sum to the cohort — which is also published. So where exactly one category is suppressed, subtracting the published ones recovers it exactly. Verified against three real schools in the 2022/23 file:
| School | URN | Withheld | Recovers to |
|---|---|---|---|
| North East Futures UTC | 145900 | School sixth form | **3 pupils** |
| Whitley Bay High School | 108638 | Further education | **18 pupils** |
| St Matthew's RC High School | 148389 | School sixth form | **4 pupils** |
22% of mainstream secondaries have exactly one suppressed category in their disadvantaged group, so this is the normal case. Three rules follow, and they are executable rather than documentary — `lib/destinations.ts` plus four dbt tests:
- **R1** Never render a derived remainder. No number, and **no bar at all** for a group with any suppression: a segment sized by the residual can be read straight off the axis.
- **R2** Never aggregate across a suppression boundary. Compute a total from components only when all are published; render a DfE-published total only when it spans 0 or ≥2 suppressed cells.
- **R3** Where a category is withheld for disadvantaged pupils it is withheld for the other-pupils group too — the two partition the whole. Applied in the mart. DfE already does this in 493 of 498 cases; this closes the remaining 5.
**`safe_numeric` is deliberately not used on destination columns.** It maps every EES sentinel to `NULL`, which is right for attainment and wrong here: one state has to print "withheld" and the other has to print nothing.
## Display
Question-led — three cards over one bar, with the cards acting as a lens on the bar rather than a summary beside it, so the grouping we chose is inspectable. The headline is deliberately *not* the sustained-destination rate, which sits between 92% and 97% for nearly every school in England; the mix is what varies.
The absence is hatched neutral, never a colour. "Activity not captured" covers independent schools and moving abroad, so a red segment would state something false — and the hatch doubles as the secondary encoding that rescues the neutral/blue pair, which separates at only ΔE 7.6 as flat fills. Every other adjacent pair clears ΔE 10.9 under protanopia.
## Verification
- Frontend: 354 tests / 39 suites, `tsc --noEmit` clean, `next build` succeeds
- Backend: 169 tests (batch-query guard grows from six tables to eight)
- Tap: 10 tests
- E2E: 5 new journeys, including one that computes the residual itself and asserts it appears nowhere on the page
Tap verified against the live API for 2022/23: **135,240 school records over 4,508 schools, exactly 30 each, zero duplicate keys, 31,382 `c` sentinels preserved.** National reference reconciles: 151,912 disadvantaged + 441,823 other = 593,735.
## Before merging
The marts are empty until the **annual EES DAG** runs. The E2E journeys fail loudly rather than skipping (following 4f01fbd), so they will be red on staging until you trigger Airflow.
## Not in scope
Compare view and rankings (the long mart shape supports both), the ethnicity/sex/SEN breakdowns that travel in the same file, and primary schools — no KS2 destination measures publication exists.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_01BvdDKvFFSZuMVDH5fEyTob
The published files suppress individual cells, not whole cohorts, and the
categories sum to the cohort — so on 22% of mainstream secondaries the
withheld figure can be recovered by subtraction. Three disclosure rules
fall out of that, and the rest of the design is downstream of them.
Verified against the EES API rather than assumed: both datasets carry
school-level rows keyed by URN, with the disadvantage split and every
destination category the display needs.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvdDKvFFSZuMVDH5fEyTob
Ordered so the disclosure guards land first and everything downstream
consumes them: lib/destinations.ts, tokens, tap, staging, marts, API,
then the two sections and the journeys.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvdDKvFFSZuMVDH5fEyTob
The destination categories sum to the cohort and DfE publishes the cohort
total, so a lone suppressed cell is recoverable by subtraction. canAggregate,
canRenderPublishedAggregate and canRenderBar are what stop a consumer doing
that; toBarSegments throws rather than leaving a readable gap.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvdDKvFFSZuMVDH5fEyTob
Activity not captured includes independent schools and moving abroad, so a
red segment would be a factual error. The hatch doubles as the secondary
encoding that rescues the neutral/blue pair, which separates at only dE 7.6
as flat fills.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvdDKvFFSZuMVDH5fEyTob
EES writes 'c' where a figure is withheld and the categories sum to the
cohort, so counts and percentages are emitted as text with the sentinel
intact. safe_numeric must never be pointed at them.
School rows and the England reference need different establishment pins:
at national level selective schools, studios and UTCs are separate
populations rather than labels, so leaving establishment open multiplies
30 rows into 190. Two queries per period, each keeping its own level.
Verified against the live API for 2022/23: 135,240 school records over
4,508 schools, exactly 30 each, no duplicate keys, 31,382 sentinels kept.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvdDKvFFSZuMVDH5fEyTob
safe_numeric maps every EES sentinel to NULL, which is right for attainment
and wrong here: one of those states has to print 'withheld' and the other
has to print nothing. A status column carries the difference.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvdDKvFFSZuMVDH5fEyTob
Disadvantaged and other-pupils partition the whole and the all-pupils
figure is published, so publishing both halves recovers the suppressed
one. The mask is applied in the mart rather than the API so no consumer
added later can reach an unmasked combination.
The R1 test is a warn, not an error: DfE publishes the recoverable
combination and the mart's job is to carry it faithfully. Refusing to
close the gap is the API's job and the frontend's.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvdDKvFFSZuMVDH5fEyTob
The serialiser carries status through and computes no totals of its own.
The only aggregates in the payload are ones DfE published itself; whether
showing one is safe depends on how many of its components are suppressed,
which the frontend decides.
The batch guard grows from six tables to eight.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvdDKvFFSZuMVDH5fEyTob
Destinations are secondary-only, so the flags go on computeSecondaryFlags
rather than computeSchoolFlags. A phase counts as present only when some
pupil group carries categories — an empty block would otherwise open a nav
entry pointing at a section that never renders.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvdDKvFFSZuMVDH5fEyTob
Question cards over one bar, with the cards acting as a lens on the bar
rather than a summary beside it — focusing a card dims everything it is
not made of, so the grouping we chose is inspectable rather than asserted.
The bar renders only when canRenderBar allows it. Where a category is
withheld the section says so and shows the table instead: the categories
sum to the cohort, so a bar drawn from the published segments leaves a
gap whose width is the withheld figure.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvdDKvFFSZuMVDH5fEyTob
The 'Post-16 destination data coming soon' note is deleted rather than
reworded: for a school with no sixth form the truthful statement is that
the question does not apply, and a placeholder there implies something is
missing. hasSixthForm and .sixthFormNote go with it — nothing else used them.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvdDKvFFSZuMVDH5fEyTob
The helper throws rather than skipping when no school returns a
destinations block: a silent skip would let a real regression in the
sections ride along unnoticed, which is why the distance journeys were
changed the same way in 4f01fbd.
The disadvantaged journey computes the residual itself and asserts it
appears nowhere on the page — the one number the section must never state.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvdDKvFFSZuMVDH5fEyTob
This PR adds KS4/16-18 destination-measures sections to secondary school pages: new backend fact tables and a serialiser, an API field, and new React sections with client-side rules meant to stop DfE-suppressed pupil counts from being reconstructed and displayed. The domain logic and test coverage for the disclosure rules are extensive, but the actual data-flow only enforces those rules at render time — the raw ingredients needed to reconstruct a suppressed figure are still shipped to the browser and via the public API. New E2E tests are also written to hard-fail rather than skip until the annual pipeline populates the marts.
🔴 Severe (blocks merge)
backend/data_loader.py: _destinations_block (and the API it feeds via app.py's destinations field) always emits the full cohort total plus every published category's pupils/percentage, even for a pupil group with exactly one suppressed category. Since the destination categories sum to the cohort (the design doc's own central finding), any consumer of GET /api/schools/{urn} — not just the rendered UI — can trivially recover the withheld figure via cohort - sum(published). The disclosure guards in the frontend (canRenderBar, canRenderPublishedAggregate) only prevent the number from being drawn on screen; they never prevent it from being computed. This republishes exactly the data DfE suppressed for small-cohort privacy, which the whole R1/R2/R3 design was meant to prevent.
nextjs-app/components/school/DestinationsView.tsx: DestinationsView is a client component that receives the entire destinations phase object (cohort + every category for all three pupil groups: all/disadvantaged/other) as a prop from the server-rendered DestinationsSection/Post16DestinationsSection. Because it's a client component, this full payload is serialized into the page's RSC/JSON data and is visible via view-source or the network tab regardless of which pupil-group tab is 'selected' or whether a bar is drawn. Anyone viewing the page can extract the suppressed-adjacent figures and derive the withheld pupil count by subtraction, even though the UI itself never draws the residual — the same underlying leak as the data_loader.py finding, but confirming it reaches the browser, not just the API.
e2e/tests/journeys.spec.ts: The new destination journeys use secondaryWithDestinations(), which throws (fails the test) rather than skipping when no school returns a populated destinations.ks4 block. Per the PR's own plan doc, the destination marts are only populated once the annual Airflow DAG runs on staging, which will not be true immediately after this merges. These tests will therefore fail deterministically on the staging E2E gate right after merge (and for as long as the DAG hasn't run), which can block the staging-to-production promotion pipeline for this and any subsequent change.
## 🤖 AI Code Review (Claude Code)
This PR adds KS4/16-18 destination-measures sections to secondary school pages: new backend fact tables and a serialiser, an API field, and new React sections with client-side rules meant to stop DfE-suppressed pupil counts from being reconstructed and displayed. The domain logic and test coverage for the disclosure rules are extensive, but the actual data-flow only enforces those rules at render time — the raw ingredients needed to reconstruct a suppressed figure are still shipped to the browser and via the public API. New E2E tests are also written to hard-fail rather than skip until the annual pipeline populates the marts.
### 🔴 Severe (blocks merge)
- **backend/data_loader.py**: `_destinations_block` (and the API it feeds via app.py's `destinations` field) always emits the full `cohort` total plus every published category's `pupils`/`percentage`, even for a pupil group with exactly one suppressed category. Since the destination categories sum to the cohort (the design doc's own central finding), any consumer of `GET /api/schools/{urn}` — not just the rendered UI — can trivially recover the withheld figure via `cohort - sum(published)`. The disclosure guards in the frontend (`canRenderBar`, `canRenderPublishedAggregate`) only prevent the number from being drawn on screen; they never prevent it from being computed. This republishes exactly the data DfE suppressed for small-cohort privacy, which the whole R1/R2/R3 design was meant to prevent.
- **nextjs-app/components/school/DestinationsView.tsx**: DestinationsView is a client component that receives the entire `destinations` phase object (cohort + every category for all three pupil groups: all/disadvantaged/other) as a prop from the server-rendered DestinationsSection/Post16DestinationsSection. Because it's a client component, this full payload is serialized into the page's RSC/JSON data and is visible via view-source or the network tab regardless of which pupil-group tab is 'selected' or whether a bar is drawn. Anyone viewing the page can extract the suppressed-adjacent figures and derive the withheld pupil count by subtraction, even though the UI itself never draws the residual — the same underlying leak as the data_loader.py finding, but confirming it reaches the browser, not just the API.
- **e2e/tests/journeys.spec.ts**: The new destination journeys use `secondaryWithDestinations()`, which throws (fails the test) rather than skipping when no school returns a populated `destinations.ks4` block. Per the PR's own plan doc, the destination marts are only populated once the annual Airflow DAG runs on staging, which will not be true immediately after this merges. These tests will therefore fail deterministically on the staging E2E gate right after merge (and for as long as the DAG hasn't run), which can block the staging-to-production promotion pipeline for this and any subsequent change.
Code review found the disclosure the whole design was meant to prevent.
R1 was written as a rendering rule and implemented as one: canRenderBar
stopped the bar being drawn, but GET /api/schools/{urn} still carried the
cohort and every published category. cohort - sum(published) returned
Whitley Bay's withheld further-education figure exactly — 18 pupils — to
any caller, and the RSC payload put it in the browser too.
app.py already stated the principle for admission_distance: this endpoint
is public and unauthenticated, so a field left in the payload is a
published field. The same reasoning applies here and did not get applied.
_mask_for_disclosure now closes both identities before serialisation —
categories sum to the cohort, and disadvantaged + other = all — by adding
secondary suppression until every row and column hides none or at least
two. My first attempt picked the smallest published cell as the companion
and a new test caught it choosing a zero, which protects nothing: the
residual still resolved to 18. The companion must carry pupils.
DfE's own aggregates are no longer served. Nothing rendered them, and one
spanning a single suppressed component names it.
Cost, measured over 262 mainstream secondaries: the all-pupils bar
survives on 94% rather than 100%. Zero lone-suppressed groups remain.
The e2e helper now tells a missing feature apart from missing data: it
fails if the API serves no destinations key at all, and skips if the key
is served but the annual DAG has not populated the marts. Failing on the
second would redden the staging gate for unrelated commits.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvdDKvFFSZuMVDH5fEyTob
Adds a new 'destination measures' feature (After Year 11 / post-16 sections) end-to-end: Meltano/dbt pipeline docs/plans, new FactKs4Destinations/FactKs5Destinations SQLAlchemy models, a data_loader serialiser with secondary-suppression disclosure control, a new API field, and (partially shown) Next.js UI. The engineering is unusually careful about not republishing DfE-suppressed pupil figures, but the core masking algorithm has a real gap that can leave a withheld figure unprotected in sparse-data edge cases.
🔴 Severe (blocks merge)
backend/data_loader.py: _mask_for_disclosure's add_companion helper only ever suppresses a cell drawn from candidates whose status is published. When a category is suppressed in exactly one pupil group/row and every remaining cell in that row/column is not_applicable (rather than published), published is empty, add_companion returns False, and the outer while changed loop exits without having fixed the 'exactly one hidden' violation — silently, with no error or log. The lone suppressed cell then remains exposed to the exact subtraction/identity attack this function exists to prevent (cohort minus the sum of the few published/no-data categories reveals it, or disadvantaged/other/all comparison reveals it). This is most reachable for sparse cohorts such as special/alternative-provision schools, which the PR's own design doc notes have far fewer applicable categories than mainstream secondaries — i.e. this isn't a purely theoretical case.
🟡 Minor
backend/tests/test_destinations_api.py: test_a_category_hidden_in_one_group_is_hidden_in_a_second builds a fixture with six destination measures (varying suppression patterns) but only asserts the cross-group masking invariant for for measure in ["school_sixth_form"] — a one-element list — so the other five measures' masking is never actually verified despite being set up for it.
## 🤖 AI Code Review (Claude Code)
Adds a new 'destination measures' feature (After Year 11 / post-16 sections) end-to-end: Meltano/dbt pipeline docs/plans, new FactKs4Destinations/FactKs5Destinations SQLAlchemy models, a data_loader serialiser with secondary-suppression disclosure control, a new API field, and (partially shown) Next.js UI. The engineering is unusually careful about not republishing DfE-suppressed pupil figures, but the core masking algorithm has a real gap that can leave a withheld figure unprotected in sparse-data edge cases.
### 🔴 Severe (blocks merge)
- **backend/data_loader.py**: `_mask_for_disclosure`'s `add_companion` helper only ever suppresses a cell drawn from candidates whose status is `published`. When a category is suppressed in exactly one pupil group/row and every remaining cell in that row/column is `not_applicable` (rather than `published`), `published` is empty, `add_companion` returns `False`, and the outer `while changed` loop exits without having fixed the 'exactly one hidden' violation — silently, with no error or log. The lone suppressed cell then remains exposed to the exact subtraction/identity attack this function exists to prevent (cohort minus the sum of the few published/no-data categories reveals it, or disadvantaged/other/all comparison reveals it). This is most reachable for sparse cohorts such as special/alternative-provision schools, which the PR's own design doc notes have far fewer applicable categories than mainstream secondaries — i.e. this isn't a purely theoretical case.
### 🟡 Minor
- **backend/tests/test_destinations_api.py**: `test_a_category_hidden_in_one_group_is_hidden_in_a_second` builds a fixture with six destination measures (varying suppression patterns) but only asserts the cross-group masking invariant for `for measure in ["school_sixth_form"]` — a one-element list — so the other five measures' masking is never actually verified despite being set up for it.
Review found _mask_for_disclosure could return with its invariant broken
and say nothing. add_companion only ever withheld a *published* cell, so a
group with one suppressed category and every other one not_applicable —
routine in special schools and AP, where few categories apply — left the
loop with the lone suppressed cell still solvable. Reproduced on a
nine-pupil cohort: one hidden cell, cohort served, residual intact.
A disclosure-control pass that fails silently is worse than none, because
everything downstream trusts it. The loop now runs until the invariant
holds and escalates when no companion exists: the pupil group is dropped
from the payload, and an empty block serialises as None so the section is
absent rather than an empty shell. disclosure_invariant_holds() is exported
so tests assert it directly instead of re-deriving it, and an exhaustive
test sweeps all 81 suppression patterns of a four-category group.
Also fixes a test that set up six measures and checked one: the loop was
`for measure in ["school_sixth_form"]`. It now checks every measure, and
against the real invariant — none hidden, or at least two, rather than
"at least two", which the five published measures would have failed.
No regression on real data: 262 mainstream secondaries, all-pupils bar
still drawable on 94%, zero invariant violations, one disadvantaged group
dropped by the new escalation.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvdDKvFFSZuMVDH5fEyTob
This PR adds KS4 and 16-18 destination measures to secondary school pages: backend models/serialiser exposing a new destinations field on the school detail endpoint, and frontend sections rendering cohort breakdowns with a bar chart. The disclosure-control logic (_mask_for_disclosure in data_loader.py, preventing DfE-suppressed figures from being recoverable via subtraction across categories or pupil groups) is carefully designed, provably terminates in a safe state, and is backed by extensive targeted and exhaustive tests. One minor display bug survives in the frontend table.
🟡 Minor
nextjs-app/components/school/DestinationsView.tsx: In the destinations table, the Share column shows the 'withheld' badge whenever isWithheld || cell.percentage === null, but cell.percentage is also null for status === 'not_applicable' categories, not just genuinely 'suppressed' ones. This mislabels a category that simply doesn't apply to the school as one DfE withheld, contradicting the status distinction this PR otherwise carefully preserves end-to-end (backend models, staging SQL, serialiser). The Pupils column in the same row correctly renders blank (not 'withheld') for not_applicable cells, so the two columns disagree on the same row.
## 🤖 AI Code Review (Claude Code)
This PR adds KS4 and 16-18 destination measures to secondary school pages: backend models/serialiser exposing a new `destinations` field on the school detail endpoint, and frontend sections rendering cohort breakdowns with a bar chart. The disclosure-control logic (`_mask_for_disclosure` in data_loader.py, preventing DfE-suppressed figures from being recoverable via subtraction across categories or pupil groups) is carefully designed, provably terminates in a safe state, and is backed by extensive targeted and exhaustive tests. One minor display bug survives in the frontend table.
### 🟡 Minor
- **nextjs-app/components/school/DestinationsView.tsx**: In the destinations table, the Share column shows the 'withheld' badge whenever `isWithheld || cell.percentage === null`, but `cell.percentage` is also null for `status === 'not_applicable'` categories, not just genuinely `'suppressed'` ones. This mislabels a category that simply doesn't apply to the school as one DfE withheld, contradicting the status distinction this PR otherwise carefully preserves end-to-end (backend models, staging SQL, serialiser). The Pupils column in the same row correctly renders blank (not 'withheld') for not_applicable cells, so the two columns disagree on the same row.
The Share column keyed off `percentage === null`, which is true for
not_applicable as well as suppressed, so a destination that does not apply
to the school was labelled as one DfE withheld — while the Pupils column
in the same row rendered blank. Two columns, one row, disagreeing about
what the row was, and one of them making a claim about DfE that wasn't
true.
Both columns now derive from `status`, which is the distinction the mart,
the SQLAlchemy model and the serialiser all preserve deliberately:
published shows the figure, suppressed shows the withheld badge,
not_applicable shows an em-dash with a title saying so.
A published count with no published percentage now derives its share from
the cohort rather than falling through to a marker — both halves are
published, so nothing withheld is involved, and it is the same derivation
the bar widths already use.
Verified the new tests fail against the old logic before keeping them.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvdDKvFFSZuMVDH5fEyTob
tudor
merged commit 73182d0c0c into main2026-08-30 20:49:15 +00:00
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Adds two sections to secondary school pages: After Year 11 (KS4 destination measures) and After the sixth form (16-18), replacing the "Post-16 destination data coming soon" placeholder that has stood in
SecondaryAdmissionsSectionsince the exam-phase taxonomy work.Design:
docs/superpowers/specs/2026-08-28-destination-measures-design.mdPlan:
docs/superpowers/plans/2026-08-28-destination-measures.mdMockup: https://claude.ai/code/artifact/5be149d6-252f-473c-9a4f-4c36b05161b0
The finding that shapes the whole PR
DfE suppresses individual cells with
c, not whole cohorts, and the destination categories sum to the cohort — which is also published. So where exactly one category is suppressed, subtracting the published ones recovers it exactly. Verified against three real schools in the 2022/23 file:22% of mainstream secondaries have exactly one suppressed category in their disadvantaged group, so this is the normal case. Three rules follow, and they are executable rather than documentary —
lib/destinations.tsplus four dbt tests:safe_numericis deliberately not used on destination columns. It maps every EES sentinel toNULL, which is right for attainment and wrong here: one state has to print "withheld" and the other has to print nothing.Display
Question-led — three cards over one bar, with the cards acting as a lens on the bar rather than a summary beside it, so the grouping we chose is inspectable. The headline is deliberately not the sustained-destination rate, which sits between 92% and 97% for nearly every school in England; the mix is what varies.
The absence is hatched neutral, never a colour. "Activity not captured" covers independent schools and moving abroad, so a red segment would state something false — and the hatch doubles as the secondary encoding that rescues the neutral/blue pair, which separates at only ΔE 7.6 as flat fills. Every other adjacent pair clears ΔE 10.9 under protanopia.
Verification
tsc --noEmitclean,next buildsucceedsTap verified against the live API for 2022/23: 135,240 school records over 4,508 schools, exactly 30 each, zero duplicate keys, 31,382
csentinels preserved. National reference reconciles: 151,912 disadvantaged + 441,823 other = 593,735.Before merging
The marts are empty until the annual EES DAG runs. The E2E journeys fail loudly rather than skipping (following
4f01fbd), so they will be red on staging until you trigger Airflow.Not in scope
Compare view and rankings (the long mart shape supports both), the ethnicity/sex/SEN breakdowns that travel in the same file, and primary schools — no KS2 destination measures publication exists.
🤖 Generated with Claude Code
https://claude.ai/code/session_01BvdDKvFFSZuMVDH5fEyTob
🤖 AI Code Review (Claude Code)
This PR adds KS4/16-18 destination-measures sections to secondary school pages: new backend fact tables and a serialiser, an API field, and new React sections with client-side rules meant to stop DfE-suppressed pupil counts from being reconstructed and displayed. The domain logic and test coverage for the disclosure rules are extensive, but the actual data-flow only enforces those rules at render time — the raw ingredients needed to reconstruct a suppressed figure are still shipped to the browser and via the public API. New E2E tests are also written to hard-fail rather than skip until the annual pipeline populates the marts.
🔴 Severe (blocks merge)
_destinations_block(and the API it feeds via app.py'sdestinationsfield) always emits the fullcohorttotal plus every published category'spupils/percentage, even for a pupil group with exactly one suppressed category. Since the destination categories sum to the cohort (the design doc's own central finding), any consumer ofGET /api/schools/{urn}— not just the rendered UI — can trivially recover the withheld figure viacohort - sum(published). The disclosure guards in the frontend (canRenderBar,canRenderPublishedAggregate) only prevent the number from being drawn on screen; they never prevent it from being computed. This republishes exactly the data DfE suppressed for small-cohort privacy, which the whole R1/R2/R3 design was meant to prevent.destinationsphase object (cohort + every category for all three pupil groups: all/disadvantaged/other) as a prop from the server-rendered DestinationsSection/Post16DestinationsSection. Because it's a client component, this full payload is serialized into the page's RSC/JSON data and is visible via view-source or the network tab regardless of which pupil-group tab is 'selected' or whether a bar is drawn. Anyone viewing the page can extract the suppressed-adjacent figures and derive the withheld pupil count by subtraction, even though the UI itself never draws the residual — the same underlying leak as the data_loader.py finding, but confirming it reaches the browser, not just the API.secondaryWithDestinations(), which throws (fails the test) rather than skipping when no school returns a populateddestinations.ks4block. Per the PR's own plan doc, the destination marts are only populated once the annual Airflow DAG runs on staging, which will not be true immediately after this merges. These tests will therefore fail deterministically on the staging E2E gate right after merge (and for as long as the DAG hasn't run), which can block the staging-to-production promotion pipeline for this and any subsequent change.Code review found the disclosure the whole design was meant to prevent. R1 was written as a rendering rule and implemented as one: canRenderBar stopped the bar being drawn, but GET /api/schools/{urn} still carried the cohort and every published category. cohort - sum(published) returned Whitley Bay's withheld further-education figure exactly — 18 pupils — to any caller, and the RSC payload put it in the browser too. app.py already stated the principle for admission_distance: this endpoint is public and unauthenticated, so a field left in the payload is a published field. The same reasoning applies here and did not get applied. _mask_for_disclosure now closes both identities before serialisation — categories sum to the cohort, and disadvantaged + other = all — by adding secondary suppression until every row and column hides none or at least two. My first attempt picked the smallest published cell as the companion and a new test caught it choosing a zero, which protects nothing: the residual still resolved to 18. The companion must carry pupils. DfE's own aggregates are no longer served. Nothing rendered them, and one spanning a single suppressed component names it. Cost, measured over 262 mainstream secondaries: the all-pupils bar survives on 94% rather than 100%. Zero lone-suppressed groups remain. The e2e helper now tells a missing feature apart from missing data: it fails if the API serves no destinations key at all, and skips if the key is served but the annual DAG has not populated the marts. Failing on the second would redden the staging gate for unrelated commits. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BvdDKvFFSZuMVDH5fEyTob🤖 AI Code Review (Claude Code)
Adds a new 'destination measures' feature (After Year 11 / post-16 sections) end-to-end: Meltano/dbt pipeline docs/plans, new FactKs4Destinations/FactKs5Destinations SQLAlchemy models, a data_loader serialiser with secondary-suppression disclosure control, a new API field, and (partially shown) Next.js UI. The engineering is unusually careful about not republishing DfE-suppressed pupil figures, but the core masking algorithm has a real gap that can leave a withheld figure unprotected in sparse-data edge cases.
🔴 Severe (blocks merge)
_mask_for_disclosure'sadd_companionhelper only ever suppresses a cell drawn from candidates whose status ispublished. When a category is suppressed in exactly one pupil group/row and every remaining cell in that row/column isnot_applicable(rather thanpublished),publishedis empty,add_companionreturnsFalse, and the outerwhile changedloop exits without having fixed the 'exactly one hidden' violation — silently, with no error or log. The lone suppressed cell then remains exposed to the exact subtraction/identity attack this function exists to prevent (cohort minus the sum of the few published/no-data categories reveals it, or disadvantaged/other/all comparison reveals it). This is most reachable for sparse cohorts such as special/alternative-provision schools, which the PR's own design doc notes have far fewer applicable categories than mainstream secondaries — i.e. this isn't a purely theoretical case.🟡 Minor
test_a_category_hidden_in_one_group_is_hidden_in_a_secondbuilds a fixture with six destination measures (varying suppression patterns) but only asserts the cross-group masking invariant forfor measure in ["school_sixth_form"]— a one-element list — so the other five measures' masking is never actually verified despite being set up for it.🤖 AI Code Review (Claude Code)
This PR adds KS4 and 16-18 destination measures to secondary school pages: backend models/serialiser exposing a new
destinationsfield on the school detail endpoint, and frontend sections rendering cohort breakdowns with a bar chart. The disclosure-control logic (_mask_for_disclosurein data_loader.py, preventing DfE-suppressed figures from being recoverable via subtraction across categories or pupil groups) is carefully designed, provably terminates in a safe state, and is backed by extensive targeted and exhaustive tests. One minor display bug survives in the frontend table.🟡 Minor
isWithheld || cell.percentage === null, butcell.percentageis also null forstatus === 'not_applicable'categories, not just genuinely'suppressed'ones. This mislabels a category that simply doesn't apply to the school as one DfE withheld, contradicting the status distinction this PR otherwise carefully preserves end-to-end (backend models, staging SQL, serialiser). The Pupils column in the same row correctly renders blank (not 'withheld') for not_applicable cells, so the two columns disagree on the same row.