fix(destinations): withhold at the API, not just in the chart

Code review found the disclosure the whole design was meant to prevent.
R1 was written as a rendering rule and implemented as one: canRenderBar
stopped the bar being drawn, but GET /api/schools/{urn} still carried the
cohort and every published category. cohort - sum(published) returned
Whitley Bay's withheld further-education figure exactly — 18 pupils — to
any caller, and the RSC payload put it in the browser too.

app.py already stated the principle for admission_distance: this endpoint
is public and unauthenticated, so a field left in the payload is a
published field. The same reasoning applies here and did not get applied.

_mask_for_disclosure now closes both identities before serialisation —
categories sum to the cohort, and disadvantaged + other = all — by adding
secondary suppression until every row and column hides none or at least
two. My first attempt picked the smallest published cell as the companion
and a new test caught it choosing a zero, which protects nothing: the
residual still resolved to 18. The companion must carry pupils.

DfE's own aggregates are no longer served. Nothing rendered them, and one
spanning a single suppressed component names it.

Cost, measured over 262 mainstream secondaries: the all-pupils bar
survives on 94% rather than 100%. Zero lone-suppressed groups remain.

The e2e helper now tells a missing feature apart from missing data: it
fails if the API serves no destinations key at all, and skips if the key
is served but the annual DAG has not populated the marts. Failing on the
second would redden the staging gate for unrelated commits.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvdDKvFFSZuMVDH5fEyTob

backend/data_loader.py

@@ -839,17 +839,100 @@ def _format_cohort_year(year) -> str | None:
     return text
 
 
+def _mask_for_disclosure(groups: dict) -> None:
+    """Add secondary suppression until no withheld figure can be recovered.
+
+    Not rendering a number is not the same as not publishing it. This endpoint
+    is public and unauthenticated, so anything left in the payload is
+    published, whatever the UI chooses to draw — the same reasoning the
+    admission_distance field carries in app.py.
+
+    Two identities let a caller solve for a withheld cell:
+
+      * within a pupil group, the categories sum to the cohort, so a group with
+        exactly ONE suppressed category gives it away as cohort - sum(rest);
+      * across groups, disadvantaged + other = all for every category, so a
+        category suppressed in exactly ONE of the three gives itself away.
+
+    DfE's own answer is secondary suppression: withhold a second cell so the
+    residual spans two unknowns and identifies neither. This does the same,
+    iterating because each new suppression can break the other identity, and
+    terminating because cells are only ever added to the suppressed set.
+
+    Mutates `groups` in place.
+    """
+    PAIRS = ("disadvantaged", "other", "all")
+
+    def cells(group_key):
+        group = groups.get(group_key)
+        return group["categories"] if group else []
+
+    def suppress(cell):
+        if cell["status"] == "published":
+            cell["status"] = "suppressed"
+            cell["pupils"] = None
+            cell["percentage"] = None
+            return True
+        return False
+
+    def add_companion(candidates):
+        """Withhold a second cell so the residual spans two unknowns.
+
+        The companion must carry pupils. Suppressing a zero looks like
+        secondary suppression and protects nothing: the residual still equals
+        the original withheld figure exactly. Where every remaining cell is
+        zero there is no companion that helps, so the whole set goes — losing
+        real data, but that beats publishing what DfE withheld.
+        """
+        published = [c for c in candidates if c["status"] == "published"]
+        useful = sorted(
+            (c for c in published if (c["pupils"] or 0) > 0),
+            key=lambda c: c["pupils"],
+        )
+        if useful:
+            return suppress(useful[0])
+        return any([suppress(c) for c in published])
+
+    changed = True
+    while changed:
+        changed = False
+
+        # Column rule: a category must be suppressed in none of the three
+        # pupil groups, or in at least two of them.
+        categories = {c["category"] for key in PAIRS for c in cells(key)}
+        for category in categories:
+            found = [
+                c for key in PAIRS for c in cells(key) if c["category"] == category
+            ]
+            hidden = [c for c in found if c["status"] == "suppressed"]
+            if len(hidden) == 1 and len(found) > 1:
+                if add_companion(found):
+                    changed = True
+
+        # Row rule: within a group, none suppressed or at least two.
+        for key in PAIRS:
+            group_cells = cells(key)
+            hidden = [c for c in group_cells if c["status"] == "suppressed"]
+            if len(hidden) == 1:
+                if add_companion(group_cells):
+                    changed = True
+
+
 def _destinations_block(rows: list) -> dict | None:
     """Shape destination rows for one phase into the API's block.
 
-    Carries `status` through untouched and emits no computed totals. The only
-    aggregates present are ones DfE published itself; whether showing one is
-    safe depends on how many of its components are suppressed, which the
-    frontend decides (lib/destinations.ts, rule R2).
+    Applies secondary suppression before returning, so no caller of this public
+    endpoint can solve for a figure DfE withheld. See _mask_for_disclosure.
 
-    Deliberately does NOT compute a residual, a "remaining pupils" figure, or
-    any total that would close a gap left by a suppressed category — the
-    categories sum to the cohort, so such a figure names the withheld cell.
+    Aggregate measures are dropped entirely. DfE publishes them, and they would
+    be useful for a "what is published for this group" fallback, but nothing
+    renders them today and an aggregate spanning exactly one suppressed
+    component names that component. An unused field that leaks is not a
+    trade-off worth carrying — re-add them with their own guard if the fallback
+    is ever built.
+
+    Deliberately computes no residual, no "remaining pupils" figure, and no
+    total that would close a gap left by a suppressed category.
     """
     if not rows:
         return None
@@ -864,23 +947,28 @@ def _destinations_block(rows: list) -> dict | None:
     for row in rows:
         group = groups.setdefault(
             row["pupil_group"],
-            {"cohort": row.get("cohort_pupils"), "categories": [], "aggregates": {}},
+            {"cohort": row.get("cohort_pupils"), "categories": []},
         )
         measure = row["destination_measure"]
+        published = row.get("status") == "published"
+        # Belt and braces: percentage is derived from the same source cell as
+        # pupils, but publishing one without the other would hand back the
+        # cohort (pupils / percentage) and with it the residual.
         cell = {
             "category": measure,
-            "pupils": row.get("pupils"),
-            "percentage": row.get("percentage"),
+            "pupils": row.get("pupils") if published else None,
+            "percentage": row.get("percentage") if published else None,
             "status": row.get("status"),
         }
         if measure in _AGGREGATE_MEASURES:
-            group["aggregates"][measure[len("agg_"):]] = cell
-        else:
-            group["categories"].append(cell)
+            continue
+        group["categories"].append(cell)
 
     if not groups:
         return None
 
+    _mask_for_disclosure(groups)
+
     return {"cohort_year": _format_cohort_year(latest_year), "groups": groups}