You spotted that SW19 is mostly Merton but partly Wandsworth, and the page named only Merton.
The cause was one field doing two jobs. _parent_authority takes the modal authority — correct for a 301 redirect target, wrong as a statement about where a place is. I used it for both.
Not a corner case
Measured against the live corpus:
Straddle a boundary
Viable outcodes
425 of 1,760 (24%)
Viable towns
263 of 783 (34%)
SW19 → Merton 26, Wandsworth 7. Bedford the town spans Bedford and Central Bedfordshire. NW6 spans Brent, Camden and Westminster.
The change
Place now carries authorities — every authority holding at least a tenth of the schools and at least two of them, largest first. The page names them all and links each. parent_authority is unchanged and still single, because a redirect needs exactly one target.
The share threshold is doing real work: GIAS carries postcode errors.EN6 lists two Shropshire schools among fourteen in Hertfordshire, and a bare "any authority present" rule would print those as though they were true. A place too small or too fragmented to clear the threshold still names its largest, so the line never goes silent about where the place is.
The frontend also falls back to the single parent when authorities is absent, so a cached API response from before this change cannot blank the line.
Verification
Backend 105 · frontend 256 · tsc --noEmit clean · next build green · 85 e2e journeys, including one that finds a straddling outcode in the live registry and asserts every named authority is linked on the page.
Six backend tests cover the rules directly: the SW19 shape, the redirect target staying single, the stray-authority threshold, the sentinel filter, and the never-silent fallback.
Note
One test initially passed a loose /and/ matcher that also matched "Wandsworth" — tightened to assert the summary line reads "Merton and Wandsworth".
You spotted that SW19 is mostly Merton but partly Wandsworth, and the page named only Merton.
The cause was one field doing two jobs. `_parent_authority` takes the **modal** authority — correct for a 301 redirect target, wrong as a statement about where a place is. I used it for both.
## Not a corner case
Measured against the live corpus:
| | Straddle a boundary |
|---|---|
| Viable outcodes | **425 of 1,760** (24%) |
| Viable towns | **263 of 783** (34%) |
`SW19` → Merton 26, Wandsworth 7. `Bedford` the town spans Bedford and Central Bedfordshire. `NW6` spans Brent, Camden and Westminster.
## The change
`Place` now carries `authorities` — every authority holding at least a tenth of the schools and at least two of them, largest first. The page names them all and links each. `parent_authority` is unchanged and still single, because a redirect needs exactly one target.
The share threshold is doing real work: **GIAS carries postcode errors.** `EN6` lists two Shropshire schools among fourteen in Hertfordshire, and a bare "any authority present" rule would print those as though they were true. A place too small or too fragmented to clear the threshold still names its largest, so the line never goes silent about where the place is.
The frontend also falls back to the single parent when `authorities` is absent, so a cached API response from before this change cannot blank the line.
## Verification
Backend 105 · frontend 256 · `tsc --noEmit` clean · `next build` green · 85 e2e journeys, including one that finds a straddling outcode in the live registry and asserts every named authority is linked on the page.
Six backend tests cover the rules directly: the SW19 shape, the redirect target staying single, the stray-authority threshold, the sentinel filter, and the never-silent fallback.
## Note
One test initially passed a loose `/and/` matcher that also matched "W**and**sworth" — tightened to assert the summary line reads "Merton and Wandsworth".
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_015mWQnpye9F299NVRCCSRvj
This PR adds a per-place "authorities" list (backend Place.authorities, /api/places/{kind}/{slug} response, PlaceView UI) so pages naming a boundary-straddling place (e.g. SW19: Merton + Wandsworth) list every authority with a meaningful share instead of just the dominant one used for redirects. It's well tested at the unit, component, and e2e level, includes a backward-compatible fallback for cached API responses missing the new field, and correctly wires all four place kinds (town, locality, authority, outcode) without touching CI/deploy config.
🟡 Minor
backend/places.py: _AUTHORITY_MAX_SHOWN caps the list at 3 authorities. If a place has 4+ authorities each clearing the 10%/2-school threshold (e.g. a near-even 4-way split), the 4th+ is silently dropped from the API response, which contradicts the stated goal of naming every authority a place meaningfully sits in. No log/telemetry marks the truncation.
backend/places.py: parent_authority is computed via group['local_authority'].mode().iloc[0] while authorities[0] (the 'largest' authority shown first on the page) is computed via value_counts(). In the rare case of an exact tie in school counts between two authorities, pandas does not guarantee mode() and value_counts() pick the same authority as top, so the 301 redirect target could disagree with the authority highlighted first on the page.
## 🤖 AI Code Review (Claude Code)
This PR adds a per-place "authorities" list (backend Place.authorities, /api/places/{kind}/{slug} response, PlaceView UI) so pages naming a boundary-straddling place (e.g. SW19: Merton + Wandsworth) list every authority with a meaningful share instead of just the dominant one used for redirects. It's well tested at the unit, component, and e2e level, includes a backward-compatible fallback for cached API responses missing the new field, and correctly wires all four place kinds (town, locality, authority, outcode) without touching CI/deploy config.
### 🟡 Minor
- **backend/places.py**: _AUTHORITY_MAX_SHOWN caps the list at 3 authorities. If a place has 4+ authorities each clearing the 10%/2-school threshold (e.g. a near-even 4-way split), the 4th+ is silently dropped from the API response, which contradicts the stated goal of naming every authority a place meaningfully sits in. No log/telemetry marks the truncation.
- **backend/places.py**: parent_authority is computed via group['local_authority'].mode().iloc[0] while authorities[0] (the 'largest' authority shown first on the page) is computed via value_counts(). In the rare case of an exact tie in school counts between two authorities, pandas does not guarantee mode() and value_counts() pick the same authority as top, so the 301 redirect target could disagree with the authority highlighted first on the page.
SW19 is mostly Merton but partly Wandsworth, and the page said only Merton.
The cause was one field doing two jobs: _parent_authority takes the modal
authority, which is right for a 301 target and wrong as a statement about
where a place is.
This is not a corner case. A quarter of viable outcodes (425 of 1,760) and a
third of viable towns (263 of 783) cross an authority boundary — Bedford the
town spans Bedford and Central Bedfordshire.
Place now carries `authorities`, every authority holding at least a tenth of
the schools and at least two of them, largest first. parent_authority stays
single and unchanged, because a redirect still needs one target.
The share threshold exists because GIAS carries postcode errors: EN6 lists two
Shropshire schools among fourteen in Hertfordshire, and a bare "any authority
present" rule would print those as though they were real. A place too small or
too fragmented to clear the threshold still names its largest, so the page
never goes silent about where it is.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015mWQnpye9F299NVRCCSRvj
Two findings from review on #121, plus a third the review prompted.
The cap at three authorities silently dropped the fourth in exactly the case
where the information matters most — a genuinely fragmented place — and
contradicted the stated goal of naming every authority a place sits in. It is
gone. The share rule was always the real limit and already bounds the list at
ten. Measured against the live corpus, one town would have been truncated
today: LONDON, split evenly between Hackney, Lambeth, Westminster and
Lewisham.
parent_authority used mode() while authorities used value_counts(), and on an
exact tie pandas does not guarantee the two pick the same name, so the 301
could have pointed somewhere other than the authority named first on the page.
The parent is now derived from authorities[0]: one computation, one answer.
It also inherits the sentinel filter, so a place can no longer redirect to
/schools/authority/does-not-apply.
Chasing the truncation case surfaced a worse bug. Places were grouped by raw
town value, but the registry is keyed by slug, and GIAS spells the same place
several ways. Five town slugs come from more than one spelling: "London"
(1,819 schools) and "LONDON" (12) both slugify to `london`, so the later group
simply overwrote the earlier one — /schools/london could have shown twelve
schools, silently, depending on row order. Weston-super-Mare was split 14/19
across two spellings and Newcastle-under-Lyme across three. Grouping is now by
slug, and the display name is the most common spelling.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015mWQnpye9F299NVRCCSRvj
Both findings fixed, the branch is rebased onto main (it was conflicting with #120), and chasing the first one surfaced a worse bug.
1. The cap — removed, not logged
You're right that it contradicted the stated goal. I removed _AUTHORITY_MAX_SHOWN entirely rather than adding telemetry: the share rule was always the real limit and already bounds the list at ten, so an arbitrary cap of three only ever made the field a lie in the case where the information matters most.
Measured against the live corpus, the cap would truncate exactly one place today — the town LONDON, split evenly between Hackney, Lambeth, Westminster and Lewisham, two schools each. Zero outcodes are affected (the maximum is three).
2. The tie — fixed by construction
parent_authority is now derived from authorities[0] rather than computed separately, so mode() and value_counts() can no longer disagree. One computation, one answer.
It also inherits the sentinel filter as a side effect, which fixes something neither of us had flagged: a place whose modal authority was Does not apply would previously have redirected to /schools/authority/does-not-apply.
3. What the truncation case exposed
Looking at why LONDON had four authorities showed that LONDON is a separate town value from London — and both slugify to london.
Places were grouped by raw town value while the registry is keyed by slug, so out[place.key] = place silently overwrote. Five town slugs come from more than one spelling:
Slug
Spellings
Schools
london
London / LONDON
1,819 / 12
weston-super-mare
Weston-super-Mare / Weston-Super-Mare
14 / 19
newcastle-under-lyme
three variants
5 / 7 / 5
newcastle-upon-tyne
two variants
146 / 7
bury-st-edmunds
two variants
39 / 13
/schools/london could have shown twelve schools instead of 1,819, silently, depending on row order. Grouping is now by slug, with the most common spelling as the display name.
Verification
Backend 110 · frontend 258 · tsc --noEmit clean · next build green · 85 e2e journeys.
Five new backend tests: no cap, the redirect target matching the first-named authority, the sentinel never being a redirect target, spellings merging rather than overwriting, and the merged place taking its most common spelling.
The rebase conflict was in the test file only — #120's alignment tests and this branch's authority tests are both additive, so both were kept.
Both findings fixed, the branch is rebased onto `main` (it was conflicting with #120), and chasing the first one surfaced a worse bug.
## 1. The cap — removed, not logged
You're right that it contradicted the stated goal. I removed `_AUTHORITY_MAX_SHOWN` entirely rather than adding telemetry: the share rule was always the real limit and already bounds the list at ten, so an arbitrary cap of three only ever made the field a lie in the case where the information matters most.
Measured against the live corpus, the cap would truncate **exactly one place today** — the town `LONDON`, split evenly between Hackney, Lambeth, Westminster and Lewisham, two schools each. Zero outcodes are affected (the maximum is three).
## 2. The tie — fixed by construction
`parent_authority` is now derived from `authorities[0]` rather than computed separately, so `mode()` and `value_counts()` can no longer disagree. One computation, one answer.
It also inherits the sentinel filter as a side effect, which fixes something neither of us had flagged: a place whose modal authority was `Does not apply` would previously have redirected to `/schools/authority/does-not-apply`.
## 3. What the truncation case exposed
Looking at why `LONDON` had four authorities showed that `LONDON` is a *separate town value* from `London` — and both slugify to `london`.
Places were grouped by **raw town value** while the registry is keyed by **slug**, so `out[place.key] = place` silently overwrote. Five town slugs come from more than one spelling:
| Slug | Spellings | Schools |
|---|---|---|
| `london` | `London` / `LONDON` | **1,819** / 12 |
| `weston-super-mare` | `Weston-super-Mare` / `Weston-Super-Mare` | 14 / 19 |
| `newcastle-under-lyme` | three variants | 5 / 7 / 5 |
| `newcastle-upon-tyne` | two variants | 146 / 7 |
| `bury-st-edmunds` | two variants | 39 / 13 |
`/schools/london` could have shown **twelve schools instead of 1,819**, silently, depending on row order. Grouping is now by slug, with the most common spelling as the display name.
## Verification
Backend 110 · frontend 258 · `tsc --noEmit` clean · `next build` green · 85 e2e journeys.
Five new backend tests: no cap, the redirect target matching the first-named authority, the sentinel never being a redirect target, spellings merging rather than overwriting, and the merged place taking its most common spelling.
The rebase conflict was in the test file only — #120's alignment tests and this branch's authority tests are both additive, so both were kept.
This PR adds a per-place authorities list (largest-authority-first, with a share/count threshold and sentinel filtering) so pages like SW19 can name every local authority they straddle instead of just one, and fixes a latent data-loss bug where GIAS's inconsistent spelling of the same town/authority name caused groups to silently overwrite each other in the place registry. The change is thorough and well-tested (backend unit tests, e2e, and frontend component tests all cover the new behavior, including a legacy-cache fallback), and derives parent_authority from the same computation as authorities to remove a prior mode()/value_counts() tie-inconsistency risk.
🟡 Minor
backend/places.py: _group() now normalizes spelling/case variants of the grouped column (town or authority name) via _slugify before grouping, fixing the overwrite bug — but _authorities() still counts local_authority values as raw strings without the same normalization. If GIAS has case/whitespace variants for the same authority (analogous to the documented 'London'/'LONDON' town case), they'll be counted as distinct authorities, potentially each falling under the 10% share threshold when combined they wouldn't, or appearing as duplicate-looking entries with the same slug (causing a duplicate React key={a.slug} in PlaceView.tsx).
backend/places.py: _authorities()'s share threshold (n / total >= 0.10) computes total from all local_authority rows including excluded sentinel values (e.g. 'Does not apply'). If sentinel rows make up a large fraction of a place's schools, genuine authorities' shares get diluted against that inflated denominator, which can push a real straddling authority below the 10% cutoff and collapse the display to a single fallback authority even though the place genuinely spans two real authorities.
## 🤖 AI Code Review (Claude Code)
This PR adds a per-place `authorities` list (largest-authority-first, with a share/count threshold and sentinel filtering) so pages like SW19 can name every local authority they straddle instead of just one, and fixes a latent data-loss bug where GIAS's inconsistent spelling of the same town/authority name caused groups to silently overwrite each other in the place registry. The change is thorough and well-tested (backend unit tests, e2e, and frontend component tests all cover the new behavior, including a legacy-cache fallback), and derives `parent_authority` from the same computation as `authorities` to remove a prior mode()/value_counts() tie-inconsistency risk.
### 🟡 Minor
- **backend/places.py**: `_group()` now normalizes spelling/case variants of the grouped column (town or authority name) via `_slugify` before grouping, fixing the overwrite bug — but `_authorities()` still counts `local_authority` values as raw strings without the same normalization. If GIAS has case/whitespace variants for the same authority (analogous to the documented 'London'/'LONDON' town case), they'll be counted as distinct authorities, potentially each falling under the 10% share threshold when combined they wouldn't, or appearing as duplicate-looking entries with the same slug (causing a duplicate React `key={a.slug}` in PlaceView.tsx).
- **backend/places.py**: `_authorities()`'s share threshold (`n / total >= 0.10`) computes `total` from all local_authority rows including excluded sentinel values (e.g. 'Does not apply'). If sentinel rows make up a large fraction of a place's schools, genuine authorities' shares get diluted against that inflated denominator, which can push a real straddling authority below the 10% cutoff and collapse the display to a single fallback authority even though the place genuinely spans two real authorities.
tudor
merged commit d4340a8fdd into main2026-08-21 21:56:51 +00:00
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
You spotted that SW19 is mostly Merton but partly Wandsworth, and the page named only Merton.
The cause was one field doing two jobs.
_parent_authoritytakes the modal authority — correct for a 301 redirect target, wrong as a statement about where a place is. I used it for both.Not a corner case
Measured against the live corpus:
SW19→ Merton 26, Wandsworth 7.Bedfordthe town spans Bedford and Central Bedfordshire.NW6spans Brent, Camden and Westminster.The change
Placenow carriesauthorities— every authority holding at least a tenth of the schools and at least two of them, largest first. The page names them all and links each.parent_authorityis unchanged and still single, because a redirect needs exactly one target.The share threshold is doing real work: GIAS carries postcode errors.
EN6lists two Shropshire schools among fourteen in Hertfordshire, and a bare "any authority present" rule would print those as though they were true. A place too small or too fragmented to clear the threshold still names its largest, so the line never goes silent about where the place is.The frontend also falls back to the single parent when
authoritiesis absent, so a cached API response from before this change cannot blank the line.Verification
Backend 105 · frontend 256 ·
tsc --noEmitclean ·next buildgreen · 85 e2e journeys, including one that finds a straddling outcode in the live registry and asserts every named authority is linked on the page.Six backend tests cover the rules directly: the SW19 shape, the redirect target staying single, the stray-authority threshold, the sentinel filter, and the never-silent fallback.
Note
One test initially passed a loose
/and/matcher that also matched "Wandsworth" — tightened to assert the summary line reads "Merton and Wandsworth".🤖 Generated with Claude Code
https://claude.ai/code/session_015mWQnpye9F299NVRCCSRvj
🤖 AI Code Review (Claude Code)
This PR adds a per-place "authorities" list (backend Place.authorities, /api/places/{kind}/{slug} response, PlaceView UI) so pages naming a boundary-straddling place (e.g. SW19: Merton + Wandsworth) list every authority with a meaningful share instead of just the dominant one used for redirects. It's well tested at the unit, component, and e2e level, includes a backward-compatible fallback for cached API responses missing the new field, and correctly wires all four place kinds (town, locality, authority, outcode) without touching CI/deploy config.
🟡 Minor
93a0a3c26ftobb2f7a5841Both findings fixed, the branch is rebased onto
main(it was conflicting with #120), and chasing the first one surfaced a worse bug.1. The cap — removed, not logged
You're right that it contradicted the stated goal. I removed
_AUTHORITY_MAX_SHOWNentirely rather than adding telemetry: the share rule was always the real limit and already bounds the list at ten, so an arbitrary cap of three only ever made the field a lie in the case where the information matters most.Measured against the live corpus, the cap would truncate exactly one place today — the town
LONDON, split evenly between Hackney, Lambeth, Westminster and Lewisham, two schools each. Zero outcodes are affected (the maximum is three).2. The tie — fixed by construction
parent_authorityis now derived fromauthorities[0]rather than computed separately, somode()andvalue_counts()can no longer disagree. One computation, one answer.It also inherits the sentinel filter as a side effect, which fixes something neither of us had flagged: a place whose modal authority was
Does not applywould previously have redirected to/schools/authority/does-not-apply.3. What the truncation case exposed
Looking at why
LONDONhad four authorities showed thatLONDONis a separate town value fromLondon— and both slugify tolondon.Places were grouped by raw town value while the registry is keyed by slug, so
out[place.key] = placesilently overwrote. Five town slugs come from more than one spelling:londonLondon/LONDONweston-super-mareWeston-super-Mare/Weston-Super-Marenewcastle-under-lymenewcastle-upon-tynebury-st-edmunds/schools/londoncould have shown twelve schools instead of 1,819, silently, depending on row order. Grouping is now by slug, with the most common spelling as the display name.Verification
Backend 110 · frontend 258 ·
tsc --noEmitclean ·next buildgreen · 85 e2e journeys.Five new backend tests: no cap, the redirect target matching the first-named authority, the sentinel never being a redirect target, spellings merging rather than overwriting, and the merged place taking its most common spelling.
The rebase conflict was in the test file only — #120's alignment tests and this branch's authority tests are both additive, so both were kept.
🤖 AI Code Review (Claude Code)
This PR adds a per-place
authoritieslist (largest-authority-first, with a share/count threshold and sentinel filtering) so pages like SW19 can name every local authority they straddle instead of just one, and fixes a latent data-loss bug where GIAS's inconsistent spelling of the same town/authority name caused groups to silently overwrite each other in the place registry. The change is thorough and well-tested (backend unit tests, e2e, and frontend component tests all cover the new behavior, including a legacy-cache fallback), and derivesparent_authorityfrom the same computation asauthoritiesto remove a prior mode()/value_counts() tie-inconsistency risk.🟡 Minor
_group()now normalizes spelling/case variants of the grouped column (town or authority name) via_slugifybefore grouping, fixing the overwrite bug — but_authorities()still countslocal_authorityvalues as raw strings without the same normalization. If GIAS has case/whitespace variants for the same authority (analogous to the documented 'London'/'LONDON' town case), they'll be counted as distinct authorities, potentially each falling under the 10% share threshold when combined they wouldn't, or appearing as duplicate-looking entries with the same slug (causing a duplicate Reactkey={a.slug}in PlaceView.tsx)._authorities()'s share threshold (n / total >= 0.10) computestotalfrom all local_authority rows including excluded sentinel values (e.g. 'Does not apply'). If sentinel rows make up a large fraction of a place's schools, genuine authorities' shares get diluted against that inflated denominator, which can push a real straddling authority below the 10% cutoff and collapse the display to a single fallback authority even though the place genuinely spans two real authorities.