Three journeys pin the fixes: a non-selective secondary's row has no
Selective tag, a school recorded with no religious character has no faith
tag, and the list and the school page agree on a secondary's pupil count.
Run against staging before the fix, all three fail.
The list API reads page_size. Nine journeys asked for per_page, which it
ignores, so each got the default 25 rows whatever it asked for.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
fact_performance's total_pupils is the cohort a year's results were
measured on. For a secondary that is Year 11 alone, and the list API sent
it as the card's "pupils": Burntwood showed 245 in search and 1,462 on its
page. The list and place payloads now carry the register's whole-school
count, and nothing when the register has none. Map popups and the compare
basket read the same field.
The header and the wellbeing section fell back to the same results figure
when the census had no record. They now fall back to the register count.
The shell took yearlyData only for that fallback, so the prop is gone and
the results array no longer ships to the client for the chrome.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The secondary row tested the admissions policy with includes('selective'),
which "Non-selective" passes, so every comprehensive (Burntwood, Graveney)
was tagged Selective. It now needs an exact "Selective".
Both rows excluded only "Does not apply" from the religious character, so a
school recorded as "None" got "Faith priority" (Putney High) or a bare
"None" chip (Abacus Belsize Primary). hasReligiousCharacter() treats both of
the register's no-faith values as no faith.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The phase filter listed GIAS phases alphabetically, so "16 plus" and
"All-through" came before Nursery. /api/filters (and the result-scoped
list) now order them Nursery, Primary, Middle deemed primary, Middle
deemed secondary, Secondary, 16 plus, then All-through, which spans the
whole path. A phase GIAS adds later follows the known ones, A-Z.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The desktop toolbar's Clear went to the bare home URL, dropping the
search along with the filters, so it landed on the home page instead of
the unfiltered results. It now clears every filter and keeps the search,
postcode and distance, as the phone's Clear all already did. It is
offered only once a filter is applied, since with only a search there is
nothing for it to clear, and is labelled Clear all to match the phone.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The close button was 40px square: a 24px icon in 8px of padding. MOBILE.md
sets 44px as the minimum, and the phone filter sheet journey, which
checks every target in the sheet, has failed on it in the staging E2E
gate since the sheet shipped. Measured on staging with the rule
injected: the button is 44x44, nothing in the sheet is under 44, and
the page does not overflow. Every Modal gets the same button, the
compare page's school search included.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The School type filter split state schools into "academy or free
school" and "council-run". The two were near-halves of one pool (11,186
and 9,316 schools), so choosing one rarely narrowed anything, and the
split did not follow the difference a parent feels most, admissions:
voluntary aided and foundation schools set their own, as academies do.
Faith, which voluntary aided mostly meant, has its own filter.
They are now one group, "State school (free)", leaving five. The old
keys academy and council resolve to state, so a link made with them
keeps working instead of falling through to the raw-label path and
returning nothing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The row repeated the search, local authority and school type above the
results list, and nothing else: no phase, distance, gender, sixth form,
admissions or faith. The filter bar already shows every one of them, in
its selects and More filters count on desktop and as removable chips on
phones, so the row was a partial second copy.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The /api/schools section still described filtering on stored
school_type_group and faith_groups columns, which the architecture
correction had already dropped: the filters look up the name columns at
request time. Also trims a trailing blank line from two test files.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The results list's chip printed the school type group's key ("post16")
rather than its label. A URL value that no option spelled the same way
(an old link's raw GIAS type, or a key in another case) left the select
reading "Any school type" while the list stayed filtered, and choosing
"Any" changed nothing. Values now match options ignoring case, as the
API does, and an unlisted one is shown as an option of its own.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The secondary-only filters were hidden for an exact set of lowercased
labels, so a change of case, hyphen or spacing in a GIAS phase label
would have left them showing. Phases are now matched on their words, as
compareGroups already reads "primary".
The E2E journey no longer counts school types: it asserts that choosing
one leaves the same list on offer, whatever the data holds. A test now
pins that a filter with no options is left out, not shown with only its
"any" option.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
School type, gender and admissions took their options from the result
set, which the filter had already narrowed: choose "Girls" and only
"Girls" was offered, so switching to "Boys" meant clearing first. They
now offer the full lists from /api/filters, as phase already did. Local
authority stays scoped to the results, so a postcode search offers the
councils nearby rather than all 153.
Whether gender, sixth form and admissions show was also decided by the
results (any secondary school in them). It is now decided by the phase
alone: hidden for Primary, Nursery and Middle deemed primary, shown
otherwise. Choosing one of those phases clears the three filters, which
would otherwise stay applied with no control showing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The sheet's controls were disabled while a filter change navigated, and
a control disabled under focus drops it to <body>, out of the dialog.
They now stay enabled, with aria-busy on the sheet instead.
The disabling had also been covering a race: updateURL built from
useSearchParams, which only catches up once a navigation lands, so a
second change made before then undid the first. It now builds on the
URL the navigation in flight is heading to.
The sheet also closes if the screen widens past phone width while it is
open, so its selects and the desktop row's are never both showing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On phones the results toolbar's filters were a sideways-scrolling row
led by "More filters", so phase showed only in part and school type not
at all, and the panel "More filters" opened held neither of them.
Phones now get a single Filters button beside the folded search summary,
counting every applied filter. It opens a bottom sheet with every filter:
distance as five segments, then phase, school type, local authority and
the secondary-only filters. Changes apply at once, as on desktop, so the
footer's "Show N schools" only closes the sheet. Applied filters show as
removable chips on a second line, which appears only when something is
applied. Desktop and tablet are unchanged.
Modal gains dialog semantics, a pinned footer and focus handling, and
moves above the pinned toolbar, the floating List/Map button and the
comparison toast, which its old z-index sat beneath.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The pinned-toolbar journeys scrolled the results of a typed B1 1BB search.
At the new half-mile default that is two schools, so the page ended before
the toolbar had anything to stay pinned over (desktop scrollY stopped at
229; on phones the scroll reached the footer). They now open the search
at a mile, which is what they were measuring.
Leaflet crossfades popups for 200ms, so for that window there are two
.sc-popup elements, and a strict locator fails at once rather than
retrying. Wait for one popup before reading it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
leaflet.css colours every link in the map `.leaflet-container a`
(#0078A8, specificity 0,1,1), which beat .btn-tertiary (0,1,0), so the
card's View button came out in Leaflet's link blue in both themes. Pull
it back onto --text-secondary next to the other Leaflet overrides.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adds "Within 0.25 miles" to the Distance control; 0.5 stays the default.
The radius shown in the results heading and drawn on the map was rounded
to one decimal place, which would have turned 0.25 into 0.3. It now keeps
two, and the list heading says "1 mile" rather than "1.0 miles".
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
New postcode searches, the near-me button and URLs without a radius now
use 0.5 miles. A postcode URL with no radius used to show "1 mile" in the
Distance control while the API applied its own 5-mile default; the page
and the map fetch now send the same default the control displays.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The results toolbar wrapped wherever it ran out of room, and the List/Map
switch only appears when there are results, so the same search took two
lines with results and one without.
From 1340px the controls never wrap away from the search, which takes
what they leave (at least 12rem); phase and type chips cap at 11rem to
fit. Between 641px and 1339px the controls always take a full line of
their own. The switch now sits in FilterBar's row via a viewSwitch slot,
so that line runs the full width instead of stopping short of it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
/api/schools only recognised primary, secondary and all-through. Any
other phase the search page offers (nursery, 16 plus, middle deemed
primary/secondary) fell through to no filter, so "Nursery" returned the
whole result set, mostly primaries. Ungrouped phases now match exactly,
and an unknown phase returns nothing rather than everything.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The phase select read its options from the result-scoped filters, which
the backend computes after applying the phase filter. With secondary
chosen only secondary and all-through were offered, so switching to
primary meant going back to "Any phase" first. Read the global phase
list instead.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Beside the map, "19 schools within 1.0 miles of SW196AR" (359px) and the
sort menu (281px, as wide as its longest option) could not share the
pane's 427px, so the sort dropped to a second line.
In the map view the heading now reads "19 schools within 1 mile": the
postcode is already in the search box just above, and the radius reads
as the toolbar's does. The sort menu is capped at 11rem, truncating a
long choice, and the header no longer wraps; on a narrow pane the count
truncates rather than pushing the sort down. List view keeps the full
heading. Tried on staging by injecting the change: 243px + 176px, one
line at 1440.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The rows' narrow layout now switches at a 608px list, not 600px. Below
769px the page pads 1rem each side, so a 640px screen gives a 608px
list: exactly the old max-width: 640px media query, where 600px left
633-640px screens on the wide layout.
- rowContainerGuard.test.ts fails if anything other than HomeView renders
SchoolRow or SecondarySchoolRow, or if one of HomeView's row lists loses
its `results` container. Outside one the rows silently keep their wide
layout on phones. (Checked: HomeView is the only importer today.)
- Picking a pin from the list beside the map now works from the keyboard:
each row carries a "Show <school> on the map" button, visually hidden
until focused, with aria-pressed for the selected school. The row itself
cannot be the button, since it holds links and buttons of its own.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The list beside the map used its own cut-down card, so it showed less
than the list view (no phase, age range, denomination, trend, local
authority) and looked different. It now renders the list view's
SchoolRow / SecondarySchoolRow, through one renderRow shared by the list
view, the map pane and the phone's bottom sheet, which also gains
Remove from compare. CompactSchoolItem and its styles are gone.
The rows switched to their stacked layout on a max-width: 640px media
query, which never fires in a ~430px pane on a desktop screen. It is now
a container query on the list the row sits in (`results`, set on the
list view, the map pane and the bottom sheet). 600px of list is a 632px
screen less padding, so phones lay out as before.
Clicking a row, outside its links and buttons, still picks its pin, and
the selected row is outlined. The pane widens to 360–460px for the
taller rows.
The bottom sheet's close button was a 30px target; it is now 44px with
the same 30px circle (MOBILE.md), and the phone journey checks it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The search summary ("SW196AR · within 1 mile Edit") ran about 40px off
the right edge on every phone, cutting "Edit" in half, and the filter row
bled past the screen with it (reported on an iPhone 17, 402px).
The phone toolbar is a column flex container, but it kept flex-wrap:
wrap from the desktop rule. In a wrapping flex container each line is as
wide as its widest item's content, not the container, so the summary's
~410px of text stretched the line past a 370px toolbar. nowrap makes
stretch mean the container's width. Verified on staging by injecting the
rule: nothing in the toolbar passes the right edge at 360 or 402px.
The E2E overflow check could not see this: .main clips on x, so
scrollWidth stayed equal to the viewport. The phone journey now measures
the toolbar's own right edges, and runs at 402px as well. It also waits
for the bottom sheet's slide-up to finish before measuring, and measures
the Map/List button again after the sheet pushes it up; both assertions
read a mid-animation or stale box before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Trial the map as the default for every postcode search, not only on
desktop (DEFAULT_LOCATION_VIEW in HomeView; results_view_changed shows
how often readers switch away). The view is now derived: the reader's
choice once made, else the default for the kind of search. That follows
a hero search through client-side navigation without syncing state.
With nothing left varying by device, the user-agent default goes
(lib/device.ts, the headers() read in page.tsx). That settles two review
points: the HTML no longer needs Vary: User-Agent, and there is no
narrow-window fallback left to flash the map before the list.
Review fix in LeafletMapInner: the card reopened only when the selection
or the school list changed, so a rebuild caused by the centre, reference
point or radius closed it for good while the page still had the school
selected. The selection effect now follows a pins-rebuilt counter.
SchoolMap memoises its fallback centre, which was a fresh array on every
render and would have rebuilt every pin each time.
Phones hide the map view's list pane, so it no longer builds up to 500
hidden cards there (decided after mount, so hydration still matches).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The map view now follows option B of the results-controls mockups:
- The list sits in a pane on the left, under the result count and sort,
and the map fills the rest of the screen below the pinned toolbar. The
toolbar's height is measured, so the split ends 1rem above the bottom
of the screen however the controls wrap.
- Pins are brand-teal dots and the selected one is coral. The search
location is an ink dot, the search radius a dashed circle with its
distance, and the view fits that circle. Tiles are muted and the zoom
sits under the fullscreen button.
- A school picked on the map or in the list opens a card on the map
(View, + Compare, following the basket), and its list card is ringed
and scrolled into view. Phones keep the bottom sheet.
- The list cards show the full name, Ofsted and school type, the
headline figure and pupils.
The map cards now follow the list rows: no England benchmark for
special schools, PRUs and AP, and no placeholder all-zero RWM (Greenmead
showed "0% RWM -62 pts vs national"). That rule moves to a shared
listRwmValue helper.
A postcode search opens on the map for desktop browsers, chosen on the
server from the user agent so the list never paints first. The client
falls back to the list below 1024px, and follows the default through
client-side navigation from the hero until the reader picks a view.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Refold the phone search, and resync its text, when the query changes
without a submit (back/forward, "Use my location"). The summary and the
unfolded input could otherwise describe two different searches.
- Cap the "More filters" panel at every width, not only on phones, so an
open panel scrolls inside the pinned toolbar on a short screen.
- Drop the phone filter row's right-edge fade once it is scrolled to the
end or does not overflow (the MOBILE.md pattern), and give focus rings
room inside the scroll clip.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The List/Map switch was a small grey control beside the results heading,
and the filters were plain dropdowns labelled "All Phases" and "Advanced".
Both scrolled away with the first result.
Search, filters and the List/Map switch now share one card pinned under
the header. Distance, phase and school type are pill controls in the row;
"Advanced" becomes "More filters" and counts only what it hides. The
switch is filled brand teal and says which view is on (aria-pressed).
On phones the search folds to a one-line summary once made, the filter
pills scroll sideways, and a floating Map/List button sits above the tab
bar in place of the toolbar switch. The selected pin's card now stacks
under that button instead of covering the tab bar.
Switching view from far down the list scrolls back to the top of the
results, and each switch is tracked as results_view_changed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review of the per-phase limit found entries reaching the basket with no
phase, and a phase-less entry holds a slot in both groups:
- Nearby-school cards added without one. The API now returns each
peer's own phase (its pool is a phase group, so an all-through school
can sit beside a primary); the button passes it through, and an older
API simply leaves the conservative both-groups count in place.
- Baskets saved before this change were never migrated. The compare
page now backfills missing phases from the data it already fetches,
and never overwrites a phase the basket has.
- "16 plus" counted against both groups; it is secondary, as the API's
PHASE_GROUPS files it.
Also rewraps the HomeView doc comment the previous commit left awkward.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A parent choosing a primary and a secondary school at once hit the old
cap of five total. The basket now holds up to five primary and five
secondary schools (ten in all), matching the compare page's phase tabs.
Schools that could land in either tab (all-through, special schools with
phase "Not applicable", unknown phase) count against both groups, so no
tab ever exceeds the five-slot chart palette and point styles.
- lib/compareLogic: compareGroups, fullGroupFor, fitToGroupLimits
- search modal disables only the full group and says which one
- rankings rows carry the phase of the tab they are ranked under
- shared ?urns= links are trimmed per group
- copy: compare metadata, homepage value prop, How it works card now
name primary and secondary schools (also better for search intent)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Rewrites every visitor-facing string that used an em dash, choosing the
punctuation for what the dash was doing: a colon before a list or
explanation, a comma for an aside, a full stop between two thoughts,
parentheses for an aside mid-sentence. Covers page titles and meta
descriptions, the home and admissions guide copy, school page headings
and notes, the compare page, metric labels and tooltips.
Two rewrites also fix the sentence around them: the closure banner no
longer repeats "proposed for closure", and the cut-off caveat's list of
priorities now parses.
A lone dash marking a missing value in a table cell stays: it is a data
convention, not prose. A Jest guard walks the source with the TypeScript
parser and fails on any other em dash in a string or JSX text node.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review catch, and a bad one: the previous commit anchored its insertion on
`padding: 0.5rem 0.75rem;` and closed .sectionNav there. Everything that
followed in the rule — margin-bottom, box-shadow, display: flex, align-items,
gap — was orphaned into .sectionNavSheetOpen, which is only applied while the
mobile jump sheet is open.
So the sticky nav lost its flex layout, spacing and shadow in the closed
state, which is virtually every page view on every school detail page. A
site-wide regression introduced by a fix for one mobile menu.
Redone by anchoring on the complete rule, closing brace included, so nothing
can be orphaned. .sectionNav is now byte-identical to main and the diff is
purely additive; .sectionNavSheetOpen carries the z-index and nothing else.
The staging experiment that validated this fix set nav.style.zIndex = '1100'
with every other declaration intact, so it was always testing this version
rather than the broken one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
On mobile the last item in "Jump to section" was painted over by the fixed
bottom tab bar and could not be tapped. Reported as nearby schools missing
from the menu; it was there, underneath the bar.
The sticky nav sets `position: sticky` with `z-index: 10`, which makes it a
stacking context. The sheet's own `z-index: 1600` therefore orders it only
inside that context — against the tab bar (z-index 1000) the nav's 10 is what
counts, so the bar wins. Verified on staging: every menu item returns itself
from elementFromPoint except the last, which returns the tab bar.
Latent rather than new. With five sections the list stopped just above the bar;
"Nearby schools" made six, and the sixth is the first to reach it. Any section
added later would have done the same.
Lifted only while the sheet is open, and only to 1100 — above the bar, below
the comparison toast (2000), the fullscreen map (5000) and the info popover
(9999). The backdrop rises with it, so tapping over the bar now dismisses the
sheet instead of navigating away.
The journey asks what a thumb asks: for each item, whether it is the topmost
element at its own centre. A bounding-box check cannot see this — the item is
in the viewport and the right size, just underneath something. Confirmed to
fail against current staging, naming "Nearby schools", before the fix.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Second failure of the same assertion, and the first fix addressed a real but
different problem. This is the one that was actually producing the number.
The arrows scroll with `behavior: 'smooth'`. The test polled for "has it moved
at all" — satisfied 50ms in, at 13px of a 1300px journey — then recorded the
offset, clicked, and recorded again. The row was still travelling throughout,
so the delta it measured was the tail of the arrow's animation, not the effect
of the selection. Hence a deterministic 659, roughly half of the 1317 this
school's row scrolls.
Measured against staging rather than reasoned about: the animation runs about
700ms, and the samples are in the helper's comment.
settledScrollLeft waits for two identical readings before trusting one. The app
was never at fault — driving staging by hand, the offset holds at 1317 across
the selection, exactly as intended.
Verified against staging both ways: main's version of this test fails there,
this version passes, along with all three mobile widths.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The assertion that adding to compare does not reset the carousel failed on
staging: 537 → 2. The app was not at fault. Playwright scrolls a target into
view before clicking, and the test clicked the FIRST card's button after
paging the row to the end — so Playwright scrolled the container back to the
start, and the assertion measured that.
Reproduced on a static page with no React on it: a snap scroller at 615,
Playwright clicks the off-screen first card, scrollLeft becomes 2. Scroll-snap
was ruled out first — mandatory, proximity and no-snap all behave identically
when the button mutates in place.
Now clicks the last card's button, which is visible at the end of the travel,
and allows a few pixels for snap and sub-pixel adjustment while still failing
on a reset to the start. The property was never actually under test before;
it is now.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The spec described the tier system as the design of record. It is gone, so
the document was describing something the code deliberately does not do.
The revision note and the "why not, having built it the other way first"
passage are kept rather than overwritten. The mistake is the instructive part:
treating a preference as a constraint inverted the ranking, and the stopping
rule added to prevent weak distant matches is what guaranteed six Catholic
schools and no community school down the road. A spec that quietly presents the
second design as the plan teaches nobody why the first one failed.
The mockup link is annotated as one revision behind rather than silently left
to look current.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The section ranks on distance and is headed "Other schools nearby", but every
identifier still called it "similar" — the exact drift that leaves a later
reader trusting a name over the behaviour.
Mechanical: files, the module, the payload key, the type, the components, the
prop. No behaviour change; the suites are unchanged in count and still green.
Free to do now because #150 has not merged, so the payload key rename needs no
lockstep deploy. Uses of "similar" that are ordinary English — progress
measures compared to similar pupils, and unrelated comments — are untouched.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Reported from staging: a Catholic primary showed six Catholic primaries, none
of them close enough to be a real option, and omitted the community school
down the road.
Three causes, compounding. Ranking put tier before distance, so a faith match
at 2.9 miles outranked a community school at 0.3. The ENOUGH=3 stopping rule —
added so a cap of six would not drag in weak distant matches — filled the row
from the best tier before it ever widened, which is what made every card
Catholic. And a 3-mile tier-1 radius is sane for a secondary and most of a city
for a primary, whose catchments are routinely under a mile.
The premise was backwards. For a parent, distance is a constraint and intake is
a preference; a school beyond a primary catchment is not a weaker option, it is
not an option. So distance now decides the order and nothing else does. The
hard filters are untouched — they were always where the defensibility lived.
Similarity survives as chips on the card: reported, so a reader applies their
own weighting, rather than ranked, so we apply ours for them.
Reach is capped per phase (primary 2, secondary 6, post-16 10) as a sanity
bound, not a target: ordering already handles density, so the cap only decides
what happens where an area is sparse. A primary with nothing inside two miles
now renders no section, which is the honest answer.
Deleted: the tier system, the stopping rule, the tier-dependent lede, the
`tier` field, the tier-3 fallback chip and its style. select_similar also stops
taking is_secondary — it reads the phase from the subject's own row, so no
caller can hand it one that disagrees with the data.
The heading is now "Other schools nearby". The hard filters still guarantee a
comparable set, but nothing ranks on likeness, so the heading no longer says it
does.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The spec said the metric follows the template the page renders. That is no
longer true for GIAS phase 6: a sixth-form college renders the primary
template but is matched, correctly, against secondaries. The rule is phase
group membership, decided once in is_secondary_phase — and the section's lede
noun comes from the school's phase rather than its template for the same
reason.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
GIAS phase 6 is "16 plus", and PHASE_GROUPS deliberately files it in the
secondary group. The payload helper decided the same question with
`"secondary" in phase_text`, which that value does not satisfy — so a
sixth-form college was handed the primary bucket and offered infant schools
as its peers, with the KS2 metric key to label them. No crash; just a page
confidently showing the wrong schools.
The decision now lives in similar_schools.is_secondary_phase, beside the
PHASE_GROUPS bucket it selects from, so the two cannot drift again. A test
pins them together.
The same binary assumption had a second output. computeSchoolFlags tests for
the substring too, so a 16-plus school renders the primary template, and the
composer was labelling the section from the template: "Other primary schools
near <sixth form college>" above a row of secondaries. The section now
derives its noun from the school's own phase, which also removes the
duplicated wording from both composers. A 16-plus school's candidates span
the whole secondary group, so no single noun fits and it gets the honest
general one.
Reported in review on #150.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two things here cannot be covered anywhere else. jsdom has no layout, so
scrollWidth and clientWidth are both 0 and the arrows' disabled state can only
be measured by a real engine. And the scroll position surviving a selection is
DOM state rather than React state, so only a real browser can prove the row
does not jump back when the footer re-renders.
MOBILE.md asks for a Playwright width check and records that it was not written
because Playwright was not in the project. It is — this suite — so the check
exists now, scoped to the page this feature touches.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Inside SchoolDetailShell rather than after it, because the sticky nav's
scroll-spy finds sections with getElementById and can only reach one that
lives in the shell. Last in the order, and last in the nav, because the two
must agree or the nav links to an anchor that was never rendered.
hasSimilarSchools is optional on NavItemsInput, matching hasLocation beside
it: absent has to mean "no section", and making it required would have
churned ten unrelated call sites for no added safety.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Server-rendered cards inside a client carousel that scrolls rather than
paginates, so all six links stay in the initial HTML and the row still works
with JavaScript off. Three client islands, split by what each needs: a school,
the whole selection, a DOM ref.
The lede claims a similar intake only when no card came from tier 3, chips
list what a school actually shares, a missing figure reads "Not published",
and the neighbour's number carries no valence colour — green and terracotta
mean "against England" everywhere else, and colouring it here would read as
ranking the neighbours.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Rides in the existing payload rather than a new endpoint: the page already
makes one server fetch for its data, and /school/[slug] regenerates weekly,
so the per-request cost is paid once per school per week.
Wrapped so a failure in selection never 500s a page that is otherwise
complete — the posture get_supplementary_data already takes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Hard filters encode claims the section may not make — a selective school is
not an alternative to a non-selective one, a special school is not comparable
to a mainstream one, a Girls school is not an option for a Boys school's
reader — so they never relax. Soft preferences describe closeness of fit, so
they relax across three tiers, and only far enough to reach three; the
remaining slots up to six fill from the tiers already opened.
PHASE_GROUPS moves to schemas.py so this module can share it without
importing app, which would be a cycle.
_mask() exists because Series.apply on an empty Series returns a DataFrame,
and using that as a mask drops every column — so the next lookup raises
KeyError instead of yielding no rows. A special school with no special school
near it empties the frame at the provision filter, which is the ordinary case
for most special schools, so this was a crash on a common path rather than an
edge case.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Checked the section against MOBILE.md at its three reference widths instead
of assuming the breakpoints were enough. Two real failures at 360px.
The arrows sat in the heading's flex row, taking 96px from a 328px card and
crushing the lede into a four-line column — for a control that swiping
already provides. Below 640px they are now gone, the header is a single
column, one card shows at 86% so the next one peeks, and the affordance is
the right-edge scroll-fade MOBILE.md already documents for horizontal
scrollers. The fade lifts at the end of the travel, so the at-end state is
computed whether or not an arrow exists to consume it.
The arrow and add-to-compare buttons were 40px against a 44px floor. Both
are 44 now. A card title's own box is shorter, but its hit area is the whole
card through the ::after overlay, so it passes on the target that actually
receives the tap.
360, 390 and 430 now all report zero overflow, no failing tap targets and no
text under 11px. MOBILE.md wanted a Playwright width check and recorded that
Playwright was not in the project; it is, so the journey now carries one for
this page.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three schools is not a neighbourhood in inner London, so the cap is six with
three visible and arrows for the rest.
Raising the cap exposes something the old cap hid. Tiers exist to reach a
usable set, and with six slots a naive loop would keep widening to fill them
— dragging in tier-3 schools ten miles away to sit beside three good matches
that had already earned the row. So tiers now stop relaxing once three are
found, and the remaining slots are filled only from the tiers already used.
Four tier-1 matches never open tier 2.
The carousel scrolls a list rather than swapping a view: all six cards are in
the initial HTML, so every link stays crawlable and the row still scrolls with
JavaScript off. The arrows' edge test carries an 8px tolerance because the
scroller's focus-ring padding is the first snap position — a row at rest
reports scrollLeft 2, and an exact test for 0 left the back arrow live and
pointing nowhere. Caught in the mockup.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The "how these schools are chosen" panel restated what the section already
shows — the phase in the lede, the shared characteristics on each card, the
distance above each name — so it cost space to say nothing new.
One line survives, and it is not a method note. A reader who sees "0.6 miles
away" and takes it for the walk has been misled by us, and no other element
on the card corrects that. The rest were claims the selection rules keep
true without narrating them.
Also records what happens past the third school: surplus matches are dropped
silently, because NearbyPlaces below already leads to the full lists.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Five tasks, each ending in a green test run and a commit: the pure
selection module, the endpoint key, the section and its two client
islands, the wiring into both templates, and the journey.
The selection logic gets its own module rather than another 200 lines in
app.py, which means the tier rules are testable against a synthetic frame
with no TestClient, no database and no monkeypatch. Moving PHASE_GROUPS
to schemas.py is what keeps that import acyclic.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A school page links outward to its places and never to another school.
This section adds that edge: three nearby schools of the same phase and a
comparable intake, each a crawlable link and each addable to the basket.
The design separates hard filters from soft preferences and never confuses
them. Selectivity, provision and opposite-sex intake are claims the section
cannot make, so they never relax, even where that means no section renders.
Gender and religious character describe closeness of fit, so they relax in
tiers — and the card states what actually survived rather than padding with
a match it did not earn.
Includes the mockup the design is drawn against, with all three tier states
live in both themes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The staging poller swallowed every failure identically, so a run that
timed out told us only that the expected release never appeared — not
whether the proxy refused us, the endpoint was down, or the containers
were still serving an older build. The public staging proxy also answers
403 to urllib's default user agent while the release endpoint is healthy,
which looked exactly like a deployment that never arrived.
Identify the poller, and report each distinct observation once: HTTP
status, connection failure type, invalid JSON, or the release identities
actually reported. The timeout error carries the last observation and the
identity it wanted. Responses and the base URL stay out of the logs —
only validated sha/build_id fields are echoed back.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Fetching every match kept scoped searches correct but left the number of
round trips in the caller's hands: a one-letter query, or a deliberately
broad one, walked the whole collection a page at a time.
Cap the candidate set at 1,000 URNs — four pages — and return the
relevance-ordered prefix when the ceiling is hit. That is still far more
than one page, so the API's own authority, phase and postcode filters
keep the matches they need, while latency and upstream load stay bounded.
A capped query is logged so a genuinely truncated search is visible.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Staging health polling asked only whether something answered HTTP 200 at
the base URL. It could not tell the new deployment from the old one, so
journeys could pass against the previous release, and concurrent merges
could move the staging tags underneath a run in flight.
Each staging run now mints a build ID and stamps all three images with
the commit and that ID, as labels and — for frontend and backend — as a
build-time JSON file that environment overrides cannot rewrite.
/release.json reports both identities uncached, and scripts/ci/release.py
polls for the expected pair before and after the journeys. Only then are
the captured build digests tagged verified-<sha>.
Promotion resolves those verified tags to immutable digests, revalidates
their labels, and refuses a mixed or incomplete set before any :prod tag
moves. The whole staging workflow shares one concurrency group with
cancellation disabled, so releases serialise.
The scripts are stdlib-only and unit-tested against mocked registry and
HTTP behaviour; PR checks now run the pipeline and CI suites too. The
runbook records what this cannot prove locally, and that the first
rollout needs a commit built by this workflow.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The home page caught every fetch failure and rendered its empty state,
so a backend outage looked like a site with no schools in it. School
pages turned any error into notFound(), which told visitors — and
crawlers — that a real school had ceased to exist. Place fetches did the
same by returning [] and null. Failures now reach a retryable error
boundary; only a genuine 404 still calls notFound().
"Load more" and the map fetch resolved against whatever state existed
when they returned, so results from an abandoned search appended
themselves to the new ones. Each fetch now carries an AbortController
and checks that its search scope is still current before touching state.
The map only records its cache key on success, so a failed load retries
instead of pinning the stale marker set.
Jest ignored .next/, whose build output otherwise shadowed real suites.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Reload cleared the caches first and rebuilt afterwards, so any failure
left the API serving nothing, and requests arriving mid-reload saw a
half-swapped state. It now builds and validates the replacement frames,
place registry, reverse index and sitemaps off the request loop, then
publishes them in one synchronous step under a lock. A failed reload
returns 503 and keeps the previous data. Sitemap regeneration takes the
same path rather than clearing the live registry up front.
Typesense search returned at most one page of hits and used an empty
list for both "no matches" and "search is down", so a genuine empty
result silently fell back to substring matching. It now pages through
every candidate and returns None only on failure; the fallback matches
literally, since a query containing regex metacharacters used to throw.
Empty datasets answer 503 rather than 200-with-nothing or a misleading
404, so callers can tell an outage from an absent school. Adds
/api/release, which reports the build identity baked into the image.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The old sync created a collection, imported batches without reading a
single import response, and swapped the alias regardless. A partial
import published a half-empty index, and two overlapping DAG runs could
prune each other's collections.
Publication now checks every import response and the final document
count before upserting the alias, and holds a session-scoped advisory
lock across the read and the publish so concurrent runs serialise.
Cleanup keeps the previous collection as a rollback pointer and is
best-effort: an uncertain alias response must never delete what might
still be live.
Also parses the Typesense URL properly instead of splitting on colons,
which mangled any host carrying a scheme and a default port.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`backend/migration.py` and `scripts/migrate_csv_to_db.py` import `School`,
`SchoolResult`, `init_db` and `set_db_schema_version` — names that no longer
exist. `scripts/geocode_schools.py` imports the same removed ORM model. None of
the three can be imported against the current backend, so they were not dormant
utilities anyone could fall back on; they were files that would fail on the
first line. `backend/version.py` existed only to hand `SCHEMA_VERSION` to that
importer, and the FastAPI lifespan performs no version-triggered import.
Three symbols go with them, each confirmed to have no caller: the unvectorised
`haversine_distance`, superseded by the inline NumPy calculation in search;
`fetcher`, an SWR helper for a dependency this project does not install; and
`kmToMiles`. `calculateDistance` stays — CutoffMapPanel uses it.
Two comments pointed at `migrate_csv_to_db.py --drop` to explain why Payload
owns its own schema. The reason survives the script: blog content must stay
clear of the school marts and Airflow's metadata. Reworded rather than deleted,
so the constraint keeps its justification.
docs/LEGACY_CODE.md records what was removed and where to find it in history. It
also records what was deliberately *not* removed, which is the more useful half:
unused UI components awaiting a design decision, manual data utilities whose
operators a repository search cannot see, and fallbacks that look obsolete but
are load-bearing — `data_loader.py`'s older-mart branches, the generated GIAS
dictionary copies, and the `legacy`-named dbt models that annual DAG selectors
explicitly include. A zero-import count is evidence, not a verdict.
The scripts that fetch DfE CSVs are marked historical and kept, pending
confirmation that nobody runs them by hand.
Checked: 190 backend tests, 429 frontend tests, `tsc --noEmit` clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016y2J6bs8gbuSJbH18w7Tan
The README still opened on "Primary School Compass", a KS2 tool for Wandsworth
and Merton served by FastAPI and vanilla JavaScript with Chart.js. Every layer
of that sentence is now wrong: coverage is England-wide across KS2, KS4,
all-through and post-16, Next.js owns the public UI, and school data comes from
dbt-built `marts.*` rather than CSVs loaded at startup. The setup instructions
walked a reader into a virtualenv and a CSV import that cannot build the current
schema, so following the docs produced an empty database and a wrong mental
model at the same time.
Replace the narrative docs with two reference documents that were checked
against the code: docs/ARCHITECTURE.md for request flow, data ownership, the
backend/frontend module boundaries and the real publication sequence, and
docs/DEVELOPMENT.md for the checks that actually run, including the container
and CI version skew that makes "just run pytest" misleading.
The env examples drifted the same way. ALLOWED_ORIGINS is a JSON array, not a
comma-separated list; the frontend needs FASTAPI_URL, DATABASE_URL and
PAYLOAD_SECRET, none of which were documented; and RATE_LIMIT_BURST,
DEFAULT_PAGE_SIZE and MAX_PAGE_SIZE were presented as tuning controls the routes
do not consult. Each is now stated as it behaves.
MIGRATION_SUMMARY.md keeps its content but gains a banner, because it reads like
setup instructions and is not.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016y2J6bs8gbuSJbH18w7Tan
The staging E2E gate's one failure. Every link to the site pasted into
a chat has been rendering bare.
Staging serves og:title, og:description, og:url, og:site_name, og:type
and twitter:card, and no og:image at all. So metadata from the layout
reaches the page; only the file convention does not.
app/opengraph-image.tsx does work — _not-found, which lives in the app
root segment, carries an og:image from it in the build output. It does
not reach the site's pages, which live in the (frontend) route group
whose own layout.tsx is the root layout. The icon conventions are not
affected: /icon.png and /apple-icon.png are both linked correctly on
the same page, verified against staging. The asymmetry is the whole
bug, and it arrived with the route-group split that Payload required.
The file stays at the app root. Moving metadata files into a route
group is what drops /robots.txt and hashes /icon.png, which CLAUDE.md
records and which this must not undo — the build still emits all four
of /robots.txt, /icon.png, /apple-icon.png and /opengraph-image. The
root layout points at the route instead, and metadataBase makes it
absolute, which the journey needs since it calls new URL() on the value.
twitter.images is set for the same reason: the card is declared
summary_large_image, and claiming a large-image card while supplying no
image is worse than claiming a summary card.
Checked before fixing that og:image was the only broken assertion in
that journey: the test aborts at line 911, so its apple-touch-icon and
maskable-icon assertions had never run. All four of those assets return
200 image/png from staging, so this does not simply move the failure
further down the test.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DXnXQKnPpZBBP61fBQiFkq
Two review findings, both confirmed before fixing.
The client fixture in test_school_details.py patched load_school_data
but not _place_registry, which is a module-level cache. A probe settled
it rather than an argument: poisoning the global with a registry built
from data the fixture never saw, then issuing the fixture's own
request, returned that other dataset's places. So the new
`places == []` assertion was satisfied by a stale registry exactly as
well as by the fixture's own data, and proved nothing. Every other test
that touches place data already reset it; the fixture predates places
existing and was never updated. It resets it now.
places_for_urn walked every place in the registry and did a tuple
membership test against each, on /api/schools/{urn}, the site's
highest-traffic endpoint. It now reads a dict built once per registry.
Measured against a synthetic corpus of 27k schools in 1,650 places:
0.118ms per request becomes 0.0001ms, with the index built once in
21ms. Production carries ~5,000 places, so the scan there is larger
again. The absolute saving per request is small; the point is that it
is repeated on every school page view and costs nothing to remove.
The index is cached against the registry by identity rather than
behind a second flag. Anything that drops _place_registry — every test
that touches place data does — gets a fresh registry object, which no
longer matches what the index was built from, so the index rebuilds
with it. A separate _place_index = None would be one more thing to
forget, and a stale reverse index is precisely the first finding's bug
wearing a different hat.
That invalidation has its own test, and the test was checked by
breaking the identity check: five tests fail without it, so three
existing ones were already relying on it too.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DXnXQKnPpZBBP61fBQiFkq
Review caught _places_payload documenting a `phase_url` the function
never returned. The docstring was not stray prose: the approved design
included the phase variant — "Primary schools in Beccles" was one of
its four example links — and it was dropped during implementation
without being mentioned. Deleting the sentence would have closed the
report while losing the feature, so the links are built instead.
These are the pages that most needed them. ~950 phase variants were
once reachable by nothing at all: absent from every sitemap and
unlinked from the place page. "Primary schools in brentwood" is the
query they exist to answer.
Membership is read from the registry's own `phase_urns` rather than
re-derived from the school's phase string. The registry already decides
which phases a place publishes and which schools are listed on each, so
asking it is both shorter and the only way the link cannot disagree
with the page it points at. It also means outcodes need no special
case: they carry empty `phase_urns` by design, because nobody searches
"primary schools in SW11", so they report no phase links on their own.
`phases` is a list rather than a single url. An all-through school is
listed on both the primary and secondary pages, so there is no tie to
break and no reason to invent one. Each entry renders directly after
its own place, so "22 primary schools in Brentwood" reads as part of
Brentwood rather than as an unrelated link further along the row.
The e2e journey now follows a phase link where the town publishes one
and asserts it resolves.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DXnXQKnPpZBBP61fBQiFkq
W2 shipped ~5,000 place pages and nothing linked into them. The
location layer pointed down at school pages; school pages pointed
nowhere on the site. Their only anchor was the school's own website, so
the ~27k pages carrying most of the site's inbound authority passed it
straight off-site, and the new corpus was reachable mainly through the
sitemap.
Three things close the loop.
A reverse index over the place registry, places_for_urn, answers which
published places contain a school. Derived from the registry rather
than stored beside it, so the two cannot disagree about which places
exist: a place below the publish threshold is absent from the registry
and therefore never offered as a link. A test asserts that invariant
across every place in a built registry.
GET /api/schools/{urn} gains a `places` array carrying the name, count
and canonical path for each. It rides on the request the page already
makes, so the school page costs no extra round trip. The frontend types
it optional and defaults it to empty, because the two images deploy
separately and a frontend ahead of the API must render without it.
The page gains a "More schools near here" module and a BreadcrumbList.
The module orders narrowest first, because a reader on a school page
wants its town before its county, while the API orders widest first for
the trail. Anchors state their destination's size — "37 schools in
Brentwood" — which is worth more to a reader and a crawler than "see
more". With no published places it renders nothing rather than an empty
heading.
The trail is rooted at the homepage, not /schools. There is no /schools
index page; the location layer lives only at /schools/[place],
/schools/authority/[la] and /schools/near/[outcode]. Rooting it at the
bare path would have opened every breadcrumb with a link to a 404.
Outcodes are omitted from the trail: "schools near CM15" is a real
query and a useful link, but nobody navigates Essex to CM15 to a
school, and a breadcrumb claiming that describes a hierarchy the site
does not have.
School pages also now declare the School type rather than
EducationalOrganization, the parent type that covers universities and
nurseries alike.
The e2e journey asserts the round trip in both directions, following a
place page's own first school so the pair is genuinely related rather
than hardcoded. A one-way link is what already existed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DXnXQKnPpZBBP61fBQiFkq
Both features ship dark. Neither is reachable in an environment where
its flag is off, and every flag in this system starts off, so a deploy
of this commit makes both disappear until someone turns them on
deliberately.
Two independent flags rather than one, which makes blog-on-about-off a
reachable state. That state is the whole reason the change is larger
than four notFound() calls: the blog leans on the About page for its
author identity. The Person entity is anchored at /about#tudor, and
that URL 404s while about_page is dark, so a post published in that
state would claim an author resolving to nothing. Worse than having no
named author. Both bylines fall back to unlinked text and the
BlogPosting attributes to the publisher instead, so every combination
of the two flags renders something correct.
Gated: /about, /blog, /blog/[slug], the RSS feed, both footer links,
and the matching content-sitemap entries. A sitemap must never
advertise a URL that 404s. With both dark it emits a valid empty
urlset rather than a 404, because robots.txt names it unconditionally.
Not gated: /admin. Posts have to be writable before the blog is worth
switching on, so flagging the panel would make the flag unflippable.
getFlags takes a revalidate rather than always using the 300s
constant. Reading a flag pins the calling route to the lowest
revalidate among its fetches, and the footer links live in the root
layout, so a naive gate there would have dropped every school and
place page from a weekly cache to a 5-minute one. The layout passes
604800, the floor those routes already declare, and the build confirms
all four SSG route families still prerender. The cost is one-way
latency: pages follow a flip in minutes, footer links within a week.
The e2e journeys follow the existing paired shape from the
admission_distance flag: a lit journey and a dark one for each flag,
reading state from whether /about and /blog respond rather than from
/api/flags, which another journey asserts is not publicly reachable.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DXnXQKnPpZBBP61fBQiFkq
Creating a post in the admin panel showed no Content editor, and saving
failed validation on the field the writer was never shown.
The admin panel does not import field components. The server hands the
client a path per field, and resolves it through the generated map at
app/(payload)/admin/importMap.js. A richText field's path is
@payloadcms/richtext-lexical/rsc#RscEntryLexicalField. The committed map
held one entry, @payloadcms/next/rsc#CollectionCards, generated before
the blog collections existed and never re-run. A path missing from the
map is not an error the panel reports: the field simply does not render,
while required is still enforced server-side on save.
next build does not regenerate the map, so the stale copy shipped in the
image and the editor was equally broken on staging and production.
Regenerated with payload generate:importmap, which adds the lexical RSC
field, cell and diff components, BlocksFeatureClient for the Callout
block, and the default toolbar features.
Two things stop it drifting again. There was no script to run, so
package.json gets generate:importmap. And a test asserts the map carries
an entry for each thing the config asks for, in the source-reading style
of the other payload suites; against the old map all five fail.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DXnXQKnPpZBBP61fBQiFkq
Staging failed on boot with 42P01, relation "payload.users" does not
exist. The schema was empty because no migration existed, and the
adapter cannot create tables itself: db-postgres/connect.js gates push
on NODE_ENV !== 'production', so it is inert in a deployed container
regardless of config.
The generated migration is schema-qualified to "payload" throughout but
does not create that schema — schemaName says where tables go, it does
not create anything. It only worked against the throwaway database used
to generate it because the schema was created there by hand, so every
real environment would have failed on the first statement. CREATE SCHEMA
IF NOT EXISTS is hand-added at the top of up(), which makes it exactly
the kind of edit a regeneration discards silently; a test asserts it is
present and ordered before the first CREATE TABLE.
payload-types.ts is now committed rather than ignored. Ignoring it meant
CI typechecked against looser types than a developer with a generated
copy, which is how a Record<string, unknown> cast passed CI and then
failed locally the moment the file appeared. The post page uses the
generated Post and Media types instead, and narrows heroImage rather
than asserting it, since the field is an id at shallow depth.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017YmbBhr8s7GusjDE12hrZM
The em dash is one of the clearest tells of machine-written text, which
is the exact impression this work exists to remove. Rewritten rather
than substituted: where a dash was carrying a real aside the sentence is
split or recast, not patched with a comma.
Covers the About page, the two Callout labels an editor sees in the
admin panel, and PUBLISHING.md, which defines the house style and should
follow it. The rule is now recorded in that house style and in the
spec's voice rules, so it survives this branch.
Code comments are left alone: they are not copy, and the surrounding
codebase uses the same punctuation throughout.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017YmbBhr8s7GusjDE12hrZM
Review findings on #140.
Drafts were reachable. Posts granted unconditional public read and the
_status filter lived only in the pages that query the collection — which
is a convenience, not a control. Payload's documentation is explicit:
"The `draft` argument alone does not restrict documents with _status:
'draft' from being returned by the API." A direct GET /cms-api/posts
would have handed every unpublished draft to any visitor. Read access
now returns a query constraint for anonymous callers, which is the
documented mechanism.
The --drop claim was asserted across four files while the spec still
listed it as an open question. Now verified rather than assumed:
run_full_migration drops exactly ["school_results", "schools"] by name,
there is no drop_all() or DROP SCHEMA anywhere in backend/, the only
other drop is schema-qualified to marts, and nothing sets search_path.
The guarantee is stronger than schema isolation alone — those two table
names do not exist in Payload — so the claim stands, but it now rests on
cited code. The spec records the evidence and closes the open item.
findPost is wrapped in React's cache(): Next calls generateMetadata and
the page separately for one request, so every post view ran the same
query against Postgres twice.
The bare .lede rule was dead — .prose p scores (0,1,1) and outranks it —
so only .prose .lede ever applied. Removed, with the specificity noted
so the surviving selector is not "simplified" back into a silent
regression.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017YmbBhr8s7GusjDE12hrZM
PUBLISHING.md carries the house style with the posts, so the standard
survives without the design doc to hand — including the rule that a post
states what a metric does not show, which is the strongest signal a
human wrote it.
CLAUDE.md gains the two constraints that are invisible from the code and
expensive to rediscover: metadata file conventions break if moved into a
route group, and the build must keep succeeding with DATABASE_URL unset.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017YmbBhr8s7GusjDE12hrZM
The rendering split is dictated by CI building with no database.
/blog, /blog/rss.xml and /content-sitemap.xml have no dynamic params, so
Next prerenders them at build time and the build fails on a missing
Payload secret — caught here, not on staging. They are force-dynamic
instead: one indexed query against Postgres on the same Docker network,
and a newly published post appears immediately rather than waiting on a
revalidation. /blog/[slug] keeps ISR, because with no
generateStaticParams there is nothing to prerender; it is generated on
first request and cached, which is exactly what the collection's
afterChange hook exists to invalidate.
RichText takes `converters`, not `blocks`, in Payload 3.88, and the
default converters must be spread or every paragraph and heading loses
its renderer and the body comes out empty.
BlogPosting references the Person and Organization by @id rather than
repeating them, so every post and the About page resolve to one author
entity instead of declaring several people with the same name.
/sitemap.xml is proxied from FastAPI, which knows nothing about Payload,
so the Next-owned URLs get their own sitemap and robots.txt lists both.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017YmbBhr8s7GusjDE12hrZM
Drafts are on so a post can be written across sittings without saving
being publishing.
afterChange and afterDelete revalidate every path a post appears on.
Blog pages are ISR because CI builds with no database, so without these
a published post would not appear until the revalidate window expired —
up to an hour of a writer concluding that publishing is broken. Payload
runs in the same process as Next, so these are direct revalidatePath
calls with no webhook and no shared secret.
Media writes to an absolute /app/media matching the compose mount; a
mismatch would write into the container filesystem, where the next
redeploy silently discards it. Alt text is required rather than
optional.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017YmbBhr8s7GusjDE12hrZM
The site had no author, no statement of why it exists and nobody
accountable for its numbers, which is most of why it reads as machine
generated.
The page states plainly that its author is not an education expert. The
credibility claim is lived experience — a parent going through primary
admissions — plus stated provenance for every figure, which is true and
cannot be undermined by someone noticing there is no teaching
qualification behind it. First name only: the Person JSON-LD carries no
familyName, worksFor or affiliation, and a test asserts it stays that
way.
The footer gains a fourth column, with a tablet breakpoint so four
columns pair up rather than crushing before the 768px collapse. The nav
is deliberately untouched — its mobile tab bar already carries four
items.
public/brand/tudor.jpg is NOT in this commit. The page references it and
will show a broken image until the photograph is supplied; a stock
portrait would defeat the entire point of the work.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017YmbBhr8s7GusjDE12hrZM
Uploads go to a named volume at /app/media. The directory is created in
the image before the mount and covered by the existing chown, because
Docker seeds a fresh named volume from the image path — a missing or
root-owned directory there fails every upload with EACCES at runtime,
long after the build passed.
PAYLOAD_SECRET uses the same :? form as AIRFLOW_ADMIN_PASSWORD: refuse
to start rather than boot with an empty secret and accept forged
sessions. Staging's must differ from production's, which the header
comment now says explicitly. Portainer prefixes volume names per stack,
so payload_media isolates itself.
prodMigrations is not wired yet — generating the initial migration needs
a reachable Postgres. Follows in its own commit.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017YmbBhr8s7GusjDE12hrZM
X-Robots-Tag rather than the robots.txt Disallow alone, for the same
reason the staging rule uses one: a Disallow blocks crawling, not
indexing, so a URL found from an external link can be indexed without
ever being fetched — and blocking the crawl means the noindex is never
seen. Both mechanisms are applied to /admin and /cms-api.
The existing CSP is frame-ancestors only, which restricts who may embed
the site rather than what a page may load, so it cannot break the panel.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017YmbBhr8s7GusjDE12hrZM
Payload 3.88 runs inside the Next app against the existing Postgres, in
its own 'payload' schema so no pipeline operation on public — the app
tables, Airflow's metadata, migrate_csv_to_db.py --drop — can reach blog
content.
Its REST API is mounted at /cms-api. /api is the FastAPI proxy's
catch-all, which would swallow every admin call and forward it to the
backend with no error. The mount points live in lib/payloadRoutes.ts so
there is one definition and a test can assert it without importing
Payload: it is ESM-only, next/jest will not transform it, and appending
transformIgnorePatterns cannot un-ignore a package. Forcing it through
transpilePackages would change how the production build bundles Payload
to serve a test, so the live proof that /api still reaches FastAPI stays
where it belongs — the e2e journeys, which call /api/schools.
The package becomes ESM ("type": "module"), which Payload's CLI requires:
richtext-lexical has top-level await and the config cannot be require()d.
Only two files needed renaming, jest.config.cjs and a build script.
The build is verified to succeed with DATABASE_URL and PAYLOAD_SECRET
both unset, which is how CI builds it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017YmbBhr8s7GusjDE12hrZM
Payload's admin panel ships its own root layout rendering html/body.
Next allows multiple root layouts only when no app/layout.tsx exists, so
the site's routes move into their own group. Route groups are invisible
to routing: every public URL is unchanged, verified against the build's
route table.
The metadata file conventions deliberately stay at the app/ root. Moving
them into the group renamed /icon.png to /icon-4usi79.png (likewise
apple-icon and opengraph-image) and dropped /robots.txt altogether,
which would have broken the /icon.png cache-control rule, the
outputFileTracingIncludes entry for the share card, and robots.txt.
darkThemeSafety reads app/globals.css off disk rather than importing it,
so it needed its own path fix — a grep for import specifiers misses it,
and it fails as an unrunnable suite rather than a failed assertion.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017YmbBhr8s7GusjDE12hrZM
withPayload() is ESM-only, so next.config.js has to become .mjs. That
file also carries the rule that keeps staging out of Google's index, so
the conversion goes in on its own, behind a test that asserts the rule
survived — along with the standalone output, the opengraph-image font
tracing and the analytics frame-ancestors CSP.
Jest resolves the .mjs config without extra configuration, so
jest.config.js is untouched.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017YmbBhr8s7GusjDE12hrZM
Nine tasks, each ending in an independently testable deliverable.
Two structural findings that the spec did not anticipate, both recorded
in the plan. Payload's admin panel ships its own root layout rendering
html/body, and Next allows multiple root layouts only when no
app/layout.tsx exists — so every existing route moves into an
app/(frontend) route group first, on its own, with the full suite as the
gate. Route groups are invisible to routing, so no public URL changes.
The second finding corrects the spec: adding /cms-api to the FastAPI
proxy's exclusion list would be dead code, because that catch-all only
ever matches /api/*. The route remap alone is sufficient.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017YmbBhr8s7GusjDE12hrZM
The site reads as synthetic because nobody is accountable for the
numbers, no editorial judgement is visible, and the voice is
institutional third person. This designs the fix: a named author
(first name, photo, explicitly not an education expert), a coded
/about page, and a blog backed by Payload CMS running inside the
existing Next app.
Also fills a hole in the SEO programme, which has eight workstreams
and no E-E-A-T or authorship signal on a YMYL corpus.
Records two collisions found while designing, both of which fail
badly if missed: Payload's default /api route fights the existing
FastAPI catch-all proxy, and withPayload() is ESM-only so
next.config.js — which carries staging's noindex header — has to
become next.config.mjs.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FT1Ls4GbgLDXoQX7NAuHGT
The annual DAG died with a BrokenPipeError from Meltano's log writer, which
is several frames from the cause: target-postgres exited first and the tap
saw its stdout close.
The tap declared primary_keys = [urn, ...] while emitting urn=None for the
national rows, and target-postgres turns primary_keys into a NOT NULL
constraint. The first national row of the run failed the insert and took
the loader with it. Every other tap in this repo keys on non-null columns.
Carrying two grains in one stream was the actual mistake, so the fix is to
separate them rather than paper over the null: four streams now, with
ees_ks4/ks5_destinations_national carrying no urn column at all — a school
identifier that is null in every row is a grain mismatch, not a column.
The staging models split the same way and the national mart reads the new
pair instead of filtering `where urn is null`.
Verified against the live API: the school stream yields 135,240 rows over
4,508 schools with no duplicate keys, no null key columns and all 31,382
suppression sentinels intact; the national streams yield 30 and 33 rows
with no urn column.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvdDKvFFSZuMVDH5fEyTob
The simple auth manager generates a random password on first start and
writes it to a file, so every restart of the api-server invalidated the
last one and the password had to be dug out of the container logs again.
The stack now writes that file itself from AIRFLOW_ADMIN_PASSWORD before
exec'ing the api-server. Airflow generates nothing when the file already
exists, so the login is whatever the stack environment says it is.
Written with python rather than echo, so json.dumps escapes a password
containing quotes, backslashes or non-ASCII correctly — verified against
`p@ss "wo\rd' £5`, which round-trips intact.
An unset AIRFLOW_ADMIN_PASSWORD raises KeyError and the container exits.
Falling back to a generated password would silently undo the point of the
change, and a compose-level `:?` gives the same refusal a readable reason.
This does mean the variable MUST be set in Portainer before the next
deploy of either stack.
Not affected by the two Docker gotchas in the upstream docs: this image
has no USER directive so it runs as root, and the file is rewritten from
the environment on every start rather than persisted on a volume.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvdDKvFFSZuMVDH5fEyTob
meltano install would resolve it from pip_url, but five of the six custom
taps are also installed explicitly and a new plugin failing to appear is
not something you want to debug from a deploy log.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvdDKvFFSZuMVDH5fEyTob
The Share column keyed off `percentage === null`, which is true for
not_applicable as well as suppressed, so a destination that does not apply
to the school was labelled as one DfE withheld — while the Pupils column
in the same row rendered blank. Two columns, one row, disagreeing about
what the row was, and one of them making a claim about DfE that wasn't
true.
Both columns now derive from `status`, which is the distinction the mart,
the SQLAlchemy model and the serialiser all preserve deliberately:
published shows the figure, suppressed shows the withheld badge,
not_applicable shows an em-dash with a title saying so.
A published count with no published percentage now derives its share from
the cohort rather than falling through to a marker — both halves are
published, so nothing withheld is involved, and it is the same derivation
the bar widths already use.
Verified the new tests fail against the old logic before keeping them.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvdDKvFFSZuMVDH5fEyTob
Review found _mask_for_disclosure could return with its invariant broken
and say nothing. add_companion only ever withheld a *published* cell, so a
group with one suppressed category and every other one not_applicable —
routine in special schools and AP, where few categories apply — left the
loop with the lone suppressed cell still solvable. Reproduced on a
nine-pupil cohort: one hidden cell, cohort served, residual intact.
A disclosure-control pass that fails silently is worse than none, because
everything downstream trusts it. The loop now runs until the invariant
holds and escalates when no companion exists: the pupil group is dropped
from the payload, and an empty block serialises as None so the section is
absent rather than an empty shell. disclosure_invariant_holds() is exported
so tests assert it directly instead of re-deriving it, and an exhaustive
test sweeps all 81 suppression patterns of a four-category group.
Also fixes a test that set up six measures and checked one: the loop was
`for measure in ["school_sixth_form"]`. It now checks every measure, and
against the real invariant — none hidden, or at least two, rather than
"at least two", which the five published measures would have failed.
No regression on real data: 262 mainstream secondaries, all-pupils bar
still drawable on 94%, zero invariant violations, one disadvantaged group
dropped by the new escalation.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvdDKvFFSZuMVDH5fEyTob
Code review found the disclosure the whole design was meant to prevent.
R1 was written as a rendering rule and implemented as one: canRenderBar
stopped the bar being drawn, but GET /api/schools/{urn} still carried the
cohort and every published category. cohort - sum(published) returned
Whitley Bay's withheld further-education figure exactly — 18 pupils — to
any caller, and the RSC payload put it in the browser too.
app.py already stated the principle for admission_distance: this endpoint
is public and unauthenticated, so a field left in the payload is a
published field. The same reasoning applies here and did not get applied.
_mask_for_disclosure now closes both identities before serialisation —
categories sum to the cohort, and disadvantaged + other = all — by adding
secondary suppression until every row and column hides none or at least
two. My first attempt picked the smallest published cell as the companion
and a new test caught it choosing a zero, which protects nothing: the
residual still resolved to 18. The companion must carry pupils.
DfE's own aggregates are no longer served. Nothing rendered them, and one
spanning a single suppressed component names it.
Cost, measured over 262 mainstream secondaries: the all-pupils bar
survives on 94% rather than 100%. Zero lone-suppressed groups remain.
The e2e helper now tells a missing feature apart from missing data: it
fails if the API serves no destinations key at all, and skips if the key
is served but the annual DAG has not populated the marts. Failing on the
second would redden the staging gate for unrelated commits.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvdDKvFFSZuMVDH5fEyTob
The helper throws rather than skipping when no school returns a
destinations block: a silent skip would let a real regression in the
sections ride along unnoticed, which is why the distance journeys were
changed the same way in 4f01fbd.
The disadvantaged journey computes the residual itself and asserts it
appears nowhere on the page — the one number the section must never state.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvdDKvFFSZuMVDH5fEyTob
The 'Post-16 destination data coming soon' note is deleted rather than
reworded: for a school with no sixth form the truthful statement is that
the question does not apply, and a placeholder there implies something is
missing. hasSixthForm and .sixthFormNote go with it — nothing else used them.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvdDKvFFSZuMVDH5fEyTob
Question cards over one bar, with the cards acting as a lens on the bar
rather than a summary beside it — focusing a card dims everything it is
not made of, so the grouping we chose is inspectable rather than asserted.
The bar renders only when canRenderBar allows it. Where a category is
withheld the section says so and shows the table instead: the categories
sum to the cohort, so a bar drawn from the published segments leaves a
gap whose width is the withheld figure.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvdDKvFFSZuMVDH5fEyTob
Destinations are secondary-only, so the flags go on computeSecondaryFlags
rather than computeSchoolFlags. A phase counts as present only when some
pupil group carries categories — an empty block would otherwise open a nav
entry pointing at a section that never renders.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvdDKvFFSZuMVDH5fEyTob
The serialiser carries status through and computes no totals of its own.
The only aggregates in the payload are ones DfE published itself; whether
showing one is safe depends on how many of its components are suppressed,
which the frontend decides.
The batch guard grows from six tables to eight.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvdDKvFFSZuMVDH5fEyTob
Disadvantaged and other-pupils partition the whole and the all-pupils
figure is published, so publishing both halves recovers the suppressed
one. The mask is applied in the mart rather than the API so no consumer
added later can reach an unmasked combination.
The R1 test is a warn, not an error: DfE publishes the recoverable
combination and the mart's job is to carry it faithfully. Refusing to
close the gap is the API's job and the frontend's.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvdDKvFFSZuMVDH5fEyTob
safe_numeric maps every EES sentinel to NULL, which is right for attainment
and wrong here: one of those states has to print 'withheld' and the other
has to print nothing. A status column carries the difference.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvdDKvFFSZuMVDH5fEyTob
