test(e2e): fix flaky compare journey — select two same-phase schools #40

Merged
tudor merged 1 commits from fix/e2e-compare-samephase into main 2026-07-14 22:30:18 +00:00
Owner

Fixes the flaky comparing two schools shows the parent-first sections side by side e2e journey.

Root cause

Not a product regression — the compare page renders fine (the mobile compare journey passed in the same run). The test picked the first two "primary" search hits without guaranteeing they're the same phase. Some "primary" search results are all-through schools, which carry KS4 data and therefore classify as secondary on the compare page's phase tabs. Comparing an all-through school (e.g. URN 137306, "Hessle High School and Penshurst Primary School") with a pure primary splits them across the Primary/Secondary tabs — only the active tab renders its school link, so a[href*="137306"] was never visible and the assertion timed out (15.6s).

Fix

Select two pure-Primary URNs via the API (/api/schools?search=primary, filtered to phase === 'Primary' with KS2 data) so both stay on one tab. Deterministic and data-invariant — uses whatever primaries the environment holds.

Verified against staging: the test went from a 15.6s timeout to passing in ~1.8s; both compare journeys pass.

Follow-up worth noting (not fixed here)

An all-through school compared with a primary is currently hidden from the Primary tab even though it has KS2 data — arguably it should appear on both tabs. That's pre-existing phase-classification behaviour (unchanged by the redesign) and a separate product decision.

🤖 Generated with Claude Code

https://claude.ai/code/session_0146VHeLAWjDVE2B5uU67jCB

Fixes the flaky `comparing two schools shows the parent-first sections side by side` e2e journey. ## Root cause Not a product regression — the compare page renders fine (the mobile compare journey passed in the same run). The test picked the first two "primary" search hits without guaranteeing they're the same phase. Some "primary" search results are **all-through** schools, which carry KS4 data and therefore classify as *secondary* on the compare page's phase tabs. Comparing an all-through school (e.g. URN 137306, "Hessle High School and Penshurst Primary School") with a pure primary splits them across the Primary/Secondary tabs — only the active tab renders its school link, so `a[href*="137306"]` was never visible and the assertion timed out (15.6s). ## Fix Select two **pure-Primary** URNs via the API (`/api/schools?search=primary`, filtered to `phase === 'Primary'` with KS2 data) so both stay on one tab. Deterministic and data-invariant — uses whatever primaries the environment holds. Verified against staging: the test went from a 15.6s timeout to passing in ~1.8s; both compare journeys pass. ## Follow-up worth noting (not fixed here) An all-through school compared with a primary is currently hidden from the Primary tab even though it has KS2 data — arguably it should appear on both tabs. That's pre-existing phase-classification behaviour (unchanged by the redesign) and a separate product decision. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_0146VHeLAWjDVE2B5uU67jCB
tudor added 1 commit 2026-07-14 22:12:50 +00:00
test(e2e): pick two same-phase schools for the compare journey
PR Checks / Frontend Typecheck + Tests (pull_request) Successful in 9m41s
PR Checks / Backend Smoke (pull_request) Successful in 7s
PR Checks / Build Backend (no push) (pull_request) Successful in 16s
PR Checks / Build Frontend (no push) (pull_request) Successful in 45s
PR Checks / Build Pipeline (no push) (pull_request) Successful in 10s
PR Checks / AI Code Review (Claude) (pull_request) Successful in 47s
06e4898c30
The compare page's phase tabs put all-through schools (which carry KS4
data) on the secondary tab, so comparing an all-through school with a
pure primary splits them across tabs and only the active tab renders its
link. The test picked the first two 'primary' search hits without
guaranteeing same phase, so it flaked whenever a search returned an
all-through school first (e.g. URN 137306). Now selects two pure-Primary
URNs via the API — deterministic and data-invariant.

Verified against staging: was a 15.6s timeout, now passes in ~1.8s.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146VHeLAWjDVE2B5uU67jCB

🤖 AI Code Review (Claude Code)

The PR reworks an e2e test to select two same-phase (pure-primary) schools via a direct API call instead of scraping search-result links, avoiding cross-tab rendering flakiness on the compare page. The change is test-only and low risk, but has one functional bug in the new helper.

🟡 Minor

  • e2e/tests/journeys.spec.ts: twoPrimaryUrns() passes per_page=50 to /api/schools, but the backend endpoint's actual pagination parameter is page_size (see backend/app.py). The unknown per_page param is silently ignored by FastAPI, so the request falls back to the default page_size of 25, which can yield fewer than 2 qualifying primary schools and make the test flaky/fail depending on data ordering.
## 🤖 AI Code Review (Claude Code) The PR reworks an e2e test to select two same-phase (pure-primary) schools via a direct API call instead of scraping search-result links, avoiding cross-tab rendering flakiness on the compare page. The change is test-only and low risk, but has one functional bug in the new helper. ### 🟡 Minor - **e2e/tests/journeys.spec.ts**: twoPrimaryUrns() passes `per_page=50` to `/api/schools`, but the backend endpoint's actual pagination parameter is `page_size` (see backend/app.py). The unknown `per_page` param is silently ignored by FastAPI, so the request falls back to the default page_size of 25, which can yield fewer than 2 qualifying primary schools and make the test flaky/fail depending on data ordering.
tudor merged commit 0c7ad0f309 into main 2026-07-14 22:30:18 +00:00
Sign in to join this conversation.
No Reviewers
No labels
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: tudor/school_compare#40