fix(analytics): the funnel source read a referrer that never changes #134

Merged
tudor merged 1 commits from fix/navigation-source-soft-nav into main 2026-08-27 08:26:46 +00:00
Owner

The staging E2E gate has been red since #132 merged — run 1064 (#132's own merge) and run 1066 after it. Same test, same failure both times:

✘ a school reached from a location page is attributed to it, not to direct
  Expected: "place"
  Received: "direct"

This is not #133. Run 1064 predates it and fails identically.

What actually broke

#132 fixed a real bug — /schools/ had no case in getNavigationSource and fell through to direct. But the mechanism underneath it never worked.

getNavigationSource read document.referrer, which the browser writes only when a document loads. Every internal navigation in this app is an App Router soft navigation: history.pushState, no new document, so document.referrer goes on naming whatever opened the tab for the whole session.

Verified on staging, in a real browser:

before: { url: "/schools/brentwood", referrer: "" }
after:  { url: "/school/149971-bentley-st-pauls-…", referrer: "" }

The URL changed. The referrer did not.

So from reported direct for essentially every in-app journey — not just the location layer. search, rankings, compare and detail were all being counted in the bucket you read as "typed the URL". The location pages were the visible half of a whole broken dimension.

The unit suite passed throughout because every case set document.referrer directly, which is a thing that only happens on a full page load. It tested the one path production almost never takes.

The fix

A module-level trail, written by RouteTrail — a render-nothing client component mounted once in the root layout, so every route is recorded, including the ones that fire no event of their own but are still somebody else's previous page.

Its lifetime is exactly right: it survives soft navigation, and it dies on a real document load — which is precisely when document.referrer becomes meaningful again. The two cover each other with no overlap and no ambiguity about which to trust.

Reading it skips entries equal to the current path rather than taking trail[length - 2]. That one choice buys three things:

  • the answer no longer depends on whether the layout effect or the page effect ran first (React orders those by tree position — not a contract worth resting a measurement on, and there is a test for it);
  • going back to a page you came from still names the page you came back from;
  • a hard load of a school page records only itself, finds nothing else, and correctly falls through to the referrer.

Verification

Frontend 328 passed (was 319), tsc clean, next build green.

Nine new unit tests, six of which failed first for the right reason (recordVisitedPath is not a function). The existing E2E journey is the real regression test — it is the only thing in the suite that ever exercised a soft navigation, and it is why this was caught at all.

Worth noting the gate cannot go green until this is on staging: the E2E job runs post-merge, so #132's journey was never provable before merge.

🤖 Generated with Claude Code

https://claude.ai/code/session_01FuPUioHpxtaiDNagQvjxyM

The staging E2E gate has been red since #132 merged — run 1064 (#132's own merge) and run 1066 after it. Same test, same failure both times: ``` ✘ a school reached from a location page is attributed to it, not to direct Expected: "place" Received: "direct" ``` **This is not #133.** Run 1064 predates it and fails identically. ## What actually broke #132 fixed a real bug — `/schools/` had no case in `getNavigationSource` and fell through to `direct`. But the mechanism underneath it never worked. `getNavigationSource` read `document.referrer`, which the browser writes **only when a document loads**. Every internal navigation in this app is an App Router soft navigation: `history.pushState`, no new document, so `document.referrer` goes on naming whatever opened the tab for the whole session. Verified on staging, in a real browser: ``` before: { url: "/schools/brentwood", referrer: "" } after: { url: "/school/149971-bentley-st-pauls-…", referrer: "" } ``` The URL changed. The referrer did not. So `from` reported `direct` for essentially every in-app journey — not just the location layer. `search`, `rankings`, `compare` and `detail` were all being counted in the bucket you read as "typed the URL". The location pages were the visible half of a whole broken dimension. The unit suite passed throughout because **every case set `document.referrer` directly**, which is a thing that only happens on a full page load. It tested the one path production almost never takes. ## The fix A module-level trail, written by `RouteTrail` — a render-nothing client component mounted once in the root layout, so every route is recorded, including the ones that fire no event of their own but are still somebody else's previous page. Its lifetime is exactly right: it survives soft navigation, and it dies on a real document load — which is precisely when `document.referrer` becomes meaningful again. The two cover each other with no overlap and no ambiguity about which to trust. Reading it **skips entries equal to the current path** rather than taking `trail[length - 2]`. That one choice buys three things: - the answer no longer depends on whether the layout effect or the page effect ran first (React orders those by tree position — not a contract worth resting a measurement on, and there is a test for it); - going back to a page you came from still names the page you came back from; - a hard load of a school page records only itself, finds nothing else, and correctly falls through to the referrer. ## Verification Frontend **328 passed** (was 319), `tsc` clean, `next build` green. Nine new unit tests, six of which failed first for the right reason (`recordVisitedPath is not a function`). The existing E2E journey is the real regression test — it is the only thing in the suite that ever exercised a soft navigation, and it is why this was caught at all. Worth noting the gate cannot go green until this is on staging: the E2E job runs post-merge, so #132's journey was never provable before merge. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01FuPUioHpxtaiDNagQvjxyM
tudor added 1 commit 2026-08-27 08:22:58 +00:00
fix(analytics): the funnel source read a referrer that never changes
PR Checks / Frontend Typecheck + Tests (pull_request) Successful in 1m5s
PR Checks / Backend Smoke (pull_request) Successful in 9s
PR Checks / Build Backend (no push) (pull_request) Successful in 10s
PR Checks / Build Frontend (no push) (pull_request) Successful in 47s
PR Checks / Build Pipeline (no push) (pull_request) Successful in 10s
PR Checks / AI Code Review (Claude) (pull_request) Successful in 1m13s
e820e7fecd
The staging E2E gate has been red since #132 merged (run 1064, and
1066 after it): "a school reached from a location page is attributed
to it, not to direct" expects `place`, receives `direct`.

#132 fixed a real bug — `/schools/` had no case and fell through to
`direct` — but the mechanism underneath it never worked.
getNavigationSource read document.referrer, which the browser writes
only when a *document* loads. Every internal navigation here is an App
Router soft navigation: history.pushState, no new document, so
document.referrer goes on naming whatever opened the tab for the whole
session.

Verified on staging: load /schools/brentwood, click a school, the URL
becomes /school/… and document.referrer is still "".

So `from` reported `direct` for essentially every in-app journey, not
just the ones through the location layer — search, rankings, compare
and detail were all being counted as "typed the URL". The unit suite
passed throughout because every case set document.referrer directly,
which only happens on a full page load.

The fix is a module-level trail written by RouteTrail, a render-nothing
client component in the root layout. Its lifetime is exactly right: it
survives soft navigation, and it dies on a real document load — which
is precisely when document.referrer becomes meaningful again, so the
two cover each other with no overlap.

Reading it skips entries equal to the current path rather than taking
the second-to-last. That makes the answer independent of whether the
layout effect or the page effect ran first — React orders those by
tree position, which is not a contract worth resting a measurement on
— and it gives the right answer both when the user returns to a page
they came from and on a hard load of a school page.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FuPUioHpxtaiDNagQvjxyM

🤖 AI Code Review (Claude Code)

Adds a client-side RouteTrail component and module-level path trail to lib/analytics.ts so in-app navigation source can be attributed correctly, since document.referrer never updates across Next.js App Router soft navigations. The ordering-independence claim (trail written by a layout effect, read by a page effect) is verified by an explicit test, and the referrer fallback path is preserved for real document loads; the change is analytics-only and does not touch backend, deploy, or CI configuration.

✅ No issues found.

## 🤖 AI Code Review (Claude Code) Adds a client-side RouteTrail component and module-level path trail to lib/analytics.ts so in-app navigation source can be attributed correctly, since document.referrer never updates across Next.js App Router soft navigations. The ordering-independence claim (trail written by a layout effect, read by a page effect) is verified by an explicit test, and the referrer fallback path is preserved for real document loads; the change is analytics-only and does not touch backend, deploy, or CI configuration. ✅ No issues found.
tudor merged commit cf9d41b476 into main 2026-08-27 08:26:46 +00:00
Sign in to join this conversation.
No Reviewers
No labels
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: tudor/school_compare#134