perf(places): index the reverse lookup, and isolate the registry in tests
PR Checks / Frontend Typecheck + Tests (pull_request) Successful in 1m11s
PR Checks / Backend Smoke (pull_request) Successful in 9s
PR Checks / Build Backend (no push) (pull_request) Successful in 18s
PR Checks / Build Frontend (no push) (pull_request) Successful in 1m17s
PR Checks / Build Pipeline (no push) (pull_request) Successful in 11s
PR Checks / AI Code Review (Claude) (pull_request) Successful in 1m47s
PR Checks / Frontend Typecheck + Tests (pull_request) Successful in 1m11s
PR Checks / Backend Smoke (pull_request) Successful in 9s
PR Checks / Build Backend (no push) (pull_request) Successful in 18s
PR Checks / Build Frontend (no push) (pull_request) Successful in 1m17s
PR Checks / Build Pipeline (no push) (pull_request) Successful in 11s
PR Checks / AI Code Review (Claude) (pull_request) Successful in 1m47s
Two review findings, both confirmed before fixing.
The client fixture in test_school_details.py patched load_school_data
but not _place_registry, which is a module-level cache. A probe settled
it rather than an argument: poisoning the global with a registry built
from data the fixture never saw, then issuing the fixture's own
request, returned that other dataset's places. So the new
`places == []` assertion was satisfied by a stale registry exactly as
well as by the fixture's own data, and proved nothing. Every other test
that touches place data already reset it; the fixture predates places
existing and was never updated. It resets it now.
places_for_urn walked every place in the registry and did a tuple
membership test against each, on /api/schools/{urn}, the site's
highest-traffic endpoint. It now reads a dict built once per registry.
Measured against a synthetic corpus of 27k schools in 1,650 places:
0.118ms per request becomes 0.0001ms, with the index built once in
21ms. Production carries ~5,000 places, so the scan there is larger
again. The absolute saving per request is small; the point is that it
is repeated on every school page view and costs nothing to remove.
The index is cached against the registry by identity rather than
behind a second flag. Anything that drops _place_registry — every test
that touches place data does — gets a fresh registry object, which no
longer matches what the index was built from, so the index rebuilds
with it. A separate _place_index = None would be one more thing to
forget, and a stale reverse index is precisely the first finding's bug
wearing a different hat.
That invalidation has its own test, and the test was checked by
breaking the identity check: five tests fail without it, so three
existing ones were already relying on it too.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DXnXQKnPpZBBP61fBQiFkq
This commit is contained in:
1 parent
d65eb58883
commit
b0d5334e06
4 files changed
+124
-18
No files matched your search
@@ -8,7 +8,8 @@ import numpy as np
|
||||
import pandas as pd
|
||||
import pytest
|
||||
|
||||
from backend.places import MIN_SCHOOLS, build_place_registry, places_for_urn
|
||||
from backend.places import (MIN_SCHOOLS, build_place_index,
|
||||
build_place_registry, places_for_urn)
|
||||
|
||||
|
||||
def _df(rows: list[dict]) -> pd.DataFrame:
|
||||
@@ -428,7 +429,7 @@ def test_an_authority_still_publishes_phase_variants():
|
||||
|
||||
def test_a_school_resolves_to_every_published_place_containing_it():
|
||||
reg = build_place_registry(_df(_town(MIN_SCHOOLS, "Brentwood", "Essex")))
|
||||
places = places_for_urn(reg, 100000)
|
||||
places = places_for_urn(build_place_index(reg), 100000)
|
||||
|
||||
kinds = {p.kind for p in places}
|
||||
assert "town" in kinds
|
||||
@@ -443,7 +444,7 @@ def test_a_school_in_an_unpublished_town_still_resolves_to_its_authority():
|
||||
_town(MIN_SCHOOLS - 1, "Tinytown", "Essex")
|
||||
+ _town(MIN_SCHOOLS, "Brentwood", "Essex", start=200000)
|
||||
))
|
||||
places = places_for_urn(reg, 100000)
|
||||
places = places_for_urn(build_place_index(reg), 100000)
|
||||
|
||||
# The town is below the threshold, so it has no page and must not be
|
||||
# offered as a link. The authority above it does, and is the right target.
|
||||
@@ -455,7 +456,7 @@ def test_an_unknown_urn_resolves_to_nothing_rather_than_raising():
|
||||
# A school page renders for any URN the API knows; the link module is not
|
||||
# entitled to take the page down when it has nothing to say.
|
||||
reg = build_place_registry(_df(_town(MIN_SCHOOLS, "Brentwood", "Essex")))
|
||||
assert places_for_urn(reg, 999999) == ()
|
||||
assert places_for_urn(build_place_index(reg), 999999) == ()
|
||||
|
||||
|
||||
def test_the_index_is_consistent_with_the_registry_it_was_built_from():
|
||||
@@ -465,7 +466,32 @@ def test_the_index_is_consistent_with_the_registry_it_was_built_from():
|
||||
_town(MIN_SCHOOLS, "Brentwood", "Essex")
|
||||
+ _town(MIN_SCHOOLS, "Bedford", "Bedford", start=300000)
|
||||
))
|
||||
index = build_place_index(reg)
|
||||
for key, place in reg.items():
|
||||
for urn in place.urns:
|
||||
assert place in places_for_urn(reg, urn), (
|
||||
assert place in places_for_urn(index, urn), (
|
||||
f"{urn} is in {key} but the index does not say so")
|
||||
|
||||
|
||||
def test_the_index_holds_no_school_the_registry_does_not():
|
||||
# The reverse direction of the invariant above. An index entry for a URN
|
||||
# no published place contains would put a link on a page for a place that
|
||||
# does not list that school.
|
||||
reg = build_place_registry(_df(
|
||||
_town(MIN_SCHOOLS, "Brentwood", "Essex")
|
||||
+ _town(MIN_SCHOOLS - 1, "Tinytown", "Essex", start=400000)
|
||||
))
|
||||
index = build_place_index(reg)
|
||||
|
||||
for urn, places in index.items():
|
||||
for place in places:
|
||||
assert urn in place.urns
|
||||
assert place.key in reg
|
||||
|
||||
|
||||
def test_the_index_preserves_the_widest_first_order():
|
||||
# The breadcrumb reads authority then town, and takes this order as given.
|
||||
reg = build_place_registry(_df(_town(MIN_SCHOOLS, "Brentwood", "Essex")))
|
||||
kinds = [p.kind for p in places_for_urn(build_place_index(reg), 100000)]
|
||||
|
||||
assert kinds.index("authority") < kinds.index("town")
|
||||
Reference in new issue
Block a user