perf(places): index the reverse lookup, and isolate the registry in tests
PR Checks / Frontend Typecheck + Tests (pull_request) Successful in 1m11s
PR Checks / Backend Smoke (pull_request) Successful in 9s
PR Checks / Build Backend (no push) (pull_request) Successful in 18s
PR Checks / Build Frontend (no push) (pull_request) Successful in 1m17s
PR Checks / Build Pipeline (no push) (pull_request) Successful in 11s
PR Checks / AI Code Review (Claude) (pull_request) Successful in 1m47s
PR Checks / Frontend Typecheck + Tests (pull_request) Successful in 1m11s
PR Checks / Backend Smoke (pull_request) Successful in 9s
PR Checks / Build Backend (no push) (pull_request) Successful in 18s
PR Checks / Build Frontend (no push) (pull_request) Successful in 1m17s
PR Checks / Build Pipeline (no push) (pull_request) Successful in 11s
PR Checks / AI Code Review (Claude) (pull_request) Successful in 1m47s
Two review findings, both confirmed before fixing.
The client fixture in test_school_details.py patched load_school_data
but not _place_registry, which is a module-level cache. A probe settled
it rather than an argument: poisoning the global with a registry built
from data the fixture never saw, then issuing the fixture's own
request, returned that other dataset's places. So the new
`places == []` assertion was satisfied by a stale registry exactly as
well as by the fixture's own data, and proved nothing. Every other test
that touches place data already reset it; the fixture predates places
existing and was never updated. It resets it now.
places_for_urn walked every place in the registry and did a tuple
membership test against each, on /api/schools/{urn}, the site's
highest-traffic endpoint. It now reads a dict built once per registry.
Measured against a synthetic corpus of 27k schools in 1,650 places:
0.118ms per request becomes 0.0001ms, with the index built once in
21ms. Production carries ~5,000 places, so the scan there is larger
again. The absolute saving per request is small; the point is that it
is repeated on every school page view and costs nothing to remove.
The index is cached against the registry by identity rather than
behind a second flag. Anything that drops _place_registry — every test
that touches place data does — gets a fresh registry object, which no
longer matches what the index was built from, so the index rebuilds
with it. A separate _place_index = None would be one more thing to
forget, and a stale reverse index is precisely the first finding's bug
wearing a different hat.
That invalidation has its own test, and the test was checked by
breaking the identity check: five tests fail without it, so three
existing ones were already relying on it too.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DXnXQKnPpZBBP61fBQiFkq
This commit is contained in:
1 parent
d65eb58883
commit
b0d5334e06
4 files changed
+124
-18
No files matched your search
+36
-11
@@ -296,21 +296,46 @@ def _locality_places(df, publishable: set[int],
|
||||
return out
|
||||
|
||||
|
||||
def places_for_urn(registry: dict[str, Place], urn: int) -> tuple[Place, ...]:
|
||||
"""Every published place containing this school, largest kind first.
|
||||
# Ordered authority → town/locality → outcode, widest first, because that is
|
||||
# the order a breadcrumb reads. The link module re-sorts for its own purposes.
|
||||
_PLACE_ORDER = {"authority": 0, "town": 1, "locality": 2, "outcode": 3}
|
||||
|
||||
|
||||
def build_place_index(registry: dict[str, Place]) -> dict[int, tuple[Place, ...]]:
|
||||
"""URN → the published places containing it, built once per registry.
|
||||
|
||||
The reverse of the registry, and the thing school pages link out through.
|
||||
Derived from the registry rather than stored beside it, so the two cannot
|
||||
disagree about which places exist: a link module must never offer a place
|
||||
whose page does not exist, and a place below the publish threshold is
|
||||
simply absent from the registry, so it is absent from here too.
|
||||
Derived from the registry rather than maintained beside it, so the two
|
||||
cannot disagree about which places exist: a place below the publish
|
||||
threshold is absent from the registry, so it is absent from here too, and
|
||||
a link is never offered for a page that does not exist.
|
||||
|
||||
Ordered authority → town/locality → outcode, widest first, because that is
|
||||
the order a breadcrumb reads and the order the link module lists.
|
||||
Built as an index rather than scanned per call because /api/schools/{urn}
|
||||
is the site's highest-traffic endpoint. Scanning meant walking every place
|
||||
and doing a tuple membership test against each — on the order of 10^5
|
||||
comparisons per request, repeated for every school page view. One pass at
|
||||
registry-build time replaces all of it with a dict lookup.
|
||||
"""
|
||||
order = {"authority": 0, "town": 1, "locality": 2, "outcode": 3}
|
||||
found = [p for p in registry.values() if urn in p.urns]
|
||||
return tuple(sorted(found, key=lambda p: (order.get(p.kind, 9), p.slug)))
|
||||
grouped: dict[int, list[Place]] = {}
|
||||
for place in registry.values():
|
||||
for urn in place.urns:
|
||||
grouped.setdefault(int(urn), []).append(place)
|
||||
|
||||
return {
|
||||
urn: tuple(sorted(places,
|
||||
key=lambda p: (_PLACE_ORDER.get(p.kind, 9), p.slug)))
|
||||
for urn, places in grouped.items()
|
||||
}
|
||||
|
||||
|
||||
def places_for_urn(index: dict[int, tuple[Place, ...]], urn: int) -> tuple[Place, ...]:
|
||||
"""The published places containing this school, widest first.
|
||||
|
||||
Empty is a real answer, not a failure: a school whose town and authority
|
||||
both fall below the publish threshold has nowhere to link, and the page
|
||||
renders without the module.
|
||||
"""
|
||||
return index.get(int(urn), ())
|
||||
|
||||
|
||||
def build_place_registry(df) -> dict[str, Place]:
|
||||
|
||||
Reference in new issue
Block a user