fix(api): bound the search candidate set instead of draining Typesense
PR Checks / Frontend Typecheck + Tests (pull_request) Successful in 1m12s
PR Checks / Backend Smoke (pull_request) Successful in 10s
PR Checks / Build Backend (no push) (pull_request) Successful in 17s
PR Checks / Build Frontend (no push) (pull_request) Successful in 1m18s
PR Checks / Build Pipeline (no push) (pull_request) Successful in 39s
PR Checks / AI Code Review (Claude) (pull_request) Failing after 6m44s
PR Checks / Frontend Typecheck + Tests (pull_request) Successful in 1m12s
PR Checks / Backend Smoke (pull_request) Successful in 10s
PR Checks / Build Backend (no push) (pull_request) Successful in 17s
PR Checks / Build Frontend (no push) (pull_request) Successful in 1m18s
PR Checks / Build Pipeline (no push) (pull_request) Successful in 39s
PR Checks / AI Code Review (Claude) (pull_request) Failing after 6m44s
Fetching every match kept scoped searches correct but left the number of round trips in the caller's hands: a one-letter query, or a deliberately broad one, walked the whole collection a page at a time. Cap the candidate set at 1,000 URNs — four pages — and return the relevance-ordered prefix when the ceiling is hit. That is still far more than one page, so the API's own authority, phase and postcode filters keep the matches they need, while latency and upstream load stay bounded. A capped query is logged so a genuinely truncated search is visible. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
1 parent
0c901cd0d1
commit
5ad1cbfb53
3 files changed
+75
-11
No files matched your search
@@ -23,6 +23,47 @@ def test_search_returns_matches_beyond_first_page(monkeypatch):
|
||||
assert pages == [1, 2]
|
||||
|
||||
|
||||
def test_search_caps_broad_queries_at_a_bounded_number_of_pages(monkeypatch):
|
||||
requests = []
|
||||
|
||||
def search(params):
|
||||
requests.append(params)
|
||||
return {
|
||||
'found': 10_000,
|
||||
'hits': [
|
||||
{'document': {'urn': 100000 + params['page'] * 1000 + i}}
|
||||
for i in range(params['per_page'])
|
||||
],
|
||||
}
|
||||
|
||||
client_for(monkeypatch, search)
|
||||
result = data_loader.search_schools_typesense('school')
|
||||
|
||||
assert len(result) == data_loader.SEARCH_MAX_CANDIDATES
|
||||
assert len(requests) == data_loader.SEARCH_MAX_CANDIDATES // data_loader.SEARCH_PAGE_SIZE
|
||||
assert all(request['per_page'] == data_loader.SEARCH_PAGE_SIZE for request in requests)
|
||||
assert requests[-1]['page'] == len(requests)
|
||||
|
||||
|
||||
def test_search_uses_a_smaller_final_page_when_the_cap_is_not_a_page_multiple(monkeypatch):
|
||||
monkeypatch.setattr(data_loader, 'SEARCH_MAX_CANDIDATES', 251)
|
||||
requests = []
|
||||
|
||||
def search(params):
|
||||
requests.append(params)
|
||||
return {
|
||||
'found': 10_000,
|
||||
'hits': [{'document': {'urn': 100000 + len(requests) * 1000 + i}}
|
||||
for i in range(params['per_page'])],
|
||||
}
|
||||
|
||||
client_for(monkeypatch, search)
|
||||
result = data_loader.search_schools_typesense('school')
|
||||
|
||||
assert len(result) == 251
|
||||
assert [request['per_page'] for request in requests] == [250, 1]
|
||||
|
||||
|
||||
def test_later_page_failure_does_not_return_partial_results(monkeypatch):
|
||||
def search(params):
|
||||
if params['page'] == 2:
|
||||
|
||||
Reference in new issue
Block a user