fix(data): drop the Welsh school GIAS does not type as Welsh
PR Checks / Frontend Typecheck + Tests (pull_request) Successful in 1m2s
PR Checks / Backend Smoke (pull_request) Successful in 7s
PR Checks / Build Backend (no push) (pull_request) Successful in 10s
PR Checks / Build Frontend (no push) (pull_request) Successful in 45s
PR Checks / Build Pipeline (no push) (pull_request) Successful in 38s
PR Checks / AI Code Review (Claude) (pull_request) Successful in 54s
PR Checks / Frontend Typecheck + Tests (pull_request) Successful in 1m2s
PR Checks / Backend Smoke (pull_request) Successful in 7s
PR Checks / Build Backend (no push) (pull_request) Successful in 10s
PR Checks / Build Frontend (no push) (pull_request) Successful in 45s
PR Checks / Build Pipeline (no push) (pull_request) Successful in 38s
PR Checks / AI Code Review (Claude) (pull_request) Successful in 54s
Beechwood College (URN 142458, Sully, CF64 5SE) survived the England-only filter. GIAS types it a Special post 16 institution (32), not a Welsh establishment (30), so filtering on establishment type alone left it behind — the last Welsh school on the site, and the reason Vale of Glamorgan was still in the authority list. The earlier verification claimed the type codes mapped onto the Welsh authorities in both directions. That was checked exhaustively for Cardiff and by count for three others; Vale of Glamorgan was never checked, and it was the one that did not hold. LA code is the reliable discriminator: GIAS gives the 22 Welsh unitary authorities the contiguous block 660-681, which English authorities never use. Filtering on postcode would have been wrong — Redbrook, Tutshill, Wyedean and two other Gloucestershire schools carry NP16/NP25 postcodes because Royal Mail areas straddle the border, and they are English schools with English data. An e2e test now pins those five so the fix cannot be simplified into a postcode filter later. The 660-681 range is documented GIAS structure this project cannot verify from its own data, so the range filters and the authority NAME checks: if the range is ever wrong, a Welsh authority reappears in assert_england_only_schools and the pipeline fails loudly. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015mWQnpye9F299NVRCCSRvj
This commit is contained in:
1 parent
c47fe38971
commit
3aad5101a8
5 files changed
+69
No files matched your search
@@ -26,3 +26,29 @@ select
|
||||
from {{ ref('dim_location') }} l
|
||||
left join {{ ref('dim_school') }} s on l.urn = s.urn
|
||||
where s.urn is null
|
||||
|
||||
union all
|
||||
|
||||
-- Belt and braces on the Welsh LA code range.
|
||||
--
|
||||
-- The range 660-681 is documented GIAS structure, not something this project
|
||||
-- can verify from its own data — local_authority_code is not exposed anywhere
|
||||
-- the app can assert on. So the range does the filtering and the authority
|
||||
-- NAME does the checking: if 660-681 is ever wrong or incomplete, a Welsh
|
||||
-- authority reappears here and the pipeline fails loudly rather than the
|
||||
-- corpus quietly regrowing a Welsh tail.
|
||||
--
|
||||
-- Beechwood College (142458) is why this exists: a Welsh school GIAS types as
|
||||
-- a Special post 16 institution, which the type filter alone let through.
|
||||
select
|
||||
l.urn,
|
||||
l.local_authority_name as school_name,
|
||||
null::integer as school_type_code
|
||||
from {{ ref('dim_location') }} l
|
||||
where l.local_authority_name in (
|
||||
'Isle of Anglesey', 'Gwynedd', 'Conwy', 'Denbighshire', 'Flintshire',
|
||||
'Wrexham', 'Powys', 'Ceredigion', 'Pembrokeshire', 'Carmarthenshire',
|
||||
'Swansea', 'Neath Port Talbot', 'Bridgend', 'Vale of Glamorgan',
|
||||
'Rhondda Cynon Taf', 'Merthyr Tydfil', 'Caerphilly', 'Blaenau Gwent',
|
||||
'Torfaen', 'Monmouthshire', 'Newport', 'Cardiff'
|
||||
)
|
||||
Reference in new issue
Block a user