Files
school_compare/pipeline/transform
Tudor Sitaru 3401654ab9
Build and Push Docker Images / Build Backend (FastAPI) (push) Successful in 17s
Build and Push Docker Images / Build Frontend (Next.js) (push) Successful in 46s
Build and Push Docker Images / Build Pipeline (Meltano + dbt + Airflow) (push) Successful in 1m21s
Build and Push Docker Images / Trigger Portainer Update (push) Successful in 0s
fix(pipeline): restore multi-year KS4 data
Two bugs prevented historical secondary school data from loading:

1. stg_ees_ks4.sql filtered breakdown_topic = 'Total' only, but EES
   releases prior to 2023/24 use breakdown_topic = 'All pupils' (matching
   the KS2 convention). All older years were silently dropped to zero rows.
   Fix: accept both values with an IN clause.

2. get_all_releases() in tap-uk-ees fetched only the first page of the
   EES releases API. Now follows all pages via the paging.totalPages field
   so no historical release is missed when more than 20 exist.

After re-running the annual EES pipeline, secondary school comparison
charts should show data across all available years (2018/19 onwards).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 09:18:55 +01:00
..