feat(census): add demographic columns to EES census tap and staging models
All checks were successful
Build and Push Docker Images / Build Backend (FastAPI) (push) Successful in 32s
Build and Push Docker Images / Build Frontend (Next.js) (push) Successful in 1m7s
Build and Push Docker Images / Build Integrator (push) Successful in 55s
Build and Push Docker Images / Build Kestra Init (push) Successful in 32s
Build and Push Docker Images / Build Pipeline (Meltano + dbt + Airflow) (push) Successful in 1m39s
Build and Push Docker Images / Trigger Portainer Update (push) Successful in 1s
All checks were successful
Build and Push Docker Images / Build Backend (FastAPI) (push) Successful in 32s
Build and Push Docker Images / Build Frontend (Next.js) (push) Successful in 1m7s
Build and Push Docker Images / Build Integrator (push) Successful in 55s
Build and Push Docker Images / Build Kestra Init (push) Successful in 32s
Build and Push Docker Images / Build Pipeline (Meltano + dbt + Airflow) (push) Successful in 1m39s
Build and Push Docker Images / Trigger Portainer Update (push) Successful in 1s
tap-uk-ees: EESCensusStream now declares 27 data columns (FSM %, EAL %, ethnicity breakdowns, pupil counts) with clean Singer field names mapped from the verbose CSV column names (e.g. '% of pupils known to be eligible for free school meals' → fsm_pct) via a new _column_renames mechanism on the base stream class. stg_ees_census: materialised as table, applies safe_numeric to all percentage/count columns, filters to numeric URNs. int_pupil_chars_merged + fact_pupil_characteristics: pass all columns through from staging (previously stubs with only 3 columns). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -1,8 +1,34 @@
|
||||
-- Mart: Pupil characteristics — one row per URN per year
|
||||
-- TODO: Expand once census data columns are verified and added to staging
|
||||
|
||||
select
|
||||
urn,
|
||||
year,
|
||||
phase_type_grouping
|
||||
phase_type_grouping,
|
||||
total_pupils,
|
||||
female_pupils,
|
||||
male_pupils,
|
||||
full_time_pupils,
|
||||
part_time_pupils,
|
||||
total_boarders,
|
||||
fsm_eligible_n,
|
||||
fsm_pct,
|
||||
eal_pct,
|
||||
young_carer_pct,
|
||||
ethnicity_white_british_pct,
|
||||
ethnicity_white_other_pct,
|
||||
ethnicity_gypsy_roma_pct,
|
||||
ethnicity_mixed_wbc_pct,
|
||||
ethnicity_mixed_wba_pct,
|
||||
ethnicity_mixed_wa_pct,
|
||||
ethnicity_mixed_other_pct,
|
||||
ethnicity_asian_indian_pct,
|
||||
ethnicity_asian_pakistani_pct,
|
||||
ethnicity_asian_bangladeshi_pct,
|
||||
ethnicity_asian_other_pct,
|
||||
ethnicity_black_caribbean_pct,
|
||||
ethnicity_black_african_pct,
|
||||
ethnicity_black_other_pct,
|
||||
ethnicity_chinese_pct,
|
||||
ethnicity_other_pct,
|
||||
ethnicity_unclassified_pct
|
||||
from {{ ref('int_pupil_chars_merged') }}
|
||||
|
||||
Reference in New Issue
Block a user