perf(api): persist KS4 national averages as a mart; stop per-request aggregation

fact_ks4_national_averages is computed once at dbt build time (covered by
the EES DAG's stg_ees_ks4+ selector). _national_averages_payload now reads
both national-averages marts instead of scanning the performance dataframe
per year on every /api/compare request (~250ms saved per call). Fallback
for the deploy-before-DAG window computes the latest year only.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146VHeLAWjDVE2B5uU67jCB
This commit is contained in:
Tudor
2026-07-14 13:05:03 +01:00
co-authored by Claude Fable 5
parent 9990f540f7
commit 52f8994401
5 changed files with 219 additions and 70 deletions
@@ -160,6 +160,12 @@ models:
- name: year
tests: [not_null, unique]
- name: fact_ks4_national_averages
description: Computed national KS4 averages (means across state schools in our dataset — not official DfE figures) — one row per academic year
columns:
- name: year
tests: [not_null, unique]
- name: fact_deprivation
description: IDACI deprivation index — one row per URN
columns:
@@ -0,0 +1,25 @@
{{ config(materialized='table') }}
-- Mart: Computed national KS4 averages — one row per academic year.
-- Unlike fact_ks2_national_averages (official DfE figures), DfE publishes no
-- KS4 national-headline dataset we ingest yet, so these are means computed
-- across the state schools in our dataset. Computed once at build time so the
-- API never has to aggregate the full performance table per request.
-- Semantics match the API's previous per-request computation: rows where
-- attainment_8_score is non-null; per-column means ignore NULLs.
select
year,
round(avg(attainment_8_score)::numeric, 2) as attainment_8_score,
round(avg(progress_8_score)::numeric, 2) as progress_8_score,
round(avg(english_maths_standard_pass_pct)::numeric, 2) as english_maths_standard_pass_pct,
round(avg(english_maths_strong_pass_pct)::numeric, 2) as english_maths_strong_pass_pct,
round(avg(ebacc_entry_pct)::numeric, 2) as ebacc_entry_pct,
round(avg(ebacc_standard_pass_pct)::numeric, 2) as ebacc_standard_pass_pct,
round(avg(ebacc_strong_pass_pct)::numeric, 2) as ebacc_strong_pass_pct,
round(avg(ebacc_avg_score)::numeric, 2) as ebacc_avg_score,
round(avg(gcse_grade_91_pct)::numeric, 2) as gcse_grade_91_pct
from {{ ref('fact_ks4_performance') }}
where attainment_8_score is not null
group by year
order by year