fix(list): read ofsted grade from fact_ofsted_inspection directly, fix dim_school schema lookup
Build and Push Docker Images / Build Backend (FastAPI) (push) Successful in 18s
Build and Push Docker Images / Build Frontend (Next.js) (push) Successful in 49s
Build and Push Docker Images / Build Pipeline (Meltano + dbt + Airflow) (push) Successful in 1m9s
Build and Push Docker Images / Trigger Portainer Update (push) Successful in 1s
Build and Push Docker Images / Build Backend (FastAPI) (push) Successful in 18s
Build and Push Docker Images / Build Frontend (Next.js) (push) Successful in 49s
Build and Push Docker Images / Build Pipeline (Meltano + dbt + Airflow) (push) Successful in 1m9s
Build and Push Docker Images / Trigger Portainer Update (push) Successful in 1s
dim_school.sql was checking for int_ofsted_latest in target.schema (wrong schema) due to the custom generate_schema_name macro using literal schema names. The model lives in 'intermediate', so ofsted_grade/date/framework were always NULL in dim_school, causing all list cards to show 'Not yet inspected'. Fix 1: data_loader.py joins marts.fact_ofsted_inspection with DISTINCT ON to get latest inspection per school — no pipeline re-run needed. Fix 2: dim_school.sql uses schema='intermediate' so future dbt runs correctly denormalise the Ofsted summary into dim_school.
This commit is contained in:
+12
-3
@@ -130,9 +130,9 @@ _MAIN_QUERY = text("""
|
|||||||
s.total_pupils AS gias_total_pupils,
|
s.total_pupils AS gias_total_pupils,
|
||||||
s.headteacher_name,
|
s.headteacher_name,
|
||||||
s.website,
|
s.website,
|
||||||
s.ofsted_grade,
|
foi.ofsted_grade,
|
||||||
s.ofsted_date,
|
foi.ofsted_date,
|
||||||
s.ofsted_framework,
|
foi.ofsted_framework,
|
||||||
l.local_authority_name AS local_authority,
|
l.local_authority_name AS local_authority,
|
||||||
l.local_authority_code,
|
l.local_authority_code,
|
||||||
l.address_line1 AS address1,
|
l.address_line1 AS address1,
|
||||||
@@ -201,6 +201,15 @@ _MAIN_QUERY = text("""
|
|||||||
FROM marts.dim_school s
|
FROM marts.dim_school s
|
||||||
JOIN marts.dim_location l ON s.urn = l.urn
|
JOIN marts.dim_location l ON s.urn = l.urn
|
||||||
LEFT JOIN marts.fact_performance p ON s.urn = p.urn
|
LEFT JOIN marts.fact_performance p ON s.urn = p.urn
|
||||||
|
LEFT JOIN (
|
||||||
|
SELECT DISTINCT ON (urn)
|
||||||
|
urn,
|
||||||
|
overall_effectiveness AS ofsted_grade,
|
||||||
|
inspection_date AS ofsted_date,
|
||||||
|
framework AS ofsted_framework
|
||||||
|
FROM marts.fact_ofsted_inspection
|
||||||
|
ORDER BY urn, inspection_date DESC NULLS LAST
|
||||||
|
) foi ON s.urn = foi.urn
|
||||||
ORDER BY s.school_name, p.year
|
ORDER BY s.school_name, p.year
|
||||||
""")
|
""")
|
||||||
|
|
||||||
|
|||||||
@@ -6,7 +6,7 @@ with schools as (
|
|||||||
|
|
||||||
{% set ofsted_relation = adapter.get_relation(
|
{% set ofsted_relation = adapter.get_relation(
|
||||||
database=target.database,
|
database=target.database,
|
||||||
schema=target.schema,
|
schema='intermediate',
|
||||||
identifier='int_ofsted_latest'
|
identifier='int_ofsted_latest'
|
||||||
) %}
|
) %}
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user