tudor/school_compare
1
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Releases Wiki Activity
226 Commits 70 Branches 0 Tags
33b395d2bd799f785590aa22a4dbeb10c77b349b
Commit Graph
3 Commits
Author SHA1 Message Date
tudorandClaude Sonnet 4.6 33b395d2bd fix(dbt): apply safe_numeric macro to fix EES suppression code 'c' errors
Build and Push Docker Images / Build Backend (FastAPI) (push) Successful in 33s
Details
Build and Push Docker Images / Build Frontend (Next.js) (push) Successful in 1m14s
Details
Build and Push Docker Images / Build Integrator (push) Successful in 58s
Details
Build and Push Docker Images / Build Kestra Init (push) Successful in 31s
Details
Build and Push Docker Images / Build Pipeline (Meltano + dbt + Airflow) (push) Successful in 1m25s
Details
Build and Push Docker Images / Trigger Portainer Update (push) Successful in 0s
Details 
Replace nullif(col, 'z') casts with safe_numeric macro across KS2, KS4,
and admissions staging models. The regex-based macro treats any non-numeric
string (z, c, x, q, u, etc.) as NULL without needing an explicit list.

Also fix FSM_eligible_percent column quoting in stg_ees_admissions — target-
postgres stores mixed-case column names quoted, so unquoted references were
being folded to fsm_eligible_percent by PostgreSQL.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-27 10:41:27 +00:00
tudorandClaude Opus 4.6 45f3e4d9fc fix(dbt): override generate_schema_name to use direct schema names
Build and Push Docker Images / Build Backend (FastAPI) (push) Successful in 34s
Details
Build and Push Docker Images / Build Frontend (Next.js) (push) Successful in 1m7s
Details
Build and Push Docker Images / Build Integrator (push) Successful in 55s
Details
Build and Push Docker Images / Build Kestra Init (push) Successful in 31s
Details
Build and Push Docker Images / Build Pipeline (Meltano + dbt + Airflow) (push) Successful in 1m28s
Details
Build and Push Docker Images / Trigger Portainer Update (push) Successful in 1s
Details 
dbt default prepends the profile schema as prefix (public_staging,
public_marts). Override to use custom schema names directly (staging,
marts) so scripts can reference marts.dim_location correctly.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-26 21:09:23 +00:00
tudorandClaude Opus 4.6 8f02b5125e feat(pipeline): add Meltano + dbt + Airflow ELT pipeline scaffold
Build and Push Docker Images / Build Backend (FastAPI) (push) Successful in 35s
Details
Build and Push Docker Images / Build Frontend (Next.js) (push) Successful in 1m9s
Details
Build and Push Docker Images / Build Integrator (push) Successful in 56s
Details
Build and Push Docker Images / Build Kestra Init (push) Successful in 32s
Details
Build and Push Docker Images / Trigger Portainer Update (push) Successful in 1s
Details 
Replaces the hand-rolled integrator with a production-grade ELT pipeline
using Meltano (Singer taps), dbt Core (medallion architecture), and
Apache Airflow (orchestration). Adds Typesense for search and PostGIS
for geospatial queries.

- 6 custom Singer taps (GIAS, EES, Ofsted, Parent View, FBIT, IDACI)
- dbt project: 12 staging, 5 intermediate, 12 mart models
- 3 Airflow DAGs (daily/monthly/annual schedules)
- Typesense sync + batch geocoding scripts
- docker-compose: add Airflow, Typesense; upgrade to PostGIS
- Portainer stack definition matching live deployment topology

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-26 08:37:53 +00:00