2 Commits

Include renames

Author	SHA1	Message	Date
TudorandClaude Opus 5	e236669fde	fix(destinations): school rows and the England reference are different grains ... The annual DAG died with a BrokenPipeError from Meltano's log writer, which is several frames from the cause: target-postgres exited first and the tap saw its stdout close. The tap declared primary_keys = [urn, ...] while emitting urn=None for the national rows, and target-postgres turns primary_keys into a NOT NULL constraint. The first national row of the run failed the insert and took the loader with it. Every other tap in this repo keys on non-null columns. Carrying two grains in one stream was the actual mistake, so the fix is to separate them rather than paper over the null: four streams now, with ees_ks4/ks5_destinations_national carrying no urn column at all — a school identifier that is null in every row is a grain mismatch, not a column. The staging models split the same way and the national mart reads the new pair instead of filtering `where urn is null`. Verified against the live API: the school stream yields 135,240 rows over 4,508 schools with no duplicate keys, no null key columns and all 31,382 suppression sentinels intact; the national streams yield 30 and 33 rows with no urn column. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BvdDKvFFSZuMVDH5fEyTob	2026-08-31 21:36:43 +01:00
TudorandClaude Opus 5	9188626051	feat(destinations): a tap that preserves the suppression sentinel ... EES writes 'c' where a figure is withheld and the categories sum to the cohort, so counts and percentages are emitted as text with the sentinel intact. safe_numeric must never be pointed at them. School rows and the England reference need different establishment pins: at national level selective schools, studios and UTCs are separate populations rather than labels, so leaving establishment open multiplies 30 rows into 190. Two queries per period, each keeping its own level. Verified against the live API for 2022/23: 135,240 school records over 4,508 schools, exactly 30 each, no duplicate keys, 31,382 sentinels kept. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BvdDKvFFSZuMVDH5fEyTob	2026-08-28 16:06:37 +01:00