tudor/school_compare
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(legacy-ks2): strip % suffix from percentage values
All checks were successful
Build and Push Docker Images / Build Backend (FastAPI) (push) Successful in 34s
Details
Build and Push Docker Images / Build Frontend (Next.js) (push) Successful in 1m11s
Details
Build and Push Docker Images / Build Pipeline (Meltano + dbt + Airflow) (push) Successful in 1m37s
Details
Build and Push Docker Images / Trigger Portainer Update (push) Successful in 1s
Details

Browse Source 
Old DfE CSVs encode percentages as "57%" not "57". The safe_numeric
macro rejects non-numeric strings, so strip the suffix before emitting.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Tudor Sitaru
2026-04-01 13:07:51 +01:00
parent fbd1de9220
commit 2b757e556d
1 changed files with 5 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
pipeline/plugins/extractors/tap-uk-ees/tap_uk_ees/tap.py
View File

@@ -592,7 +592,11 @@ class LegacyKS2Stream(Stream):
for _, row in df.iterrows():
record = {"year": year_code}
for old_col, new_col in _LEGACY_KS2_COLUMN_MAP.items():
record[new_col] = row.get(old_col, "")
val = row.get(old_col, "")
# Strip % suffix — old DfE CSVs use "57%" not "57"
if isinstance(val, str) and val.endswith("%"):
val = val[:-1]
record[new_col] = val
yield record