tudor/school_compare
1
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request 'fix: convert NaN/NULL to None and restore record properties structure in tap.py' (#48) from feature/ingest-independent-schools into main
Stage (build -> staging -> E2E gate) / Build Backend (FastAPI) (push) Successful in 14s
Details
Stage (build -> staging -> E2E gate) / Build Frontend (Next.js) (push) Successful in 50s
Details
Stage (build -> staging -> E2E gate) / Build Pipeline (Meltano + dbt + Airflow) (push) Successful in 1m22s
Details
Stage (build -> staging -> E2E gate) / Deploy to Staging (push) Successful in 1s
Details
Stage (build -> staging -> E2E gate) / E2E Journeys against Staging (push) Failing after 41s
Details

Browse Source 
Reviewed-on: #48
This commit was merged in pull request #48.
This commit is contained in:
tudor
2026-07-16 07:40:28 +00:00
parent 674470ceb6 95f10bf352
commit 4bfcd9ba9a
1 changed files with 2 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files
pipeline/plugins/extractors/tap-uk-ofsted/tap_uk_ofsted/tap.py
+2 -5
View File
@@ -237,18 +237,15 @@ class OfstedInspectionsStream(Stream):
for _, row in df.iterrows(): for _, row in df.iterrows():
record = {} record = {}
for key in self.schema["properties"].keys():
record[key] = None
for field, col in col_map.items(): for field, col in col_map.items():
val = row.get(col, None) val = row.get(col, None)
if val == 'NULL': if pd.isna(val) or val == 'NULL':
val = None val = None
record[field] = val record[field] = val
# Cast URN # Cast URN
try: try:
record["urn"] = int(record["urn"]) record["urn"] = int(record.get("urn"))
except (ValueError, KeyError, TypeError): except (ValueError, KeyError, TypeError):
continue continue