fix(taps): align with integrator resilience patterns
All checks were successful
Build and Push Docker Images / Build Backend (FastAPI) (push) Successful in 32s
Build and Push Docker Images / Build Frontend (Next.js) (push) Successful in 1m5s
Build and Push Docker Images / Build Integrator (push) Successful in 56s
Build and Push Docker Images / Build Kestra Init (push) Successful in 32s
Build and Push Docker Images / Build Pipeline (Meltano + dbt + Airflow) (push) Successful in 1m7s
Build and Push Docker Images / Trigger Portainer Update (push) Successful in 1s
All checks were successful
Build and Push Docker Images / Build Backend (FastAPI) (push) Successful in 32s
Build and Push Docker Images / Build Frontend (Next.js) (push) Successful in 1m5s
Build and Push Docker Images / Build Integrator (push) Successful in 56s
Build and Push Docker Images / Build Kestra Init (push) Successful in 32s
Build and Push Docker Images / Build Pipeline (Meltano + dbt + Airflow) (push) Successful in 1m7s
Build and Push Docker Images / Trigger Portainer Update (push) Successful in 1s
Port critical patterns from the working integrator into Singer taps: - GIAS: add 404 fallback to yesterday's date, increase timeout to 300s, use latin-1 encoding, use dated URL for links (static URL returns 500) - FBIT: add GIAS date fallback, increase timeout, fix encoding to latin-1 - IDACI: use dated GIAS URL with fallback instead of undated static URL, fix encoding to latin-1, increase timeout to 300s - Ofsted: try utf-8-sig then fall back to latin-1 encoding Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -129,7 +129,11 @@ class OfstedInspectionsStream(Stream):
|
||||
df = pd.read_excel(io.BytesIO(resp.content), engine="odf", dtype=str)
|
||||
else:
|
||||
# Detect header row (may not be row 0)
|
||||
text = resp.content.decode("utf-8-sig", errors="replace")
|
||||
# Try utf-8-sig first, fall back to latin-1 (matches integrator)
|
||||
try:
|
||||
text = resp.content.decode("utf-8-sig")
|
||||
except UnicodeDecodeError:
|
||||
text = resp.content.decode("latin-1")
|
||||
lines = text.split("\n")
|
||||
header_idx = 0
|
||||
for i, line in enumerate(lines[:20]):
|
||||
|
||||
Reference in New Issue
Block a user