Tudor
f1fb847164
All checks were successful
Build and Push Docker Images / Build Backend (FastAPI) (push) Successful in 47s
Build and Push Docker Images / Build Frontend (Next.js) (push) Successful in 1m13s
Build and Push Docker Images / Build Integrator (push) Successful in 40s
Build and Push Docker Images / Trigger Portainer Update (push) Successful in 0s
- backend: POST /api/admin/reimport-ks2 runs full CSV migration in a thread - backend/docker-compose: ADMIN_API_KEY env var (default: changeme) so the key is stable across restarts and the integrator can call the endpoint - integrator: sources/ks2.py triggers the backend endpoint (900s timeout) - integrator: flows/ks2.yml Kestra flow (manual trigger, no schedule) To re-ingest after a DB wipe: trigger the ks2-reimport flow from the Kestra UI at http://localhost:8080. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
23 lines
603 B
YAML
23 lines
603 B
YAML
id: ks2-reimport
|
|
namespace: schoolcompare.data
|
|
description: Re-import KS2 attainment data from bundled CSV files (use after DB wipe)
|
|
|
|
# No scheduled trigger — run manually from the Kestra UI when needed.
|
|
|
|
tasks:
|
|
- id: reimport
|
|
type: io.kestra.plugin.core.http.Request
|
|
uri: http://integrator:8001/run/ks2?action=load
|
|
method: POST
|
|
allowFailed: false
|
|
timeout: PT20M # full migration takes ~10 min; give headroom
|
|
|
|
errors:
|
|
- id: notify-failure
|
|
type: io.kestra.plugin.core.log.Log
|
|
message: "KS2 re-import FAILED: {{ error.message }}"
|
|
|
|
retry:
|
|
maxAttempts: 2
|
|
delay: PT5M
|