integrator/flows/ofsted.yml

id: ofsted-monthly-update
namespace: schoolcompare.data
description: Download and load Ofsted Monthly Management Information CSV

triggers:
  - id: monthly-schedule
    type: io.kestra.plugin.core.trigger.Schedule
    cron: "0 2 1 * *"   # 1st of each month at 02:00

tasks:
  - id: download
    type: io.kestra.plugin.core.http.Request
    uri: http://integrator:8001/run/ofsted?action=download
    method: POST
    allowFailed: false
    timeout: PT10M

  - id: load
    type: io.kestra.plugin.core.http.Request
    uri: http://integrator:8001/run/ofsted?action=load
    method: POST
    allowFailed: false
    timeout: PT30M

errors:
  - id: notify-failure
    type: io.kestra.plugin.core.log.Log
    message: "Ofsted update FAILED: {{ error.message }}"

retry:
  maxAttempts: 3
  delay: PT10M
feat(data): integrate 9 UK government data sources via Kestra Adds a full data integration pipeline for enriching school profiles with supplementary data from Ofsted, GIAS, EES, IDACI, and FBIT. Backend: - Bump SCHEMA_VERSION to 3; add 8 new DB tables (ofsted_inspections, ofsted_parent_view, school_census, admissions, sen_detail, phonics, school_deprivation, school_finance) plus GIAS columns on schools - Expose all supplementary data via GET /api/schools/{urn} - Enrich school list responses with ofsted_grade + ofsted_date Integrator (new service): - FastAPI HTTP microservice; Kestra calls POST /run/{source} - 9 source modules: ofsted, gias, parent_view, census, admissions, sen_detail, phonics, idaci, finance - 9 Kestra flow YAMLs with scheduled triggers and 3× retry Frontend: - SchoolRow: colour-coded Ofsted badge (Outstanding/Good/RI/Inadequate) - SchoolDetailView: 7 new sections — Ofsted sub-judgements, Parent View survey bars, Admissions, Pupils & Inclusion / SEN, Phonics, Deprivation Context, Finances - types.ts: 8 new interfaces + extended School/SchoolDetailsResponse Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> 2026-03-24 11:44:04 +00:00			`id: ofsted-monthly-update`
			`namespace: schoolcompare.data`
			`description: Download and load Ofsted Monthly Management Information CSV`

			`triggers:`
			`- id: monthly-schedule`
			`type: io.kestra.plugin.core.trigger.Schedule`
			`cron: "0 2 1 * *" # 1st of each month at 02:00`

			`tasks:`
			`- id: download`
			`type: io.kestra.plugin.core.http.Request`
			`uri: http://integrator:8001/run/ofsted?action=download`
			`method: POST`
			`allowFailed: false`
			`timeout: PT10M`

			`- id: load`
			`type: io.kestra.plugin.core.http.Request`
			`uri: http://integrator:8001/run/ofsted?action=load`
			`method: POST`
			`allowFailed: false`
			`timeout: PT30M`

			`errors:`
			`- id: notify-failure`
			`type: io.kestra.plugin.core.log.Log`
			`message: "Ofsted update FAILED: {{ error.message }}"`

			`retry:`
			`maxAttempts: 3`
			`delay: PT10M`