TudorandClaude Opus 5 e236669fde
PR Checks / Frontend Typecheck + Tests (pull_request) Successful in 1m4s
PR Checks / Backend Smoke (pull_request) Successful in 9s
PR Checks / Build Backend (no push) (pull_request) Successful in 11s
PR Checks / Build Frontend (no push) (pull_request) Successful in 45s
PR Checks / Build Pipeline (no push) (pull_request) Successful in 52s
PR Checks / AI Code Review (Claude) (pull_request) Successful in 2m27s
fix(destinations): school rows and the England reference are different grains
The annual DAG died with a BrokenPipeError from Meltano's log writer, which
is several frames from the cause: target-postgres exited first and the tap
saw its stdout close.

The tap declared primary_keys = [urn, ...] while emitting urn=None for the
national rows, and target-postgres turns primary_keys into a NOT NULL
constraint. The first national row of the run failed the insert and took
the loader with it. Every other tap in this repo keys on non-null columns.

Carrying two grains in one stream was the actual mistake, so the fix is to
separate them rather than paper over the null: four streams now, with
ees_ks4/ks5_destinations_national carrying no urn column at all — a school
identifier that is null in every row is a grain mismatch, not a column.
The staging models split the same way and the national mart reads the new
pair instead of filtering `where urn is null`.

Verified against the live API: the school stream yields 135,240 rows over
4,508 schools with no duplicate keys, no null key columns and all 31,382
suppression sentinels intact; the national streams yield 30 and 33 rows
with no urn column.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvdDKvFFSZuMVDH5fEyTob
2026-08-31 21:36:43 +01:00
2026-01-06 19:05:22 +00:00
2026-04-07 16:17:56 +01:00
2026-01-07 16:20:49 +00:00
2026-01-07 16:20:49 +00:00
2026-04-01 15:05:21 +01:00

Primary School Compass 🧒📚

A modern web application for comparing primary school (KS2) performance data in Wandsworth and Merton over the last 5 years. Built with FastAPI and vanilla JavaScript with Chart.js visualizations.

Python FastAPI License

Features

  • 📊 Interactive Charts - Visualize KS2 performance trends over time
  • 🔍 Smart Search - Find primary schools by name in Wandsworth & Merton
  • ⚖️ Side-by-Side Comparison - Compare up to 5 schools simultaneously
  • 🏆 Rankings - View top-performing primary schools by various KS2 metrics
  • 📱 Responsive Design - Works beautifully on desktop and mobile

Key Metrics (KS2)

The application tracks these Key Stage 2 performance indicators:

Metric Description
Reading Progress Progress in reading from KS1 to KS2
Writing Progress Progress in writing from KS1 to KS2
Maths Progress Progress in maths from KS1 to KS2
Reading Expected % Percentage meeting expected standard in reading
Writing Expected % Percentage meeting expected standard in writing
Maths Expected % Percentage meeting expected standard in maths
Reading, Writing & Maths Combined % Percentage meeting expected standard in all three subjects

Quick Start

1. Clone and Setup

cd school_results

# Create virtual environment
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

2. Run the Application

# Start the server
python -m uvicorn backend.app:app --reload --port 8000

Then open http://localhost:8000 in your browser.

The app will run with sample data by default, showing 110 primary schools (66 in Wandsworth, 44 in Merton) with 5 years of KS2 performance data.

3. (Optional) Use Real Data

To use real UK school performance data:

  1. Visit Compare School Performance - Download Data

  2. Download Key Stage 2 data for the years you want (2019-2024)

    • Select "Key Stage 2" as the data type
  3. Place the CSV files in the data/ folder

  4. Restart the server - it will automatically load and filter to Wandsworth & Merton schools

Note: The app only displays schools in Wandsworth and Merton. Data from other areas will be filtered out.

See the helper script for more details:

python scripts/download_data.py

Project Structure

school_results/
├── backend/
│   └── app.py           # FastAPI application with all API endpoints
├── frontend/
│   ├── index.html       # Main HTML page
│   ├── styles.css       # Styling (warm, editorial design)
│   └── app.js           # Frontend JavaScript
├── data/
│   └── .gitkeep         # Place CSV data files here
├── scripts/
│   └── download_data.py # Helper for downloading/processing data
├── requirements.txt     # Python dependencies
└── README.md

API Endpoints

Endpoint Description
GET /api/schools List schools with optional search/filter
GET /api/schools/{urn} Get detailed data for a specific school
GET /api/compare?urns=... Compare multiple schools
GET /api/rankings Get school rankings by metric
GET /api/filters Get available filter options
GET /api/metrics Get available performance metrics

Example API Usage

# Search for schools
curl "http://localhost:8000/api/schools?search=academy"

# Get school details
curl "http://localhost:8000/api/schools/100001"

# Compare schools
curl "http://localhost:8000/api/compare?urns=100001,100002,100003"

# Get rankings
curl "http://localhost:8000/api/rankings?metric=rwm_expected_pct&year=2024"

Data Format

If using your own CSV data, ensure it includes these columns (or similar):

Column Type Description
URN Integer Unique Reference Number
SCHNAME String School name
LA String Local Authority (must be Wandsworth or Merton)
READPROG Float Reading progress score
WRITPROG Float Writing progress score
MATPROG Float Maths progress score
PTRWM_EXP Float % meeting expected standard in reading, writing & maths
PTREAD_EXP Float % meeting expected standard in reading
PTWRIT_EXP Float % meeting expected standard in writing
PTMAT_EXP Float % meeting expected standard in maths

The application normalizes column names automatically and filters to only show Wandsworth and Merton schools.

Technology Stack

  • Backend: FastAPI (Python) - High-performance async API framework
  • Frontend: Vanilla JavaScript with Chart.js
  • Styling: Custom CSS with CSS variables for theming
  • Data: Pandas for CSV processing

Design Philosophy

The UI features a warm, editorial design inspired by quality publications:

  • Typography: DM Sans for body text, Playfair Display for headings
  • Color Palette: Warm cream background with coral and teal accents
  • Interactions: Smooth animations and hover effects
  • Charts: Clean, readable data visualizations

Development

# Run with auto-reload
python -m uvicorn backend.app:app --reload --port 8000

# Or run directly
python backend/app.py

Coverage

This application is specifically designed for:

  • School Phase: Primary schools only (Key Stage 2)
  • Geographic Area: Wandsworth and Merton (London boroughs)
  • Time Period: Last 5 years of data (2020-2024)

Note: 2021 data shows as unavailable because SATs were cancelled due to COVID-19.

Data Source

Data is sourced from the UK Government's Compare School Performance service, which provides official school performance data for England.

Important: When using real data, please comply with the terms of use and data protection regulations.

Scheduled Jobs

Geocoding Schools (Cron Job)

School postcodes are geocoded by a scheduled job, not on-demand. This improves performance and reduces API calls.

Setup the cron job (runs weekly on Sunday at 2am):

# Edit crontab
crontab -e

# Add this line (adjust paths as needed):
0 2 * * 0 cd /path/to/school_compare && /path/to/venv/bin/python scripts/geocode_schools.py >> /var/log/geocode_schools.log 2>&1

Manual run:

# Geocode only schools missing coordinates
python scripts/geocode_schools.py

# Force re-geocode all schools
python scripts/geocode_schools.py --force

License

MIT License - feel free to use this project for educational purposes.


Built with ❤️ for Wandsworth & Merton families

S
Description
No description provided
Readme
53 MiB
0 Stars 1 Watchers 0 Forks
Languages
TypeScript 56.2%
Python 25.7%
CSS 13.2%
HTML 4.1%
JavaScript 0.6%
Other 0.2%