school_compare/claude.md

SchoolCompare.co.uk - Project Context

Overview

SchoolCompare is a web application for comparing UK primary school (KS2) performance data. It allows users to:

• Search and browse schools by name, location (postcode), or local authority
• Compare multiple schools side-by-side with charts and tables
• View school rankings by various KS2 metrics
• See historical performance trends across years

Architecture

Backend (Python/FastAPI)

• Framework: FastAPI with uvicorn
• Database: PostgreSQL with SQLAlchemy ORM
• Data Source: UK Government "Compare School Performance" CSV downloads

Key files:

• backend/app.py - Main FastAPI application, API routes
• backend/config.py - Configuration via pydantic-settings (env vars, .env file)
• backend/database.py - SQLAlchemy engine, session management
• backend/models.py - Database models (School, SchoolResult)
• backend/data_loader.py - Data queries, geocoding, legacy DataFrame compatibility
• backend/schemas.py - Column mappings, metric definitions, LA code mappings

Frontend (Vanilla JS)

• Single-page application with hash-based routing
• Chart.js for data visualization
• No build step required

Key files:

• frontend/index.html - Main HTML structure
• frontend/app.js - All application logic, API calls, rendering
• frontend/styles.css - Styling (CSS variables, responsive design)

Database Schema

schools                          school_results
├── id (PK)                      ├── id (PK)
├── urn (unique, indexed)        ├── school_id (FK → schools.id)
├── school_name                  ├── year (indexed)
├── local_authority              ├── rwm_expected_pct
├── school_type                  ├── reading_expected_pct
├── postcode                     ├── ... (all KS2 metrics)
├── latitude, longitude          └── unique(school_id, year)
└── results → SchoolResult[]

Configuration

Environment variables (or .env file):

• DATABASE_URL - PostgreSQL connection string (default: postgresql://schoolcompare:schoolcompare@localhost:5432/schoolcompare)
• HOST, PORT - Server binding (default: 0.0.0.0:80)
• ALLOWED_ORIGINS - CORS origins

Running Locally

1. Start PostgreSQL:

   docker compose up -d db
   

2. Run migration to import CSV data:

   python scripts/migrate_csv_to_db.py --drop
   # Add --geocode to geocode postcodes (slower, adds lat/long)
   

3. Start the app:

   uvicorn backend.app:app --host 0.0.0.0 --port 8000
   

Docker Deployment

docker compose up -d

This starts:

• db - PostgreSQL 16 with persistent volume
• app - FastAPI application on port 80

Data

• Source: UK Government Compare School Performance downloads
• Location: data/ directory with year folders (e.g., 2023-2024/england_ks2final.csv)
• The scripts/download_data.py can fetch data from the government website

Key Features

• Location Search: Enter postcode to find nearby schools (uses postcodes.io API)
• Multi-school Comparison: Select multiple schools, view metrics across years
• Rankings: Top schools by any KS2 metric, filterable by local authority
• Variability Analysis: Shows standard deviation of scores across years

API Endpoints

• GET /api/schools - List/search schools (supports pagination, location search)
• GET /api/schools/{urn} - School details with all yearly data
• GET /api/compare?urns=123,456 - Compare multiple schools
• GET /api/rankings - School rankings by metric
• GET /api/filters - Available filter options (LAs, types, years)
• GET /api/metrics - Metric definitions (single source of truth)
• GET /api/data-info - Database stats

SDLC

Full details in docs/DEPLOY.md. The short version:

• Never push to main directly. Work on a feature branch and open a PR; branch protection requires the PR checks (typecheck, tests, builds, AI review) to pass before merge.
• Merging to main deploys automatically: images are built once, deployed to the staging Portainer stack, verified by the Playwright journeys in e2e/, and only then retagged :prod and rolled out to production.
• If you change user-facing behaviour, update or extend the e2e/ journey tests in the same PR — they are the promotion gate.

Recent Changes

• Added staging environment + automated staging→prod pipeline (Gitea Actions)
• Migrated from CSV file storage to PostgreSQL database
• Added location-based search using postcode geocoding
• Added local authority filter to rankings
• Improved frontend with featured schools, loading states, API caching

Important

• Do not attempt to start a local server to test the application, it does not work