TudorandClaude Opus 5
1d8858fbda
`backend/migration.py` and `scripts/migrate_csv_to_db.py` import `School`, `SchoolResult`, `init_db` and `set_db_schema_version` — names that no longer exist. `scripts/geocode_schools.py` imports the same removed ORM model. None of the three can be imported against the current backend, so they were not dormant utilities anyone could fall back on; they were files that would fail on the first line. `backend/version.py` existed only to hand `SCHEMA_VERSION` to that importer, and the FastAPI lifespan performs no version-triggered import. Three symbols go with them, each confirmed to have no caller: the unvectorised `haversine_distance`, superseded by the inline NumPy calculation in search; `fetcher`, an SWR helper for a dependency this project does not install; and `kmToMiles`. `calculateDistance` stays — CutoffMapPanel uses it. Two comments pointed at `migrate_csv_to_db.py --drop` to explain why Payload owns its own schema. The reason survives the script: blog content must stay clear of the school marts and Airflow's metadata. Reworded rather than deleted, so the constraint keeps its justification. docs/LEGACY_CODE.md records what was removed and where to find it in history. It also records what was deliberately *not* removed, which is the more useful half: unused UI components awaiting a design decision, manual data utilities whose operators a repository search cannot see, and fallbacks that look obsolete but are load-bearing — `data_loader.py`'s older-mart branches, the generated GIAS dictionary copies, and the `legacy`-named dbt models that annual DAG selectors explicitly include. A zero-import count is evidence, not a verdict. The scripts that fetch DfE CSVs are marked historical and kept, pending confirmation that nobody runs them by hand. Checked: 190 backend tests, 429 frontend tests, `tsc --noEmit` clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016y2J6bs8gbuSJbH18w7Tan
SchoolCompare
SchoolCompare compares schools across England: primary (KS2), secondary (KS4), all-through and post-16 provision, with coverage depending on the source dataset. It provides school search, postcode maps, comparisons, rankings, place pages, Ofsted information, admissions and destination measures. Editorial content lives in a Payload CMS blog.
Start here
- Architecture and data flow
- Development and validation
- Deployment and promotion
- Legacy and unused-code inventory
- Frontend conventions
- CMS publishing
Repository map
| Path | Responsibility |
|---|---|
backend/ |
FastAPI routes, cached school data, read-only SQLAlchemy mappings, feature flags |
nextjs-app/ |
Next.js App Router, React UI, Payload CMS, frontend tests |
pipeline/plugins/extractors/ |
Custom Singer taps for GIAS, EES, Ofsted and other datasets |
pipeline/transform/ |
dbt staging/intermediate models, marts, seeds and data tests |
pipeline/dags/ |
Airflow extraction, transformation and publication workflows |
pipeline/scripts/ |
Search indexing, code generation and operational diagnostics |
e2e/ |
Playwright journeys against a running environment |
.gitea/workflows/ |
PR checks, staging deployment and manual production promotion |
scripts/ |
CI review tooling and historical data utilities; see the legacy inventory |
docs/superpowers/, mockups/ |
Design history and prototypes, not application entry points |
Runtime
The public site is Next.js, not the FastAPI root page. Browser /api/*
requests pass through a Next.js route handler to FastAPI. Server-rendered pages
call FastAPI directly using FASTAPI_URL, including its /api suffix.
PostgreSQL/PostGIS stores school data. Meltano/Singer extracts source data;
dbt builds marts.*; FastAPI reads those tables. Typesense serves text search
and autocomplete. Payload runs inside Next.js and owns a separate payload
database schema and uploaded media.
There is no automatic CSV import or sample dataset on startup. A working school-data environment needs populated marts from the pipeline or an approved database snapshot. See development before choosing a setup.
Validation
cd nextjs-app
npm ci
npm run typecheck
npm test -- --runInBand
Backend checks, pipeline validation, runtime versions and E2E requirements are
listed in DEVELOPMENT.md. No npm run lint script is
currently defined.
Deployment
Work on a feature branch and open a PR. Merging to main builds images and
deploys staging. Production promotion is a separate, human-triggered Gitea
workflow. Use DEPLOY.md and the Portainer compose files as the
operational references. The generic compose examples still reference :latest,
which the current release workflow does not publish.