TudorandClaude Opus 5
eaf5e5d180
The README still opened on "Primary School Compass", a KS2 tool for Wandsworth and Merton served by FastAPI and vanilla JavaScript with Chart.js. Every layer of that sentence is now wrong: coverage is England-wide across KS2, KS4, all-through and post-16, Next.js owns the public UI, and school data comes from dbt-built `marts.*` rather than CSVs loaded at startup. The setup instructions walked a reader into a virtualenv and a CSV import that cannot build the current schema, so following the docs produced an empty database and a wrong mental model at the same time. Replace the narrative docs with two reference documents that were checked against the code: docs/ARCHITECTURE.md for request flow, data ownership, the backend/frontend module boundaries and the real publication sequence, and docs/DEVELOPMENT.md for the checks that actually run, including the container and CI version skew that makes "just run pytest" misleading. The env examples drifted the same way. ALLOWED_ORIGINS is a JSON array, not a comma-separated list; the frontend needs FASTAPI_URL, DATABASE_URL and PAYLOAD_SECRET, none of which were documented; and RATE_LIMIT_BURST, DEFAULT_PAGE_SIZE and MAX_PAGE_SIZE were presented as tuning controls the routes do not consult. Each is now stated as it behaves. MIGRATION_SUMMARY.md keeps its content but gains a banner, because it reads like setup instructions and is not. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016y2J6bs8gbuSJbH18w7Tan
95 lines
3.8 KiB
Markdown
95 lines
3.8 KiB
Markdown
# Development and validation
|
|
|
|
## Prerequisites and environment boundaries
|
|
|
|
Use a feature branch. The deployed stack is the integration environment; do not
|
|
assume a local server can run from a fresh checkout. This cleanup did not start
|
|
local servers or provision databases. Unit tests use fixtures and mocks.
|
|
|
|
The current versions are not yet aligned:
|
|
|
|
| Component | Container | PR checks |
|
|
|---|---|---|
|
|
| Backend | Python 3.11 | Python 3.12 |
|
|
| Frontend | Node 24 | Node 22 |
|
|
| Pipeline | Python 3.13 | Pipeline image build |
|
|
|
|
Use the component's container version when reproducing deployment behaviour.
|
|
The backend dependency pins predate Python 3.14; do not assume the system Python
|
|
can install or run them. Version alignment is a separate maintenance task.
|
|
|
|
## Frontend checks
|
|
|
|
```sh
|
|
cd nextjs-app
|
|
npm ci
|
|
npm run typecheck
|
|
npm test -- --runInBand
|
|
```
|
|
|
|
`npm run build` is the production build check. There is no `lint` script.
|
|
Tests live in `__tests__/` and use Jest/React Testing Library. These checks do not
|
|
prove that live PostgreSQL queries, Typesense or a deployed proxy work.
|
|
|
|
The frontend `.env.example` documents runtime variables. Browser traffic normally
|
|
uses `/api`; `FASTAPI_URL` is an absolute server-side URL ending in `/api`.
|
|
Payload additionally needs `DATABASE_URL` and `PAYLOAD_SECRET` when used at runtime.
|
|
Never commit credentials or real `.env` files.
|
|
|
|
## Backend checks
|
|
|
|
From the repository root, using an available Python 3.11 or 3.12 interpreter:
|
|
|
|
```sh
|
|
python3.11 -m venv /tmp/schoolcompare-backend-venv
|
|
/tmp/schoolcompare-backend-venv/bin/python -m pip install -r requirements.txt pytest 'httpx<0.28'
|
|
/tmp/schoolcompare-backend-venv/bin/python -m pytest backend/tests -q
|
|
```
|
|
|
|
Substitute `python3.12` if matching PR CI. The test dependencies above match the
|
|
current workflow; they are not yet captured in a dedicated development lockfile.
|
|
Backend configuration is defined in `backend/config.py`; `.env.example` documents
|
|
commonly used values. `ALLOWED_ORIGINS` uses a JSON array, not a comma-separated string.
|
|
|
|
## Data and pipeline work
|
|
|
|
The app needs populated `marts.*` tables. A new Postgres instance alone is not a
|
|
working school-data environment. Use the existing managed pipeline or an approved
|
|
snapshot; the removed CSV importer cannot build the current schema.
|
|
|
|
The pipeline container includes Meltano, dbt/Postgres, Airflow and the custom taps.
|
|
Airflow commands/selectors live in `pipeline/dags/`. Schema tests live in
|
|
`pipeline/transform/tests/` and model YAML files. Run the relevant `dbt build`
|
|
selector in an isolated data environment for model changes; it writes tables and
|
|
is not a read-only smoke test. Prefer `python -m dbt.cli.main` as the DAGs do.
|
|
|
|
GIAS dictionaries are generated together by
|
|
`pipeline/scripts/generate_gias_codes.py`. The backend and pipeline copies are
|
|
intentional; `backend/tests/test_gias_codes.py` checks that they stay identical.
|
|
|
|
For Payload collection/editor changes, run `npm run generate:importmap` in
|
|
`nextjs-app/` and include the generated map. Preserve CMS migrations and the
|
|
separate `payload` schema. See [publishing](../nextjs-app/docs/PUBLISHING.md).
|
|
|
|
## End-to-end checks
|
|
|
|
Against an existing, authorised test environment:
|
|
|
|
```sh
|
|
cd e2e
|
|
npm ci
|
|
npx playwright install chromium
|
|
BASE_URL=https://your-test-environment.example npx playwright test
|
|
```
|
|
|
|
The suite does not start a web server. CI installs Chromium with system dependencies
|
|
and runs against staging. Use the configured staging target: `docs/DEPLOY.md`
|
|
records the public staging proxy limitation. User-visible behaviour changes should
|
|
update the corresponding journeys.
|
|
|
|
## Before requesting review
|
|
|
|
Run checks relevant to the change, inspect `git diff --check`, and report checks
|
|
that could not run. Do not publish or promote as part of local validation.
|
|
[DEPLOY.md](DEPLOY.md) documents the PR and human promotion gates.
|