0 Fork 0

Files

T

TudorandClaude Opus 5 eaf5e5d180 docs: describe the system that exists, not the one we started with

The README still opened on "Primary School Compass", a KS2 tool for Wandsworth
and Merton served by FastAPI and vanilla JavaScript with Chart.js. Every layer
of that sentence is now wrong: coverage is England-wide across KS2, KS4,
all-through and post-16, Next.js owns the public UI, and school data comes from
dbt-built `marts.*` rather than CSVs loaded at startup. The setup instructions
walked a reader into a virtualenv and a CSV import that cannot build the current
schema, so following the docs produced an empty database and a wrong mental
model at the same time.

Replace the narrative docs with two reference documents that were checked
against the code: docs/ARCHITECTURE.md for request flow, data ownership, the
backend/frontend module boundaries and the real publication sequence, and
docs/DEVELOPMENT.md for the checks that actually run, including the container
and CI version skew that makes "just run pytest" misleading.

The env examples drifted the same way. ALLOWED_ORIGINS is a JSON array, not a
comma-separated list; the frontend needs FASTAPI_URL, DATABASE_URL and
PAYLOAD_SECRET, none of which were documented; and RATE_LIMIT_BURST,
DEFAULT_PAGE_SIZE and MAX_PAGE_SIZE were presented as tuning controls the routes
do not consult. Each is now stated as it behaves.

MIGRATION_SUMMARY.md keeps its content but gains a banner, because it reads like
setup instructions and is not.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016y2J6bs8gbuSJbH18w7Tan

2026-09-14 23:01:15 +01:00

3.8 KiB

Raw Blame History

Development and validation

Prerequisites and environment boundaries

Use a feature branch. The deployed stack is the integration environment; do not assume a local server can run from a fresh checkout. This cleanup did not start local servers or provision databases. Unit tests use fixtures and mocks.

The current versions are not yet aligned:

Component	Container	PR checks
Backend	Python 3.11	Python 3.12
Frontend	Node 24	Node 22
Pipeline	Python 3.13	Pipeline image build

Use the component's container version when reproducing deployment behaviour. The backend dependency pins predate Python 3.14; do not assume the system Python can install or run them. Version alignment is a separate maintenance task.

Frontend checks

cd nextjs-app
npm ci
npm run typecheck
npm test -- --runInBand

npm run build is the production build check. There is no lint script. Tests live in __tests__/ and use Jest/React Testing Library. These checks do not prove that live PostgreSQL queries, Typesense or a deployed proxy work.

The frontend .env.example documents runtime variables. Browser traffic normally uses /api; FASTAPI_URL is an absolute server-side URL ending in /api. Payload additionally needs DATABASE_URL and PAYLOAD_SECRET when used at runtime. Never commit credentials or real .env files.

Backend checks

From the repository root, using an available Python 3.11 or 3.12 interpreter:

python3.11 -m venv /tmp/schoolcompare-backend-venv
/tmp/schoolcompare-backend-venv/bin/python -m pip install -r requirements.txt pytest 'httpx<0.28'
/tmp/schoolcompare-backend-venv/bin/python -m pytest backend/tests -q

Substitute python3.12 if matching PR CI. The test dependencies above match the current workflow; they are not yet captured in a dedicated development lockfile. Backend configuration is defined in backend/config.py; .env.example documents commonly used values. ALLOWED_ORIGINS uses a JSON array, not a comma-separated string.

Data and pipeline work

The app needs populated marts.* tables. A new Postgres instance alone is not a working school-data environment. Use the existing managed pipeline or an approved snapshot; the removed CSV importer cannot build the current schema.

The pipeline container includes Meltano, dbt/Postgres, Airflow and the custom taps. Airflow commands/selectors live in pipeline/dags/. Schema tests live in pipeline/transform/tests/ and model YAML files. Run the relevant dbt build selector in an isolated data environment for model changes; it writes tables and is not a read-only smoke test. Prefer python -m dbt.cli.main as the DAGs do.

GIAS dictionaries are generated together by pipeline/scripts/generate_gias_codes.py. The backend and pipeline copies are intentional; backend/tests/test_gias_codes.py checks that they stay identical.

For Payload collection/editor changes, run npm run generate:importmap in nextjs-app/ and include the generated map. Preserve CMS migrations and the separate payload schema. See publishing.

End-to-end checks

Against an existing, authorised test environment:

cd e2e
npm ci
npx playwright install chromium
BASE_URL=https://your-test-environment.example npx playwright test

The suite does not start a web server. CI installs Chromium with system dependencies and runs against staging. Use the configured staging target: docs/DEPLOY.md records the public staging proxy limitation. User-visible behaviour changes should update the corresponding journeys.

Before requesting review

Run checks relevant to the change, inspect git diff --check, and report checks that could not run. Do not publish or promote as part of local validation. DEPLOY.md documents the PR and human promotion gates.

3.8 KiB Raw Blame History