Compare commits

...
Author SHA1 Message Date
TudorandClaude Fable 5 9f8dba227c fix(compare): keep chart data when a client refetch fails
PR Checks / Frontend Typecheck + Tests (pull_request) Successful in 9m38s
PR Checks / Backend Smoke (pull_request) Successful in 5s
PR Checks / Build Backend (no push) (pull_request) Successful in 10s
PR Checks / Build Frontend (no push) (pull_request) Successful in 47s
PR Checks / Build Pipeline (no push) (pull_request) Successful in 9s
PR Checks / AI Code Review (Claude) (pull_request) Successful in 1m9s
The compare view refetches /api/compare on the client after SSR; on any
failure the catch nulled comparisonData, destroying the working
SSR-provided chart. A transient error (or staging's broken external /api
proxy) should not blank a comparison the user is already viewing — keep
the existing data instead.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 23:08:22 +01:00
tudor 18cd805c6c Merge pull request 'feat(compare): readable comparison chart on mobile' (#11) from feat/compare-chart-mobile-readability into main
Deploy (staging -> E2E gate -> production) / Build Backend (FastAPI) (push) Successful in 13s
Deploy (staging -> E2E gate -> production) / Build Frontend (Next.js) (push) Successful in 49s
Deploy (staging -> E2E gate -> production) / Build Pipeline (Meltano + dbt + Airflow) (push) Successful in 12s
Deploy (staging -> E2E gate -> production) / Deploy to Staging (push) Successful in 1s
Deploy (staging -> E2E gate -> production) / E2E Journeys against Staging (push) Failing after 1m6s
Deploy (staging -> E2E gate -> production) / Promote to Production (push) Has been skipped
Reviewed-on: #11
2026-07-05 21:26:01 +00:00
TudorandClaude Fable 5 22769b6295 feat(compare): readable comparison chart on mobile
PR Checks / Frontend Typecheck + Tests (pull_request) Successful in 9m36s
PR Checks / Backend Smoke (pull_request) Successful in 5s
PR Checks / Build Backend (no push) (pull_request) Successful in 10s
PR Checks / Build Frontend (no push) (pull_request) Successful in 44s
PR Checks / Build Pipeline (no push) (pull_request) Successful in 10s
PR Checks / AI Code Review (Claude) (pull_request) Successful in 2m49s
The compare chart squashed clustered schools into a thin band (y pinned
0-100) under an in-chart title + per-school legend that ate ~40% of a
300px card, leaving converging lines indistinguishable on phones.

- Auto-fit the y-axis to the data on all viewports (computeYBounds in
  lib/utils: padded + min-span for percentages, symmetric around 0 for
  progress, fitted for scores; negative pct-named trend metrics are not
  zero-clamped).
- Distinct point style per school (circle/triangle/rect/rectRot/star)
  as secondary encoding for convergence and colour-blindness.
- Mobile: drop in-chart title/legend/axis titles; add a chip row (colour
  dot + name) that doubles as tap-to-focus — highlights one school's
  line and dims the rest. Chart card 300px -> 340px, nearly all plot.
- Fix a latent colour mismatch: datasets were built from Object.entries
  whose integer-like URN keys enumerate in ascending numeric order,
  desyncing line colours from card colours; the chart now receives the
  ordered school list.
- Union years across schools instead of taking the first school's.
- Extract PerformanceChart's matchMedia pattern into hooks/useIsMobile.

Unit tests for metricKind/computeYBounds; e2e journey covers the mobile
chips and focus toggle.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 22:02:26 +01:00
tudor 90f2a02e75 Merge pull request 'fix(school): make hero map fullscreen work on iOS Safari' (#10) from fix/hero-map-ios-fullscreen into main
Deploy (staging -> E2E gate -> production) / Build Backend (FastAPI) (push) Successful in 13s
Deploy (staging -> E2E gate -> production) / Build Frontend (Next.js) (push) Successful in 52s
Deploy (staging -> E2E gate -> production) / Build Pipeline (Meltano + dbt + Airflow) (push) Successful in 12s
Deploy (staging -> E2E gate -> production) / Deploy to Staging (push) Successful in 1s
Deploy (staging -> E2E gate -> production) / E2E Journeys against Staging (push) Successful in 34s
Deploy (staging -> E2E gate -> production) / Promote to Production (push) Successful in 8s
Reviewed-on: #10
2026-07-05 20:52:32 +00:00
TudorandClaude Fable 5 d52d384cf2 fix(school): make hero map fullscreen work on iOS Safari
PR Checks / Frontend Typecheck + Tests (pull_request) Successful in 9m39s
PR Checks / Backend Smoke (pull_request) Successful in 5s
PR Checks / Build Backend (no push) (pull_request) Successful in 10s
PR Checks / Build Frontend (no push) (pull_request) Successful in 47s
PR Checks / Build Pipeline (no push) (pull_request) Successful in 9s
PR Checks / AI Code Review (Claude) (pull_request) Successful in 4m15s
iOS Safari has no Element.requestFullscreen (fullscreen is video-only),
so tapping the map band or 'View on map' silently did nothing on
iPhones. Fall back to a fixed-position CSS overlay driven by state when
the Fullscreen API is missing or its promise rejects, locking body
scroll while open. The Leaflet map already re-measures via the shared
isFullscreen flag. New e2e journey simulates the iOS condition by
deleting the API and asserts the overlay opens and closes; it fails
against the current production build.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 21:34:22 +01:00
tudor ff606dad71 Merge pull request 'fix(e2e): pick the latest explicit year in the rankings year test' (#8) from fix/e2e-rankings-year-pick into main
Deploy (staging -> E2E gate -> production) / Build Backend (FastAPI) (push) Successful in 12s
Deploy (staging -> E2E gate -> production) / Build Frontend (Next.js) (push) Successful in 52s
Deploy (staging -> E2E gate -> production) / Build Pipeline (Meltano + dbt + Airflow) (push) Successful in 12s
Deploy (staging -> E2E gate -> production) / Deploy to Staging (push) Successful in 1s
Deploy (staging -> E2E gate -> production) / E2E Journeys against Staging (push) Successful in 34s
Deploy (staging -> E2E gate -> production) / Promote to Production (push) Successful in 10s
Reviewed-on: #8
2026-07-05 13:54:55 +00:00
TudorandClaude Fable 5 acec8135e1 fix(e2e): pick the latest explicit year in the rankings year test
PR Checks / Frontend Typecheck + Tests (pull_request) Successful in 9m39s
PR Checks / Backend Smoke (pull_request) Successful in 5s
PR Checks / Build Backend (no push) (pull_request) Successful in 10s
PR Checks / Build Frontend (no push) (pull_request) Successful in 41s
PR Checks / Build Pipeline (no push) (pull_request) Successful in 9s
PR Checks / AI Code Review (Claude) (pull_request) Successful in 1m24s
Staging doesn't always carry the full data history, so selecting the
oldest year legitimately returns no rows and fails the promotion gate.
Select the most recent explicit year instead: the default view already
proved it has rows, so an empty table after selecting it can only mean
the year query param was rejected — the regression this test guards.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 14:29:43 +01:00
tudor 0a370e3b63 Merge pull request 'fix(api): accept academic-year codes in rankings year filter' (#7) from fix/rankings-year-validation into main
Deploy (staging -> E2E gate -> production) / Build Backend (FastAPI) (push) Successful in 21s
Deploy (staging -> E2E gate -> production) / Build Frontend (Next.js) (push) Successful in 46s
Deploy (staging -> E2E gate -> production) / Build Pipeline (Meltano + dbt + Airflow) (push) Successful in 12s
Deploy (staging -> E2E gate -> production) / Deploy to Staging (push) Successful in 1s
Deploy (staging -> E2E gate -> production) / E2E Journeys against Staging (push) Failing after 1m3s
Deploy (staging -> E2E gate -> production) / Promote to Production (push) Has been skipped
Reviewed-on: #7
2026-07-05 10:46:04 +00:00
TudorandClaude Fable 5 6c872ce726 fix(api): accept academic-year codes in rankings year filter
PR Checks / Frontend Typecheck + Tests (pull_request) Successful in 9m36s
PR Checks / Backend Smoke (pull_request) Successful in 5s
PR Checks / Build Backend (no push) (pull_request) Successful in 20s
PR Checks / Build Frontend (no push) (pull_request) Successful in 52s
PR Checks / Build Pipeline (no push) (pull_request) Successful in 10s
PR Checks / AI Code Review (Claude) (pull_request) Successful in 1m44s
The rankings endpoint validated year with le=2100, but the database
stores academic-year codes like 201819, so any explicit year selection
returned a 422 and the rankings page rendered its empty state. Widen
the bound to cover the codes and extend the e2e journey to pick a
specific year and assert the table stays populated.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 22:13:56 +01:00
tudor 23b4e1c453 Merge pull request 'fix(e2e): scroll detail-page chart into view before asserting' (#5) from fix/e2e-detail-chart-scroll into main
Deploy (staging -> E2E gate -> production) / Build Backend (FastAPI) (push) Successful in 13s
Deploy (staging -> E2E gate -> production) / Build Frontend (Next.js) (push) Successful in 49s
Deploy (staging -> E2E gate -> production) / Build Pipeline (Meltano + dbt + Airflow) (push) Successful in 12s
Deploy (staging -> E2E gate -> production) / Deploy to Staging (push) Successful in 1s
Deploy (staging -> E2E gate -> production) / E2E Journeys against Staging (push) Successful in 33s
Deploy (staging -> E2E gate -> production) / Promote to Production (push) Successful in 8s
2026-07-03 13:56:00 +00:00
TudorandClaude Fable 5 deeef23131 fix(e2e): assert on a visible canvas — the first canvas is hidden by design
PR Checks / Frontend Typecheck + Tests (pull_request) Successful in 9m41s
PR Checks / Backend Smoke (pull_request) Successful in 5s
PR Checks / Build Backend (no push) (pull_request) Successful in 10s
PR Checks / Build Frontend (no push) (pull_request) Successful in 47s
PR Checks / Build Pipeline (no push) (pull_request) Successful in 10s
PR Checks / AI Code Review (Claude) (pull_request) Successful in 26s
The admissions card stacks year/trend views in one grid cell and keeps the
inactive view visibility:hidden; its canvas is first in the DOM. Use
canvas:visible instead of scrolling. Verified: full suite passes against prod.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PqGhF93UrpDNvXBLMjJENL
2026-07-03 14:40:13 +01:00
TudorandClaude Fable 5 4ece55b031 fix(e2e): scroll the detail-page chart into view before asserting visibility
PR Checks / Frontend Typecheck + Tests (pull_request) Successful in 9m41s
PR Checks / Backend Smoke (pull_request) Successful in 5s
PR Checks / Build Backend (no push) (pull_request) Successful in 10s
PR Checks / Build Frontend (no push) (pull_request) Successful in 55s
PR Checks / Build Pipeline (no push) (pull_request) Successful in 10s
PR Checks / AI Code Review (Claude) (pull_request) Successful in 34s
The canvas renders below the fold and stays 'hidden' to Playwright until
scrolled to; wait for attachment, scroll, then assert.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PqGhF93UrpDNvXBLMjJENL
2026-07-03 14:34:17 +01:00
tudor 515494dbf0 Merge pull request 'fix(analytics): restrict Umami to production hostnames' (#4) from fix/umami-prod-domains-only into main
Deploy (staging -> E2E gate -> production) / Build Backend (FastAPI) (push) Successful in 13s
Deploy (staging -> E2E gate -> production) / Build Frontend (Next.js) (push) Successful in 50s
Deploy (staging -> E2E gate -> production) / Build Pipeline (Meltano + dbt + Airflow) (push) Successful in 12s
Deploy (staging -> E2E gate -> production) / Deploy to Staging (push) Successful in 0s
Deploy (staging -> E2E gate -> production) / E2E Journeys against Staging (push) Failing after 1m7s
Deploy (staging -> E2E gate -> production) / Promote to Production (push) Has been skipped
2026-07-03 13:17:53 +00:00
TudorandClaude Fable 5 5772c54ccd fix(ci): use the documented GITHUB_TOKEN name for the run-scoped token
PR Checks / Frontend Typecheck + Tests (pull_request) Successful in 9m37s
PR Checks / Backend Smoke (pull_request) Successful in 5s
PR Checks / Build Backend (no push) (pull_request) Successful in 11s
PR Checks / Build Frontend (no push) (pull_request) Successful in 45s
PR Checks / Build Pipeline (no push) (pull_request) Successful in 9s
PR Checks / AI Code Review (Claude) (pull_request) Successful in 1m27s
GITEA_TOKEN worked (the review comment posted with it) but GITHUB_TOKEN is
the documented name in Gitea Actions; use it to keep the reviewer and the
docs aligned.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PqGhF93UrpDNvXBLMjJENL
2026-07-03 13:53:57 +01:00
TudorandClaude Fable 5 c62ba0ca25 fix(ci): post AI review comments with the run-scoped Gitea token
PR Checks / Frontend Typecheck + Tests (pull_request) Successful in 9m37s
PR Checks / Backend Smoke (pull_request) Successful in 5s
PR Checks / Build Backend (no push) (pull_request) Successful in 10s
PR Checks / Build Frontend (no push) (pull_request) Successful in 45s
PR Checks / Build Pipeline (no push) (pull_request) Successful in 10s
PR Checks / AI Code Review (Claude) (pull_request) Failing after 1m56s
REGISTRY_TOKEN lacks issue-write scope (403 on comment post). Gitea Actions
auto-provides a repo-scoped per-run token as secrets.GITEA_TOKEN — no
user-managed secret needed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PqGhF93UrpDNvXBLMjJENL
2026-07-03 13:39:59 +01:00
TudorandClaude Fable 5 b5a63e82d4 fix(analytics): restrict Umami to production hostnames
PR Checks / Frontend Typecheck + Tests (pull_request) Successful in 9m37s
PR Checks / Backend Smoke (pull_request) Successful in 4s
PR Checks / Build Backend (no push) (pull_request) Successful in 11s
PR Checks / Build Frontend (no push) (pull_request) Successful in 48s
PR Checks / Build Pipeline (no push) (pull_request) Successful in 10s
PR Checks / AI Code Review (Claude) (pull_request) Failing after 36s
The same frontend image runs on staging and prod (build once, promote), so a
build-time env var can't tell them apart. Umami's data-domains attribute
scopes the tracker client-side: events only fire when location.hostname is a
production domain, so staging traffic and the E2E journeys never pollute the
dashboards.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PqGhF93UrpDNvXBLMjJENL
2026-07-03 13:24:14 +01:00
tudor f5de745a8b Merge pull request 'fix(ci): AI review via Claude Code CLI; reuse REGISTRY_TOKEN' (#3) from fix/ai-review-claude-code into main
Deploy (staging -> E2E gate -> production) / Build Backend (FastAPI) (push) Successful in 18s
Deploy (staging -> E2E gate -> production) / Build Frontend (Next.js) (push) Successful in 46s
Deploy (staging -> E2E gate -> production) / Build Pipeline (Meltano + dbt + Airflow) (push) Successful in 12s
Deploy (staging -> E2E gate -> production) / Deploy to Staging (push) Failing after 1s
Deploy (staging -> E2E gate -> production) / E2E Journeys against Staging (push) Has been skipped
Deploy (staging -> E2E gate -> production) / Promote to Production (push) Has been skipped
2026-07-03 08:50:26 +00:00
TudorandClaude Fable 5 d0895c71df fix(ci): AI review via Claude Code CLI; reuse REGISTRY_TOKEN for PR comments
PR Checks / Frontend Typecheck + Tests (pull_request) Successful in 9m39s
PR Checks / Backend Smoke (pull_request) Successful in 5s
PR Checks / Build Backend (no push) (pull_request) Successful in 16s
PR Checks / Build Frontend (no push) (pull_request) Successful in 42s
PR Checks / Build Pipeline (no push) (pull_request) Successful in 9s
PR Checks / AI Code Review (Claude) (pull_request) Failing after 27s
- ai_review.py now pipes the diff through headless Claude Code (claude -p,
  --output-format json) authenticated with CLAUDE_CODE_OAUTH_TOKEN from
  'claude setup-token' — subscription auth, no Anthropic API billing
- stdlib-only script (urllib instead of requests/anthropic)
- PR comments posted with the existing REGISTRY_TOKEN secret; the separate
  GITEA_TOKEN secret is no longer needed

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PqGhF93UrpDNvXBLMjJENL
2026-07-03 08:28:20 +01:00
tudor df0bf1c4d6 Merge pull request 'feat(sdlc): staging environment + automated staging→prod pipeline' (#2) from feat/sdlc-staging-pipeline into main
Deploy (staging -> E2E gate -> production) / Build Backend (FastAPI) (push) Successful in 19s
Deploy (staging -> E2E gate -> production) / Build Frontend (Next.js) (push) Successful in 46s
Deploy (staging -> E2E gate -> production) / Build Pipeline (Meltano + dbt + Airflow) (push) Successful in 12s
Deploy (staging -> E2E gate -> production) / Deploy to Staging (push) Successful in 1s
Deploy (staging -> E2E gate -> production) / E2E Journeys against Staging (push) Failing after 1m10s
Deploy (staging -> E2E gate -> production) / Promote to Production (push) Has been skipped
2026-07-03 06:17:40 +00:00
TudorandClaude Fable 5 4a52735356 feat(sdlc): staging environment + automated staging→prod pipeline
PR Checks / Frontend Typecheck + Tests (pull_request) Successful in 10m0s
PR Checks / Backend Smoke (pull_request) Successful in 48s
PR Checks / Build Backend (no push) (pull_request) Successful in 17s
PR Checks / Build Frontend (no push) (pull_request) Successful in 41s
PR Checks / Build Pipeline (no push) (pull_request) Successful in 10s
PR Checks / AI Code Review (Claude) (pull_request) Successful in 24s
- pr-checks.yml: PR gate — frontend typecheck+jest, backend import smoke,
  image builds (no push), Claude AI review posted as PR comment (severe
  findings block merge)
- deploy.yml (replaces build-and-push.yml): merge to main builds+pushes
  images tagged sha-<sha>/staging, deploys the staging Portainer stack via
  webhook, runs Playwright E2E journeys against staging, then retags the
  verified images :prod (previous kept as :prod-previous) and deploys prod
- docker-compose.portainer.staging.yml: second Portainer stack — :staging
  images, sc_staging_* names, own macvlan IPs, Airflow on 8081; data
  bootstrapped from source via the staging Airflow DAGs
- prod compose now pins :prod instead of :latest (only the promotion step
  moves it; :latest is no longer published)
- e2e/: 6 Playwright journeys (search, postcode, detail, compare, rankings)
  driven by BASE_URL — the promotion gate
- scripts/ci/ai_review.py: Claude review with structured JSON findings
- docs/DEPLOY.md: full SDLC doc incl. one-time setup checklist and rollback
- replaced removed 'next lint' with tsc typecheck; fixed stale jest tests
  (slug URLs, N/A formatting, stable trend, fake-timer setup)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PqGhF93UrpDNvXBLMjJENL
2026-07-03 06:50:40 +01:00
27 changed files with 1424 additions and 109 deletions
@@ -1,12 +1,9 @@
name: Build and Push Docker Images name: Deploy (staging -> E2E gate -> production)
on: on:
push: push:
branches: branches:
- main - main
pull_request:
branches:
- main
env: env:
REGISTRY: privaterepo.sitaru.org REGISTRY: privaterepo.sitaru.org
@@ -45,17 +42,15 @@ jobs:
with: with:
images: ${{ env.REGISTRY }}/${{ env.BACKEND_IMAGE_NAME }} images: ${{ env.REGISTRY }}/${{ env.BACKEND_IMAGE_NAME }}
tags: | tags: |
type=ref,event=branch type=sha
type=ref,event=pr type=raw,value=staging
type=sha,prefix=backend-
type=raw,value=latest,enable=${{ gitea.ref == 'refs/heads/main' }}
- name: Build and push Backend Docker image - name: Build and push Backend Docker image
uses: docker/build-push-action@v5 uses: docker/build-push-action@v5
with: with:
context: . context: .
file: ./Dockerfile file: ./Dockerfile
push: ${{ gitea.event_name != 'pull_request' }} push: true
tags: ${{ steps.meta-backend.outputs.tags }} tags: ${{ steps.meta-backend.outputs.tags }}
labels: ${{ steps.meta-backend.outputs.labels }} labels: ${{ steps.meta-backend.outputs.labels }}
cache-from: type=registry,ref=${{ env.REGISTRY }}/${{ env.BACKEND_IMAGE_NAME }}:buildcache cache-from: type=registry,ref=${{ env.REGISTRY }}/${{ env.BACKEND_IMAGE_NAME }}:buildcache
@@ -91,24 +86,20 @@ jobs:
with: with:
images: ${{ env.REGISTRY }}/${{ env.FRONTEND_IMAGE_NAME }} images: ${{ env.REGISTRY }}/${{ env.FRONTEND_IMAGE_NAME }}
tags: | tags: |
type=ref,event=branch type=sha
type=ref,event=pr type=raw,value=staging
type=sha,prefix=frontend-
type=raw,value=latest,enable=${{ gitea.ref == 'refs/heads/main' }}
- name: Build and push Frontend Docker image - name: Build and push Frontend Docker image
uses: docker/build-push-action@v5 uses: docker/build-push-action@v5
with: with:
context: ./nextjs-app context: ./nextjs-app
file: ./nextjs-app/Dockerfile file: ./nextjs-app/Dockerfile
push: ${{ gitea.event_name != 'pull_request' }} push: true
tags: ${{ steps.meta-frontend.outputs.tags }} tags: ${{ steps.meta-frontend.outputs.tags }}
labels: ${{ steps.meta-frontend.outputs.labels }} labels: ${{ steps.meta-frontend.outputs.labels }}
build-args: | build-args: |
FASTAPI_URL=http://backend:80/api FASTAPI_URL=http://backend:80/api
# Cache disabled due to registry size limits # Cache disabled due to registry size limits
# cache-from: type=registry,ref=${{ env.REGISTRY }}/${{ env.FRONTEND_IMAGE_NAME }}:buildcache
# cache-to: type=registry,ref=${{ env.REGISTRY }}/${{ env.FRONTEND_IMAGE_NAME }}:buildcache,mode=max
build-pipeline: build-pipeline:
name: Build Pipeline (Meltano + dbt + Airflow) name: Build Pipeline (Meltano + dbt + Airflow)
@@ -140,28 +131,110 @@ jobs:
with: with:
images: ${{ env.REGISTRY }}/${{ env.PIPELINE_IMAGE_NAME }} images: ${{ env.REGISTRY }}/${{ env.PIPELINE_IMAGE_NAME }}
tags: | tags: |
type=ref,event=branch type=sha
type=ref,event=pr type=raw,value=staging
type=sha,prefix=pipeline-
type=raw,value=latest,enable=${{ gitea.ref == 'refs/heads/main' }}
- name: Build and push Pipeline Docker image - name: Build and push Pipeline Docker image
uses: docker/build-push-action@v5 uses: docker/build-push-action@v5
with: with:
context: ./pipeline context: ./pipeline
file: ./pipeline/Dockerfile file: ./pipeline/Dockerfile
push: ${{ gitea.event_name != 'pull_request' }} push: true
tags: ${{ steps.meta-pipeline.outputs.tags }} tags: ${{ steps.meta-pipeline.outputs.tags }}
labels: ${{ steps.meta-pipeline.outputs.labels }} labels: ${{ steps.meta-pipeline.outputs.labels }}
cache-from: type=registry,ref=${{ env.REGISTRY }}/${{ env.PIPELINE_IMAGE_NAME }}:buildcache cache-from: type=registry,ref=${{ env.REGISTRY }}/${{ env.PIPELINE_IMAGE_NAME }}:buildcache
cache-to: type=registry,ref=${{ env.REGISTRY }}/${{ env.PIPELINE_IMAGE_NAME }}:buildcache,mode=max cache-to: type=registry,ref=${{ env.REGISTRY }}/${{ env.PIPELINE_IMAGE_NAME }}:buildcache,mode=max
trigger-deployment: deploy-staging:
name: Trigger Portainer Update name: Deploy to Staging
runs-on: ubuntu-latest runs-on: ubuntu-latest
needs: [build-backend, build-frontend, build-pipeline] needs: [build-backend, build-frontend, build-pipeline]
if: gitea.event_name != 'pull_request'
steps: steps:
- name: Trigger Portainer stack update - name: Trigger staging stack update
run: curl -fsSk -X POST "${{ secrets.PORTAINER_STAGING_WEBHOOK }}"
- name: Wait for staging to become healthy
run: | run: |
curl -X POST -k "https://10.0.1.224:9443/api/stacks/webhooks/863fc57c-bf24-4c63-9001-bdf9912fba73" echo "Polling ${STAGING_BASE_URL} for up to 5 minutes..."
for i in $(seq 1 60); do
if curl -fsS -o /dev/null --max-time 10 "${STAGING_BASE_URL}/"; then
echo "Staging is up (attempt $i)"
exit 0
fi
sleep 5
done
echo "Staging did not become healthy in time" >&2
exit 1
env:
STAGING_BASE_URL: ${{ secrets.STAGING_BASE_URL }}
e2e-staging:
name: E2E Journeys against Staging
runs-on: ubuntu-latest
needs: [deploy-staging]
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: 22
- name: Install Playwright
working-directory: e2e
run: |
npm ci
npx playwright install --with-deps chromium
- name: Run E2E journeys
working-directory: e2e
run: npx playwright test
env:
BASE_URL: ${{ secrets.STAGING_BASE_URL }}
promote-prod:
name: Promote to Production
runs-on: ubuntu-latest
needs: [e2e-staging]
steps:
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Log in to Gitea Container Registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ gitea.actor }}
password: ${{ secrets.REGISTRY_TOKEN }}
- name: Retag verified images as prod
run: |
SHORT_SHA="sha-$(echo "${{ gitea.sha }}" | cut -c1-7)"
for IMAGE in \
"${REGISTRY}/${BACKEND_IMAGE_NAME}" \
"${REGISTRY}/${FRONTEND_IMAGE_NAME}" \
"${REGISTRY}/${PIPELINE_IMAGE_NAME}"; do
# Keep a rollback pointer before moving :prod
docker buildx imagetools create -t "${IMAGE}:prod-previous" "${IMAGE}:prod" || true
docker buildx imagetools create -t "${IMAGE}:prod" "${IMAGE}:${SHORT_SHA}"
echo "Promoted ${IMAGE}:${SHORT_SHA} -> :prod"
done
- name: Trigger production stack update
run: curl -fsSk -X POST "${{ secrets.PORTAINER_PROD_WEBHOOK }}"
- name: Wait for production to become healthy
run: |
echo "Polling ${PROD_BASE_URL} for up to 5 minutes..."
for i in $(seq 1 60); do
if curl -fsS -o /dev/null --max-time 10 "${PROD_BASE_URL}/"; then
echo "Production is up (attempt $i)"
exit 0
fi
sleep 5
done
echo "Production did not become healthy in time" >&2
exit 1
env:
PROD_BASE_URL: ${{ secrets.PROD_BASE_URL }}
+188
View File
@@ -0,0 +1,188 @@
name: PR Checks
on:
pull_request:
branches:
- main
env:
REGISTRY: privaterepo.sitaru.org
BACKEND_IMAGE_NAME: ${{ gitea.repository }}-backend
FRONTEND_IMAGE_NAME: ${{ gitea.repository }}-frontend
PIPELINE_IMAGE_NAME: ${{ gitea.repository }}-pipeline
jobs:
frontend-checks:
name: Frontend Typecheck + Tests
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: 22
cache: npm
cache-dependency-path: nextjs-app/package-lock.json
- name: Install dependencies
working-directory: nextjs-app
run: npm ci
- name: Typecheck
working-directory: nextjs-app
run: npm run typecheck
- name: Unit tests
working-directory: nextjs-app
run: npm test
backend-checks:
name: Backend Smoke
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install dependencies
run: pip install -r requirements.txt
- name: Import smoke test
run: python -c "from backend.app import app; print('backend imports OK')"
build-backend:
name: Build Backend (no push)
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
buildkitd-config-inline: |
[registry."docker.io"]
mirrors = ["10.0.1.224:6000"]
[registry."10.0.1.224:6000"]
http = true
insecure = true
- name: Log in to Gitea Container Registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ gitea.actor }}
password: ${{ secrets.REGISTRY_TOKEN }}
- name: Build Backend Docker image
uses: docker/build-push-action@v5
with:
context: .
file: ./Dockerfile
push: false
cache-from: type=registry,ref=${{ env.REGISTRY }}/${{ env.BACKEND_IMAGE_NAME }}:buildcache
build-frontend:
name: Build Frontend (no push)
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
buildkitd-config-inline: |
[registry."docker.io"]
mirrors = ["10.0.1.224:6000"]
[registry."10.0.1.224:6000"]
http = true
insecure = true
- name: Log in to Gitea Container Registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ gitea.actor }}
password: ${{ secrets.REGISTRY_TOKEN }}
- name: Build Frontend Docker image
uses: docker/build-push-action@v5
with:
context: ./nextjs-app
file: ./nextjs-app/Dockerfile
push: false
build-args: |
FASTAPI_URL=http://backend:80/api
build-pipeline:
name: Build Pipeline (no push)
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
buildkitd-config-inline: |
[registry."docker.io"]
mirrors = ["10.0.1.224:6000"]
[registry."10.0.1.224:6000"]
http = true
insecure = true
- name: Log in to Gitea Container Registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ gitea.actor }}
password: ${{ secrets.REGISTRY_TOKEN }}
- name: Build Pipeline Docker image
uses: docker/build-push-action@v5
with:
context: ./pipeline
file: ./pipeline/Dockerfile
push: false
cache-from: type=registry,ref=${{ env.REGISTRY }}/${{ env.PIPELINE_IMAGE_NAME }}:buildcache
ai-review:
name: AI Code Review (Claude)
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: 22
- name: Install Claude Code
run: npm install -g @anthropic-ai/claude-code
- name: Review PR diff with Claude Code
env:
CLAUDE_CODE_OAUTH_TOKEN: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
# Auto-provided per-run token from Gitea Actions (repo-scoped).
# GITHUB_TOKEN is the documented name; GITEA_TOKEN is its alias.
GITEA_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITEA_SERVER_URL: ${{ gitea.server_url }}
GITEA_REPOSITORY: ${{ gitea.repository }}
PR_NUMBER: ${{ gitea.event.pull_request.number }}
BASE_REF: ${{ gitea.event.pull_request.base.ref }}
run: python scripts/ci/ai_review.py
+4 -1
View File
@@ -834,7 +834,10 @@ async def get_rankings(
request: Request, request: Request,
metric: str = Query("rwm_expected_pct", description="Metric to rank by", max_length=50), metric: str = Query("rwm_expected_pct", description="Metric to rank by", max_length=50),
year: Optional[int] = Query( year: Optional[int] = Query(
None, description="Specific year (defaults to most recent)", ge=2000, le=2100 None,
description="Academic year code, e.g. 201819 (defaults to most recent)",
ge=2000,
le=210100,
), ),
limit: int = Query(20, ge=1, le=100, description="Number of schools to return"), limit: int = Query(20, ge=1, le=100, description="Number of schools to return"),
local_authority: Optional[str] = Query( local_authority: Optional[str] = Query(
+14
View File
@@ -105,8 +105,22 @@ This starts:
- `GET /api/metrics` - Metric definitions (single source of truth) - `GET /api/metrics` - Metric definitions (single source of truth)
- `GET /api/data-info` - Database stats - `GET /api/data-info` - Database stats
## SDLC
Full details in `docs/DEPLOY.md`. The short version:
- **Never push to `main` directly.** Work on a feature branch and open a PR;
branch protection requires the PR checks (typecheck, tests, builds, AI review)
to pass before merge.
- Merging to `main` deploys automatically: images are built once, deployed to
the **staging** Portainer stack, verified by the Playwright journeys in
`e2e/`, and only then retagged `:prod` and rolled out to production.
- If you change user-facing behaviour, update or extend the `e2e/` journey
tests in the same PR — they are the promotion gate.
## Recent Changes ## Recent Changes
- Added staging environment + automated staging→prod pipeline (Gitea Actions)
- Migrated from CSV file storage to PostgreSQL database - Migrated from CSV file storage to PostgreSQL database
- Added location-based search using postcode geocoding - Added location-based search using postcode geocoding
- Added local authority filter to rankings - Added local authority filter to rankings
+214
View File
@@ -0,0 +1,214 @@
# Portainer Stack Definition for School Compare — STAGING
#
# Deploy this as a *separate* Portainer stack (e.g. "schoolcompare-staging")
# alongside the production stack. Differences from production:
# - images pinned to :staging (pushed by every merge to main, before the E2E gate)
# - sc_staging_* container names
# - own macvlan IPs (STAGING_DB_IP / STAGING_FRONTEND_IP env vars)
# - Airflow UI published on 8081 (prod uses 8080)
# - volumes are isolated automatically: Portainer prefixes volume names with
# the stack name, so this stack gets its own postgres/typesense/airflow data
#
# Portainer environment variables (set in Portainer UI -> Stack -> Environment):
# DB_USERNAME — PostgreSQL username
# DB_PASSWORD — PostgreSQL password
# DB_DATABASE_NAME — PostgreSQL database name
# ADMIN_API_KEY — Backend admin API key
# TYPESENSE_API_KEY — Typesense admin API key
# TYPESENSE_SEARCH_KEY — Typesense search-only key (exposed to frontend)
# AIRFLOW_ADMIN_USER — Airflow admin username (password auto-generated, see api-server logs)
# STAGING_DB_IP — macvlan IP for staging Postgres (default 10.0.1.190)
# STAGING_FRONTEND_IP — macvlan IP for staging frontend (default 10.0.1.151)
services:
# ── PostgreSQL ────────────────────────────────────────────────────────
sc_database:
container_name: sc_staging_postgres
image: postgis/postgis:18-3.6-alpine
environment:
POSTGRES_PASSWORD: ${DB_PASSWORD}
POSTGRES_USER: ${DB_USERNAME}
POSTGRES_DB: ${DB_DATABASE_NAME}
volumes:
- postgres_data:/var/lib/postgresql
shm_size: 128mb
networks:
backend: {}
macvlan:
ipv4_address: ${STAGING_DB_IP:-10.0.1.190}
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres"]
interval: 10s
timeout: 5s
retries: 5
start_period: 10s
restart: unless-stopped
# ── FastAPI Backend ───────────────────────────────────────────────────
backend:
image: privaterepo.sitaru.org/tudor/school_compare-backend:staging
container_name: sc_staging_backend
environment:
DATABASE_URL: postgresql://${DB_USERNAME}:${DB_PASSWORD}@sc_database:5432/${DB_DATABASE_NAME}
PYTHONUNBUFFERED: 1
ADMIN_API_KEY: ${ADMIN_API_KEY:-changeme}
TYPESENSE_URL: http://typesense:8108
TYPESENSE_API_KEY: ${TYPESENSE_API_KEY:-changeme}
depends_on:
sc_database:
condition: service_healthy
networks:
- backend
restart: unless-stopped
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:80/api/data-info"]
interval: 30s
timeout: 10s
retries: 3
start_period: 30s
# ── Next.js Frontend ──────────────────────────────────────────────────
frontend:
image: privaterepo.sitaru.org/tudor/school_compare-frontend:staging
container_name: sc_staging_nextjs
environment:
- NODE_ENV=production
- NEXT_PUBLIC_API_URL=http://localhost:8000/api
- FASTAPI_URL=http://backend:80/api
- TYPESENSE_URL=http://typesense:8108
- TYPESENSE_API_KEY=${TYPESENSE_SEARCH_KEY:-changeme}
depends_on:
backend:
condition: service_healthy
networks:
backend: {}
macvlan:
ipv4_address: ${STAGING_FRONTEND_IP:-10.0.1.151}
restart: unless-stopped
healthcheck:
test: ["CMD", "node", "-e", "require('http').get('http://localhost:3000/', (r) => {process.exit(r.statusCode === 200 ? 0 : 1)})"]
interval: 30s
timeout: 10s
retries: 3
start_period: 40s
# ── Typesense Search Engine ───────────────────────────────────────────
typesense:
image: typesense/typesense:30.1
container_name: sc_staging_typesense
environment:
TYPESENSE_API_KEY: ${TYPESENSE_API_KEY:-changeme}
TYPESENSE_DATA_DIR: /data
volumes:
- typesense_data:/data
networks:
- backend
restart: unless-stopped
healthcheck:
test: ["CMD-SHELL", "cat < /dev/tcp/localhost/8108"]
interval: 15s
timeout: 5s
retries: 5
start_period: 10s
# ── Airflow API Server + UI (staging: http://<host>:8081) ─────────────
airflow-api-server:
image: privaterepo.sitaru.org/tudor/school_compare-pipeline:staging
container_name: sc_staging_airflow_api
command: airflow api-server --port 8080
ports:
- "8081:8080"
environment:
AIRFLOW__CORE__EXECUTOR: LocalExecutor
AIRFLOW__DATABASE__SQL_ALCHEMY_CONN: postgresql+psycopg2://${DB_USERNAME}:${DB_PASSWORD}@sc_database:5432/${DB_DATABASE_NAME}
AIRFLOW__CORE__DAGS_FOLDER: /opt/pipeline/dags
AIRFLOW__CORE__LOAD_EXAMPLES: "false"
AIRFLOW__CORE__EXECUTION_API_SERVER_URL: http://airflow-api-server:8080/execution/
AIRFLOW__API_AUTH__JWT_SECRET: "school-compare-staging-airflow-jwt-secret-key-long-enough-for-sha512"
AIRFLOW__API_AUTH__JWT_ISSUER: airflow
AIRFLOW__CORE__SIMPLE_AUTH_MANAGER_USERS: "${AIRFLOW_ADMIN_USER:-admin}:admin"
AIRFLOW__LOGGING__BASE_LOG_FOLDER: /opt/airflow/logs
PG_HOST: sc_database
PG_PORT: "5432"
PG_USER: ${DB_USERNAME}
PG_PASSWORD: ${DB_PASSWORD}
PG_DATABASE: ${DB_DATABASE_NAME}
TYPESENSE_URL: http://typesense:8108
TYPESENSE_API_KEY: ${TYPESENSE_API_KEY:-changeme}
volumes:
- airflow_logs:/opt/airflow/logs
depends_on:
sc_database:
condition: service_healthy
networks:
- backend
restart: unless-stopped
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8080/api/v2/monitor/health"]
interval: 30s
timeout: 10s
retries: 5
start_period: 60s
# ── Airflow Scheduler ──────────────────────────────────────────────
airflow-scheduler:
image: privaterepo.sitaru.org/tudor/school_compare-pipeline:staging
container_name: sc_staging_airflow_scheduler
command: airflow scheduler
environment:
AIRFLOW__CORE__EXECUTOR: LocalExecutor
AIRFLOW__DATABASE__SQL_ALCHEMY_CONN: postgresql+psycopg2://${DB_USERNAME}:${DB_PASSWORD}@sc_database:5432/${DB_DATABASE_NAME}
AIRFLOW__CORE__DAGS_FOLDER: /opt/pipeline/dags
AIRFLOW__CORE__LOAD_EXAMPLES: "false"
AIRFLOW__CORE__EXECUTION_API_SERVER_URL: http://airflow-api-server:8080/execution/
AIRFLOW__API_AUTH__JWT_SECRET: "school-compare-staging-airflow-jwt-secret-key-long-enough-for-sha512"
AIRFLOW__API_AUTH__JWT_ISSUER: airflow
AIRFLOW__LOGGING__BASE_LOG_FOLDER: /opt/airflow/logs
PG_HOST: sc_database
PG_PORT: "5432"
PG_USER: ${DB_USERNAME}
PG_PASSWORD: ${DB_PASSWORD}
PG_DATABASE: ${DB_DATABASE_NAME}
TYPESENSE_URL: http://typesense:8108
TYPESENSE_API_KEY: ${TYPESENSE_API_KEY:-changeme}
volumes:
- airflow_logs:/opt/airflow/logs
depends_on:
sc_database:
condition: service_healthy
networks:
- backend
restart: unless-stopped
# ── Airflow DB Init (one-shot) ───────────────────────────────────────
airflow-init:
image: privaterepo.sitaru.org/tudor/school_compare-pipeline:staging
container_name: sc_staging_airflow_init
command: bash -c "airflow db migrate && airflow dags reserialize"
environment:
AIRFLOW__CORE__EXECUTOR: LocalExecutor
AIRFLOW__DATABASE__SQL_ALCHEMY_CONN: postgresql+psycopg2://${DB_USERNAME}:${DB_PASSWORD}@sc_database:5432/${DB_DATABASE_NAME}
AIRFLOW__CORE__DAGS_FOLDER: /opt/pipeline/dags
AIRFLOW__CORE__LOAD_EXAMPLES: "false"
AIRFLOW__CORE__EXECUTION_API_SERVER_URL: http://airflow-api-server:8080/execution/
AIRFLOW__API_AUTH__JWT_SECRET: "school-compare-staging-airflow-jwt-secret-key-long-enough-for-sha512"
AIRFLOW__API_AUTH__JWT_ISSUER: airflow
depends_on:
sc_database:
condition: service_healthy
networks:
- backend
restart: "no"
networks:
backend:
driver: bridge
macvlan:
external:
name: macvlan
volumes:
postgres_data:
typesense_data:
airflow_logs:
+5 -5
View File
@@ -36,7 +36,7 @@ services:
# ── FastAPI Backend ─────────────────────────────────────────────────── # ── FastAPI Backend ───────────────────────────────────────────────────
backend: backend:
image: privaterepo.sitaru.org/tudor/school_compare-backend:latest image: privaterepo.sitaru.org/tudor/school_compare-backend:prod
container_name: schoolcompare_backend container_name: schoolcompare_backend
environment: environment:
DATABASE_URL: postgresql://${DB_USERNAME}:${DB_PASSWORD}@sc_database:5432/${DB_DATABASE_NAME} DATABASE_URL: postgresql://${DB_USERNAME}:${DB_PASSWORD}@sc_database:5432/${DB_DATABASE_NAME}
@@ -59,7 +59,7 @@ services:
# ── Next.js Frontend ────────────────────────────────────────────────── # ── Next.js Frontend ──────────────────────────────────────────────────
frontend: frontend:
image: privaterepo.sitaru.org/tudor/school_compare-frontend:latest image: privaterepo.sitaru.org/tudor/school_compare-frontend:prod
container_name: schoolcompare_nextjs container_name: schoolcompare_nextjs
environment: environment:
- NODE_ENV=production - NODE_ENV=production
@@ -103,7 +103,7 @@ services:
# ── Airflow API Server + UI ─────────────────────────────────────────── # ── Airflow API Server + UI ───────────────────────────────────────────
airflow-api-server: airflow-api-server:
image: privaterepo.sitaru.org/tudor/school_compare-pipeline:latest image: privaterepo.sitaru.org/tudor/school_compare-pipeline:prod
container_name: schoolcompare_airflow_api container_name: schoolcompare_airflow_api
command: airflow api-server --port 8080 command: airflow api-server --port 8080
ports: ports:
@@ -142,7 +142,7 @@ services:
# ── Airflow Scheduler ────────────────────────────────────────────── # ── Airflow Scheduler ──────────────────────────────────────────────
airflow-scheduler: airflow-scheduler:
image: privaterepo.sitaru.org/tudor/school_compare-pipeline:latest image: privaterepo.sitaru.org/tudor/school_compare-pipeline:prod
container_name: schoolcompare_airflow_scheduler container_name: schoolcompare_airflow_scheduler
command: airflow scheduler command: airflow scheduler
environment: environment:
@@ -172,7 +172,7 @@ services:
# ── Airflow DB Init (one-shot) ─────────────────────────────────────── # ── Airflow DB Init (one-shot) ───────────────────────────────────────
airflow-init: airflow-init:
image: privaterepo.sitaru.org/tudor/school_compare-pipeline:latest image: privaterepo.sitaru.org/tudor/school_compare-pipeline:prod
container_name: schoolcompare_airflow_init container_name: schoolcompare_airflow_init
command: bash -c "airflow db migrate && airflow dags delete school_data_daily -y 2>/dev/null; airflow dags delete school_data_monthly_ofsted -y 2>/dev/null; airflow dags delete school_data_annual_ees -y 2>/dev/null; airflow dags reserialize" command: bash -c "airflow db migrate && airflow dags delete school_data_daily -y 2>/dev/null; airflow dags delete school_data_monthly_ofsted -y 2>/dev/null; airflow dags delete school_data_annual_ees -y 2>/dev/null; airflow dags reserialize"
environment: environment:
+132
View File
@@ -0,0 +1,132 @@
# SDLC & Deployment Pipeline
SchoolCompare uses a fully automated staging → production pipeline on Gitea
Actions. AI writes the code on feature branches; the pipeline verifies every
change on a staging environment before promoting the exact same images to
production. Human input is directional only: feature requests, PR review if
desired, and intervention when a gate fails.
## The flow
```
feature branch (AI-authored)
│ PR to main
PR checks (.gitea/workflows/pr-checks.yml)
typecheck + unit tests + backend smoke + image builds (no push)
+ Claude code review posted as a PR comment (severe findings fail the check)
│ merge (branch protection requires green checks)
Deploy pipeline (.gitea/workflows/deploy.yml)
1. build & push images → tags sha-<sha>, staging
2. staging Portainer webhook → wait for staging health
3. Playwright E2E journeys against staging
4. retag sha-<sha> → :prod (same bytes — build once, promote the image)
previous :prod saved as :prod-previous
5. prod Portainer webhook → wait for prod health
```
Key principle: **build once, promote the exact image**. Production pins `:prod`,
which only moves after the E2E gate passes on staging. Nothing tags `:latest`
anymore.
## Branch & PR workflow
- `main` is protected: no direct pushes, PRs require green status checks.
- All work (human or AI) happens on feature branches → PR to `main`.
- Merging to `main` **is** the release action. If staging or the E2E gate
fails, production is untouched.
## Environments
| | Production | Staging |
|---|---|---|
| Portainer stack file | `docker-compose.portainer.yml` | `docker-compose.portainer.staging.yml` |
| Image tag | `:prod` | `:staging` |
| Container prefix | `sc_` / `schoolcompare_` | `sc_staging_` |
| Frontend macvlan IP | 10.0.1.150 | `STAGING_FRONTEND_IP` (default 10.0.1.151) |
| Postgres macvlan IP | 10.0.1.189 | `STAGING_DB_IP` (default 10.0.1.190) |
| Airflow UI port | 8080 | 8081 |
| Volumes | stack-prefixed | stack-prefixed (fully isolated) |
Staging gets `:staging` images on every merge to main — even ones that later
fail the E2E gate. That's the point: staging absorbs the risk.
## Gitea repository secrets
| Secret | Purpose |
|---|---|
| `REGISTRY_TOKEN` | push images to privaterepo.sitaru.org (already set) |
| `CLAUDE_CODE_OAUTH_TOKEN` | Claude Code subscription auth for the PR review — generate with `claude setup-token` on your machine |
| `PORTAINER_STAGING_WEBHOOK` | staging stack redeploy webhook URL |
| `PORTAINER_PROD_WEBHOOK` | production stack redeploy webhook URL |
| `STAGING_BASE_URL` | e.g. `http://10.0.1.151:3000` — health poll + E2E target |
| `PROD_BASE_URL` | e.g. `http://10.0.1.150:3000` — post-promotion health poll |
## One-time setup checklist
1. **Create the staging stack** in Portainer from
`docker-compose.portainer.staging.yml` (stack name e.g.
`schoolcompare-staging`). Set the same environment variables as prod plus
`STAGING_DB_IP` / `STAGING_FRONTEND_IP` if the defaults clash.
2. **Enable webhooks** on both stacks (Portainer → Stack → Webhook) and store
the URLs as `PORTAINER_STAGING_WEBHOOK` / `PORTAINER_PROD_WEBHOOK`. Remove
the old hardcoded webhook usage (now gone from the workflows).
3. **Add the remaining secrets** listed above in Gitea → repo → Settings →
Actions → Secrets.
4. **Protect `main`** in Gitea → Settings → Branches: require PRs, require the
pr-checks status checks (frontend, backend, builds, ai-review) to pass.
5. **Bootstrap staging data via Airflow** (no prod dump — staging populates
itself from source, exercising the pipeline image end-to-end):
- Open the staging Airflow UI (`http://<host>:8081`) and trigger, in order:
`school_data_daily`, `school_data_monthly_ofsted`,
`school_data_monthly_parent_view`, then the manual-schedule
`school_data_annual_ees` and `school_data_annual_idaci`.
- First runs download from government sources (GIAS, Ofsted, EES, IDACI),
run dbt, and sync Typesense — expect the initial backfill to take a while.
- The scheduled DAGs then keep staging fresh exactly like prod.
6. **Switch the prod stack to `:prod` tags** — the repo's
`docker-compose.portainer.yml` is already updated; redeploy the prod stack
from it. Until the first pipeline run promotes an image, tag the current
images manually: `docker buildx imagetools create -t <image>:prod <image>:latest`
for each of the three images.
## Rollback
Every promotion first re-points `:prod-previous` at the outgoing `:prod`.
To roll back:
```bash
for img in backend frontend pipeline; do
docker buildx imagetools create \
-t privaterepo.sitaru.org/tudor/school_compare-$img:prod \
privaterepo.sitaru.org/tudor/school_compare-$img:prod-previous
done
curl -fsSk -X POST "$PORTAINER_PROD_WEBHOOK"
```
Or promote any older build directly: `imagetools create -t <image>:prod <image>:sha-<shortsha>`.
## E2E suite
Lives in `e2e/` (own package — CI installs it without the app's node_modules).
Journeys: home + name search, postcode search, school detail, two-school
comparison, rankings table. Run locally against any environment:
```bash
cd e2e && npm ci
BASE_URL=http://10.0.1.151:3000 npx playwright test
```
Tests assert data invariants (results exist, charts render), not exact
numbers, so scheduled data refreshes don't break the gate.
## AI code review
`scripts/ci/ai_review.py` pipes the PR diff through headless Claude Code
(`claude -p`, authenticated with the subscription OAuth token — no API
billing), posts the structured findings as a PR comment using the per-run
token Gitea Actions provides automatically (`secrets.GITEA_TOKEN` — no setup
needed), and fails the check only when a finding is rated
**severe** (would break prod, leak data, or corrupt data). Minor findings are
informational and never block a merge.
+3
View File
@@ -0,0 +1,3 @@
node_modules/
test-results/
playwright-report/
+78
View File
@@ -0,0 +1,78 @@
{
"name": "schoolcompare-e2e",
"version": "1.0.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "schoolcompare-e2e",
"version": "1.0.0",
"devDependencies": {
"@playwright/test": "^1.49.0"
}
},
"node_modules/@playwright/test": {
"version": "1.61.1",
"resolved": "https://registry.npmjs.org/@playwright/test/-/test-1.61.1.tgz",
"integrity": "sha512-8nKv6+0RJSL9FE4jYOEGXnPeM/Hg12qZpmqzZjRh3qM0Y7c3z1mrOTfFLids72RDQYVh9WpLEfR5WdpNX4fkig==",
"dev": true,
"license": "Apache-2.0",
"dependencies": {
"playwright": "1.61.1"
},
"bin": {
"playwright": "cli.js"
},
"engines": {
"node": ">=18"
}
},
"node_modules/fsevents": {
"version": "2.3.2",
"resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.2.tgz",
"integrity": "sha512-xiqMQR4xAeHTuB9uWm+fFRcIOgKBMiOBP+eXiyT7jsgVCq1bkVygt00oASowB7EdtpOHaaPgKt812P9ab+DDKA==",
"dev": true,
"hasInstallScript": true,
"license": "MIT",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": "^8.16.0 || ^10.6.0 || >=11.0.0"
}
},
"node_modules/playwright": {
"version": "1.61.1",
"resolved": "https://registry.npmjs.org/playwright/-/playwright-1.61.1.tgz",
"integrity": "sha512-DWnY5o3YbLWK4GovuAVwpqL+1VwGNdUGrRr++8j8PtQQzvAVZUIMjKQ90fY689sEJZJBbZVw1rXaOKSTitkzPQ==",
"dev": true,
"license": "Apache-2.0",
"dependencies": {
"playwright-core": "1.61.1"
},
"bin": {
"playwright": "cli.js"
},
"engines": {
"node": ">=18"
},
"optionalDependencies": {
"fsevents": "2.3.2"
}
},
"node_modules/playwright-core": {
"version": "1.61.1",
"resolved": "https://registry.npmjs.org/playwright-core/-/playwright-core-1.61.1.tgz",
"integrity": "sha512-h7Qlt6m4REp25qvIdvbDtVmD4LqVXfpRxhORv9L0jzETM05p4fuPJ3dKyuSXQxDSbXnmS79HAgi9589lGSpLkg==",
"dev": true,
"license": "Apache-2.0",
"bin": {
"playwright-core": "cli.js"
},
"engines": {
"node": ">=18"
}
}
}
}
+12
View File
@@ -0,0 +1,12 @@
{
"name": "schoolcompare-e2e",
"version": "1.0.0",
"private": true,
"description": "Journey tests run against staging as the production promotion gate",
"scripts": {
"test": "playwright test"
},
"devDependencies": {
"@playwright/test": "^1.49.0"
}
}
+13
View File
@@ -0,0 +1,13 @@
import { defineConfig } from '@playwright/test';
export default defineConfig({
testDir: './tests',
timeout: 60_000,
retries: 1,
workers: 2,
reporter: process.env.CI ? 'list' : 'html',
use: {
baseURL: process.env.BASE_URL || 'http://localhost:3000',
trace: 'retain-on-failure',
},
});
+147
View File
@@ -0,0 +1,147 @@
import { test, expect, Page } from '@playwright/test';
/**
* Journey tests for SchoolCompare, run against the staging environment as the
* gate before promotion to production. They assert stable data invariants
* (results exist, key UI renders) rather than exact numbers, so routine data
* refreshes don't break the pipeline.
*/
async function searchByName(page: Page, query: string) {
await page.goto('/');
const searchInput = page.getByPlaceholder('School name or postcode').first();
await searchInput.fill(query);
await searchInput.press('Enter');
await page.waitForURL(/search=|postcode=/);
}
function schoolLinks(page: Page) {
return page.locator('a[href^="/school/"]');
}
test('home page loads with hero search', async ({ page }) => {
await page.goto('/');
await expect(page.locator('h1').first()).toBeVisible();
await expect(page.getByPlaceholder('School name or postcode').first()).toBeVisible();
});
test('searching by name returns school results', async ({ page }) => {
await searchByName(page, 'primary');
await expect(schoolLinks(page).first()).toBeVisible({ timeout: 15_000 });
expect(await schoolLinks(page).count()).toBeGreaterThan(1);
});
test('searching by postcode returns nearby schools', async ({ page }) => {
await searchByName(page, 'B1 1BB');
await expect(schoolLinks(page).first()).toBeVisible({ timeout: 15_000 });
});
test('school detail page renders name and performance data', async ({ page }) => {
await searchByName(page, 'primary');
const firstSchool = schoolLinks(page).first();
await expect(firstSchool).toBeVisible({ timeout: 15_000 });
await firstSchool.click();
await page.waitForURL(/\/school\//);
await expect(page.locator('h1').first()).toBeVisible();
// The detail page renders at least one *visible* chart canvas. Plain
// .first() is wrong here: the admissions card stacks its year/trend views
// in one grid cell and keeps the inactive view's canvas visibility:hidden
// by design, and that canvas comes first in the DOM.
await expect(page.locator('canvas:visible').first()).toBeVisible({ timeout: 15_000 });
});
test('school hero map opens fullscreen on mobile without the Fullscreen API', async ({ page }) => {
// iOS Safari has no Element.requestFullscreen; the map must fall back to a
// CSS overlay. Simulate that by removing the API before any page script runs.
await page.setViewportSize({ width: 390, height: 844 });
await page.addInitScript(() => {
// @ts-expect-error deliberate API removal
delete Element.prototype.requestFullscreen;
});
await searchByName(page, 'primary');
const firstSchool = schoolLinks(page).first();
await expect(firstSchool).toBeVisible({ timeout: 15_000 });
await firstSchool.click();
await page.waitForURL(/\/school\//);
const openMap = page.getByRole('button', { name: 'Open full map' });
await expect(openMap).toBeVisible({ timeout: 15_000 });
await openMap.click();
const closeMap = page.getByRole('button', { name: 'Close map' });
await expect(closeMap).toBeVisible();
await closeMap.click();
await expect(openMap).toBeVisible();
});
test('comparing two schools shows both side by side', async ({ page }) => {
// Collect two school URNs from search results, then load the share URL
await searchByName(page, 'primary');
await expect(schoolLinks(page).first()).toBeVisible({ timeout: 15_000 });
const hrefs = await schoolLinks(page).evaluateAll((links) =>
links.map((l) => (l as HTMLAnchorElement).getAttribute('href') || '')
);
const urns = [...new Set(hrefs.map((h) => h.match(/\/school\/(\d+)/)?.[1]).filter(Boolean))];
expect(urns.length).toBeGreaterThanOrEqual(2);
await page.goto(`/compare?urns=${urns[0]},${urns[1]}`);
// Both schools' detail links should render in the comparison view
await expect(page.locator(`a[href*="${urns[0]}"]`).first()).toBeVisible({ timeout: 15_000 });
await expect(page.locator(`a[href*="${urns[1]}"]`).first()).toBeVisible();
});
test('compare chart on mobile shows school chips with tap-to-focus', async ({ page }) => {
await page.setViewportSize({ width: 390, height: 844 });
await searchByName(page, 'primary');
await expect(schoolLinks(page).first()).toBeVisible({ timeout: 15_000 });
const hrefs = await schoolLinks(page).evaluateAll((links) =>
links.map((l) => (l as HTMLAnchorElement).getAttribute('href') || '')
);
const urns = [...new Set(hrefs.map((h) => h.match(/\/school\/(\d+)/)?.[1]).filter(Boolean))];
expect(urns.length).toBeGreaterThanOrEqual(2);
await page.goto(`/compare?urns=${urns[0]},${urns[1]}`);
await expect(page.locator('canvas:visible').first()).toBeVisible({ timeout: 15_000 });
// The mobile chart legend renders one chip per school inside the chart card.
const chipGroup = page.getByRole('group', { name: /highlight a school/i });
const chips = chipGroup.getByRole('button');
await expect(chips).toHaveCount(2);
// Tapping a chip focuses that school's line; tapping again releases it.
await chips.first().click();
await expect(chips.first()).toHaveAttribute('aria-pressed', 'true');
await chips.first().click();
await expect(chips.first()).toHaveAttribute('aria-pressed', 'false');
});
test('rankings page loads a populated table', async ({ page }) => {
await page.goto('/rankings');
await expect(page.getByRole('heading', { name: /rankings/i }).first()).toBeVisible();
const rows = page.locator('table tbody tr');
await expect(rows.first()).toBeVisible({ timeout: 15_000 });
expect(await rows.count()).toBeGreaterThan(5);
});
test('rankings stay populated after picking a specific year', async ({ page }) => {
// Years are academic-year codes (e.g. 201819); the API must accept them
// as the `year` query param rather than rejecting with a 422.
await page.goto('/rankings');
const yearSelect = page.locator('#year-select');
await expect(yearSelect).toBeVisible({ timeout: 15_000 });
// Pick the last option — the most recent explicit year. The default view
// already proved this year has rows, so an empty table after selecting it
// can only mean the year param was rejected. (The oldest year is no good
// here: staging doesn't always carry the full data history.)
const yearValue = await yearSelect.locator('option').last().getAttribute('value');
expect(yearValue).toBeTruthy();
await yearSelect.selectOption(yearValue!);
await page.waitForURL(/year=/);
const rows = page.locator('table tbody tr');
await expect(rows.first()).toBeVisible({ timeout: 15_000 });
expect(await rows.count()).toBeGreaterThan(5);
});
@@ -34,24 +34,24 @@ describe('SchoolCard', () => {
render(<SchoolCard school={mockSchool} />); render(<SchoolCard school={mockSchool} />);
const link = screen.getByRole('link', { name: /test primary school/i }); const link = screen.getByRole('link', { name: /test primary school/i });
expect(link).toHaveAttribute('href', '/school/100001'); expect(link).toHaveAttribute('href', '/school/100001-test-primary-school');
}); });
it('calls onAddToCompare when Add to Compare button is clicked', () => { it('calls onAddToCompare when the Compare button is clicked', () => {
const mockAddToCompare = jest.fn(); const mockAddToCompare = jest.fn();
render(<SchoolCard school={mockSchool} onAddToCompare={mockAddToCompare} />); render(<SchoolCard school={mockSchool} onAddToCompare={mockAddToCompare} />);
const addButton = screen.getByText('Add to Compare'); const addButton = screen.getByText('+ Compare');
fireEvent.click(addButton); fireEvent.click(addButton);
expect(mockAddToCompare).toHaveBeenCalledWith(mockSchool); expect(mockAddToCompare).toHaveBeenCalledWith(mockSchool);
expect(mockAddToCompare).toHaveBeenCalledTimes(1); expect(mockAddToCompare).toHaveBeenCalledTimes(1);
}); });
it('does not render Add to Compare button when handler not provided', () => { it('does not render the Compare button when handler not provided', () => {
render(<SchoolCard school={mockSchool} />); render(<SchoolCard school={mockSchool} />);
expect(screen.queryByText('Add to Compare')).not.toBeInTheDocument(); expect(screen.queryByText('+ Compare')).not.toBeInTheDocument();
}); });
it('displays trend indicator for positive change', () => { it('displays trend indicator for positive change', () => {
+66 -9
View File
@@ -9,6 +9,8 @@ import {
isValidPostcode, isValidPostcode,
debounce, debounce,
buildOfstedListBadge, buildOfstedListBadge,
metricKind,
computeYBounds,
} from '@/lib/utils'; } from '@/lib/utils';
describe('formatPercentage', () => { describe('formatPercentage', () => {
@@ -19,7 +21,7 @@ describe('formatPercentage', () => {
}); });
it('handles null values', () => { it('handles null values', () => {
expect(formatPercentage(null)).toBe('-'); expect(formatPercentage(null)).toBe('N/A');
}); });
}); });
@@ -31,7 +33,7 @@ describe('formatProgress', () => {
}); });
it('handles null values', () => { it('handles null values', () => {
expect(formatProgress(null)).toBe('-'); expect(formatProgress(null)).toBe('N/A');
}); });
}); });
@@ -44,16 +46,16 @@ describe('calculateTrend', () => {
expect(calculateTrend(70, 75)).toBe('down'); expect(calculateTrend(70, 75)).toBe('down');
}); });
it('calculates same trend', () => { it('calculates stable trend', () => {
expect(calculateTrend(75, 75)).toBe('same'); expect(calculateTrend(75, 75)).toBe('stable');
}); });
it('handles null previous value', () => { it('handles null previous value', () => {
expect(calculateTrend(75, null)).toBe('same'); expect(calculateTrend(75, null)).toBe('stable');
}); });
it('handles null current value', () => { it('handles null current value', () => {
expect(calculateTrend(null, 75)).toBe('same'); expect(calculateTrend(null, 75)).toBe('stable');
}); });
}); });
@@ -72,7 +74,13 @@ describe('isValidPostcode', () => {
}); });
describe('debounce', () => { describe('debounce', () => {
jest.useFakeTimers(); beforeEach(() => {
jest.useFakeTimers();
});
afterEach(() => {
jest.useRealTimers();
});
it('delays function execution', () => { it('delays function execution', () => {
const mockFn = jest.fn(); const mockFn = jest.fn();
@@ -100,8 +108,6 @@ describe('debounce', () => {
expect(mockFn).toHaveBeenCalledWith('third'); expect(mockFn).toHaveBeenCalledWith('third');
expect(mockFn).toHaveBeenCalledTimes(1); expect(mockFn).toHaveBeenCalledTimes(1);
}); });
jest.useRealTimers();
}); });
describe('buildOfstedListBadge', () => { describe('buildOfstedListBadge', () => {
@@ -155,3 +161,54 @@ describe('buildOfstedListBadge', () => {
expect(badge.cssClass).toBe('ofstedPending'); expect(badge.cssClass).toBe('ofstedPending');
}); });
}); });
describe('metricKind', () => {
it('classifies metrics by key', () => {
expect(metricKind('rwm_expected_pct')).toBe('percentage');
expect(metricKind('absence_rate')).toBe('percentage');
expect(metricKind('reading_progress')).toBe('progress');
expect(metricKind('progress_8_score')).toBe('progress');
expect(metricKind('attainment_8_score')).toBe('score');
expect(metricKind('reading_avg_score')).toBe('score');
});
});
describe('computeYBounds', () => {
it('tightens clustered percentages instead of framing 0-100', () => {
const b = computeYBounds([86, 86, 86, 80, 96], 'percentage');
expect(b.min).toBeGreaterThanOrEqual(0);
expect(b.max).toBeLessThanOrEqual(100);
expect(b.min).toBeGreaterThan(50);
expect(b.max! - b.min!).toBeGreaterThanOrEqual(10);
});
it('never widens percentages beyond 0-100 for non-negative data', () => {
const b = computeYBounds([2, 5, 98], 'percentage');
expect(b.min).toBe(0);
expect(b.max).toBe(100);
});
it('does not clamp to zero when pct-named trend data is negative', () => {
const b = computeYBounds([-12, -3, 4], 'percentage');
expect(b.min).toBeLessThan(-12);
});
it('keeps progress bounds symmetric around zero', () => {
const b = computeYBounds([-1.2, 0.4, 2.1], 'progress');
expect(b.min).toBe(-b.max!);
expect(b.min).toBeLessThanOrEqual(-1.2);
expect(b.max).toBeGreaterThanOrEqual(2.1);
});
it('fits score metrics without a fixed frame', () => {
const b = computeYBounds([42.3, 48.9, 51.2], 'score');
expect(b.min).toBeGreaterThanOrEqual(0);
expect(b.min).toBeLessThanOrEqual(42.3);
expect(b.max).toBeGreaterThanOrEqual(51.2);
});
it('returns empty bounds when there is no numeric data', () => {
expect(computeYBounds([null, undefined, NaN], 'percentage')).toEqual({});
expect(computeYBounds([], 'progress')).toEqual({});
});
});
+3
View File
@@ -76,9 +76,12 @@ export default function RootLayout({
<head> <head>
<link rel="preconnect" href="https://analytics.schoolcompare.co.uk" /> <link rel="preconnect" href="https://analytics.schoolcompare.co.uk" />
<link rel="preconnect" href="https://api.postcodes.io" /> <link rel="preconnect" href="https://api.postcodes.io" />
{/* data-domains: the tracker only fires on the production hostnames,
so staging (same image, different host) never pollutes Umami */}
<Script <Script
src="https://analytics.schoolcompare.co.uk/script.js" src="https://analytics.schoolcompare.co.uk/script.js"
data-website-id="d7fb0c95-bb6c-4336-8209-bd10077e50dd" data-website-id="d7fb0c95-bb6c-4336-8209-bd10077e50dd"
data-domains="schoolcompare.co.uk,www.schoolcompare.co.uk"
data-performance="true" data-performance="true"
strategy="afterInteractive" strategy="afterInteractive"
/> />
@@ -0,0 +1,63 @@
/* Chart wrapper: chips (mobile) above, canvas filling the rest of the
parent .chartContainer, whose fixed height drives Chart.js sizing via
maintainAspectRatio: false. */
.wrapper {
display: flex;
flex-direction: column;
height: 100%;
}
.canvasBox {
position: relative;
flex: 1 1 auto;
min-height: 0;
}
/* School chips: mobile-only legend + tap-to-focus control. Desktop keeps
Chart.js's built-in legend (with per-school point shapes). */
.chips {
display: none;
}
@media (max-width: 640px) {
.chips {
display: flex;
flex-wrap: wrap;
gap: 6px;
padding-bottom: 8px;
}
.chip {
display: inline-flex;
align-items: center;
gap: 6px;
min-height: 44px;
max-width: 100%;
padding: 4px 10px;
border: 1px solid rgba(0, 0, 0, .12);
border-radius: 999px;
background: transparent;
cursor: pointer;
font-size: 12px;
font-weight: 600;
}
.chip[aria-pressed="true"] {
background: rgba(0, 0, 0, .06);
border-color: rgba(0, 0, 0, .35);
}
.chipDot {
flex: 0 0 auto;
width: 10px;
height: 10px;
border-radius: 50%;
}
.chipName {
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
max-width: 9rem;
}
}
+116 -45
View File
@@ -1,47 +1,82 @@
/** /**
* ComparisonChart Component * ComparisonChart Component
* Multi-school comparison chart using Chart.js * Multi-school comparison chart using Chart.js.
*
* Desktop: built-in legend (point-style markers double as per-school shapes).
* Mobile (≤640px): the in-chart legend and axis titles are dropped in favour
* of a chip row above the canvas; tapping a chip highlights that school's
* line and dims the rest. The y-axis auto-fits the data on all viewports so
* clustered schools stay distinguishable.
*/ */
'use client'; 'use client';
import { useEffect, useState } from 'react';
import { Line } from 'react-chartjs-2'; import { Line } from 'react-chartjs-2';
import { ChartOptions } from 'chart.js'; import { ChartOptions, ChartDataset, PointStyle } from 'chart.js';
import '@/lib/chartSetup'; import '@/lib/chartSetup';
import type { ComparisonData } from '@/lib/types'; import type { ComparisonData } from '@/lib/types';
import { CHART_COLORS, formatAcademicYear } from '@/lib/utils'; import {
CHART_COLORS,
CHART_TEXT_COLORS,
computeYBounds,
formatAcademicYear,
metricKind,
rgbToRgba,
} from '@/lib/utils';
import { useIsMobile } from '@/hooks/useIsMobile';
import { track } from '@/lib/analytics';
import styles from './ComparisonChart.module.css';
interface ComparisonChartProps { interface ComparisonChartProps {
comparisonData: Record<string, ComparisonData>; comparisonData: Record<string, ComparisonData>;
/** Ordered as displayed in the school cards, so colours match by index. */
schools: Array<{ urn: number; school_name: string }>;
metric: string; metric: string;
metricLabel: string; metricLabel: string;
} }
export function ComparisonChart({ comparisonData, metric, metricLabel }: ComparisonChartProps) { // One shape per basket slot (MAX_SCHOOLS = 5) — secondary encoding so
// Get all schools and their data // converging lines stay tellable apart without relying on hue alone.
const schools = Object.entries(comparisonData); const POINT_STYLES: PointStyle[] = ['circle', 'triangle', 'rect', 'rectRot', 'star'];
export function ComparisonChart({ comparisonData, schools, metric, metricLabel }: ComparisonChartProps) {
const isMobile = useIsMobile();
const [focusedUrn, setFocusedUrn] = useState<number | null>(null);
// A focused school that leaves the basket must not linger.
const urnKey = schools.map((s) => s.urn).join(',');
useEffect(() => {
setFocusedUrn(null);
}, [urnKey]);
if (schools.length === 0) { if (schools.length === 0) {
return <div>No data available</div>; return <div>No data available</div>;
} }
// Get years from first school (assuming all schools have same years) // Union of years across all schools — coverage differs between them.
const years = schools[0][1].yearly_data.map((d) => d.year).sort((a, b) => a - b); const years = [
...new Set(schools.flatMap((s) => comparisonData[String(s.urn)]?.yearly_data.map((d) => d.year) ?? [])),
].sort((a, b) => a - b);
// Create datasets for each school const datasets: ChartDataset<'line'>[] = schools.map((school, index) => {
const datasets = schools.map(([urn, data], index) => { const data = comparisonData[String(school.urn)];
const schoolInfo = data.school_info;
const color = CHART_COLORS[index % CHART_COLORS.length]; const color = CHART_COLORS[index % CHART_COLORS.length];
const dimmed = focusedUrn !== null && focusedUrn !== school.urn;
return { return {
label: schoolInfo.school_name, label: school.school_name,
data: years.map((year) => { data: years.map((year) => {
const yearData = data.yearly_data.find((d) => d.year === year); const yearData = data?.yearly_data.find((d) => d.year === year);
if (!yearData) return null; if (!yearData) return null;
return yearData[metric as keyof typeof yearData] as number | null; return yearData[metric as keyof typeof yearData] as number | null;
}), }),
borderColor: color, borderColor: dimmed ? rgbToRgba(color, 0.2) : color,
backgroundColor: color.replace('rgb', 'rgba').replace(')', ', 0.1)'), backgroundColor: dimmed ? 'transparent' : rgbToRgba(color, 0.1),
borderWidth: focusedUrn === school.urn ? 3 : dimmed ? 1.5 : 2,
pointStyle: POINT_STYLES[index % POINT_STYLES.length],
pointRadius: dimmed ? 2 : isMobile ? 3 : 4,
pointHoverRadius: isMobile ? 5 : 6,
tension: 0.3, tension: 0.3,
spanGaps: true, spanGaps: true,
}; };
@@ -52,9 +87,11 @@ export function ComparisonChart({ comparisonData, metric, metricLabel }: Compari
datasets, datasets,
}; };
// Determine if metric is a progress score or percentage const kind = metricKind(metric);
const isProgressScore = metric.includes('progress'); const yBounds = computeYBounds(
const isPercentage = metric.includes('pct') || metric.includes('rate'); datasets.flatMap((ds) => ds.data as Array<number | null>),
kind,
);
const options: ChartOptions<'line'> = { const options: ChartOptions<'line'> = {
responsive: true, responsive: true,
@@ -65,6 +102,7 @@ export function ComparisonChart({ comparisonData, metric, metricLabel }: Compari
}, },
plugins: { plugins: {
legend: { legend: {
display: !isMobile,
position: 'top' as const, position: 'top' as const,
labels: { labels: {
usePointStyle: true, usePointStyle: true,
@@ -74,26 +112,22 @@ export function ComparisonChart({ comparisonData, metric, metricLabel }: Compari
}, },
}, },
}, },
// No in-chart title: the section heading and metric selector above the
// chart already state the metric.
title: { title: {
display: true, display: false,
text: `${metricLabel} - Comparison`,
font: {
size: 16,
weight: 'bold',
},
padding: {
bottom: 20,
},
}, },
tooltip: { tooltip: {
backgroundColor: 'rgba(0, 0, 0, 0.8)', backgroundColor: 'rgba(0, 0, 0, 0.8)',
padding: 12, padding: isMobile ? 10 : 12,
titleFont: { titleFont: {
size: 14, size: isMobile ? 12 : 14,
}, },
bodyFont: { bodyFont: {
size: 13, size: isMobile ? 11 : 13,
}, },
usePointStyle: true,
itemSort: (a, b) => (b.parsed.y ?? -Infinity) - (a.parsed.y ?? -Infinity),
callbacks: { callbacks: {
label: function (context) { label: function (context) {
let label = context.dataset.label || ''; let label = context.dataset.label || '';
@@ -101,13 +135,7 @@ export function ComparisonChart({ comparisonData, metric, metricLabel }: Compari
label += ': '; label += ': ';
} }
if (context.parsed.y !== null) { if (context.parsed.y !== null) {
if (isProgressScore) { label += context.parsed.y.toFixed(1) + (kind === 'percentage' ? '%' : '');
label += context.parsed.y.toFixed(1);
} else if (isPercentage) {
label += context.parsed.y.toFixed(1) + '%';
} else {
label += context.parsed.y.toFixed(1);
}
} else { } else {
label += 'N/A'; label += 'N/A';
} }
@@ -121,17 +149,18 @@ export function ComparisonChart({ comparisonData, metric, metricLabel }: Compari
type: 'linear' as const, type: 'linear' as const,
display: true, display: true,
title: { title: {
display: true, display: !isMobile,
text: isPercentage ? 'Percentage (%)' : isProgressScore ? 'Progress Score' : 'Value', text: kind === 'percentage' ? 'Percentage (%)' : kind === 'progress' ? 'Progress Score' : 'Value',
font: { font: {
size: 12, size: 12,
weight: 'bold', weight: 'bold',
}, },
}, },
...(isPercentage && { ...yBounds,
min: 0, ticks: {
max: 100, font: { size: isMobile ? 10 : 12 },
}), ...(isMobile && { maxTicksLimit: 5 }),
},
grid: { grid: {
color: 'rgba(0, 0, 0, 0.05)', color: 'rgba(0, 0, 0, 0.05)',
}, },
@@ -141,16 +170,58 @@ export function ComparisonChart({ comparisonData, metric, metricLabel }: Compari
display: false, display: false,
}, },
title: { title: {
display: true, display: !isMobile,
text: 'Year', text: 'Year',
font: { font: {
size: 12, size: 12,
weight: 'bold', weight: 'bold',
}, },
}, },
ticks: {
font: { size: isMobile ? 10 : 12 },
...(isMobile && { maxRotation: 0, autoSkip: true, maxTicksLimit: 4 }),
},
}, },
}, },
}; };
return <Line data={chartData} options={options} />; const toggleFocus = (urn: number) => {
const next = focusedUrn === urn ? null : urn;
setFocusedUrn(next);
if (next !== null) track('compare_focus_school', { urn: next });
};
return (
<div className={styles.wrapper}>
{/* Mobile legend + focus control; a single series needs no legend. */}
{schools.length > 1 && (
<div className={styles.chips} role="group" aria-label="Highlight a school on the chart">
{schools.map((school, index) => (
<button
key={school.urn}
type="button"
className={styles.chip}
aria-pressed={focusedUrn === school.urn}
onClick={() => toggleFocus(school.urn)}
>
<span
className={styles.chipDot}
style={{ background: CHART_COLORS[index % CHART_COLORS.length] }}
aria-hidden="true"
/>
<span
className={styles.chipName}
style={{ color: CHART_TEXT_COLORS[index % CHART_TEXT_COLORS.length] }}
>
{school.school_name}
</span>
</button>
))}
</div>
)}
<div className={styles.canvasBox}>
<Line data={chartData} options={options} aria-label={`${metricLabel} comparison chart`} />
</div>
</div>
);
} }
@@ -454,7 +454,10 @@
} }
.chartContainer { .chartContainer {
height: 300px; /* Taller than desktop's proportion would suggest: the chip legend row
sits inside, and the in-chart title/legend/axis titles are gone, so
nearly all of this is plot area. */
height: 340px;
} }
.comparisonTable { .comparisonTable {
+4 -1
View File
@@ -111,8 +111,10 @@ export function ComparisonView({
setComparisonData(data.comparison); setComparisonData(data.comparison);
}) })
.catch((err) => { .catch((err) => {
// Keep whatever we already have (SSR data or a previous fetch) rather
// than blanking the chart — a transient refetch failure shouldn't
// destroy a working comparison the user is looking at.
console.error('Failed to fetch comparison:', err); console.error('Failed to fetch comparison:', err);
setComparisonData(null);
}); });
} else { } else {
setComparisonData(null); setComparisonData(null);
@@ -429,6 +431,7 @@ export function ComparisonView({
<div className={styles.chartContainer}> <div className={styles.chartContainer}>
<ComparisonChart <ComparisonChart
comparisonData={activeComparisonData} comparisonData={activeComparisonData}
schools={activeSchools}
metric={selectedMetric} metric={selectedMetric}
metricLabel={metricLabel} metricLabel={metricLabel}
/> />
+3 -11
View File
@@ -10,12 +10,13 @@
'use client'; 'use client';
import { useEffect, useMemo, useState } from 'react'; import { useMemo, useState } from 'react';
import { Line } from 'react-chartjs-2'; import { Line } from 'react-chartjs-2';
import { ChartOptions, ChartDataset } from 'chart.js'; import { ChartOptions, ChartDataset } from 'chart.js';
import '@/lib/chartSetup'; import '@/lib/chartSetup';
import type { SchoolResult } from '@/lib/types'; import type { SchoolResult } from '@/lib/types';
import { formatAcademicYear } from '@/lib/utils'; import { formatAcademicYear } from '@/lib/utils';
import { useIsMobile } from '@/hooks/useIsMobile';
import { track } from '@/lib/analytics'; import { track } from '@/lib/analytics';
import styles from './PerformanceChart.module.css'; import styles from './PerformanceChart.module.css';
@@ -68,16 +69,7 @@ export function PerformanceChart({
const sortedData = [...data].sort((a, b) => a.year - b.year); const sortedData = [...data].sort((a, b) => a.year - b.year);
const years = sortedData.map(d => formatAcademicYear(d.year)); const years = sortedData.map(d => formatAcademicYear(d.year));
// ── Mobile detection ───────────────────────────────────────────────── const isMobile = useIsMobile();
// Hydration-safe: SSR renders desktop; client flips to mobile after mount.
const [isMobile, setIsMobile] = useState(false);
useEffect(() => {
const mq = window.matchMedia('(max-width: 640px)');
const update = () => setIsMobile(mq.matches);
update();
mq.addEventListener('change', update);
return () => mq.removeEventListener('change', update);
}, []);
// ── Build per-year national averages ───────────────────────────────── // ── Build per-year national averages ─────────────────────────────────
const natRefRwm: (number | null)[] = sortedData.map(d => { const natRefRwm: (number | null)[] = sortedData.map(d => {
@@ -34,6 +34,15 @@
background: #fff; background: #fff;
} }
/* Fallback fullscreen (iOS Safari — no Element.requestFullscreen): the API
can't promote the element, so pin it over the page ourselves. Above the
comparison toast (3000) and everything else except modals (9999+). */
.wrapper[data-fs-fallback] {
position: fixed;
inset: 0;
z-index: 5000;
}
.skeleton { .skeleton {
width: 100%; width: 100%;
height: 100%; height: 100%;
+29 -4
View File
@@ -29,25 +29,50 @@ interface SchoolHeroMapProps {
export const SchoolHeroMap = forwardRef<SchoolHeroMapHandle, SchoolHeroMapProps>( export const SchoolHeroMap = forwardRef<SchoolHeroMapHandle, SchoolHeroMapProps>(
function SchoolHeroMap({ lat, lng }, ref) { function SchoolHeroMap({ lat, lng }, ref) {
const wrapperRef = useRef<HTMLDivElement>(null); const wrapperRef = useRef<HTMLDivElement>(null);
const [isFullscreen, setIsFullscreen] = useState(false); const [nativeFullscreen, setNativeFullscreen] = useState(false);
// iOS Safari has no Element.requestFullscreen — fall back to a
// fixed-position overlay driven by state instead of the Fullscreen API.
const [fallbackFullscreen, setFallbackFullscreen] = useState(false);
const isFullscreen = nativeFullscreen || fallbackFullscreen;
const open = useCallback(() => { const open = useCallback(() => {
wrapperRef.current?.requestFullscreen?.().catch(() => {}); const el = wrapperRef.current;
if (!el) return;
if (el.requestFullscreen) {
el.requestFullscreen().catch(() => setFallbackFullscreen(true));
} else {
setFallbackFullscreen(true);
}
}, []); }, []);
const close = useCallback(() => { const close = useCallback(() => {
if (document.fullscreenElement) document.exitFullscreen().catch(() => {}); if (document.fullscreenElement) document.exitFullscreen().catch(() => {});
setFallbackFullscreen(false);
}, []); }, []);
useImperativeHandle(ref, () => ({ open }), [open]); useImperativeHandle(ref, () => ({ open }), [open]);
useEffect(() => { useEffect(() => {
const onChange = () => setIsFullscreen(!!document.fullscreenElement); const onChange = () => setNativeFullscreen(!!document.fullscreenElement);
document.addEventListener('fullscreenchange', onChange); document.addEventListener('fullscreenchange', onChange);
return () => document.removeEventListener('fullscreenchange', onChange); return () => document.removeEventListener('fullscreenchange', onChange);
}, []); }, []);
// The fallback overlay sits on top of the page rather than replacing it,
// so lock body scroll while it is up.
useEffect(() => {
if (!fallbackFullscreen) return;
const prev = document.body.style.overflow;
document.body.style.overflow = 'hidden';
return () => { document.body.style.overflow = prev; };
}, [fallbackFullscreen]);
return ( return (
<div ref={wrapperRef} className={styles.wrapper} data-fullscreen={isFullscreen || undefined}> <div
ref={wrapperRef}
className={styles.wrapper}
data-fullscreen={isFullscreen || undefined}
data-fs-fallback={fallbackFullscreen || undefined}
>
<LeafletHeroMap lat={lat} lng={lng} interactive={isFullscreen} /> <LeafletHeroMap lat={lat} lng={lng} interactive={isFullscreen} />
{isFullscreen ? ( {isFullscreen ? (
+23
View File
@@ -0,0 +1,23 @@
/**
* Viewport hook shared by the chart components.
* Hydration-safe: SSR and the first client render report desktop; the
* media-query subscription flips the value after mount.
*/
'use client';
import { useEffect, useState } from 'react';
export function useIsMobile(maxWidth = 640): boolean {
const [isMobile, setIsMobile] = useState(false);
useEffect(() => {
const mq = window.matchMedia(`(max-width: ${maxWidth}px)`);
const update = () => setIsMobile(mq.matches);
update();
mq.addEventListener('change', update);
return () => mq.removeEventListener('change', update);
}, [maxWidth]);
return isMobile;
}
+1
View File
@@ -29,6 +29,7 @@ export type EventName =
| 'compare_viewed' | 'compare_viewed'
| 'compare_metric_changed' | 'compare_metric_changed'
| 'compare_shared' | 'compare_shared'
| 'compare_focus_school'
// Operational // Operational
| 'api_error' | 'api_error'
| 'results_load_more'; | 'results_load_more';
+51
View File
@@ -317,6 +317,57 @@ export function getTrendColor(trend: 'up' | 'down' | 'stable'): string {
} }
} }
/**
* Broad shape of a KS2/KS4 metric, used to scale chart axes and format values.
*/
export type MetricKind = 'percentage' | 'progress' | 'score';
export function metricKind(metric: string): MetricKind {
if (metric.includes('progress')) return 'progress';
if (metric.includes('pct') || metric.includes('rate')) return 'percentage';
return 'score';
}
/**
* Fit a chart y-axis to the data instead of a fixed frame, so clustered
* series remain distinguishable. Padding keeps a minimum span so noise is
* not magnified into drama.
*
* - percentage: pad and snap to 5s; cap at 100; floor at 0 only when the
* data is non-negative (some trend metrics have `pct` in the key but hold
* negative year-over-year deltas).
* - progress: symmetric around 0 so the zero line always shows.
* - score (Attainment 8, scaled scores): pad and snap to integers; floor at
* 0 only when the data is non-negative.
*/
export function computeYBounds(
values: Array<number | null | undefined>,
kind: MetricKind,
): { min?: number; max?: number } {
const nums = values.filter((v): v is number => typeof v === 'number' && Number.isFinite(v));
if (nums.length === 0) return {};
const lo = Math.min(...nums);
const hi = Math.max(...nums);
if (kind === 'progress') {
const reach = Math.max(2, Math.ceil(Math.max(Math.abs(lo), Math.abs(hi)) + 0.5));
return { min: -reach, max: reach };
}
if (kind === 'percentage') {
const pad = Math.max(5, Math.round((hi - lo) * 0.2));
const min = Math.floor((lo - pad) / 5) * 5;
const max = Math.min(100, Math.ceil((hi + pad) / 5) * 5);
return { min: lo >= 0 ? Math.max(0, min) : min, max };
}
// score
const pad = Math.max(2, (hi - lo) * 0.2);
const min = Math.floor(lo - pad);
return { min: lo >= 0 ? Math.max(0, min) : min, max: Math.ceil(hi + pad) };
}
// ============================================================================ // ============================================================================
// Local Storage Utilities // Local Storage Utilities
// ============================================================================ // ============================================================================
+1 -1
View File
@@ -7,7 +7,7 @@
"dev": "next dev", "dev": "next dev",
"build": "next build", "build": "next build",
"start": "next start", "start": "next start",
"lint": "next lint", "typecheck": "tsc --noEmit",
"test": "jest", "test": "jest",
"test:watch": "jest --watch", "test:watch": "jest --watch",
"test:coverage": "jest --coverage" "test:coverage": "jest --coverage"
+137
View File
@@ -0,0 +1,137 @@
#!/usr/bin/env python3
"""AI code review for Gitea pull requests, powered by Claude Code.
Reads the PR diff (base branch vs HEAD), asks Claude Code (headless `claude -p`)
to review it, posts the findings as a PR comment via the Gitea API, and exits
non-zero only when the review contains at least one severe finding — so the
job can gate merges without blocking on nitpicks.
Uses only the Python standard library; the review itself runs through the
Claude Code CLI, authenticated with a subscription OAuth token.
Required environment:
CLAUDE_CODE_OAUTH_TOKEN token from `claude setup-token` (subscription auth)
GITEA_TOKEN Gitea access token for posting PR comments
GITEA_SERVER_URL e.g. https://privaterepo.sitaru.org
GITEA_REPOSITORY owner/repo
PR_NUMBER pull request index
BASE_REF base branch name (e.g. main)
"""
import json
import os
import subprocess
import sys
import urllib.request
MAX_DIFF_CHARS = 150_000
PROMPT = """You are reviewing a pull request for SchoolCompare, a UK school
comparison site (FastAPI backend, Next.js frontend, Airflow/dbt data pipeline,
deployed via Gitea Actions to a staging-then-production Docker setup).
The PR diff is provided on stdin.
Report correctness bugs, security issues, data-loss risks, and broken deploy/CI
configuration. Mark a finding "severe" only if it would break production, leak
data, or corrupt data — severe findings block the merge. Everything else
(style, performance suggestions, minor cleanups) is "minor". Do not invent
findings: an empty findings list is a perfectly good review of a clean diff.
Respond with ONLY a JSON object (no markdown fences, no prose) of this shape:
{
"summary": "two or three sentences on what the change does and its health",
"findings": [
{"severity": "severe" | "minor", "file": "path", "issue": "description"}
]
}"""
def get_diff(base_ref: str) -> str:
subprocess.run(
["git", "fetch", "origin", base_ref],
check=True,
capture_output=True,
)
diff = subprocess.run(
["git", "diff", f"origin/{base_ref}...HEAD"],
check=True,
capture_output=True,
text=True,
).stdout
if len(diff) > MAX_DIFF_CHARS:
diff = diff[:MAX_DIFF_CHARS] + "\n\n[diff truncated for review]"
return diff
def review(diff: str) -> dict:
proc = subprocess.run(
["claude", "-p", PROMPT, "--output-format", "json"],
input=diff,
capture_output=True,
text=True,
timeout=900,
)
if proc.returncode != 0:
raise RuntimeError(f"claude CLI failed:\n{proc.stderr}")
envelope = json.loads(proc.stdout)
result = envelope["result"].strip()
# Defensive: strip markdown fences if the model added them anyway
if result.startswith("```"):
result = result.split("\n", 1)[1].rsplit("```", 1)[0]
return json.loads(result)
def format_comment(result: dict) -> str:
lines = ["## 🤖 AI Code Review (Claude Code)", "", result["summary"], ""]
severe = [f for f in result["findings"] if f["severity"] == "severe"]
minor = [f for f in result["findings"] if f["severity"] == "minor"]
if severe:
lines.append("### 🔴 Severe (blocks merge)")
lines += [f"- **{f['file']}**: {f['issue']}" for f in severe]
lines.append("")
if minor:
lines.append("### 🟡 Minor")
lines += [f"- **{f['file']}**: {f['issue']}" for f in minor]
lines.append("")
if not result["findings"]:
lines.append("✅ No issues found.")
return "\n".join(lines)
def post_comment(body: str) -> None:
server = os.environ["GITEA_SERVER_URL"].rstrip("/")
repo = os.environ["GITEA_REPOSITORY"]
pr = os.environ["PR_NUMBER"]
req = urllib.request.Request(
f"{server}/api/v1/repos/{repo}/issues/{pr}/comments",
data=json.dumps({"body": body}).encode(),
headers={
"Authorization": f"token {os.environ['GITEA_TOKEN']}",
"Content-Type": "application/json",
},
method="POST",
)
with urllib.request.urlopen(req, timeout=30) as resp:
if resp.status >= 300:
raise RuntimeError(f"Comment post failed: HTTP {resp.status}")
def main() -> int:
diff = get_diff(os.environ["BASE_REF"])
if not diff.strip():
print("Empty diff, nothing to review")
return 0
result = review(diff)
comment = format_comment(result)
print(comment)
post_comment(comment)
severe = [f for f in result["findings"] if f["severity"] == "severe"]
if severe:
print(f"\n{len(severe)} severe finding(s) — failing the check", file=sys.stderr)
return 1
return 0
if __name__ == "__main__":
sys.exit(main())