Compare commits

...
Author SHA1 Message Date
TudorandClaude Fable 5 d0100cce69 feat(ci): comment-triggered PR fix-ups via @claude
PR Checks / Frontend Typecheck + Tests (pull_request) Successful in 9m41s
PR Checks / Backend Smoke (pull_request) Successful in 5s
PR Checks / Build Backend (no push) (pull_request) Successful in 16s
PR Checks / Build Frontend (no push) (pull_request) Successful in 47s
PR Checks / Build Pipeline (no push) (pull_request) Successful in 10s
PR Checks / AI Code Review (Claude) (pull_request) Failing after 2m30s
Commenting '@claude <instruction>' on a PR runs headless Claude Code on the
PR branch (subscription auth), pushes the resulting commit — re-running the
PR checks — and replies with a summary. Owner-only trigger; runs unsandboxed
inside the ephemeral runner container per explicit maintainer sign-off.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PqGhF93UrpDNvXBLMjJENL
2026-07-03 16:26:09 +01:00
tudor 23b4e1c453 Merge pull request 'fix(e2e): scroll detail-page chart into view before asserting' (#5) from fix/e2e-detail-chart-scroll into main
Deploy (staging -> E2E gate -> production) / Build Backend (FastAPI) (push) Successful in 13s
Deploy (staging -> E2E gate -> production) / Build Frontend (Next.js) (push) Successful in 49s
Deploy (staging -> E2E gate -> production) / Build Pipeline (Meltano + dbt + Airflow) (push) Successful in 12s
Deploy (staging -> E2E gate -> production) / Deploy to Staging (push) Successful in 1s
Deploy (staging -> E2E gate -> production) / E2E Journeys against Staging (push) Successful in 33s
Deploy (staging -> E2E gate -> production) / Promote to Production (push) Successful in 8s
2026-07-03 13:56:00 +00:00
TudorandClaude Fable 5 deeef23131 fix(e2e): assert on a visible canvas — the first canvas is hidden by design
PR Checks / Frontend Typecheck + Tests (pull_request) Successful in 9m41s
PR Checks / Backend Smoke (pull_request) Successful in 5s
PR Checks / Build Backend (no push) (pull_request) Successful in 10s
PR Checks / Build Frontend (no push) (pull_request) Successful in 47s
PR Checks / Build Pipeline (no push) (pull_request) Successful in 10s
PR Checks / AI Code Review (Claude) (pull_request) Successful in 26s
The admissions card stacks year/trend views in one grid cell and keeps the
inactive view visibility:hidden; its canvas is first in the DOM. Use
canvas:visible instead of scrolling. Verified: full suite passes against prod.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PqGhF93UrpDNvXBLMjJENL
2026-07-03 14:40:13 +01:00
TudorandClaude Fable 5 4ece55b031 fix(e2e): scroll the detail-page chart into view before asserting visibility
PR Checks / Frontend Typecheck + Tests (pull_request) Successful in 9m41s
PR Checks / Backend Smoke (pull_request) Successful in 5s
PR Checks / Build Backend (no push) (pull_request) Successful in 10s
PR Checks / Build Frontend (no push) (pull_request) Successful in 55s
PR Checks / Build Pipeline (no push) (pull_request) Successful in 10s
PR Checks / AI Code Review (Claude) (pull_request) Successful in 34s
The canvas renders below the fold and stays 'hidden' to Playwright until
scrolled to; wait for attachment, scroll, then assert.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PqGhF93UrpDNvXBLMjJENL
2026-07-03 14:34:17 +01:00
tudor 515494dbf0 Merge pull request 'fix(analytics): restrict Umami to production hostnames' (#4) from fix/umami-prod-domains-only into main
Deploy (staging -> E2E gate -> production) / Build Backend (FastAPI) (push) Successful in 13s
Deploy (staging -> E2E gate -> production) / Build Frontend (Next.js) (push) Successful in 50s
Deploy (staging -> E2E gate -> production) / Build Pipeline (Meltano + dbt + Airflow) (push) Successful in 12s
Deploy (staging -> E2E gate -> production) / Deploy to Staging (push) Successful in 0s
Deploy (staging -> E2E gate -> production) / E2E Journeys against Staging (push) Failing after 1m7s
Deploy (staging -> E2E gate -> production) / Promote to Production (push) Has been skipped
2026-07-03 13:17:53 +00:00
TudorandClaude Fable 5 5772c54ccd fix(ci): use the documented GITHUB_TOKEN name for the run-scoped token
PR Checks / Frontend Typecheck + Tests (pull_request) Successful in 9m37s
PR Checks / Backend Smoke (pull_request) Successful in 5s
PR Checks / Build Backend (no push) (pull_request) Successful in 11s
PR Checks / Build Frontend (no push) (pull_request) Successful in 45s
PR Checks / Build Pipeline (no push) (pull_request) Successful in 9s
PR Checks / AI Code Review (Claude) (pull_request) Successful in 1m27s
GITEA_TOKEN worked (the review comment posted with it) but GITHUB_TOKEN is
the documented name in Gitea Actions; use it to keep the reviewer and the
docs aligned.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PqGhF93UrpDNvXBLMjJENL
2026-07-03 13:53:57 +01:00
TudorandClaude Fable 5 c62ba0ca25 fix(ci): post AI review comments with the run-scoped Gitea token
PR Checks / Frontend Typecheck + Tests (pull_request) Successful in 9m37s
PR Checks / Backend Smoke (pull_request) Successful in 5s
PR Checks / Build Backend (no push) (pull_request) Successful in 10s
PR Checks / Build Frontend (no push) (pull_request) Successful in 45s
PR Checks / Build Pipeline (no push) (pull_request) Successful in 10s
PR Checks / AI Code Review (Claude) (pull_request) Failing after 1m56s
REGISTRY_TOKEN lacks issue-write scope (403 on comment post). Gitea Actions
auto-provides a repo-scoped per-run token as secrets.GITEA_TOKEN — no
user-managed secret needed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PqGhF93UrpDNvXBLMjJENL
2026-07-03 13:39:59 +01:00
TudorandClaude Fable 5 b5a63e82d4 fix(analytics): restrict Umami to production hostnames
PR Checks / Frontend Typecheck + Tests (pull_request) Successful in 9m37s
PR Checks / Backend Smoke (pull_request) Successful in 4s
PR Checks / Build Backend (no push) (pull_request) Successful in 11s
PR Checks / Build Frontend (no push) (pull_request) Successful in 48s
PR Checks / Build Pipeline (no push) (pull_request) Successful in 10s
PR Checks / AI Code Review (Claude) (pull_request) Failing after 36s
The same frontend image runs on staging and prod (build once, promote), so a
build-time env var can't tell them apart. Umami's data-domains attribute
scopes the tracker client-side: events only fire when location.hostname is a
production domain, so staging traffic and the E2E journeys never pollute the
dashboards.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PqGhF93UrpDNvXBLMjJENL
2026-07-03 13:24:14 +01:00
tudor f5de745a8b Merge pull request 'fix(ci): AI review via Claude Code CLI; reuse REGISTRY_TOKEN' (#3) from fix/ai-review-claude-code into main
Deploy (staging -> E2E gate -> production) / Build Backend (FastAPI) (push) Successful in 18s
Deploy (staging -> E2E gate -> production) / Build Frontend (Next.js) (push) Successful in 46s
Deploy (staging -> E2E gate -> production) / Build Pipeline (Meltano + dbt + Airflow) (push) Successful in 12s
Deploy (staging -> E2E gate -> production) / Deploy to Staging (push) Failing after 1s
Deploy (staging -> E2E gate -> production) / E2E Journeys against Staging (push) Has been skipped
Deploy (staging -> E2E gate -> production) / Promote to Production (push) Has been skipped
2026-07-03 08:50:26 +00:00
TudorandClaude Fable 5 d0895c71df fix(ci): AI review via Claude Code CLI; reuse REGISTRY_TOKEN for PR comments
PR Checks / Frontend Typecheck + Tests (pull_request) Successful in 9m39s
PR Checks / Backend Smoke (pull_request) Successful in 5s
PR Checks / Build Backend (no push) (pull_request) Successful in 16s
PR Checks / Build Frontend (no push) (pull_request) Successful in 42s
PR Checks / Build Pipeline (no push) (pull_request) Successful in 9s
PR Checks / AI Code Review (Claude) (pull_request) Failing after 27s
- ai_review.py now pipes the diff through headless Claude Code (claude -p,
  --output-format json) authenticated with CLAUDE_CODE_OAUTH_TOKEN from
  'claude setup-token' — subscription auth, no Anthropic API billing
- stdlib-only script (urllib instead of requests/anthropic)
- PR comments posted with the existing REGISTRY_TOKEN secret; the separate
  GITEA_TOKEN secret is no longer needed

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PqGhF93UrpDNvXBLMjJENL
2026-07-03 08:28:20 +01:00
tudor df0bf1c4d6 Merge pull request 'feat(sdlc): staging environment + automated staging→prod pipeline' (#2) from feat/sdlc-staging-pipeline into main
Deploy (staging -> E2E gate -> production) / Build Backend (FastAPI) (push) Successful in 19s
Deploy (staging -> E2E gate -> production) / Build Frontend (Next.js) (push) Successful in 46s
Deploy (staging -> E2E gate -> production) / Build Pipeline (Meltano + dbt + Airflow) (push) Successful in 12s
Deploy (staging -> E2E gate -> production) / Deploy to Staging (push) Successful in 1s
Deploy (staging -> E2E gate -> production) / E2E Journeys against Staging (push) Failing after 1m10s
Deploy (staging -> E2E gate -> production) / Promote to Production (push) Has been skipped
2026-07-03 06:17:40 +00:00
TudorandClaude Fable 5 4a52735356 feat(sdlc): staging environment + automated staging→prod pipeline
PR Checks / Frontend Typecheck + Tests (pull_request) Successful in 10m0s
PR Checks / Backend Smoke (pull_request) Successful in 48s
PR Checks / Build Backend (no push) (pull_request) Successful in 17s
PR Checks / Build Frontend (no push) (pull_request) Successful in 41s
PR Checks / Build Pipeline (no push) (pull_request) Successful in 10s
PR Checks / AI Code Review (Claude) (pull_request) Successful in 24s
- pr-checks.yml: PR gate — frontend typecheck+jest, backend import smoke,
  image builds (no push), Claude AI review posted as PR comment (severe
  findings block merge)
- deploy.yml (replaces build-and-push.yml): merge to main builds+pushes
  images tagged sha-<sha>/staging, deploys the staging Portainer stack via
  webhook, runs Playwright E2E journeys against staging, then retags the
  verified images :prod (previous kept as :prod-previous) and deploys prod
- docker-compose.portainer.staging.yml: second Portainer stack — :staging
  images, sc_staging_* names, own macvlan IPs, Airflow on 8081; data
  bootstrapped from source via the staging Airflow DAGs
- prod compose now pins :prod instead of :latest (only the promotion step
  moves it; :latest is no longer published)
- e2e/: 6 Playwright journeys (search, postcode, detail, compare, rankings)
  driven by BASE_URL — the promotion gate
- scripts/ci/ai_review.py: Claude review with structured JSON findings
- docs/DEPLOY.md: full SDLC doc incl. one-time setup checklist and rollback
- replaced removed 'next lint' with tsc typecheck; fixed stale jest tests
  (slug URLs, N/A formatting, stable trend, fake-timer setup)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PqGhF93UrpDNvXBLMjJENL
2026-07-03 06:50:40 +01:00
TudorandClaude Fable 5 f2ed49c0a1 feat(home): compact value-prop line on the mobile hero (P1.7)
Build and Push Docker Images / Build Backend (FastAPI) (push) Successful in 13s
Build and Push Docker Images / Build Frontend (Next.js) (push) Successful in 48s
Build and Push Docker Images / Build Pipeline (Meltano + dbt + Airflow) (push) Successful in 13s
Build and Push Docker Images / Trigger Portainer Update (push) Successful in 1s
Phones showed only the poetic h1 and a bare search box — no coverage,
scope, or freshness statement above the fold for the 63%-of-entries,
56%-mobile audience. One compact line ('24,000+ English schools — SATs,
GCSEs, Ofsted & admissions, side by side. Updated for 2026/27') now
replaces the hidden eyebrow + full paragraph at ≤640px, costing ~2 short
lines. Desktop copy unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 21:56:21 +01:00
TudorandClaude Fable 5 f24b8044f8 fix(rankings): long metric labels wrap instead of widening the table
Build and Push Docker Images / Build Backend (FastAPI) (push) Successful in 15s
Build and Push Docker Images / Build Frontend (Next.js) (push) Successful in 47s
Build and Push Docker Images / Build Pipeline (Meltano + dbt + Airflow) (push) Successful in 12s
Build and Push Docker Images / Trigger Portainer Update (push) Successful in 0s
Table auto-layout sizes columns by unwrapped header text, so labels like
'Reading, Writing & Maths Combined Higher %' pushed the value column —
and the table — past the viewport. The label now renders inside a block
span capped at 110px (84px on phones), forcing multiline and keeping the
table fully in view at any metric.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 21:40:30 +01:00
22 changed files with 1214 additions and 53 deletions
@@ -1,12 +1,9 @@
name: Build and Push Docker Images
name: Deploy (staging -> E2E gate -> production)
on:
push:
branches:
- main
pull_request:
branches:
- main
env:
REGISTRY: privaterepo.sitaru.org
@@ -45,17 +42,15 @@ jobs:
with:
images: ${{ env.REGISTRY }}/${{ env.BACKEND_IMAGE_NAME }}
tags: |
type=ref,event=branch
type=ref,event=pr
type=sha,prefix=backend-
type=raw,value=latest,enable=${{ gitea.ref == 'refs/heads/main' }}
type=sha
type=raw,value=staging
- name: Build and push Backend Docker image
uses: docker/build-push-action@v5
with:
context: .
file: ./Dockerfile
push: ${{ gitea.event_name != 'pull_request' }}
push: true
tags: ${{ steps.meta-backend.outputs.tags }}
labels: ${{ steps.meta-backend.outputs.labels }}
cache-from: type=registry,ref=${{ env.REGISTRY }}/${{ env.BACKEND_IMAGE_NAME }}:buildcache
@@ -91,24 +86,20 @@ jobs:
with:
images: ${{ env.REGISTRY }}/${{ env.FRONTEND_IMAGE_NAME }}
tags: |
type=ref,event=branch
type=ref,event=pr
type=sha,prefix=frontend-
type=raw,value=latest,enable=${{ gitea.ref == 'refs/heads/main' }}
type=sha
type=raw,value=staging
- name: Build and push Frontend Docker image
uses: docker/build-push-action@v5
with:
context: ./nextjs-app
file: ./nextjs-app/Dockerfile
push: ${{ gitea.event_name != 'pull_request' }}
push: true
tags: ${{ steps.meta-frontend.outputs.tags }}
labels: ${{ steps.meta-frontend.outputs.labels }}
build-args: |
FASTAPI_URL=http://backend:80/api
# Cache disabled due to registry size limits
# cache-from: type=registry,ref=${{ env.REGISTRY }}/${{ env.FRONTEND_IMAGE_NAME }}:buildcache
# cache-to: type=registry,ref=${{ env.REGISTRY }}/${{ env.FRONTEND_IMAGE_NAME }}:buildcache,mode=max
build-pipeline:
name: Build Pipeline (Meltano + dbt + Airflow)
@@ -140,28 +131,110 @@ jobs:
with:
images: ${{ env.REGISTRY }}/${{ env.PIPELINE_IMAGE_NAME }}
tags: |
type=ref,event=branch
type=ref,event=pr
type=sha,prefix=pipeline-
type=raw,value=latest,enable=${{ gitea.ref == 'refs/heads/main' }}
type=sha
type=raw,value=staging
- name: Build and push Pipeline Docker image
uses: docker/build-push-action@v5
with:
context: ./pipeline
file: ./pipeline/Dockerfile
push: ${{ gitea.event_name != 'pull_request' }}
push: true
tags: ${{ steps.meta-pipeline.outputs.tags }}
labels: ${{ steps.meta-pipeline.outputs.labels }}
cache-from: type=registry,ref=${{ env.REGISTRY }}/${{ env.PIPELINE_IMAGE_NAME }}:buildcache
cache-to: type=registry,ref=${{ env.REGISTRY }}/${{ env.PIPELINE_IMAGE_NAME }}:buildcache,mode=max
trigger-deployment:
name: Trigger Portainer Update
deploy-staging:
name: Deploy to Staging
runs-on: ubuntu-latest
needs: [build-backend, build-frontend, build-pipeline]
if: gitea.event_name != 'pull_request'
steps:
- name: Trigger Portainer stack update
- name: Trigger staging stack update
run: curl -fsSk -X POST "${{ secrets.PORTAINER_STAGING_WEBHOOK }}"
- name: Wait for staging to become healthy
run: |
curl -X POST -k "https://10.0.1.224:9443/api/stacks/webhooks/863fc57c-bf24-4c63-9001-bdf9912fba73"
echo "Polling ${STAGING_BASE_URL} for up to 5 minutes..."
for i in $(seq 1 60); do
if curl -fsS -o /dev/null --max-time 10 "${STAGING_BASE_URL}/"; then
echo "Staging is up (attempt $i)"
exit 0
fi
sleep 5
done
echo "Staging did not become healthy in time" >&2
exit 1
env:
STAGING_BASE_URL: ${{ secrets.STAGING_BASE_URL }}
e2e-staging:
name: E2E Journeys against Staging
runs-on: ubuntu-latest
needs: [deploy-staging]
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: 22
- name: Install Playwright
working-directory: e2e
run: |
npm ci
npx playwright install --with-deps chromium
- name: Run E2E journeys
working-directory: e2e
run: npx playwright test
env:
BASE_URL: ${{ secrets.STAGING_BASE_URL }}
promote-prod:
name: Promote to Production
runs-on: ubuntu-latest
needs: [e2e-staging]
steps:
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Log in to Gitea Container Registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ gitea.actor }}
password: ${{ secrets.REGISTRY_TOKEN }}
- name: Retag verified images as prod
run: |
SHORT_SHA="sha-$(echo "${{ gitea.sha }}" | cut -c1-7)"
for IMAGE in \
"${REGISTRY}/${BACKEND_IMAGE_NAME}" \
"${REGISTRY}/${FRONTEND_IMAGE_NAME}" \
"${REGISTRY}/${PIPELINE_IMAGE_NAME}"; do
# Keep a rollback pointer before moving :prod
docker buildx imagetools create -t "${IMAGE}:prod-previous" "${IMAGE}:prod" || true
docker buildx imagetools create -t "${IMAGE}:prod" "${IMAGE}:${SHORT_SHA}"
echo "Promoted ${IMAGE}:${SHORT_SHA} -> :prod"
done
- name: Trigger production stack update
run: curl -fsSk -X POST "${{ secrets.PORTAINER_PROD_WEBHOOK }}"
- name: Wait for production to become healthy
run: |
echo "Polling ${PROD_BASE_URL} for up to 5 minutes..."
for i in $(seq 1 60); do
if curl -fsS -o /dev/null --max-time 10 "${PROD_BASE_URL}/"; then
echo "Production is up (attempt $i)"
exit 0
fi
sleep 5
done
echo "Production did not become healthy in time" >&2
exit 1
env:
PROD_BASE_URL: ${{ secrets.PROD_BASE_URL }}
+188
View File
@@ -0,0 +1,188 @@
name: PR Checks
on:
pull_request:
branches:
- main
env:
REGISTRY: privaterepo.sitaru.org
BACKEND_IMAGE_NAME: ${{ gitea.repository }}-backend
FRONTEND_IMAGE_NAME: ${{ gitea.repository }}-frontend
PIPELINE_IMAGE_NAME: ${{ gitea.repository }}-pipeline
jobs:
frontend-checks:
name: Frontend Typecheck + Tests
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: 22
cache: npm
cache-dependency-path: nextjs-app/package-lock.json
- name: Install dependencies
working-directory: nextjs-app
run: npm ci
- name: Typecheck
working-directory: nextjs-app
run: npm run typecheck
- name: Unit tests
working-directory: nextjs-app
run: npm test
backend-checks:
name: Backend Smoke
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install dependencies
run: pip install -r requirements.txt
- name: Import smoke test
run: python -c "from backend.app import app; print('backend imports OK')"
build-backend:
name: Build Backend (no push)
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
buildkitd-config-inline: |
[registry."docker.io"]
mirrors = ["10.0.1.224:6000"]
[registry."10.0.1.224:6000"]
http = true
insecure = true
- name: Log in to Gitea Container Registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ gitea.actor }}
password: ${{ secrets.REGISTRY_TOKEN }}
- name: Build Backend Docker image
uses: docker/build-push-action@v5
with:
context: .
file: ./Dockerfile
push: false
cache-from: type=registry,ref=${{ env.REGISTRY }}/${{ env.BACKEND_IMAGE_NAME }}:buildcache
build-frontend:
name: Build Frontend (no push)
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
buildkitd-config-inline: |
[registry."docker.io"]
mirrors = ["10.0.1.224:6000"]
[registry."10.0.1.224:6000"]
http = true
insecure = true
- name: Log in to Gitea Container Registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ gitea.actor }}
password: ${{ secrets.REGISTRY_TOKEN }}
- name: Build Frontend Docker image
uses: docker/build-push-action@v5
with:
context: ./nextjs-app
file: ./nextjs-app/Dockerfile
push: false
build-args: |
FASTAPI_URL=http://backend:80/api
build-pipeline:
name: Build Pipeline (no push)
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
buildkitd-config-inline: |
[registry."docker.io"]
mirrors = ["10.0.1.224:6000"]
[registry."10.0.1.224:6000"]
http = true
insecure = true
- name: Log in to Gitea Container Registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ gitea.actor }}
password: ${{ secrets.REGISTRY_TOKEN }}
- name: Build Pipeline Docker image
uses: docker/build-push-action@v5
with:
context: ./pipeline
file: ./pipeline/Dockerfile
push: false
cache-from: type=registry,ref=${{ env.REGISTRY }}/${{ env.PIPELINE_IMAGE_NAME }}:buildcache
ai-review:
name: AI Code Review (Claude)
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: 22
- name: Install Claude Code
run: npm install -g @anthropic-ai/claude-code
- name: Review PR diff with Claude Code
env:
CLAUDE_CODE_OAUTH_TOKEN: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
# Auto-provided per-run token from Gitea Actions (repo-scoped).
# GITHUB_TOKEN is the documented name; GITEA_TOKEN is its alias.
GITEA_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITEA_SERVER_URL: ${{ gitea.server_url }}
GITEA_REPOSITORY: ${{ gitea.repository }}
PR_NUMBER: ${{ gitea.event.pull_request.number }}
BASE_REF: ${{ gitea.event.pull_request.base.ref }}
run: python scripts/ci/ai_review.py
+47
View File
@@ -0,0 +1,47 @@
name: PR Comment Agent
on:
issue_comment:
types: [created]
jobs:
ai-fixup:
name: Claude Fix-up (@claude comment)
runs-on: ubuntu-latest
# Only PR comments, only from the repo owner, only when addressed to @claude.
# The owner guard matters: the job pushes code to the PR branch.
if: >-
gitea.event.issue.pull_request &&
startsWith(gitea.event.comment.body, '@claude') &&
gitea.event.comment.user.login == gitea.repository_owner
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: 22
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install Claude Code
run: npm install -g @anthropic-ai/claude-code
- name: Apply the requested fix-up
env:
CLAUDE_CODE_OAUTH_TOKEN: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
GITEA_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITEA_SERVER_URL: ${{ gitea.server_url }}
GITEA_REPOSITORY: ${{ gitea.repository }}
PR_NUMBER: ${{ gitea.event.issue.number }}
COMMENT_BODY: ${{ gitea.event.comment.body }}
# Claude Code runs as root inside the runner container; this flag
# acknowledges the container *is* the sandbox.
IS_SANDBOX: "1"
run: python scripts/ci/ai_fixup.py
+14
View File
@@ -105,8 +105,22 @@ This starts:
- `GET /api/metrics` - Metric definitions (single source of truth)
- `GET /api/data-info` - Database stats
## SDLC
Full details in `docs/DEPLOY.md`. The short version:
- **Never push to `main` directly.** Work on a feature branch and open a PR;
branch protection requires the PR checks (typecheck, tests, builds, AI review)
to pass before merge.
- Merging to `main` deploys automatically: images are built once, deployed to
the **staging** Portainer stack, verified by the Playwright journeys in
`e2e/`, and only then retagged `:prod` and rolled out to production.
- If you change user-facing behaviour, update or extend the `e2e/` journey
tests in the same PR — they are the promotion gate.
## Recent Changes
- Added staging environment + automated staging→prod pipeline (Gitea Actions)
- Migrated from CSV file storage to PostgreSQL database
- Added location-based search using postcode geocoding
- Added local authority filter to rankings
+214
View File
@@ -0,0 +1,214 @@
# Portainer Stack Definition for School Compare — STAGING
#
# Deploy this as a *separate* Portainer stack (e.g. "schoolcompare-staging")
# alongside the production stack. Differences from production:
# - images pinned to :staging (pushed by every merge to main, before the E2E gate)
# - sc_staging_* container names
# - own macvlan IPs (STAGING_DB_IP / STAGING_FRONTEND_IP env vars)
# - Airflow UI published on 8081 (prod uses 8080)
# - volumes are isolated automatically: Portainer prefixes volume names with
# the stack name, so this stack gets its own postgres/typesense/airflow data
#
# Portainer environment variables (set in Portainer UI -> Stack -> Environment):
# DB_USERNAME — PostgreSQL username
# DB_PASSWORD — PostgreSQL password
# DB_DATABASE_NAME — PostgreSQL database name
# ADMIN_API_KEY — Backend admin API key
# TYPESENSE_API_KEY — Typesense admin API key
# TYPESENSE_SEARCH_KEY — Typesense search-only key (exposed to frontend)
# AIRFLOW_ADMIN_USER — Airflow admin username (password auto-generated, see api-server logs)
# STAGING_DB_IP — macvlan IP for staging Postgres (default 10.0.1.190)
# STAGING_FRONTEND_IP — macvlan IP for staging frontend (default 10.0.1.151)
services:
# ── PostgreSQL ────────────────────────────────────────────────────────
sc_database:
container_name: sc_staging_postgres
image: postgis/postgis:18-3.6-alpine
environment:
POSTGRES_PASSWORD: ${DB_PASSWORD}
POSTGRES_USER: ${DB_USERNAME}
POSTGRES_DB: ${DB_DATABASE_NAME}
volumes:
- postgres_data:/var/lib/postgresql
shm_size: 128mb
networks:
backend: {}
macvlan:
ipv4_address: ${STAGING_DB_IP:-10.0.1.190}
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres"]
interval: 10s
timeout: 5s
retries: 5
start_period: 10s
restart: unless-stopped
# ── FastAPI Backend ───────────────────────────────────────────────────
backend:
image: privaterepo.sitaru.org/tudor/school_compare-backend:staging
container_name: sc_staging_backend
environment:
DATABASE_URL: postgresql://${DB_USERNAME}:${DB_PASSWORD}@sc_database:5432/${DB_DATABASE_NAME}
PYTHONUNBUFFERED: 1
ADMIN_API_KEY: ${ADMIN_API_KEY:-changeme}
TYPESENSE_URL: http://typesense:8108
TYPESENSE_API_KEY: ${TYPESENSE_API_KEY:-changeme}
depends_on:
sc_database:
condition: service_healthy
networks:
- backend
restart: unless-stopped
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:80/api/data-info"]
interval: 30s
timeout: 10s
retries: 3
start_period: 30s
# ── Next.js Frontend ──────────────────────────────────────────────────
frontend:
image: privaterepo.sitaru.org/tudor/school_compare-frontend:staging
container_name: sc_staging_nextjs
environment:
- NODE_ENV=production
- NEXT_PUBLIC_API_URL=http://localhost:8000/api
- FASTAPI_URL=http://backend:80/api
- TYPESENSE_URL=http://typesense:8108
- TYPESENSE_API_KEY=${TYPESENSE_SEARCH_KEY:-changeme}
depends_on:
backend:
condition: service_healthy
networks:
backend: {}
macvlan:
ipv4_address: ${STAGING_FRONTEND_IP:-10.0.1.151}
restart: unless-stopped
healthcheck:
test: ["CMD", "node", "-e", "require('http').get('http://localhost:3000/', (r) => {process.exit(r.statusCode === 200 ? 0 : 1)})"]
interval: 30s
timeout: 10s
retries: 3
start_period: 40s
# ── Typesense Search Engine ───────────────────────────────────────────
typesense:
image: typesense/typesense:30.1
container_name: sc_staging_typesense
environment:
TYPESENSE_API_KEY: ${TYPESENSE_API_KEY:-changeme}
TYPESENSE_DATA_DIR: /data
volumes:
- typesense_data:/data
networks:
- backend
restart: unless-stopped
healthcheck:
test: ["CMD-SHELL", "cat < /dev/tcp/localhost/8108"]
interval: 15s
timeout: 5s
retries: 5
start_period: 10s
# ── Airflow API Server + UI (staging: http://<host>:8081) ─────────────
airflow-api-server:
image: privaterepo.sitaru.org/tudor/school_compare-pipeline:staging
container_name: sc_staging_airflow_api
command: airflow api-server --port 8080
ports:
- "8081:8080"
environment:
AIRFLOW__CORE__EXECUTOR: LocalExecutor
AIRFLOW__DATABASE__SQL_ALCHEMY_CONN: postgresql+psycopg2://${DB_USERNAME}:${DB_PASSWORD}@sc_database:5432/${DB_DATABASE_NAME}
AIRFLOW__CORE__DAGS_FOLDER: /opt/pipeline/dags
AIRFLOW__CORE__LOAD_EXAMPLES: "false"
AIRFLOW__CORE__EXECUTION_API_SERVER_URL: http://airflow-api-server:8080/execution/
AIRFLOW__API_AUTH__JWT_SECRET: "school-compare-staging-airflow-jwt-secret-key-long-enough-for-sha512"
AIRFLOW__API_AUTH__JWT_ISSUER: airflow
AIRFLOW__CORE__SIMPLE_AUTH_MANAGER_USERS: "${AIRFLOW_ADMIN_USER:-admin}:admin"
AIRFLOW__LOGGING__BASE_LOG_FOLDER: /opt/airflow/logs
PG_HOST: sc_database
PG_PORT: "5432"
PG_USER: ${DB_USERNAME}
PG_PASSWORD: ${DB_PASSWORD}
PG_DATABASE: ${DB_DATABASE_NAME}
TYPESENSE_URL: http://typesense:8108
TYPESENSE_API_KEY: ${TYPESENSE_API_KEY:-changeme}
volumes:
- airflow_logs:/opt/airflow/logs
depends_on:
sc_database:
condition: service_healthy
networks:
- backend
restart: unless-stopped
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8080/api/v2/monitor/health"]
interval: 30s
timeout: 10s
retries: 5
start_period: 60s
# ── Airflow Scheduler ──────────────────────────────────────────────
airflow-scheduler:
image: privaterepo.sitaru.org/tudor/school_compare-pipeline:staging
container_name: sc_staging_airflow_scheduler
command: airflow scheduler
environment:
AIRFLOW__CORE__EXECUTOR: LocalExecutor
AIRFLOW__DATABASE__SQL_ALCHEMY_CONN: postgresql+psycopg2://${DB_USERNAME}:${DB_PASSWORD}@sc_database:5432/${DB_DATABASE_NAME}
AIRFLOW__CORE__DAGS_FOLDER: /opt/pipeline/dags
AIRFLOW__CORE__LOAD_EXAMPLES: "false"
AIRFLOW__CORE__EXECUTION_API_SERVER_URL: http://airflow-api-server:8080/execution/
AIRFLOW__API_AUTH__JWT_SECRET: "school-compare-staging-airflow-jwt-secret-key-long-enough-for-sha512"
AIRFLOW__API_AUTH__JWT_ISSUER: airflow
AIRFLOW__LOGGING__BASE_LOG_FOLDER: /opt/airflow/logs
PG_HOST: sc_database
PG_PORT: "5432"
PG_USER: ${DB_USERNAME}
PG_PASSWORD: ${DB_PASSWORD}
PG_DATABASE: ${DB_DATABASE_NAME}
TYPESENSE_URL: http://typesense:8108
TYPESENSE_API_KEY: ${TYPESENSE_API_KEY:-changeme}
volumes:
- airflow_logs:/opt/airflow/logs
depends_on:
sc_database:
condition: service_healthy
networks:
- backend
restart: unless-stopped
# ── Airflow DB Init (one-shot) ───────────────────────────────────────
airflow-init:
image: privaterepo.sitaru.org/tudor/school_compare-pipeline:staging
container_name: sc_staging_airflow_init
command: bash -c "airflow db migrate && airflow dags reserialize"
environment:
AIRFLOW__CORE__EXECUTOR: LocalExecutor
AIRFLOW__DATABASE__SQL_ALCHEMY_CONN: postgresql+psycopg2://${DB_USERNAME}:${DB_PASSWORD}@sc_database:5432/${DB_DATABASE_NAME}
AIRFLOW__CORE__DAGS_FOLDER: /opt/pipeline/dags
AIRFLOW__CORE__LOAD_EXAMPLES: "false"
AIRFLOW__CORE__EXECUTION_API_SERVER_URL: http://airflow-api-server:8080/execution/
AIRFLOW__API_AUTH__JWT_SECRET: "school-compare-staging-airflow-jwt-secret-key-long-enough-for-sha512"
AIRFLOW__API_AUTH__JWT_ISSUER: airflow
depends_on:
sc_database:
condition: service_healthy
networks:
- backend
restart: "no"
networks:
backend:
driver: bridge
macvlan:
external:
name: macvlan
volumes:
postgres_data:
typesense_data:
airflow_logs:
+5 -5
View File
@@ -36,7 +36,7 @@ services:
# ── FastAPI Backend ───────────────────────────────────────────────────
backend:
image: privaterepo.sitaru.org/tudor/school_compare-backend:latest
image: privaterepo.sitaru.org/tudor/school_compare-backend:prod
container_name: schoolcompare_backend
environment:
DATABASE_URL: postgresql://${DB_USERNAME}:${DB_PASSWORD}@sc_database:5432/${DB_DATABASE_NAME}
@@ -59,7 +59,7 @@ services:
# ── Next.js Frontend ──────────────────────────────────────────────────
frontend:
image: privaterepo.sitaru.org/tudor/school_compare-frontend:latest
image: privaterepo.sitaru.org/tudor/school_compare-frontend:prod
container_name: schoolcompare_nextjs
environment:
- NODE_ENV=production
@@ -103,7 +103,7 @@ services:
# ── Airflow API Server + UI ───────────────────────────────────────────
airflow-api-server:
image: privaterepo.sitaru.org/tudor/school_compare-pipeline:latest
image: privaterepo.sitaru.org/tudor/school_compare-pipeline:prod
container_name: schoolcompare_airflow_api
command: airflow api-server --port 8080
ports:
@@ -142,7 +142,7 @@ services:
# ── Airflow Scheduler ──────────────────────────────────────────────
airflow-scheduler:
image: privaterepo.sitaru.org/tudor/school_compare-pipeline:latest
image: privaterepo.sitaru.org/tudor/school_compare-pipeline:prod
container_name: schoolcompare_airflow_scheduler
command: airflow scheduler
environment:
@@ -172,7 +172,7 @@ services:
# ── Airflow DB Init (one-shot) ───────────────────────────────────────
airflow-init:
image: privaterepo.sitaru.org/tudor/school_compare-pipeline:latest
image: privaterepo.sitaru.org/tudor/school_compare-pipeline:prod
container_name: schoolcompare_airflow_init
command: bash -c "airflow db migrate && airflow dags delete school_data_daily -y 2>/dev/null; airflow dags delete school_data_monthly_ofsted -y 2>/dev/null; airflow dags delete school_data_annual_ees -y 2>/dev/null; airflow dags reserialize"
environment:
+150
View File
@@ -0,0 +1,150 @@
# SDLC & Deployment Pipeline
SchoolCompare uses a fully automated staging → production pipeline on Gitea
Actions. AI writes the code on feature branches; the pipeline verifies every
change on a staging environment before promoting the exact same images to
production. Human input is directional only: feature requests, PR review if
desired, and intervention when a gate fails.
## The flow
```
feature branch (AI-authored)
│ PR to main
PR checks (.gitea/workflows/pr-checks.yml)
typecheck + unit tests + backend smoke + image builds (no push)
+ Claude code review posted as a PR comment (severe findings fail the check)
│ merge (branch protection requires green checks)
Deploy pipeline (.gitea/workflows/deploy.yml)
1. build & push images → tags sha-<sha>, staging
2. staging Portainer webhook → wait for staging health
3. Playwright E2E journeys against staging
4. retag sha-<sha> → :prod (same bytes — build once, promote the image)
previous :prod saved as :prod-previous
5. prod Portainer webhook → wait for prod health
```
Key principle: **build once, promote the exact image**. Production pins `:prod`,
which only moves after the E2E gate passes on staging. Nothing tags `:latest`
anymore.
## Branch & PR workflow
- `main` is protected: no direct pushes, PRs require green status checks.
- All work (human or AI) happens on feature branches → PR to `main`.
- Merging to `main` **is** the release action. If staging or the E2E gate
fails, production is untouched.
## Environments
| | Production | Staging |
|---|---|---|
| Portainer stack file | `docker-compose.portainer.yml` | `docker-compose.portainer.staging.yml` |
| Image tag | `:prod` | `:staging` |
| Container prefix | `sc_` / `schoolcompare_` | `sc_staging_` |
| Frontend macvlan IP | 10.0.1.150 | `STAGING_FRONTEND_IP` (default 10.0.1.151) |
| Postgres macvlan IP | 10.0.1.189 | `STAGING_DB_IP` (default 10.0.1.190) |
| Airflow UI port | 8080 | 8081 |
| Volumes | stack-prefixed | stack-prefixed (fully isolated) |
Staging gets `:staging` images on every merge to main — even ones that later
fail the E2E gate. That's the point: staging absorbs the risk.
## Gitea repository secrets
| Secret | Purpose |
|---|---|
| `REGISTRY_TOKEN` | push images to privaterepo.sitaru.org (already set) |
| `CLAUDE_CODE_OAUTH_TOKEN` | Claude Code subscription auth for the PR review — generate with `claude setup-token` on your machine |
| `PORTAINER_STAGING_WEBHOOK` | staging stack redeploy webhook URL |
| `PORTAINER_PROD_WEBHOOK` | production stack redeploy webhook URL |
| `STAGING_BASE_URL` | e.g. `http://10.0.1.151:3000` — health poll + E2E target |
| `PROD_BASE_URL` | e.g. `http://10.0.1.150:3000` — post-promotion health poll |
## One-time setup checklist
1. **Create the staging stack** in Portainer from
`docker-compose.portainer.staging.yml` (stack name e.g.
`schoolcompare-staging`). Set the same environment variables as prod plus
`STAGING_DB_IP` / `STAGING_FRONTEND_IP` if the defaults clash.
2. **Enable webhooks** on both stacks (Portainer → Stack → Webhook) and store
the URLs as `PORTAINER_STAGING_WEBHOOK` / `PORTAINER_PROD_WEBHOOK`. Remove
the old hardcoded webhook usage (now gone from the workflows).
3. **Add the remaining secrets** listed above in Gitea → repo → Settings →
Actions → Secrets.
4. **Protect `main`** in Gitea → Settings → Branches: require PRs, require the
pr-checks status checks (frontend, backend, builds, ai-review) to pass.
5. **Bootstrap staging data via Airflow** (no prod dump — staging populates
itself from source, exercising the pipeline image end-to-end):
- Open the staging Airflow UI (`http://<host>:8081`) and trigger, in order:
`school_data_daily`, `school_data_monthly_ofsted`,
`school_data_monthly_parent_view`, then the manual-schedule
`school_data_annual_ees` and `school_data_annual_idaci`.
- First runs download from government sources (GIAS, Ofsted, EES, IDACI),
run dbt, and sync Typesense — expect the initial backfill to take a while.
- The scheduled DAGs then keep staging fresh exactly like prod.
6. **Switch the prod stack to `:prod` tags** — the repo's
`docker-compose.portainer.yml` is already updated; redeploy the prod stack
from it. Until the first pipeline run promotes an image, tag the current
images manually: `docker buildx imagetools create -t <image>:prod <image>:latest`
for each of the three images.
## Rollback
Every promotion first re-points `:prod-previous` at the outgoing `:prod`.
To roll back:
```bash
for img in backend frontend pipeline; do
docker buildx imagetools create \
-t privaterepo.sitaru.org/tudor/school_compare-$img:prod \
privaterepo.sitaru.org/tudor/school_compare-$img:prod-previous
done
curl -fsSk -X POST "$PORTAINER_PROD_WEBHOOK"
```
Or promote any older build directly: `imagetools create -t <image>:prod <image>:sha-<shortsha>`.
## E2E suite
Lives in `e2e/` (own package — CI installs it without the app's node_modules).
Journeys: home + name search, postcode search, school detail, two-school
comparison, rankings table. Run locally against any environment:
```bash
cd e2e && npm ci
BASE_URL=http://10.0.1.151:3000 npx playwright test
```
Tests assert data invariants (results exist, charts render), not exact
numbers, so scheduled data refreshes don't break the gate.
## AI code review
`scripts/ci/ai_review.py` pipes the PR diff through headless Claude Code
(`claude -p`, authenticated with the subscription OAuth token — no API
billing), posts the structured findings as a PR comment using the per-run
token Gitea Actions provides automatically (`secrets.GITEA_TOKEN` — no setup
needed), and fails the check only when a finding is rated
**severe** (would break prod, leak data, or corrupt data). Minor findings are
informational and never block a merge.
## Comment-triggered fix-ups (@claude)
Comment `@claude <instruction>` on any PR and `.gitea/workflows/pr-comment.yml`
runs `scripts/ci/ai_fixup.py`: it checks out the PR branch, hands the
instruction to headless Claude Code (same subscription auth as the reviewer),
commits and pushes whatever changed, and replies on the PR with a summary.
The push re-runs the PR checks automatically.
Guard rails:
- Only comments from the **repo owner** trigger it (the job pushes code).
- Only comments starting with `@claude` — the bot's own replies never re-trigger.
- Each comment is one full agentic session on the Claude subscription; batch
related asks into one comment rather than several small ones.
Caveat: if the checks don't re-run after the bot's push, Gitea is suppressing
workflows for pushes made with the run token — create a personal access token
secret and swap it in for the push, or re-run the checks manually.
+3
View File
@@ -0,0 +1,3 @@
node_modules/
test-results/
playwright-report/
+78
View File
@@ -0,0 +1,78 @@
{
"name": "schoolcompare-e2e",
"version": "1.0.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "schoolcompare-e2e",
"version": "1.0.0",
"devDependencies": {
"@playwright/test": "^1.49.0"
}
},
"node_modules/@playwright/test": {
"version": "1.61.1",
"resolved": "https://registry.npmjs.org/@playwright/test/-/test-1.61.1.tgz",
"integrity": "sha512-8nKv6+0RJSL9FE4jYOEGXnPeM/Hg12qZpmqzZjRh3qM0Y7c3z1mrOTfFLids72RDQYVh9WpLEfR5WdpNX4fkig==",
"dev": true,
"license": "Apache-2.0",
"dependencies": {
"playwright": "1.61.1"
},
"bin": {
"playwright": "cli.js"
},
"engines": {
"node": ">=18"
}
},
"node_modules/fsevents": {
"version": "2.3.2",
"resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.2.tgz",
"integrity": "sha512-xiqMQR4xAeHTuB9uWm+fFRcIOgKBMiOBP+eXiyT7jsgVCq1bkVygt00oASowB7EdtpOHaaPgKt812P9ab+DDKA==",
"dev": true,
"hasInstallScript": true,
"license": "MIT",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": "^8.16.0 || ^10.6.0 || >=11.0.0"
}
},
"node_modules/playwright": {
"version": "1.61.1",
"resolved": "https://registry.npmjs.org/playwright/-/playwright-1.61.1.tgz",
"integrity": "sha512-DWnY5o3YbLWK4GovuAVwpqL+1VwGNdUGrRr++8j8PtQQzvAVZUIMjKQ90fY689sEJZJBbZVw1rXaOKSTitkzPQ==",
"dev": true,
"license": "Apache-2.0",
"dependencies": {
"playwright-core": "1.61.1"
},
"bin": {
"playwright": "cli.js"
},
"engines": {
"node": ">=18"
},
"optionalDependencies": {
"fsevents": "2.3.2"
}
},
"node_modules/playwright-core": {
"version": "1.61.1",
"resolved": "https://registry.npmjs.org/playwright-core/-/playwright-core-1.61.1.tgz",
"integrity": "sha512-h7Qlt6m4REp25qvIdvbDtVmD4LqVXfpRxhORv9L0jzETM05p4fuPJ3dKyuSXQxDSbXnmS79HAgi9589lGSpLkg==",
"dev": true,
"license": "Apache-2.0",
"bin": {
"playwright-core": "cli.js"
},
"engines": {
"node": ">=18"
}
}
}
}
+12
View File
@@ -0,0 +1,12 @@
{
"name": "schoolcompare-e2e",
"version": "1.0.0",
"private": true,
"description": "Journey tests run against staging as the production promotion gate",
"scripts": {
"test": "playwright test"
},
"devDependencies": {
"@playwright/test": "^1.49.0"
}
}
+13
View File
@@ -0,0 +1,13 @@
import { defineConfig } from '@playwright/test';
export default defineConfig({
testDir: './tests',
timeout: 60_000,
retries: 1,
workers: 2,
reporter: process.env.CI ? 'list' : 'html',
use: {
baseURL: process.env.BASE_URL || 'http://localhost:3000',
trace: 'retain-on-failure',
},
});
+75
View File
@@ -0,0 +1,75 @@
import { test, expect, Page } from '@playwright/test';
/**
* Journey tests for SchoolCompare, run against the staging environment as the
* gate before promotion to production. They assert stable data invariants
* (results exist, key UI renders) rather than exact numbers, so routine data
* refreshes don't break the pipeline.
*/
async function searchByName(page: Page, query: string) {
await page.goto('/');
const searchInput = page.getByPlaceholder('School name or postcode').first();
await searchInput.fill(query);
await searchInput.press('Enter');
await page.waitForURL(/search=|postcode=/);
}
function schoolLinks(page: Page) {
return page.locator('a[href^="/school/"]');
}
test('home page loads with hero search', async ({ page }) => {
await page.goto('/');
await expect(page.locator('h1').first()).toBeVisible();
await expect(page.getByPlaceholder('School name or postcode').first()).toBeVisible();
});
test('searching by name returns school results', async ({ page }) => {
await searchByName(page, 'primary');
await expect(schoolLinks(page).first()).toBeVisible({ timeout: 15_000 });
expect(await schoolLinks(page).count()).toBeGreaterThan(1);
});
test('searching by postcode returns nearby schools', async ({ page }) => {
await searchByName(page, 'B1 1BB');
await expect(schoolLinks(page).first()).toBeVisible({ timeout: 15_000 });
});
test('school detail page renders name and performance data', async ({ page }) => {
await searchByName(page, 'primary');
const firstSchool = schoolLinks(page).first();
await expect(firstSchool).toBeVisible({ timeout: 15_000 });
await firstSchool.click();
await page.waitForURL(/\/school\//);
await expect(page.locator('h1').first()).toBeVisible();
// The detail page renders at least one *visible* chart canvas. Plain
// .first() is wrong here: the admissions card stacks its year/trend views
// in one grid cell and keeps the inactive view's canvas visibility:hidden
// by design, and that canvas comes first in the DOM.
await expect(page.locator('canvas:visible').first()).toBeVisible({ timeout: 15_000 });
});
test('comparing two schools shows both side by side', async ({ page }) => {
// Collect two school URNs from search results, then load the share URL
await searchByName(page, 'primary');
await expect(schoolLinks(page).first()).toBeVisible({ timeout: 15_000 });
const hrefs = await schoolLinks(page).evaluateAll((links) =>
links.map((l) => (l as HTMLAnchorElement).getAttribute('href') || '')
);
const urns = [...new Set(hrefs.map((h) => h.match(/\/school\/(\d+)/)?.[1]).filter(Boolean))];
expect(urns.length).toBeGreaterThanOrEqual(2);
await page.goto(`/compare?urns=${urns[0]},${urns[1]}`);
// Both schools' detail links should render in the comparison view
await expect(page.locator(`a[href*="${urns[0]}"]`).first()).toBeVisible({ timeout: 15_000 });
await expect(page.locator(`a[href*="${urns[1]}"]`).first()).toBeVisible();
});
test('rankings page loads a populated table', async ({ page }) => {
await page.goto('/rankings');
await expect(page.getByRole('heading', { name: /rankings/i }).first()).toBeVisible();
const rows = page.locator('table tbody tr');
await expect(rows.first()).toBeVisible({ timeout: 15_000 });
expect(await rows.count()).toBeGreaterThan(5);
});
@@ -34,24 +34,24 @@ describe('SchoolCard', () => {
render(<SchoolCard school={mockSchool} />);
const link = screen.getByRole('link', { name: /test primary school/i });
expect(link).toHaveAttribute('href', '/school/100001');
expect(link).toHaveAttribute('href', '/school/100001-test-primary-school');
});
it('calls onAddToCompare when Add to Compare button is clicked', () => {
it('calls onAddToCompare when the Compare button is clicked', () => {
const mockAddToCompare = jest.fn();
render(<SchoolCard school={mockSchool} onAddToCompare={mockAddToCompare} />);
const addButton = screen.getByText('Add to Compare');
const addButton = screen.getByText('+ Compare');
fireEvent.click(addButton);
expect(mockAddToCompare).toHaveBeenCalledWith(mockSchool);
expect(mockAddToCompare).toHaveBeenCalledTimes(1);
});
it('does not render Add to Compare button when handler not provided', () => {
it('does not render the Compare button when handler not provided', () => {
render(<SchoolCard school={mockSchool} />);
expect(screen.queryByText('Add to Compare')).not.toBeInTheDocument();
expect(screen.queryByText('+ Compare')).not.toBeInTheDocument();
});
it('displays trend indicator for positive change', () => {
+13 -9
View File
@@ -19,7 +19,7 @@ describe('formatPercentage', () => {
});
it('handles null values', () => {
expect(formatPercentage(null)).toBe('-');
expect(formatPercentage(null)).toBe('N/A');
});
});
@@ -31,7 +31,7 @@ describe('formatProgress', () => {
});
it('handles null values', () => {
expect(formatProgress(null)).toBe('-');
expect(formatProgress(null)).toBe('N/A');
});
});
@@ -44,16 +44,16 @@ describe('calculateTrend', () => {
expect(calculateTrend(70, 75)).toBe('down');
});
it('calculates same trend', () => {
expect(calculateTrend(75, 75)).toBe('same');
it('calculates stable trend', () => {
expect(calculateTrend(75, 75)).toBe('stable');
});
it('handles null previous value', () => {
expect(calculateTrend(75, null)).toBe('same');
expect(calculateTrend(75, null)).toBe('stable');
});
it('handles null current value', () => {
expect(calculateTrend(null, 75)).toBe('same');
expect(calculateTrend(null, 75)).toBe('stable');
});
});
@@ -72,7 +72,13 @@ describe('isValidPostcode', () => {
});
describe('debounce', () => {
jest.useFakeTimers();
beforeEach(() => {
jest.useFakeTimers();
});
afterEach(() => {
jest.useRealTimers();
});
it('delays function execution', () => {
const mockFn = jest.fn();
@@ -100,8 +106,6 @@ describe('debounce', () => {
expect(mockFn).toHaveBeenCalledWith('third');
expect(mockFn).toHaveBeenCalledTimes(1);
});
jest.useRealTimers();
});
describe('buildOfstedListBadge', () => {
+3
View File
@@ -76,9 +76,12 @@ export default function RootLayout({
<head>
<link rel="preconnect" href="https://analytics.schoolcompare.co.uk" />
<link rel="preconnect" href="https://api.postcodes.io" />
{/* data-domains: the tracker only fires on the production hostnames,
so staging (same image, different host) never pollutes Umami */}
<Script
src="https://analytics.schoolcompare.co.uk/script.js"
data-website-id="d7fb0c95-bb6c-4336-8209-bd10077e50dd"
data-domains="schoolcompare.co.uk,www.schoolcompare.co.uk"
data-performance="true"
strategy="afterInteractive"
/>
+20 -4
View File
@@ -62,6 +62,11 @@
font-weight: 700;
}
/* Copy variants: full paragraph ≥641px, compact merged line on phones. */
.heroDescriptionCompact {
display: none;
}
@media (max-width: 768px) {
.heroSection {
padding-top: 1.5rem;
@@ -74,18 +79,29 @@
}
}
/* Above the fold on phones, every line costs. Drop the eyebrow tag and the
long descriptive paragraph — the h1 already names the product, and the
search input is the primary action users came to perform. */
/* Above the fold on phones, every line costs. Drop the eyebrow tag and swap
the long descriptive paragraph for one compact line that carries coverage
("24,000+ schools") and freshness ("updated for 2026/27") — a first-time
visitor otherwise sees only a poetic h1 and a bare box, with no reason to
trust the data (audit P1.7, feeds the 46% home exit rate). */
@media (max-width: 640px) {
.heroSection {
padding-top: 0.75rem;
margin-bottom: 1rem;
}
.heroEyebrow,
.heroDescription {
.heroDescriptionFull {
display: none;
}
.heroDescriptionCompact {
display: block;
}
.heroDescription {
font-size: 0.85rem;
line-height: 1.4;
max-width: 320px;
margin-bottom: 0.25rem;
}
.heroTitle {
font-size: 1.65rem;
}
+9 -1
View File
@@ -267,7 +267,15 @@ export function HomeView({ initialSchools, filters, totalSchools, howItWorks, ed
Every school in England, <em className={styles.heroEmph}>compared.</em>
</h1>
<p className={styles.heroDescription}>
<strong>24,000+ primary and secondary schools</strong> with Key Stage 2 SATs, GCSE results, Ofsted grades, progress scores and admissions data side by side, in one place.
{/* Full copy for larger screens; a compact merged line (coverage +
freshness, standing in for the hidden eyebrow too) on phones,
where every line above the fold costs. */}
<span className={styles.heroDescriptionFull}>
<strong>24,000+ primary and secondary schools</strong> with Key Stage 2 SATs, GCSE results, Ofsted grades, progress scores and admissions data side by side, in one place.
</span>
<span className={styles.heroDescriptionCompact}>
<strong>24,000+ English schools</strong> SATs, GCSEs, Ofsted &amp; admissions, side by side. Updated for 2026/27.
</span>
</p>
</div>
)}
+17 -1
View File
@@ -187,6 +187,18 @@
text-align: center;
}
/* Table auto-layout sizes columns by the unwrapped text, so a long metric
label ("Reading, Writing & Maths Combined Higher %") widened the table
past the viewport. A block inner span is what actually caps the measured
width and forces the label onto multiple lines. */
.valueHeaderText {
display: block;
max-width: 110px;
margin: 0 auto;
white-space: normal;
line-height: 1.2;
}
.actionHeader {
width: 120px;
text-align: center;
@@ -411,7 +423,7 @@
}
/* Long metric labels like "Reading, Writing & Maths Combined %" used to
force the whole column wide; let them wrap onto 2 short lines with a
force the whole column wide; let them wrap onto short lines with a
tighter font so the value cell can stay compact. */
.valueHeader {
font-size: 0.625rem;
@@ -420,6 +432,10 @@
letter-spacing: 0.03em;
}
.valueHeaderText {
max-width: 84px;
}
.rankHeader {
width: 40px;
}
+5 -1
View File
@@ -233,7 +233,11 @@ export function RankingsView({
<th className={styles.schoolHeader}>School</th>
<th className={styles.areaHeader}>Area</th>
<th className={styles.typeHeader}>Type</th>
<th className={styles.valueHeader}>{metricLabel}</th>
<th className={styles.valueHeader}>
{/* Inner block caps the column's measured width so long
labels wrap instead of widening the table off-screen. */}
<span className={styles.valueHeaderText}>{metricLabel}</span>
</th>
<th className={styles.actionHeader}>Action</th>
</tr>
</thead>
+1 -1
View File
@@ -7,7 +7,7 @@
"dev": "next dev",
"build": "next build",
"start": "next start",
"lint": "next lint",
"typecheck": "tsc --noEmit",
"test": "jest",
"test:watch": "jest --watch",
"test:coverage": "jest --coverage"
+106
View File
@@ -0,0 +1,106 @@
#!/usr/bin/env python3
"""Comment-triggered PR fix-ups, powered by Claude Code.
Runs when a maintainer comments `@claude <instruction>` on a pull request.
Checks out the PR head branch, hands the instruction to headless Claude Code
(subscription OAuth auth — no API billing), commits and pushes whatever it
changed (which re-runs the PR checks), and replies on the PR with a summary.
Uses only the Python standard library plus the `claude` CLI.
Required environment:
CLAUDE_CODE_OAUTH_TOKEN token from `claude setup-token`
GITEA_TOKEN run-scoped token (checkout auth handles the push)
GITEA_SERVER_URL e.g. https://privaterepo.sitaru.org
GITEA_REPOSITORY owner/repo
PR_NUMBER pull request index
COMMENT_BODY the triggering comment text
"""
import json
import os
import subprocess
import sys
import urllib.request
TRIGGER = "@claude"
CLAUDE_TIMEOUT_S = 2400 # 40 min ceiling for one fix-up session
def api(path: str, payload: dict | None = None) -> dict:
server = os.environ["GITEA_SERVER_URL"].rstrip("/")
repo = os.environ["GITEA_REPOSITORY"]
req = urllib.request.Request(
f"{server}/api/v1/repos/{repo}{path}",
data=json.dumps(payload).encode() if payload is not None else None,
headers={
"Authorization": f"token {os.environ['GITEA_TOKEN']}",
"Content-Type": "application/json",
},
method="POST" if payload is not None else "GET",
)
with urllib.request.urlopen(req, timeout=30) as resp:
return json.loads(resp.read())
def run(*cmd: str, **kwargs) -> subprocess.CompletedProcess:
return subprocess.run(cmd, check=True, capture_output=True, text=True, **kwargs)
def main() -> int:
pr_number = os.environ["PR_NUMBER"]
instruction = os.environ["COMMENT_BODY"].strip()
if instruction.lower().startswith(TRIGGER):
instruction = instruction[len(TRIGGER):].strip()
if not instruction:
print("Empty instruction after trigger word; nothing to do")
return 0
pr = api(f"/pulls/{pr_number}")
head_ref = pr["head"]["ref"]
base_ref = pr["base"]["ref"]
run("git", "fetch", "origin", head_ref, base_ref)
run("git", "checkout", head_ref)
prompt = f"""You are working on pull request #{pr_number} ("{pr['title']}")
in the SchoolCompare repository. The PR branch is checked out; its base is
{base_ref}. A maintainer left this instruction on the PR:
{instruction}
Implement exactly what was asked, following the conventions in CLAUDE.md.
Run any relevant tests or typechecks you can. Do NOT commit or push — the
harness handles that. When done, summarise in a few sentences what you
changed and how you verified it."""
proc = subprocess.run(
["claude", "-p", prompt, "--dangerously-skip-permissions", "--output-format", "json"],
capture_output=True,
text=True,
timeout=CLAUDE_TIMEOUT_S,
)
if proc.returncode != 0:
raise RuntimeError(f"claude CLI failed:\n{proc.stderr[-2000:]}")
summary = json.loads(proc.stdout)["result"].strip()
changed = run("git", "status", "--porcelain").stdout.strip()
if changed:
run("git", "config", "user.name", "Claude (CI)")
run("git", "config", "user.email", "noreply@anthropic.com")
run("git", "add", "-A")
title = instruction.splitlines()[0][:60]
run("git", "commit", "-m", f"ai: {title}\n\nRequested via PR comment; applied by Claude Code in CI.")
run("git", "push", "origin", head_ref)
sha = run("git", "rev-parse", "--short", "HEAD").stdout.strip()
reply = f"## 🤖 Claude fix-up applied (`{sha}`)\n\n{summary}\n\n_PR checks re-run automatically on the new commit._"
else:
reply = f"## 🤖 Claude fix-up — no changes made\n\n{summary}"
api(f"/issues/{pr_number}/comments", {"body": reply})
print(reply)
return 0
if __name__ == "__main__":
sys.exit(main())
+137
View File
@@ -0,0 +1,137 @@
#!/usr/bin/env python3
"""AI code review for Gitea pull requests, powered by Claude Code.
Reads the PR diff (base branch vs HEAD), asks Claude Code (headless `claude -p`)
to review it, posts the findings as a PR comment via the Gitea API, and exits
non-zero only when the review contains at least one severe finding — so the
job can gate merges without blocking on nitpicks.
Uses only the Python standard library; the review itself runs through the
Claude Code CLI, authenticated with a subscription OAuth token.
Required environment:
CLAUDE_CODE_OAUTH_TOKEN token from `claude setup-token` (subscription auth)
GITEA_TOKEN Gitea access token for posting PR comments
GITEA_SERVER_URL e.g. https://privaterepo.sitaru.org
GITEA_REPOSITORY owner/repo
PR_NUMBER pull request index
BASE_REF base branch name (e.g. main)
"""
import json
import os
import subprocess
import sys
import urllib.request
MAX_DIFF_CHARS = 150_000
PROMPT = """You are reviewing a pull request for SchoolCompare, a UK school
comparison site (FastAPI backend, Next.js frontend, Airflow/dbt data pipeline,
deployed via Gitea Actions to a staging-then-production Docker setup).
The PR diff is provided on stdin.
Report correctness bugs, security issues, data-loss risks, and broken deploy/CI
configuration. Mark a finding "severe" only if it would break production, leak
data, or corrupt data — severe findings block the merge. Everything else
(style, performance suggestions, minor cleanups) is "minor". Do not invent
findings: an empty findings list is a perfectly good review of a clean diff.
Respond with ONLY a JSON object (no markdown fences, no prose) of this shape:
{
"summary": "two or three sentences on what the change does and its health",
"findings": [
{"severity": "severe" | "minor", "file": "path", "issue": "description"}
]
}"""
def get_diff(base_ref: str) -> str:
subprocess.run(
["git", "fetch", "origin", base_ref],
check=True,
capture_output=True,
)
diff = subprocess.run(
["git", "diff", f"origin/{base_ref}...HEAD"],
check=True,
capture_output=True,
text=True,
).stdout
if len(diff) > MAX_DIFF_CHARS:
diff = diff[:MAX_DIFF_CHARS] + "\n\n[diff truncated for review]"
return diff
def review(diff: str) -> dict:
proc = subprocess.run(
["claude", "-p", PROMPT, "--output-format", "json"],
input=diff,
capture_output=True,
text=True,
timeout=900,
)
if proc.returncode != 0:
raise RuntimeError(f"claude CLI failed:\n{proc.stderr}")
envelope = json.loads(proc.stdout)
result = envelope["result"].strip()
# Defensive: strip markdown fences if the model added them anyway
if result.startswith("```"):
result = result.split("\n", 1)[1].rsplit("```", 1)[0]
return json.loads(result)
def format_comment(result: dict) -> str:
lines = ["## 🤖 AI Code Review (Claude Code)", "", result["summary"], ""]
severe = [f for f in result["findings"] if f["severity"] == "severe"]
minor = [f for f in result["findings"] if f["severity"] == "minor"]
if severe:
lines.append("### 🔴 Severe (blocks merge)")
lines += [f"- **{f['file']}**: {f['issue']}" for f in severe]
lines.append("")
if minor:
lines.append("### 🟡 Minor")
lines += [f"- **{f['file']}**: {f['issue']}" for f in minor]
lines.append("")
if not result["findings"]:
lines.append("✅ No issues found.")
return "\n".join(lines)
def post_comment(body: str) -> None:
server = os.environ["GITEA_SERVER_URL"].rstrip("/")
repo = os.environ["GITEA_REPOSITORY"]
pr = os.environ["PR_NUMBER"]
req = urllib.request.Request(
f"{server}/api/v1/repos/{repo}/issues/{pr}/comments",
data=json.dumps({"body": body}).encode(),
headers={
"Authorization": f"token {os.environ['GITEA_TOKEN']}",
"Content-Type": "application/json",
},
method="POST",
)
with urllib.request.urlopen(req, timeout=30) as resp:
if resp.status >= 300:
raise RuntimeError(f"Comment post failed: HTTP {resp.status}")
def main() -> int:
diff = get_diff(os.environ["BASE_REF"])
if not diff.strip():
print("Empty diff, nothing to review")
return 0
result = review(diff)
comment = format_comment(result)
print(comment)
post_comment(comment)
severe = [f for f in result["findings"] if f["severity"] == "severe"]
if severe:
print(f"\n{len(severe)} severe finding(s) — failing the check", file=sys.stderr)
return 1
return 0
if __name__ == "__main__":
sys.exit(main())