fix(cms): regenerate the import map so the Content field renders

Creating a post in the admin panel showed no Content editor, and saving
failed validation on the field the writer was never shown.

The admin panel does not import field components. The server hands the
client a path per field, and resolves it through the generated map at
app/(payload)/admin/importMap.js. A richText field's path is
@payloadcms/richtext-lexical/rsc#RscEntryLexicalField. The committed map
held one entry, @payloadcms/next/rsc#CollectionCards, generated before
the blog collections existed and never re-run. A path missing from the
map is not an error the panel reports: the field simply does not render,
while required is still enforced server-side on save.

next build does not regenerate the map, so the stale copy shipped in the
image and the editor was equally broken on staging and production.

Regenerated with payload generate:importmap, which adds the lexical RSC
field, cell and diff components, BlocksFeatureClient for the Callout
block, and the default toolbar features.

Two things stop it drifting again. There was no script to run, so
package.json gets generate:importmap. And a test asserts the map carries
an entry for each thing the config asks for, in the source-reading style
of the other payload suites; against the old map all five fail.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DXnXQKnPpZBBP61fBQiFkq

claude.md

@@ -23,6 +23,52 @@ Key files:
 - `backend/data_loader.py` - Data queries, geocoding, legacy DataFrame compatibility
 - `backend/schemas.py` - Column mappings, metric definitions, LA code mappings
 
+### Content / CMS (Payload)
+
+Payload CMS runs **inside** the Next.js app — one image, one container, no
+separate service. It powers `/blog`; `/about` is a plain coded page.
+
+- **Admin panel:** `/admin`. The only authenticated surface on the site.
+  `noindex` via both `robots.txt` and `X-Robots-Tag`.
+- **CMS API:** `/cms-api`, **not** `/api`. `/api/*` is a catch-all proxy to
+  FastAPI (`app/(frontend)/api/[...path]`) which would silently swallow every
+  admin call and forward it to the backend. Mount points are defined once in
+  `lib/payloadRoutes.ts`.
+- **Database:** the existing Postgres, in its own `payload` schema, so no
+  pipeline operation on `public` — including
+  `scripts/migrate_csv_to_db.py --drop` — can reach blog content.
+- **Uploads:** the `payload_media` Docker volume at `/app/media`. Not
+  reproducible from the pipeline; must be backed up.
+- **New env vars:** `DATABASE_URL` and `PAYLOAD_SECRET` on the frontend service.
+  Staging must use a different `PAYLOAD_SECRET` from production.
+- Publishing workflow and house style: `nextjs-app/docs/PUBLISHING.md`.
+- **Admin field components resolve through a generated import map**
+  (`app/(payload)/admin/importMap.js`). Payload hands the client a *path* per
+  field and looks it up there; a missing entry renders no field and reports no
+  error, while `required` still blocks the save. After adding or changing any
+  field, editor or lexical feature, run `npm run generate:importmap` in
+  `nextjs-app/` and commit the result.
+
+### Two route groups
+
+`nextjs-app/app/` has no root `layout.tsx`. It cannot: Payload's admin panel
+ships its own root layout rendering `<html>`/`<body>`, and Next permits
+multiple root layouts only when no `app/layout.tsx` exists.
+
+- `app/(frontend)/` — the site. Its `layout.tsx` is the site's root layout.
+- `app/(payload)/` — the admin panel and `/cms-api`.
+
+Route groups are invisible to routing, so every public URL is unchanged.
+
+**The metadata file conventions stay at the `app/` root** — `robots.ts`,
+`opengraph-image.tsx`, `icon.png`, `apple-icon.png`. Inside a route group Next
+treats them as segment-scoped: it renames `/icon.png` to `/icon-<hash>.png` and
+drops `/robots.txt` entirely. Route handlers are unaffected.
+
+The build must succeed with `DATABASE_URL` unset, because CI builds it that
+way. Never call `getCachedPayload()` at module scope, and never add
+`generateStaticParams` to a DB-backed route.
+
 ### Frontend (Vanilla JS)
 - Single-page application with hash-based routing
 - Chart.js for data visualization

docker-compose.portainer.staging.yml

@@ -18,7 +18,14 @@
 #   TYPESENSE_SEARCH_KEY   — Typesense search-only key (exposed to frontend)
 #   UNLEASH_URL            — http://<unleash-ip>:4242/api (empty = all flags off)
 #   UNLEASH_API_TOKEN      — Unleash *client* token, environment: development
-#   AIRFLOW_ADMIN_USER     — Airflow admin username (password auto-generated, see api-server logs)
+#   PAYLOAD_SECRET         — Payload CMS encryption secret. REQUIRED: long and
+#                            random, and DIFFERENT from production's. Sharing
+#                            it would let a staging session authenticate
+#                            against production.
+#   AIRFLOW_ADMIN_USER     — Airflow admin username (default: admin)
+#   AIRFLOW_ADMIN_PASSWORD — Airflow admin password. REQUIRED: the api-server
+#                            refuses to start without it, rather than falling
+#                            back to a generated one that changes on restart.
 #   STAGING_DB_IP          — macvlan IP for staging Postgres (default 10.0.1.190)
 #   STAGING_FRONTEND_IP    — macvlan IP for staging frontend (default 10.0.1.151)
 
@@ -86,9 +93,20 @@ services:
       - FASTAPI_URL=http://backend:80/api
       - TYPESENSE_URL=http://typesense:8108
       - TYPESENSE_API_KEY=${TYPESENSE_SEARCH_KEY:-changeme}
+      # Payload CMS runs inside this container, in the `payload` schema of the
+      # staging database. Staging has its own stack, its own Postgres and its
+      # own admin account — never production's.
+      - DATABASE_URL=postgresql://${DB_USERNAME}:${DB_PASSWORD}@sc_database:5432/${DB_DATABASE_NAME}
+      - PAYLOAD_SECRET=${PAYLOAD_SECRET:?set PAYLOAD_SECRET in the staging Portainer stack environment}
+    volumes:
+      # Portainer prefixes volume names with the stack name, so this is
+      # automatically isolated from production's media.
+      - payload_media:/app/media
     depends_on:
       backend:
         condition: service_healthy
+      sc_database:
+        condition: service_healthy
     networks:
       backend: {}
       macvlan:
@@ -124,7 +142,23 @@ services:
   airflow-api-server:
     image: privaterepo.sitaru.org/tudor/school_compare-pipeline:staging
     container_name: sc_staging_airflow_api
-    command: airflow api-server --port 8080
+    # The simple auth manager generates a random password on first start and
+    # writes it to a file, so every container restart invalidates the last one.
+    # Writing the file ourselves from an environment variable makes the login
+    # deterministic. Airflow does not generate anything when the file exists.
+    #
+    # Built with python rather than echo/printf so a password containing quotes,
+    # backslashes or spaces is escaped correctly by json.dumps. An unset
+    # AIRFLOW_ADMIN_PASSWORD raises KeyError and the container exits: falling
+    # back to a generated password would silently undo the point of this.
+    command:
+      - bash
+      - -c
+      - |
+        set -euo pipefail
+        mkdir -p /opt/airflow
+        python -c "import json, os, pathlib; pathlib.Path('/opt/airflow/simple_auth_manager_passwords.json').write_text(json.dumps({os.environ.get('AIRFLOW_ADMIN_USER', 'admin'): os.environ['AIRFLOW_ADMIN_PASSWORD']}))"
+        exec airflow api-server --port 8080
     ports:
       - "8081:8080"
     environment:
@@ -136,6 +170,8 @@ services:
       AIRFLOW__API_AUTH__JWT_SECRET: "school-compare-staging-airflow-jwt-secret-key-long-enough-for-sha512"
       AIRFLOW__API_AUTH__JWT_ISSUER: airflow
       AIRFLOW__CORE__SIMPLE_AUTH_MANAGER_USERS: "${AIRFLOW_ADMIN_USER:-admin}:admin"
+      AIRFLOW__CORE__SIMPLE_AUTH_MANAGER_PASSWORDS_FILE: /opt/airflow/simple_auth_manager_passwords.json
+      AIRFLOW_ADMIN_PASSWORD: ${AIRFLOW_ADMIN_PASSWORD:?set AIRFLOW_ADMIN_PASSWORD in the Portainer stack environment}
       AIRFLOW__LOGGING__BASE_LOG_FOLDER: /opt/airflow/logs
       PG_HOST: sc_database
       PG_PORT: "5432"
@@ -221,3 +257,4 @@ volumes:
   typesense_data:
   airflow_logs:
   unleash_cache:
+  payload_media:

docker-compose.portainer.yml

@@ -9,7 +9,13 @@
 #   TYPESENSE_SEARCH_KEY   — Typesense search-only key (exposed to frontend)
 #   UNLEASH_URL            — http://<unleash-ip>:4242/api (empty = all flags off)
 #   UNLEASH_API_TOKEN      — Unleash *client* token, environment: production
-#   AIRFLOW_ADMIN_USER     — Airflow admin username (password auto-generated, see api-server logs)
+#   PAYLOAD_SECRET         — Payload CMS encryption secret. REQUIRED: long and
+#                            random. Changing it invalidates every admin
+#                            session. Staging MUST use a different value.
+#   AIRFLOW_ADMIN_USER     — Airflow admin username (default: admin)
+#   AIRFLOW_ADMIN_PASSWORD — Airflow admin password. REQUIRED: the api-server
+#                            refuses to start without it, rather than falling
+#                            back to a generated one that changes on restart.
 
 services:
 
@@ -75,9 +81,21 @@ services:
       - FASTAPI_URL=http://backend:80/api
       - TYPESENSE_URL=http://typesense:8108
       - TYPESENSE_API_KEY=${TYPESENSE_SEARCH_KEY:-changeme}
+      # Payload CMS runs inside this container. It reaches Postgres over the
+      # `backend` network and keeps its tables in the `payload` schema, so no
+      # pipeline operation on `public` can touch blog content.
+      - DATABASE_URL=postgresql://${DB_USERNAME}:${DB_PASSWORD}@sc_database:5432/${DB_DATABASE_NAME}
+      # Same :? form as AIRFLOW_ADMIN_PASSWORD: refuse to start rather than
+      # boot with an empty secret and silently accept forged sessions.
+      - PAYLOAD_SECRET=${PAYLOAD_SECRET:?set PAYLOAD_SECRET in the Portainer stack environment}
+    volumes:
+      # Blog images. Not reproducible from the pipeline — must be backed up.
+      - payload_media:/app/media
     depends_on:
       backend:
         condition: service_healthy
+      sc_database:
+        condition: service_healthy
     networks:
       backend: {}
       macvlan:
@@ -113,7 +131,23 @@ services:
   airflow-api-server:
     image: privaterepo.sitaru.org/tudor/school_compare-pipeline:prod
     container_name: schoolcompare_airflow_api
-    command: airflow api-server --port 8080
+    # The simple auth manager generates a random password on first start and
+    # writes it to a file, so every container restart invalidates the last one.
+    # Writing the file ourselves from an environment variable makes the login
+    # deterministic. Airflow does not generate anything when the file exists.
+    #
+    # Built with python rather than echo/printf so a password containing quotes,
+    # backslashes or spaces is escaped correctly by json.dumps. An unset
+    # AIRFLOW_ADMIN_PASSWORD raises KeyError and the container exits: falling
+    # back to a generated password would silently undo the point of this.
+    command:
+      - bash
+      - -c
+      - |
+        set -euo pipefail
+        mkdir -p /opt/airflow
+        python -c "import json, os, pathlib; pathlib.Path('/opt/airflow/simple_auth_manager_passwords.json').write_text(json.dumps({os.environ.get('AIRFLOW_ADMIN_USER', 'admin'): os.environ['AIRFLOW_ADMIN_PASSWORD']}))"
+        exec airflow api-server --port 8080
     ports:
       - "8080:8080"
     environment:
@@ -125,6 +159,8 @@ services:
       AIRFLOW__API_AUTH__JWT_SECRET: "school-compare-airflow-jwt-secret-key-long-enough-for-sha512"
       AIRFLOW__API_AUTH__JWT_ISSUER: airflow
       AIRFLOW__CORE__SIMPLE_AUTH_MANAGER_USERS: "${AIRFLOW_ADMIN_USER:-admin}:admin"
+      AIRFLOW__CORE__SIMPLE_AUTH_MANAGER_PASSWORDS_FILE: /opt/airflow/simple_auth_manager_passwords.json
+      AIRFLOW_ADMIN_PASSWORD: ${AIRFLOW_ADMIN_PASSWORD:?set AIRFLOW_ADMIN_PASSWORD in the Portainer stack environment}
       AIRFLOW__LOGGING__BASE_LOG_FOLDER: /opt/airflow/logs
       PG_HOST: sc_database
       PG_PORT: "5432"
@@ -210,3 +246,4 @@ volumes:
   typesense_data:
   airflow_logs:
   unleash_cache:
+  payload_media:

docker-compose.yml

@@ -105,7 +105,23 @@ services:
   airflow-api-server:
     image: privaterepo.sitaru.org/tudor/school_compare-pipeline:latest
     container_name: schoolcompare_airflow_api
-    command: airflow api-server --port 8080
+    # The simple auth manager generates a random password on first start and
+    # writes it to a file, so every container restart invalidates the last one.
+    # Writing the file ourselves from an environment variable makes the login
+    # deterministic. Airflow does not generate anything when the file exists.
+    #
+    # Built with python rather than echo/printf so a password containing quotes,
+    # backslashes or spaces is escaped correctly by json.dumps. An unset
+    # AIRFLOW_ADMIN_PASSWORD raises KeyError and the container exits: falling
+    # back to a generated password would silently undo the point of this.
+    command:
+      - bash
+      - -c
+      - |
+        set -euo pipefail
+        mkdir -p /opt/airflow
+        python -c "import json, os, pathlib; pathlib.Path('/opt/airflow/simple_auth_manager_passwords.json').write_text(json.dumps({os.environ.get('AIRFLOW_ADMIN_USER', 'admin'): os.environ['AIRFLOW_ADMIN_PASSWORD']}))"
+        exec airflow api-server --port 8080
     ports:
       - "8080:8080"
     environment: &airflow-env
@@ -117,6 +133,8 @@ services:
       AIRFLOW__API_AUTH__JWT_SECRET: "school-compare-airflow-jwt-secret-key-long-enough-for-sha512"
       AIRFLOW__API_AUTH__JWT_ISSUER: airflow
       AIRFLOW__CORE__SIMPLE_AUTH_MANAGER_USERS: "admin:admin"
+      AIRFLOW__CORE__SIMPLE_AUTH_MANAGER_PASSWORDS_FILE: /opt/airflow/simple_auth_manager_passwords.json
+      AIRFLOW_ADMIN_PASSWORD: ${AIRFLOW_ADMIN_PASSWORD:-admin}
       PG_HOST: db
       PG_PORT: "5432"
       PG_USER: schoolcompare

docs/DEPLOY.md

@@ -98,6 +98,12 @@ fail the E2E gate. That's the point: staging absorbs the risk.
    pr-checks status checks (frontend, backend, builds, ai-review) to pass.
 5. **Bootstrap staging data via Airflow** (no prod dump — staging populates
    itself from source, exercising the pipeline image end-to-end):
+   - Set `AIRFLOW_ADMIN_PASSWORD` in the stack environment first. The
+     api-server refuses to start without it. Airflow's simple auth manager
+     otherwise generates a password on first start and writes it to a file, so
+     the login changes every time the container restarts; the stack writes that
+     file itself from this variable instead. `AIRFLOW_ADMIN_USER` defaults to
+     `admin`.
    - Open the staging Airflow UI (`http://<host>:8081`) and trigger, in order:
      `school_data_daily`, `school_data_monthly_ofsted`, then the manual-schedule
      `school_data_annual_ees` and `school_data_annual_idaci`.

docs/superpowers/specs/2026-09-02-about-and-blog-design.md

@@ -0,0 +1,368 @@
+# Giving schoolcompare a human author: an About page and a blog
+
+**Date:** 2026-09-02
+**Status:** Design — awaiting review
+**Scope:** A named author for the site, an `/about` page, and a Payload-CMS-backed
+blog at `/blog`.
+
+## Why
+
+The site reads as synthetic. Not because of its tone, but because of three
+specific absences:
+
+1. **Nobody is accountable for the numbers.** There is no author, no statement
+   of why the site exists, and no one who can be wrong. The only human trace on
+   the entire site is `contact@schoolcompare.co.uk` in the footer.
+2. **No visible judgement.** Every figure is presented as though it fell out of
+   a machine. Hundreds of editorial decisions went into this codebase — which
+   metrics to show, when a benchmark is invalid, what to suppress — and not one
+   of them is visible to a reader. `isSpecialSchool()` silently drops the
+   England comparison for special schools and PRUs because that comparison is
+   meaningless; nowhere does the site *say* so.
+3. **The voice is institutional third person.** "schoolcompare brings it all
+   into one place." "Built for parents, governors, journalists." That is
+   brochure register, and it is precisely the register that machine-generated
+   content defaults to.
+
+There is a second, independent reason. The SEO programme
+(`2026-08-20-seo-programme-design.md`) defines eight workstreams and none of
+them address E-E-A-T or authorship. School performance data is YMYL territory;
+an anonymous site republishing DfE figures has no authorship signal at all. This
+work fills that hole, and the blog gives W6 (explainer content) somewhere to
+live.
+
+### The failure mode to avoid
+
+The standard fix — a stock photo and "Hi, I'm Tudor, and I'm passionate about
+education!" — reads as *more* synthetic than the current coldness. Manufactured
+warmth is a stronger machine-tell than plain institutional voice. Everything
+here has to be specific, occasionally awkward, and willing to be unflattering,
+or it makes the problem worse.
+
+## Positioning
+
+The author is **Tudor**: first name only, real photograph, no surname, no
+employer named.
+
+The credibility claim is deliberately **not** educational expertise. The About
+page states plainly: *"I'm not an education expert."* Authority comes from two
+things that are actually true:
+
+- **Experience.** A parent going through primary admissions in south-west London
+  right now. Google's E-E-A-T leads with Experience, and lived experience of the
+  thing is exactly what the DfE's own service lacks.
+- **Method.** Every number's provenance is stated, so a reader can check the
+  site rather than trust it.
+
+This is more durable than borrowed expertise: it cannot be undermined by someone
+noticing the author has no teaching qualification.
+
+**Consequence for the design.** A `Person` entity with no surname is a weak
+search signal and cannot be corroborated off-site. The credibility load
+therefore shifts onto the methodology being visibly rigorous. That is a design
+constraint, not a caveat — it is why the About page carries a substantial
+"how this is built and where it can be wrong" section rather than a short bio.
+
+### Voice rules
+
+Applied to About and every post. Recorded here so the voice does not drift.
+
+- First person singular. "I built", not "we provide".
+- Concrete over general. "when we were looking at schools in Wandsworth" beats
+  any amount of stated warmth.
+- State limits before someone else finds them. Every post that presents a
+  metric says what it does not show.
+- No mission statements, no "passionate about", no invented team.
+- No em dashes. One of the clearest tells of machine-written prose, which is
+  the exact problem this work exists to fix.
+- Short sentences. The existing code comments in this repo are already written
+  this way; the prose should match.
+
+## Scope
+
+**In:**
+
+- `/about` — a coded page (not CMS-managed).
+- `/blog` and `/blog/[slug]` — Payload-backed, with an index and post pages.
+- Payload CMS installed into the existing Next application.
+- Footer and navigation links to both.
+- `Person`, `Organization`, `BlogPosting`, `BreadcrumbList` JSON-LD.
+- RSS feed and sitemap integration.
+- One first post, so the blog does not launch empty.
+
+**Out (deliberately):**
+
+- Rewriting existing homepage/how-it-works copy into first person. Worth doing,
+  but it would double the review surface of this PR. Separate change.
+- In-product signed notes on school pages (the "distributed humanity" idea).
+  Revisit once About and the blog exist.
+- Comments, newsletter, author accounts beyond one.
+- A team page. There is no team.
+
+## Architecture
+
+### Topology
+
+Payload 3 installs **into the existing Next application** and serves `/admin`
+from the same container. One image, one deploy, no new service. This is
+Payload 3's native model and it makes on-demand revalidation trivial, because
+the CMS hooks run in the same process as the Next cache.
+
+Accepted costs: the public site's image now carries Payload, so a CMS security
+patch redeploys the whole site; and the image grows substantially.
+
+### Two collisions that must be handled
+
+**1. `/api` is already taken.** `app/api/[...path]/route.ts` is a catch-all that
+proxies `/api/*` to FastAPI at runtime. Payload's default API route is also
+`/api`. Left alone, these fight, and the failure is not clean — the catch-all
+would swallow Payload's admin API calls and forward them to FastAPI.
+
+Payload's API route is therefore remapped:
+
+```ts
+routes: { api: '/cms-api', admin: '/admin' }
+```
+
+with its route group at `app/(payload)/cms-api/[...slug]/route.ts`. The
+`/cms-api` prefix must also be added to the FastAPI proxy's excluded-paths list
+as a defensive second line.
+
+**2. `next.config.js` is CommonJS.** Payload's `withPayload()` wrapper is ESM
+only. The config must become `next.config.mjs`, converting `module.exports` to
+`export default` and wrapping the export. All existing content — the standalone
+output, `outputFileTracingIncludes`, the staging `X-Robots-Tag` header block,
+the CSP — carries over unchanged. This is mechanical but it touches the file
+that controls staging's noindex, so it needs care and an explicit test.
+
+### Database
+
+Payload uses the existing `sc_database` Postgres instance, in its **own
+`payload` schema**:
+
+```ts
+db: postgresAdapter({
+  pool: { connectionString: process.env.DATABASE_URL },
+  schemaName: 'payload',
+})
+```
+
+The frontend container is already on the `backend` Docker network, so it can
+reach `sc_database:5432` with no networking change. It needs a new
+`DATABASE_URL` environment variable.
+
+Schema isolation is not cosmetic. `public` currently holds the application
+tables and Airflow's metadata, and `scripts/migrate_csv_to_db.py --drop` exists
+to drop and reimport. Blog content living in its own schema means no data
+pipeline operation can destroy it.
+
+**Verified 2026-09-02** (this was an open question when the spec was written).
+`--drop` calls `run_full_migration()` in `backend/migration.py`, which drops
+exactly two tables by name:
+
+```python
+ks2_tables = ["school_results", "schools"]
+for tname in ks2_tables:
+    if tname in existing:
+        Base.metadata.tables[tname].drop(bind=engine)
+```
+
+There is no `Base.metadata.drop_all()` anywhere in `backend/`, and no
+`DROP SCHEMA`. The only other drop is `_apply_schema_drops()`, a single
+schema-qualified `DROP TABLE IF EXISTS marts.fact_parent_view CASCADE`.
+Nothing sets `search_path`, so the SQLAlchemy metadata resolves to `public`,
+and `inspector.get_table_names()` does not even enumerate other schemas.
+
+So the guarantee is stronger than schema isolation alone: `--drop` targets two
+named tables that Payload does not have, and would not reach `posts`, `media`
+or `users` even if they shared a schema. The `payload` schema remains the right
+choice — it protects against a *future* broadening of that script rather than
+today's behaviour — but the safety claim rests on verified code, not on
+assumption.
+
+Putting CMS tables in this instance is consistent with existing practice —
+Airflow already stores its metadata there.
+
+### Migrations
+
+Payload's Postgres adapter auto-pushes schema in development and requires
+explicit migrations in production. Use `prodMigrations`, which runs pending
+migrations during server initialisation:
+
+```ts
+db: postgresAdapter({ /* ... */, prodMigrations: migrations })
+```
+
+This is preferred over a one-shot init container (the `airflow-init` pattern)
+because the app is a single long-running process and there is no ordering
+problem to solve. Migration files are generated with `payload migrate:create`
+and committed, so schema changes travel through the same PR and staging gate as
+code.
+
+### Media
+
+Uploads go to a Docker named volume, consistent with `postgres_data`,
+`typesense_data` and `airflow_logs`.
+
+- `staticDir` must be an **absolute** path in Payload 3: `/app/media`.
+- The container runs as `nextjs` (uid 1001). The Dockerfile must
+  `mkdir -p /app/media && chown nextjs:nodejs /app/media` **before** the volume
+  is mounted, or Docker will create the mountpoint root-owned and every upload
+  will fail with EACCES.
+- `sharp` moves from `devDependencies` to `dependencies` — Payload needs it at
+  runtime to generate `imageSizes`.
+- The volume must be added to the backup routine alongside Postgres. A blog
+  post's images are not reproducible from the pipeline.
+
+### Rendering
+
+**Constraint:** CI builds the image with no database reachable. Blog pages
+therefore cannot use build-time `generateStaticParams` — that would either fail
+the build or bake in an empty post list.
+
+Instead: ISR. Post and index pages declare a `revalidate` window and render on
+first request, with Payload `afterChange` / `afterDelete` hooks calling
+`revalidatePath('/blog')` and `revalidatePath('/blog/' + slug)` for immediate
+publication. Because Payload runs in the same process, the hook calls
+`revalidatePath` from `next/cache` directly — no webhook, no shared secret.
+
+The ISR cache lives on container disk and is cleared by a redeploy. For a
+single container serving a handful of posts this is fine.
+
+### Collections
+
+- **`posts`** — `title`, `slug`, `publishedAt`, `excerpt`, `heroImage`
+  (relation to `media`), `content` (Lexical rich text), `seo` group
+  (`metaTitle`, `metaDescription`), `_status` (drafts enabled).
+- **`media`** — upload collection, `alt` required, `imageSizes` for thumbnail
+  and hero widths, public read access.
+- **`users`** — Payload's auth collection. One account. Public creation
+  disabled.
+
+Drafts are enabled so posts can be written over several sittings and previewed
+before publication.
+
+**Payload Blocks** are how posts embed live product components — a real trend
+chart or comparison table inside a post, rendered from live data rather than
+screenshotted. This is the main thing the CMS has to earn back against
+file-based MDX, and it directly serves the goal: showing judgement in context.
+Ship with one block (a callout/aside for "what this number doesn't tell you");
+add a live-chart block once a post needs it.
+
+### Security
+
+`/admin` is the first authenticated surface on this site. Public, hardened:
+
+- `PAYLOAD_SECRET` — long, random, set in the Portainer stack environment, never
+  committed. The same variable must exist in staging with a *different* value.
+- Strong unique password on the single admin account.
+- Login rate limiting via Payload's `maxLoginAttempts` / `lockTime`.
+- `X-Robots-Tag: noindex, nofollow` on `/admin/*` and `/cms-api/*`, and a
+  `robots.ts` disallow. The admin panel must never be indexed.
+- Public user creation disabled; no open registration.
+- Verify the existing CSP `frame-ancestors` directive does not break the admin
+  panel.
+
+Residual risk, accepted: a future Payload authentication CVE is live against the
+public internet. Mitigation is prompt patching, which the staging→prod pipeline
+already supports. If this becomes uncomfortable, restricting `/admin` at the
+proxy to LAN/VPN is a one-line change later.
+
+Staging note: staging runs the same image on `stx.`, so it gets its own admin
+panel and its own database. It must have its own `PAYLOAD_SECRET` and its own
+credentials — never production's.
+
+## Deployment changes
+
+- `nextjs-app/Dockerfile` — create and chown `/app/media`; ensure Payload's
+  admin bundle and `sharp` survive standalone output file tracing.
+- `docker-compose.portainer.yml` and the staging equivalent — add
+  `DATABASE_URL` and `PAYLOAD_SECRET` to the `frontend` service, add a
+  `payload_media` volume mounted at `/app/media`, and add
+  `depends_on: sc_database`.
+- Document both new environment variables in the compose header comment block,
+  which is where this stack records its configuration.
+
+## SEO
+
+- `Person` (Tudor, with photo) and `Organization` JSON-LD on `/about`.
+- `BlogPosting` + `BreadcrumbList` on post pages, with `author` referencing the
+  same `Person`.
+- Canonical URLs on `/blog` and every post.
+- Posts and `/about` added to the existing sitemap (`app/sitemap.xml/route.ts`
+  and `app/sitemaps/[...parts]`). Post URLs come from Payload at request time.
+- RSS feed at `/blog/rss.xml`.
+- Footer links to both pages, under a new "About" column.
+
+**Navigation is deliberately left alone.** `Navigation.tsx` renders a bottom tab
+bar on mobile that already carries four items (Search, Compare, Rankings,
+Admissions). A fifth tab makes each one cramped at 320px, and About and Blog are
+both lower-intent than any of the four. Both live in the footer; About
+additionally gets a byline link from every post, which is where a reader who
+cares actually asks the question. Revisit only if analytics show people hunting
+for it.
+
+## Testing
+
+Unit (Jest):
+
+- Post rendering, including a post with no hero image and one with no excerpt.
+- Slug generation and collision handling.
+- JSON-LD shape for `BlogPosting` and `Person`.
+- The `next.config.mjs` conversion preserves the staging `X-Robots-Tag` rule —
+  this guards the riskiest mechanical change in the plan.
+
+E2E (Playwright, `e2e/`, required by CLAUDE.md for user-facing change):
+
+- `/about` renders, shows the author name and photo, and is reachable from the
+  footer and nav.
+- `/blog` lists at least one post; clicking through reaches the post.
+- A post page renders title, date, body and byline.
+- `/admin` responds with `noindex` and does not leak a stack trace when
+  unauthenticated.
+
+Note the known constraint: new journeys cannot be proven in PR checks, because
+the staging E2E gate runs post-merge.
+
+## Risks
+
+| Risk | Mitigation |
+|---|---|
+| `next.config.mjs` conversion silently drops the staging noindex header, making staging a crawlable duplicate | Unit test asserting the header rule; verify on staging before promotion |
+| Payload API route collides with the FastAPI `/api` proxy | Remap to `/cms-api`; add to the proxy's exclusion list |
+| Media volume mounts root-owned; all uploads fail with EACCES | `mkdir`+`chown` in the Dockerfile before the mount; test an upload on staging |
+| Build fails or bakes empty content because CI has no DB | No build-time DB access; ISR only |
+| A pipeline `--drop` destroys blog content | Separate `payload` schema; verify `--drop` blast radius before building |
+| Media volume not backed up; images unrecoverable | Add `payload_media` to the backup routine |
+| Payload auth CVE exposed publicly | Prompt patching; proxy restriction available as a fallback |
+| Blog launches empty or goes stale | Ship with one post; cadence is explicitly "a few times a year", so no cadence is promised anywhere on the page — no dates implying a schedule |
+
+## Sequence
+
+Each step is independently reviewable and mergeable.
+
+1. **Payload foundation** — install, `next.config.mjs` conversion, `payload`
+   schema, `/cms-api` remap, `users` collection, `/admin` hardening, compose and
+   Dockerfile changes. No public-facing change yet. Verify on staging that the
+   site is unchanged and `/admin` works.
+2. **`/about`** — coded page, photo, `Person`/`Organization` JSON-LD, footer and
+   nav links, e2e journey. Independently valuable and does not depend on the
+   blog.
+3. **Blog** — `posts` and `media` collections, `/blog` index and post pages, ISR
+   plus revalidation hooks, RSS, sitemap, structured data, e2e journeys.
+4. **First post** — written in the admin panel, published through the normal
+   flow, proving the whole path end to end.
+
+Step 1 carries all the infrastructure risk and none of the visible benefit, so
+it should be verified on staging carefully before step 2 starts.
+
+## Dependencies on Tudor
+
+- **A photograph.** Blocks step 2. Nothing else in the plan is blocked by it.
+- **The first post's subject.** Blocks step 4 only. Suggested: what school
+  performance data cannot tell you — it demonstrates judgement, is genuinely
+  useful, and is the kind of thing an anonymous or machine-written site will not
+  publish.
+- ~~Confirmation that `scripts/migrate_csv_to_db.py --drop` is schema-scoped.~~
+  **Resolved 2026-09-02** — verified in `backend/migration.py`; see the
+  Database section. No action needed.

e2e/tests/journeys.spec.ts

@@ -2554,3 +2554,71 @@ test('the destinations section never claims a pupil stayed at this school', asyn
   const text = (await section.textContent()) ?? '';
   expect(text).not.toMatch(/stayed on (here|at this school)/i);
 });
+
+/**
+ * The About page and the blog exist to give the site a named human author.
+ * These journeys assert the load-bearing parts of that — a name, a face, the
+ * honesty claim, and a resolvable Person entity — rather than exact copy,
+ * which will be edited.
+ */
+test('the about page names a human author and is reachable from the footer', async ({ page }) => {
+  await page.goto('/');
+  const aboutLink = page.locator('footer a[href="/about"]');
+  await expect(aboutLink).toBeVisible();
+  await aboutLink.click();
+  await page.waitForURL(/\/about$/);
+
+  await expect(page.getByRole('heading', { level: 1 })).toContainText('Tudor');
+  await expect(page.locator('img[alt*="Tudor"]')).toBeVisible();
+
+  // The credibility claim is lived experience plus stated provenance, not
+  // expertise. If this sentence ever disappears the positioning has drifted.
+  await expect(page.getByText(/not an education expert/i)).toBeVisible();
+
+  const jsonLd = await page
+    .locator('script[type="application/ld+json"]')
+    .first()
+    .textContent();
+  expect(jsonLd).toContain('"Person"');
+  // First name only — a surname here would be the one place it leaks.
+  expect(jsonLd).not.toMatch(/familyName/);
+});
+
+test('the blog lists posts and each one renders with a byline', async ({ page }) => {
+  await page.goto('/blog');
+  await expect(page.getByRole('heading', { level: 1 })).toBeVisible();
+
+  const postLinks = page.locator('a[href^="/blog/"]');
+  // Data invariant: staging must carry at least one published post. If this
+  // fails, the environment has no content rather than the code being broken.
+  expect(await postLinks.count()).toBeGreaterThan(0);
+
+  await postLinks.first().click();
+  await page.waitForURL(/\/blog\/.+/);
+  await expect(page.getByRole('heading', { level: 1 })).toBeVisible();
+  await expect(page.getByText(/^By Tudor/)).toBeVisible();
+
+  const jsonLd = await page
+    .locator('script[type="application/ld+json"]')
+    .first()
+    .textContent();
+  expect(jsonLd).toContain('"BlogPosting"');
+});
+
+test('the admin panel is not indexable', async ({ page }) => {
+  const response = await page.request.get('/admin');
+  expect(response.headers()['x-robots-tag']).toContain('noindex');
+});
+
+test('the content sitemap lists the about page and is advertised in robots', async ({ page }) => {
+  const sitemap = await page.request.get('/content-sitemap.xml');
+  expect(sitemap.ok()).toBeTruthy();
+  expect(await sitemap.text()).toContain('/about');
+
+  // The school corpus sitemap is proxied from FastAPI; this one is Next's.
+  // robots.txt must advertise both or the blog never gets discovered.
+  const robots = await page.request.get('/robots.txt');
+  const body = await robots.text();
+  expect(body).toContain('/sitemap.xml');
+  expect(body).toContain('/content-sitemap.xml');
+});

nextjs-app/.gitignore

@@ -39,3 +39,4 @@ yarn-error.log*
 # typescript
 *.tsbuildinfo
 next-env.d.ts
+

nextjs-app/Dockerfile

@@ -53,6 +53,13 @@ COPY --from=builder /app/.next/static ./.next/static
 # a miss here is a silent 500 on /opengraph-image, not a build failure.
 COPY --from=builder /app/assets ./assets
 
+# Payload writes uploads here, and the compose file mounts a named volume over
+# it. The directory must exist and be owned by the runtime user BEFORE the
+# mount: Docker seeds a fresh named volume from the image path, so a missing or
+# root-owned directory here makes every upload fail with EACCES at runtime,
+# long after the build passed. The chown below covers it.
+RUN mkdir -p /app/media
+
 # Set correct permissions
 RUN chown -R nextjs:nodejs /app
 

nextjs-app/__tests__/api/proxyDenylist.test.ts

@@ -8,7 +8,7 @@
 // environment provides — under jsdom this suite fails on import, not on an
 // assertion.
 import { NextRequest } from 'next/server';
-import { GET } from '@/app/api/[...path]/route';
+import { GET } from '@/app/(frontend)/api/[...path]/route';
 
 function request(path: string) {
   return new NextRequest(`http://localhost:3000/api/${path}`);

nextjs-app/__tests__/app/aboutMetadata.test.ts

@@ -0,0 +1,34 @@
+import { metadata } from '@/app/(frontend)/about/page';
+import { personJsonLd, organizationJsonLd } from '@/lib/jsonld';
+
+describe('/about metadata', () => {
+  it('canonicalises to the bare path', () => {
+    expect(metadata.alternates?.canonical)
+      .toBe('https://www.schoolcompare.co.uk/about');
+  });
+});
+
+describe('author structured data', () => {
+  it('describes a Person with a first name and a photo', () => {
+    const person = personJsonLd();
+    expect(person['@type']).toBe('Person');
+    expect(person.name).toBe('Tudor');
+    expect(person.image).toBe('https://www.schoolcompare.co.uk/brand/tudor.jpg');
+    expect(person.url).toBe('https://www.schoolcompare.co.uk/about');
+  });
+
+  it('never publishes a surname or an employer', () => {
+    // Author identity constraint: first name only. A surname here would be
+    // the one place it leaks, since JSON-LD is machine-read and archived.
+    const serialised = JSON.stringify(personJsonLd());
+    expect(serialised).not.toMatch(/familyName|Sitaru/i);
+    expect(serialised).not.toMatch(/worksFor|affiliation/i);
+  });
+
+  it('describes the site as an Organization the Person authors for', () => {
+    const org = organizationJsonLd();
+    expect(org['@type']).toBe('Organization');
+    expect(org.name).toBe('schoolcompare');
+    expect(org.url).toBe('https://www.schoolcompare.co.uk');
+  });
+});

nextjs-app/__tests__/app/blogMetadata.test.ts

@@ -0,0 +1,53 @@
+/**
+ * The blog index imports getCachedPayload, which pulls in Payload — ESM-only,
+ * and next/jest will not transform node_modules. Mocking that one module keeps
+ * the page's metadata testable without loading the CMS; the mock is never
+ * called, because `metadata` is a static export evaluated at import time.
+ */
+jest.mock('@/lib/payload', () => ({ getCachedPayload: jest.fn() }));
+
+import { metadata } from '@/app/(frontend)/blog/page';
+import { blogPostingJsonLd, breadcrumbJsonLd } from '@/lib/jsonld';
+
+const post = {
+  title: 'What the data cannot tell you',
+  slug: 'what-the-data-cannot-tell-you',
+  excerpt: 'Results describe one year group on a handful of days.',
+  publishedAt: '2026-09-15T00:00:00.000Z',
+};
+
+describe('/blog metadata', () => {
+  it('canonicalises to the bare path', () => {
+    expect(metadata.alternates?.canonical)
+      .toBe('https://www.schoolcompare.co.uk/blog');
+  });
+});
+
+describe('BlogPosting structured data', () => {
+  it('names the same Person entity the about page declares', () => {
+    // By @id, not by repeating the person: search engines must resolve every
+    // post and the about page to one author entity, or the site has several.
+    const ld = blogPostingJsonLd(post);
+    expect(ld['@type']).toBe('BlogPosting');
+    expect(ld.author['@id']).toBe('https://www.schoolcompare.co.uk/about#tudor');
+    expect(ld.publisher['@id']).toBe('https://www.schoolcompare.co.uk#organization');
+  });
+
+  it('carries a self-referencing canonical url and the publish date', () => {
+    const ld = blogPostingJsonLd(post);
+    expect(ld.url).toBe(
+      'https://www.schoolcompare.co.uk/blog/what-the-data-cannot-tell-you',
+    );
+    expect(ld.datePublished).toBe('2026-09-15T00:00:00.000Z');
+  });
+});
+
+describe('breadcrumbs', () => {
+  it('places the post under the blog index', () => {
+    const ld = breadcrumbJsonLd(post);
+    expect(ld.itemListElement[0].item).toBe('https://www.schoolcompare.co.uk/blog');
+    expect(ld.itemListElement[1].item).toBe(
+      'https://www.schoolcompare.co.uk/blog/what-the-data-cannot-tell-you',
+    );
+  });
+});

nextjs-app/__tests__/app/metadata.test.ts

@@ -1,7 +1,7 @@
-import { metadata as homeMetadata } from '@/app/page';
-import { metadata as rankingsMetadata } from '@/app/rankings/page';
-import { metadata as admissionsMetadata } from '@/app/admissions/page';
-import { generateMetadata as compareMetadata } from '@/app/compare/page';
+import { metadata as homeMetadata } from '@/app/(frontend)/page';
+import { metadata as rankingsMetadata } from '@/app/(frontend)/rankings/page';
+import { metadata as admissionsMetadata } from '@/app/(frontend)/admissions/page';
+import { generateMetadata as compareMetadata } from '@/app/(frontend)/compare/page';
 
 describe('canonical URLs', () => {
   it('the homepage canonicalises to the bare root', () => {

nextjs-app/__tests__/app/nextConfig.test.ts

@@ -0,0 +1,66 @@
+/**
+ * next.config.mjs carries the staging noindex rule. Breaking it turns
+ * stx.schoolcompare.co.uk into a fully crawlable duplicate of production,
+ * and nothing else in the suite would notice.
+ *
+ * The non-null assertions are deliberate: every key asserted here is optional
+ * on NextConfig, and a missing one is precisely the regression under test, so
+ * the assertion below should fail the test rather than the compile.
+ */
+import nextConfig from '@/next.config.mjs';
+
+async function headerRules() {
+  return nextConfig.headers!();
+}
+
+describe('next.config.mjs', () => {
+  it('keeps the staging host out of the index', async () => {
+    const headers = await headerRules();
+    const stagingRule = headers.find((rule) =>
+      rule.has?.some(
+        (cond) => cond.type === 'host' && cond.value === 'stx.schoolcompare.co.uk',
+      ),
+    );
+    expect(stagingRule).toBeDefined();
+    expect(stagingRule!.headers).toContainEqual({
+      key: 'X-Robots-Tag',
+      value: 'noindex, nofollow',
+    });
+  });
+
+  it('still emits standalone output for the Docker runner', () => {
+    expect(nextConfig.output).toBe('standalone');
+  });
+
+  it('still traces the share-card fonts into the standalone bundle', () => {
+    expect(nextConfig.outputFileTracingIncludes!['/opengraph-image']).toEqual([
+      './assets/**',
+    ]);
+  });
+
+  it('still allows the analytics subdomain to frame the site', async () => {
+    const headers = await headerRules();
+    const csp = headers
+      .flatMap((rule) => rule.headers)
+      .find((header) => header.key === 'Content-Security-Policy');
+    expect(csp).toBeDefined();
+    expect(csp!.value).toContain('https://analytics.schoolcompare.co.uk');
+  });
+});
+
+describe('admin surface', () => {
+  it('serves noindex on the admin panel and the CMS API', async () => {
+    // robots.txt disallows these too, but a Disallow only blocks crawling — a
+    // URL found from an external link can still be indexed without ever being
+    // fetched. This header is what actually keeps them out.
+    const headers = await headerRules();
+    for (const source of ['/admin/:path*', '/cms-api/:path*']) {
+      const rule = headers.find((entry) => entry.source === source);
+      expect(rule).toBeDefined();
+      expect(rule!.headers).toContainEqual({
+        key: 'X-Robots-Tag',
+        value: 'noindex, nofollow',
+      });
+    }
+  });
+});

nextjs-app/__tests__/app/placeMetadata.test.ts

@@ -1,4 +1,4 @@
-import { generateMetadata as placeMeta } from '@/app/schools/[place]/page';
+import { generateMetadata as placeMeta } from '@/app/(frontend)/schools/[place]/page';
 
 jest.mock('@/lib/places', () => ({
   ...jest.requireActual('@/lib/places'),

nextjs-app/__tests__/app/robots.test.ts

@@ -0,0 +1,20 @@
+import robots from '@/app/robots';
+
+describe('robots.txt', () => {
+  it('disallows the admin panel and the CMS API', () => {
+    const rules = robots().rules;
+    const rule = Array.isArray(rules) ? rules[0] : rules;
+    expect(rule.disallow).toEqual(
+      expect.arrayContaining(['/api/', '/_next/', '/admin/', '/cms-api/']),
+    );
+  });
+});
+
+describe('sitemap discovery', () => {
+  it('lists both the proxied school sitemap and the Next-owned content sitemap', () => {
+    expect(robots().sitemap).toEqual([
+      'https://www.schoolcompare.co.uk/sitemap.xml',
+      'https://www.schoolcompare.co.uk/content-sitemap.xml',
+    ]);
+  });
+});

nextjs-app/__tests__/components/darkThemeSafety.test.ts

@@ -109,7 +109,7 @@ describe('dark-theme safety', () => {
  * simply missed.
  */
 describe('third-party surfaces under themed text', () => {
-  const GLOBALS = path.join(__dirname, '..', '..', 'app', 'globals.css');
+  const GLOBALS = path.join(__dirname, '..', '..', 'app', '(frontend)', 'globals.css');
 
   /** Leaflet surfaces our own code writes token-coloured text onto. */
   const LEAFLET_POPUP_SURFACES = [
@@ -171,7 +171,7 @@ describe('third-party surfaces under themed text', () => {
  */
 describe('destination tokens', () => {
   const css = fs.readFileSync(
-    path.join(__dirname, '..', '..', 'app', 'globals.css'), 'utf8');
+    path.join(__dirname, '..', '..', 'app', '(frontend)', 'globals.css'), 'utf8');
 
   const TOKENS = [
     '--dest-sixthform', '--dest-sfcollege', '--dest-fecollege',

nextjs-app/__tests__/payload/collections.test.ts

@@ -0,0 +1,70 @@
+/**
+ * Payload is ESM-only and next/jest will not transform it, so the collections
+ * cannot be imported and their sanitised config inspected here (see
+ * lib/payloadRoutes.ts for the full reasoning). These assert the source of the
+ * collection definitions instead — enough to catch the settings whose loss is
+ * silent, and cheap. Behaviour is proved by the e2e journeys against staging.
+ */
+import fs from 'fs';
+import path from 'path';
+
+const read = (file: string) =>
+  fs.readFileSync(path.join(__dirname, '..', '..', 'collections', file), 'utf8');
+
+const POSTS = read('Posts.ts');
+const MEDIA = read('Media.ts');
+const CONFIG = fs.readFileSync(
+  path.join(__dirname, '..', '..', 'payload.config.ts'),
+  'utf8',
+);
+
+describe('posts collection', () => {
+  it('supports drafts, so saving is not publishing', () => {
+    expect(POSTS).toMatch(/drafts:\s*true/);
+  });
+
+  it('has a unique, indexed slug for stable URLs', () => {
+    const slugField = POSTS.slice(POSTS.indexOf("name: 'slug'"));
+    expect(slugField).toMatch(/unique:\s*true/);
+    expect(slugField).toMatch(/index:\s*true/);
+  });
+
+  it('hides drafts from anonymous readers at the access layer', () => {
+    // Payload's docs are explicit: "The `draft` argument alone does not
+    // restrict documents with _status: 'draft' from being returned by the
+    // API." The blog pages' where-clause is not enforcement — a direct GET
+    // /cms-api/posts would return unpublished drafts to anyone. Access
+    // control returning a query constraint is the only thing that stops it.
+    expect(POSTS).toMatch(/_status:\s*\{\s*equals:\s*'published'\s*\}/);
+    expect(POSTS).toMatch(/if\s*\(req\.user\)\s*return true/);
+  });
+
+  it('revalidates the post page when a post changes or is deleted', () => {
+    // /blog/[slug] is ISR — generated on first request and cached — so an edit
+    // to an already-published post would otherwise not appear until the
+    // revalidate window expired, up to an hour of a writer concluding that
+    // saving is broken. The index and feeds are force-dynamic and need no hook.
+    expect(POSTS).toContain('afterChange');
+    expect(POSTS).toContain('afterDelete');
+    expect(POSTS).toMatch(/revalidatePath\(`\/blog\/\$\{[^}]+\}`\)/);
+  });
+});
+
+describe('media collection', () => {
+  it('writes uploads to the mounted volume, by absolute path', () => {
+    // Must match the payload_media mount in docker-compose.portainer.yml.
+    // Payload 3 requires staticDir to be absolute.
+    expect(MEDIA).toMatch(/staticDir:\s*'\/app\/media'/);
+  });
+
+  it('requires alt text on every upload', () => {
+    const altField = MEDIA.slice(MEDIA.indexOf("name: 'alt'"));
+    expect(altField).toMatch(/required:\s*true/);
+  });
+});
+
+describe('payload config', () => {
+  it('registers every collection', () => {
+    expect(CONFIG).toMatch(/collections:\s*\[Users,\s*Posts,\s*Media\]/);
+  });
+});

nextjs-app/__tests__/payload/import-map.test.ts

@@ -0,0 +1,67 @@
+/**
+ * The admin panel does not import field components directly. Payload sends the
+ * client a *path* for each one — a richText field's is
+ * `@payloadcms/richtext-lexical/rsc#RscEntryLexicalField` — and resolves it
+ * through this generated map. An entry that is missing from the map is not an
+ * error the panel reports: the field simply does not render.
+ *
+ * That failure is quietly awful, because `required: true` is enforced on the
+ * server regardless. A writer gets a new-post form with no Content editor and
+ * a save that refuses on a field they were never shown.
+ *
+ * The map is generated by `npx payload generate:importmap`, so it drifts every
+ * time a field or a lexical feature is added and nobody re-runs it. These
+ * assert the entries the current config needs.
+ */
+import fs from 'fs';
+import path from 'path';
+
+const MAP = fs.readFileSync(
+  path.join(__dirname, '..', '..', 'app', '(payload)', 'admin', 'importMap.js'),
+  'utf8',
+);
+const POSTS = fs.readFileSync(
+  path.join(__dirname, '..', '..', 'collections', 'Posts.ts'),
+  'utf8',
+);
+
+describe('admin import map', () => {
+  it('resolves the richText field, so Content renders in the editor', () => {
+    // Guarded because Posts.content is required: without this entry the field
+    // is invisible and the post is unsaveable.
+    expect(POSTS).toMatch(/type:\s*'richText'/);
+    expect(MAP).toContain('@payloadcms/richtext-lexical/rsc#RscEntryLexicalField');
+  });
+
+  it('resolves the richText cell, so the list view can render the column', () => {
+    expect(MAP).toContain('@payloadcms/richtext-lexical/rsc#RscEntryLexicalCell');
+  });
+
+  it('resolves the diff component, which the drafts UI needs', () => {
+    // versions.drafts is on, so the panel offers version comparison.
+    expect(POSTS).toMatch(/drafts:\s*true/);
+    expect(MAP).toContain('@payloadcms/richtext-lexical/rsc#LexicalDiffComponent');
+  });
+
+  it('resolves BlocksFeature, so the Callout block is insertable', () => {
+    expect(POSTS).toContain('BlocksFeature');
+    expect(MAP).toContain('@payloadcms/richtext-lexical/client#BlocksFeatureClient');
+  });
+
+  it('resolves the default toolbar features the editor is built with', () => {
+    // defaultFeatures is spread into the editor config; each one contributes a
+    // client component the toolbar cannot render without.
+    for (const feature of [
+      'BoldFeatureClient',
+      'ItalicFeatureClient',
+      'HeadingFeatureClient',
+      'LinkFeatureClient',
+      'UploadFeatureClient',
+      'UnorderedListFeatureClient',
+      'OrderedListFeatureClient',
+      'InlineToolbarFeatureClient',
+    ]) {
+      expect(MAP).toContain(`@payloadcms/richtext-lexical/client#${feature}`);
+    }
+  });
+});

nextjs-app/__tests__/payload/migrations.test.ts

@@ -0,0 +1,52 @@
+/**
+ * The generated migration is schema-qualified to "payload" throughout but does
+ * not create that schema — `schemaName` says where tables go, it does not
+ * create anything. On staging and production, which have never run it, the
+ * whole migration fails with `schema "payload" does not exist`.
+ *
+ * The CREATE SCHEMA is therefore hand-added, which makes it exactly the kind
+ * of edit a regeneration silently discards. This is the guard.
+ */
+import fs from 'fs';
+import path from 'path';
+
+const DIR = path.join(__dirname, '..', '..', 'migrations');
+
+function migrationFiles() {
+  return fs
+    .readdirSync(DIR)
+    .filter((f) => f.endsWith('.ts') && f !== 'index.ts');
+}
+
+describe('payload migrations', () => {
+  it('ships at least one migration, so a container has tables to find', () => {
+    expect(migrationFiles().length).toBeGreaterThan(0);
+  });
+
+  it('creates the payload schema before creating anything in it', () => {
+    const initial = migrationFiles().find((f) => f.includes('initial'))!;
+    const sql = fs.readFileSync(path.join(DIR, initial), 'utf8');
+
+    expect(sql).toMatch(/CREATE SCHEMA IF NOT EXISTS "payload"/);
+
+    // Ordering matters: the schema must be created before the first object
+    // that lives in it, or the migration fails on its first statement.
+    expect(sql.indexOf('CREATE SCHEMA IF NOT EXISTS "payload"'))
+      .toBeLessThan(sql.indexOf('CREATE TABLE "payload"'));
+  });
+
+  it('creates the tables the app queries on boot', () => {
+    const initial = migrationFiles().find((f) => f.includes('initial'))!;
+    const sql = fs.readFileSync(path.join(DIR, initial), 'utf8');
+    for (const table of ['users', 'posts', '_posts_v', 'media', 'payload_migrations']) {
+      expect(sql).toContain(`CREATE TABLE "payload"."${table}"`);
+    }
+  });
+
+  it('is wired into the adapter, so it runs on server init', () => {
+    const config = fs.readFileSync(
+      path.join(__dirname, '..', '..', 'payload.config.ts'), 'utf8',
+    );
+    expect(config).toMatch(/prodMigrations:\s*migrations/);
+  });
+});

nextjs-app/__tests__/payload/routes.test.ts

@@ -0,0 +1,45 @@
+/**
+ * Guards the one thing about Payload's mounting that fails silently.
+ *
+ * payload.config.ts itself cannot be imported here — Payload is ESM-only and
+ * next/jest will not transform it — so this asserts the shared constants and
+ * that the config actually wires them in, by reading its source. The live
+ * proof that /api still reaches FastAPI is the e2e journeys, which call
+ * /api/schools against the running app.
+ */
+import fs from 'fs';
+import path from 'path';
+import { PAYLOAD_API_ROUTE, PAYLOAD_ADMIN_ROUTE } from '@/lib/payloadRoutes';
+
+const CONFIG = fs.readFileSync(
+  path.join(__dirname, '..', '..', 'payload.config.ts'),
+  'utf8',
+);
+
+describe('payload mount points', () => {
+  it('serves the CMS API from /cms-api, never /api', () => {
+    // /api is the FastAPI proxy's catch-all. Payload's default would be
+    // swallowed by it and forwarded to the backend, silently.
+    expect(PAYLOAD_API_ROUTE).toBe('/cms-api');
+    expect(PAYLOAD_API_ROUTE).not.toBe('/api');
+  });
+
+  it('serves the admin panel from /admin', () => {
+    expect(PAYLOAD_ADMIN_ROUTE).toBe('/admin');
+  });
+
+  it('wires both constants into the Payload config', () => {
+    expect(CONFIG).toContain('PAYLOAD_API_ROUTE');
+    expect(CONFIG).toContain('PAYLOAD_ADMIN_ROUTE');
+  });
+
+  it('never hardcodes a routes block that could drift from the constants', () => {
+    expect(CONFIG).not.toMatch(/routes:\s*\{[^}]*api:\s*['"]/);
+  });
+
+  it('isolates CMS tables in their own postgres schema', () => {
+    // Blog content must sit outside `public`, where the app tables, Airflow's
+    // metadata and scripts/migrate_csv_to_db.py --drop all live.
+    expect(CONFIG).toMatch(/schemaName:\s*['"]payload['"]/);
+  });
+});

nextjs-app/__tests__/support/renderSchoolDetail.tsx

@@ -21,7 +21,7 @@ import {
 import { nationalAveragesFixture } from './schoolFixtures';
 
 // The shell calls useComparison(), which throws outside the provider. In the
-// app this wrapper comes from app/layout.tsx.
+// app this wrapper comes from app/(frontend)/layout.tsx.
 function withProviders(ui: ReactNode) {
   return <ComparisonProvider>{ui}</ComparisonProvider>;
 }

nextjs-app/app/(frontend)/about/About.module.css

@@ -0,0 +1,82 @@
+.page {
+  max-width: 42rem;
+  margin: 0 auto;
+  padding: 2.5rem 1.25rem 4rem;
+}
+
+.header {
+  display: flex;
+  align-items: center;
+  gap: 1.25rem;
+  margin-bottom: 2rem;
+}
+
+.portrait {
+  border-radius: 50%;
+  border: 2px solid var(--border);
+  object-fit: cover;
+  flex-shrink: 0;
+}
+
+.kicker {
+  font-family: var(--font-ui);
+  font-size: 0.75rem;
+  font-weight: 600;
+  text-transform: uppercase;
+  letter-spacing: 0.06em;
+  color: var(--brand);
+  margin: 0 0 0.35rem;
+}
+
+.heading {
+  font-family: var(--font-display);
+  font-size: clamp(1.5rem, 4vw, 2rem);
+  font-weight: 700;
+  line-height: 1.2;
+  color: var(--text-primary);
+  margin: 0;
+}
+
+.subheading {
+  font-family: var(--font-display);
+  font-size: 1.15rem;
+  font-weight: 600;
+  color: var(--text-primary);
+  margin: 2.25rem 0 0.75rem;
+}
+
+.prose p {
+  font-family: var(--font-ui);
+  font-size: 1rem;
+  line-height: 1.7;
+  color: var(--text-secondary);
+  margin: 0 0 1.1rem;
+}
+
+/* The opening paragraph carries the page. Larger, and in the primary ink
+   rather than the secondary, so it reads as a voice rather than as body copy.
+
+   Must stay in the descendant form: `.prose p` scores (0,1,1) and would beat a
+   bare `.lede` at (0,1,0), so simplifying this selector silently reverts the
+   lede to ordinary body copy. */
+.prose .lede {
+  font-size: 1.125rem;
+  color: var(--text-primary);
+}
+
+.link {
+  color: var(--brand);
+  font-weight: 600;
+}
+
+.link:hover {
+  color: var(--brand-strong);
+}
+
+@media (max-width: 480px) {
+  .header {
+    flex-direction: column;
+    align-items: flex-start;
+    gap: 1rem;
+  }
+}

nextjs-app/app/(frontend)/about/page.tsx

@@ -0,0 +1,130 @@
+import type { Metadata } from 'next';
+import Image from 'next/image';
+import { absoluteUrl } from '@/lib/site';
+import { personJsonLd, organizationJsonLd } from '@/lib/jsonld';
+import styles from './About.module.css';
+
+export const metadata: Metadata = {
+  title: 'About',
+  description:
+    'Who builds schoolcompare, why it exists, and where its numbers come from.',
+  alternates: { canonical: absoluteUrl('/about') },
+};
+
+export default function AboutPage() {
+  const jsonLd = {
+    '@context': 'https://schema.org',
+    '@graph': [personJsonLd(), organizationJsonLd()],
+  };
+
+  return (
+    <div className={styles.page}>
+      <script
+        type="application/ld+json"
+        dangerouslySetInnerHTML={{ __html: JSON.stringify(jsonLd) }}
+      />
+
+      <header className={styles.header}>
+        <Image
+          src="/brand/tudor.jpg"
+          alt="Tudor, who builds schoolcompare"
+          width={96}
+          height={96}
+          className={styles.portrait}
+          priority
+        />
+        <div>
+          <p className={styles.kicker}>Who&apos;s behind this</p>
+          <h1 className={styles.heading}>I&apos;m Tudor. I built this site.</h1>
+        </div>
+      </header>
+
+      <div className={styles.prose}>
+        <p className={styles.lede}>
+          I&apos;m a parent in south-west London. When we started looking at
+          primary schools, I found the information I needed was all published,
+          and almost impossible to hold in one place.
+        </p>
+
+        <p>
+          SATs results were in one government table. Ofsted judgements were in a
+          separate service, in a format that had just changed. Admissions
+          distances were buried in council PDFs, a different one per borough,
+          each with its own layout. I ended up building a spreadsheet, and then
+          I got tired of the spreadsheet.
+        </p>
+
+        <p>
+          So I built this instead. It pulls the official figures into one place
+          and puts them side by side, which is what I wanted and could not find.
+        </p>
+
+        <h2 className={styles.subheading}>I&apos;m not an education expert</h2>
+
+        <p>
+          I want to be straightforward about that. I&apos;m not a teacher, a
+          governor, or an education researcher. I have no qualification that
+          makes my opinion about a school worth more than yours.
+        </p>
+
+        <p>
+          What I do have is the problem itself. I&apos;m going through primary
+          admissions right now, and I work with data for a living. That
+          combination is enough to take published figures and present them
+          honestly. It is not enough to tell you which school is right for your
+          child, and this site never tries to.
+        </p>
+
+        <h2 className={styles.subheading}>Where the numbers come from</h2>
+
+        <p>
+          Everything here is official published data: Key Stage 2 and Key Stage
+          4 results and school characteristics from the Department for
+          Education, inspection outcomes from Ofsted, and admissions data from
+          local authorities. Nothing is estimated, modelled or filled in. Where
+          a figure is missing, the page says so rather than showing a guess.
+        </p>
+
+        <p>
+          This is an independent site. It is not affiliated with the Department
+          for Education or with Ofsted, and nobody pays to appear on it or to
+          rank higher.
+        </p>
+
+        <h2 className={styles.subheading}>What the data can&apos;t tell you</h2>
+
+        <p>
+          A school is not its results. The figures here describe one year group,
+          on a handful of days, measured in a way that suits national statistics
+          rather than your child. A small cohort makes percentages swing wildly.
+          In a class of thirty, one pupil is worth more than three points.
+          Results say nothing at all about whether a child will be happy
+          somewhere.
+        </p>
+
+        <p>
+          I try to build that honesty into the site rather than just say it
+          here. Special schools and pupil referral units are never compared
+          against a mainstream national average, because that comparison is
+          meaningless and makes good schools look like failing ones. Where a
+          number is unreliable, the aim is for the page to tell you before you
+          draw a conclusion from it.
+        </p>
+
+        <h2 className={styles.subheading}>If something&apos;s wrong</h2>
+
+        <p>
+          Tell me and I&apos;ll fix it. If a figure looks wrong, or a page gives
+          a misleading impression of a school, I genuinely want to know.
+          It&apos;s the fastest way this gets better.
+        </p>
+
+        <p>
+          <a href="mailto:contact@schoolcompare.co.uk" className={styles.link}>
+            contact@schoolcompare.co.uk
+          </a>
+        </p>
+      </div>
+    </div>
+  );
+}

nextjs-app/app/(frontend)/blog/Blog.module.css

@@ -0,0 +1,76 @@
+.page {
+  max-width: 42rem;
+  margin: 0 auto;
+  padding: 2.5rem 1.25rem 4rem;
+}
+
+.header { margin-bottom: 2.5rem; }
+
+.kicker {
+  font-family: var(--font-ui);
+  font-size: 0.75rem;
+  font-weight: 600;
+  text-transform: uppercase;
+  letter-spacing: 0.06em;
+  color: var(--brand);
+  margin: 0 0 0.35rem;
+}
+
+.heading {
+  font-family: var(--font-display);
+  font-size: clamp(1.5rem, 4vw, 2rem);
+  font-weight: 700;
+  line-height: 1.2;
+  color: var(--text-primary);
+  margin: 0 0 0.75rem;
+}
+
+.standfirst {
+  font-family: var(--font-ui);
+  font-size: 1.05rem;
+  line-height: 1.65;
+  color: var(--text-secondary);
+  margin: 0;
+}
+
+.list { list-style: none; padding: 0; margin: 0; }
+
+.item {
+  padding: 1.5rem 0;
+  border-top: 1px solid var(--border);
+}
+
+.date {
+  font-family: var(--font-ui);
+  font-size: 0.8rem;
+  color: var(--text-muted);
+  /* Inter's tabular numerals keep a column of dates aligned. */
+  font-variant-numeric: tabular-nums;
+}
+
+.itemTitle {
+  font-family: var(--font-display);
+  font-size: 1.25rem;
+  font-weight: 600;
+  line-height: 1.3;
+  margin: 0.35rem 0 0.5rem;
+}
+
+.itemLink { color: var(--text-primary); text-decoration: none; }
+.itemLink:hover { color: var(--brand); }
+
+.excerpt {
+  font-family: var(--font-ui);
+  font-size: 0.95rem;
+  line-height: 1.65;
+  color: var(--text-secondary);
+  margin: 0;
+}
+
+.empty {
+  font-family: var(--font-ui);
+  color: var(--text-muted);
+}
+
+.link { color: var(--brand); font-weight: 600; }
+.link:hover { color: var(--brand-strong); }

nextjs-app/app/(frontend)/blog/[slug]/Post.module.css

@@ -0,0 +1,87 @@
+.page {
+  max-width: 42rem;
+  margin: 0 auto;
+  padding: 2.5rem 1.25rem 4rem;
+}
+
+.crumb {
+  font-family: var(--font-ui);
+  font-size: 0.85rem;
+  margin-bottom: 1.25rem;
+}
+
+.heading {
+  font-family: var(--font-display);
+  font-size: clamp(1.6rem, 5vw, 2.25rem);
+  font-weight: 700;
+  line-height: 1.2;
+  color: var(--text-primary);
+  margin: 0 0 0.75rem;
+}
+
+.byline {
+  font-family: var(--font-ui);
+  font-size: 0.9rem;
+  color: var(--text-muted);
+  margin: 0 0 2rem;
+}
+
+.hero {
+  width: 100%;
+  height: auto;
+  border-radius: 10px;
+  border: 1px solid var(--border);
+  margin-bottom: 2rem;
+}
+
+/* Rich-text output: the editor emits plain elements, so these are styled by
+   descendant selector rather than by class. */
+.prose p {
+  font-family: var(--font-ui);
+  font-size: 1rem;
+  line-height: 1.7;
+  color: var(--text-secondary);
+  margin: 0 0 1.1rem;
+}
+
+.prose h2 {
+  font-family: var(--font-display);
+  font-size: 1.25rem;
+  font-weight: 600;
+  color: var(--text-primary);
+  margin: 2.25rem 0 0.75rem;
+}
+
+.prose h3 {
+  font-family: var(--font-display);
+  font-size: 1.05rem;
+  font-weight: 600;
+  color: var(--text-primary);
+  margin: 1.75rem 0 0.6rem;
+}
+
+.prose ul,
+.prose ol {
+  font-family: var(--font-ui);
+  font-size: 1rem;
+  line-height: 1.7;
+  color: var(--text-secondary);
+  padding-left: 1.35rem;
+  margin: 0 0 1.1rem;
+}
+
+.prose li { margin-bottom: 0.4rem; }
+
+.prose a { color: var(--brand); font-weight: 500; }
+.prose a:hover { color: var(--brand-strong); }
+
+.prose blockquote {
+  border-left: 3px solid var(--border-strong);
+  padding-left: 1rem;
+  margin: 1.5rem 0;
+  color: var(--text-muted);
+  font-style: italic;
+}
+
+.link { color: var(--brand); font-weight: 600; }
+.link:hover { color: var(--brand-strong); }

nextjs-app/app/(frontend)/blog/[slug]/page.tsx

@@ -0,0 +1,176 @@
+import { cache } from 'react';
+import type { Metadata } from 'next';
+import Link from 'next/link';
+import { notFound } from 'next/navigation';
+import { RichText } from '@payloadcms/richtext-lexical/react';
+import type { JSXConvertersFunction } from '@payloadcms/richtext-lexical/react';
+import { getCachedPayload } from '@/lib/payload';
+import type { Post, Media } from '@/payload-types';
+import { absoluteUrl } from '@/lib/site';
+import {
+  blogPostingJsonLd,
+  breadcrumbJsonLd,
+  personJsonLd,
+  organizationJsonLd,
+} from '@/lib/jsonld';
+import { CalloutBlock } from '@/components/blog/CalloutBlock';
+import styles from './Post.module.css';
+
+/*
+ * ISR. Unlike the index, this route has a dynamic param and no
+ * generateStaticParams, so there is nothing for the build to prerender: each
+ * post is generated on first request and cached until the collection's
+ * afterChange hook revalidates it. That hook is what makes an edit to an
+ * already-published post appear immediately.
+ */
+export const revalidate = 3600;
+
+/**
+ * heroImage is `number | Media | null`: an id when the query is shallow, the
+ * populated document at depth 1. Both pages query at depth 1, but narrowing
+ * rather than asserting keeps it correct if that ever changes.
+ */
+function heroOf(post: Post): Media | null {
+  return typeof post.heroImage === 'object' && post.heroImage !== null
+    ? post.heroImage
+    : null;
+}
+
+/**
+ * Spreads the default converters and adds the one custom block.
+ *
+ * Without the spread, every default node type — paragraphs, headings, links —
+ * loses its renderer and the post body comes out empty.
+ */
+const calloutConverters: JSXConvertersFunction = ({ defaultConverters }) => ({
+  ...defaultConverters,
+  blocks: {
+    // Annotated because the generic block converter cannot infer a custom
+    // block's field shape; String() guards the values regardless.
+    callout: ({ node }: { node: { fields: Record<string, unknown> } }) => (
+      <CalloutBlock
+        tone={String(node.fields.tone ?? 'caveat')}
+        body={String(node.fields.body ?? '')}
+      />
+    ),
+  },
+});
+
+/**
+ * Wrapped in React's cache() because Next calls generateMetadata and the page
+ * component separately for the same request — without it, every post view runs
+ * this query against Postgres twice. cache() dedupes within a single request
+ * only, so it never serves one visitor's request from another's.
+ */
+const findPost = cache(async (slug: string) => {
+  const payload = await getCachedPayload();
+  const { docs } = await payload.find({
+    collection: 'posts',
+    where: { slug: { equals: slug }, _status: { equals: 'published' } },
+    limit: 1,
+    depth: 1,
+  });
+  return docs[0] ?? null;
+});
+
+function summarise(post: Post) {
+  return {
+    title: post.title,
+    slug: post.slug,
+    excerpt: post.excerpt,
+    publishedAt: post.publishedAt,
+  };
+}
+
+export async function generateMetadata(
+  { params }: { params: Promise<{ slug: string }> },
+): Promise<Metadata> {
+  const { slug } = await params;
+  const post = await findPost(slug);
+  if (!post) return { title: 'Not found' };
+
+  const hero = heroOf(post);
+
+  return {
+    title: post.title,
+    description: post.excerpt,
+    alternates: { canonical: absoluteUrl(`/blog/${post.slug}`) },
+    openGraph: {
+      type: 'article',
+      title: post.title,
+      description: post.excerpt,
+      url: absoluteUrl(`/blog/${post.slug}`),
+      publishedTime: post.publishedAt,
+      // A post with a hero image shares that; one without falls through to the
+      // generated share card at app/opengraph-image.tsx.
+      ...(hero?.url ? { images: [{ url: hero.url }] } : {}),
+    },
+  };
+}
+
+export default async function PostPage(
+  { params }: { params: Promise<{ slug: string }> },
+) {
+  const { slug } = await params;
+  const post = await findPost(slug);
+  if (!post) notFound();
+
+  const summary = summarise(post);
+  const hero = heroOf(post);
+
+  const jsonLd = {
+    '@context': 'https://schema.org',
+    '@graph': [
+      blogPostingJsonLd(summary),
+      breadcrumbJsonLd(summary),
+      personJsonLd(),
+      organizationJsonLd(),
+    ],
+  };
+
+  return (
+    <article className={styles.page}>
+      <script
+        type="application/ld+json"
+        dangerouslySetInnerHTML={{ __html: JSON.stringify(jsonLd) }}
+      />
+
+      <nav className={styles.crumb}>
+        <Link href="/blog" className={styles.link}>Blog</Link>
+      </nav>
+
+      <h1 className={styles.heading}>{summary.title}</h1>
+
+      <p className={styles.byline}>
+        By <Link href="/about" className={styles.link}>Tudor</Link>
+        {' · '}
+        <time dateTime={summary.publishedAt}>
+          {new Date(summary.publishedAt).toLocaleDateString('en-GB', {
+            day: 'numeric',
+            month: 'long',
+            year: 'numeric',
+          })}
+        </time>
+      </p>
+
+      {/*
+        A plain <img>, not next/image: Payload already generated the sized
+        derivatives on upload (Media's imageSizes), so routing it through the
+        optimizer would resize an image that is already the right size.
+      */}
+      {hero?.url && (
+        <img
+          className={styles.hero}
+          src={hero.url}
+          alt={hero.alt ?? ''}
+          width={hero.width ?? undefined}
+          height={hero.height ?? undefined}
+        />
+      )}
+
+      <div className={styles.prose}>
+        <RichText data={post.content} converters={calloutConverters} />
+      </div>
+    </article>
+  );
+}

nextjs-app/app/(frontend)/blog/page.tsx

@@ -0,0 +1,76 @@
+import type { Metadata } from 'next';
+import Link from 'next/link';
+import { getCachedPayload } from '@/lib/payload';
+import { absoluteUrl } from '@/lib/site';
+import styles from './Blog.module.css';
+
+/*
+ * Dynamic, not ISR.
+ *
+ * This route has no dynamic params, so Next prerenders it at build time — and
+ * CI builds the image with no database reachable, which fails the build. It is
+ * a single indexed query against Postgres on the same Docker network, so
+ * rendering per request is cheap, and it means a newly published post appears
+ * here immediately rather than waiting on a revalidation.
+ */
+export const dynamic = 'force-dynamic';
+
+export const metadata: Metadata = {
+  title: 'Blog',
+  description:
+    'Notes on what school performance data shows, and what it does not.',
+  alternates: { canonical: absoluteUrl('/blog') },
+};
+
+function formatDate(value: string) {
+  return new Date(value).toLocaleDateString('en-GB', {
+    day: 'numeric',
+    month: 'long',
+    year: 'numeric',
+  });
+}
+
+export default async function BlogIndexPage() {
+  const payload = await getCachedPayload();
+  const { docs } = await payload.find({
+    collection: 'posts',
+    where: { _status: { equals: 'published' } },
+    sort: '-publishedAt',
+    limit: 50,
+    depth: 0,
+  });
+
+  return (
+    <div className={styles.page}>
+      <header className={styles.header}>
+        <p className={styles.kicker}>Blog</p>
+        <h1 className={styles.heading}>Notes on the numbers</h1>
+        <p className={styles.standfirst}>
+          What school performance data shows, what it doesn&apos;t, and how to
+          read it without being misled. Written by{' '}
+          <Link href="/about" className={styles.link}>Tudor</Link>.
+        </p>
+      </header>
+
+      {docs.length === 0 ? (
+        <p className={styles.empty}>No posts yet.</p>
+      ) : (
+        <ul className={styles.list}>
+          {docs.map((post) => (
+            <li key={post.id} className={styles.item}>
+              <time className={styles.date} dateTime={String(post.publishedAt)}>
+                {formatDate(String(post.publishedAt))}
+              </time>
+              <h2 className={styles.itemTitle}>
+                <Link href={`/blog/${post.slug}`} className={styles.itemLink}>
+                  {post.title}
+                </Link>
+              </h2>
+              <p className={styles.excerpt}>{post.excerpt}</p>
+            </li>
+          ))}
+        </ul>
+      )}
+    </div>
+  );
+}

nextjs-app/app/(frontend)/blog/rss.xml/route.ts

@@ -0,0 +1,52 @@
+import { getCachedPayload } from '@/lib/payload';
+import { absoluteUrl } from '@/lib/site';
+
+/*
+ * Dynamic, not ISR.
+ *
+ * This route has no dynamic params, so Next prerenders it at build time — and
+ * CI builds the image with no database reachable, which fails the build. It is
+ * a single indexed query against Postgres on the same Docker network, so
+ * rendering per request is cheap, and it means a newly published post appears
+ * here immediately rather than waiting on a revalidation.
+ */
+export const dynamic = 'force-dynamic';
+
+function escapeXml(value: string): string {
+  return value.replace(/[<>&'"]/g, (char) =>
+    ({ '<': '&lt;', '>': '&gt;', '&': '&amp;', "'": '&apos;', '"': '&quot;' }[char]!));
+}
+
+export async function GET() {
+  const payload = await getCachedPayload();
+  const { docs } = await payload.find({
+    collection: 'posts',
+    where: { _status: { equals: 'published' } },
+    sort: '-publishedAt',
+    limit: 50,
+    depth: 0,
+  });
+
+  const items = docs.map((post) => `
+    <item>
+      <title>${escapeXml(String(post.title))}</title>
+      <link>${absoluteUrl(`/blog/${post.slug}`)}</link>
+      <guid isPermaLink="true">${absoluteUrl(`/blog/${post.slug}`)}</guid>
+      <description>${escapeXml(String(post.excerpt))}</description>
+      <pubDate>${new Date(String(post.publishedAt)).toUTCString()}</pubDate>
+    </item>`).join('');
+
+  const xml = `<?xml version="1.0" encoding="UTF-8"?>
+<rss version="2.0">
+  <channel>
+    <title>schoolcompare blog</title>
+    <link>${absoluteUrl('/blog')}</link>
+    <description>What school performance data shows, and what it does not.</description>
+    <language>en-GB</language>${items}
+  </channel>
+</rss>`;
+
+  return new Response(xml, {
+    headers: { 'Content-Type': 'application/rss+xml; charset=utf-8' },
+  });
+}

nextjs-app/app/(frontend)/content-sitemap.xml/route.ts

@@ -0,0 +1,52 @@
+/*
+ * A second sitemap for the URLs Next owns.
+ *
+ * /sitemap.xml is proxied from FastAPI (app/(frontend)/sitemap.xml), which
+ * knows nothing about Payload — the backend and frontend ship as separate
+ * images. Rather than teach it, the Next-owned URLs get their own sitemap and
+ * robots.txt lists both.
+ */
+import { getCachedPayload } from '@/lib/payload';
+import { absoluteUrl } from '@/lib/site';
+
+/*
+ * Dynamic, not ISR.
+ *
+ * This route has no dynamic params, so Next prerenders it at build time — and
+ * CI builds the image with no database reachable, which fails the build. It is
+ * a single indexed query against Postgres on the same Docker network, so
+ * rendering per request is cheap, and it means a newly published post appears
+ * here immediately rather than waiting on a revalidation.
+ */
+export const dynamic = 'force-dynamic';
+
+export async function GET() {
+  const payload = await getCachedPayload();
+  const { docs } = await payload.find({
+    collection: 'posts',
+    where: { _status: { equals: 'published' } },
+    sort: '-publishedAt',
+    limit: 500,
+    depth: 0,
+  });
+
+  const urls: Array<{ loc: string; lastmod: string | null }> = [
+    { loc: absoluteUrl('/about'), lastmod: null },
+    { loc: absoluteUrl('/blog'), lastmod: null },
+    ...docs.map((post) => ({
+      loc: absoluteUrl(`/blog/${post.slug}`),
+      lastmod: new Date(String(post.updatedAt ?? post.publishedAt)).toISOString(),
+    })),
+  ];
+
+  const xml = `<?xml version="1.0" encoding="UTF-8"?>
+<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
+${urls.map(({ loc, lastmod }) =>
+  `  <url><loc>${loc}</loc>${lastmod ? `<lastmod>${lastmod}</lastmod>` : ''}</url>`,
+).join('\n')}
+</urlset>`;
+
+  return new Response(xml, {
+    headers: { 'Content-Type': 'application/xml; charset=utf-8' },
+  });
+}

nextjs-app/app/(payload)/admin/[[...segments]]/not-found.tsx

@@ -0,0 +1,16 @@
+import type { Metadata } from 'next';
+import config from '@payload-config';
+import { NotFoundPage, generatePageMetadata } from '@payloadcms/next/views';
+import { importMap } from '../importMap.js';
+
+type Args = {
+  params: Promise<{ segments: string[] }>;
+  searchParams: Promise<{ [key: string]: string | string[] }>;
+};
+
+export const generateMetadata = ({ params, searchParams }: Args): Promise<Metadata> =>
+  generatePageMetadata({ config, params, searchParams });
+
+export default function NotFound({ params, searchParams }: Args) {
+  return NotFoundPage({ config, importMap, params, searchParams });
+}

nextjs-app/app/(payload)/admin/[[...segments]]/page.tsx

@@ -0,0 +1,16 @@
+import type { Metadata } from 'next';
+import config from '@payload-config';
+import { RootPage, generatePageMetadata } from '@payloadcms/next/views';
+import { importMap } from '../importMap.js';
+
+type Args = {
+  params: Promise<{ segments: string[] }>;
+  searchParams: Promise<{ [key: string]: string | string[] }>;
+};
+
+export const generateMetadata = ({ params, searchParams }: Args): Promise<Metadata> =>
+  generatePageMetadata({ config, params, searchParams });
+
+export default function Page({ params, searchParams }: Args) {
+  return RootPage({ config, importMap, params, searchParams });
+}

nextjs-app/app/(payload)/admin/importMap.js

@@ -0,0 +1,54 @@
+import { RscEntryLexicalCell as RscEntryLexicalCell_44fe37237e0ebf4470c9990d8cb7b07e } from '@payloadcms/richtext-lexical/rsc'
+import { RscEntryLexicalField as RscEntryLexicalField_44fe37237e0ebf4470c9990d8cb7b07e } from '@payloadcms/richtext-lexical/rsc'
+import { LexicalDiffComponent as LexicalDiffComponent_44fe37237e0ebf4470c9990d8cb7b07e } from '@payloadcms/richtext-lexical/rsc'
+import { BlocksFeatureClient as BlocksFeatureClient_e70f5e05f09f93e00b997edb1ef0c864 } from '@payloadcms/richtext-lexical/client'
+import { BoldFeatureClient as BoldFeatureClient_e70f5e05f09f93e00b997edb1ef0c864 } from '@payloadcms/richtext-lexical/client'
+import { ItalicFeatureClient as ItalicFeatureClient_e70f5e05f09f93e00b997edb1ef0c864 } from '@payloadcms/richtext-lexical/client'
+import { UnderlineFeatureClient as UnderlineFeatureClient_e70f5e05f09f93e00b997edb1ef0c864 } from '@payloadcms/richtext-lexical/client'
+import { StrikethroughFeatureClient as StrikethroughFeatureClient_e70f5e05f09f93e00b997edb1ef0c864 } from '@payloadcms/richtext-lexical/client'
+import { SubscriptFeatureClient as SubscriptFeatureClient_e70f5e05f09f93e00b997edb1ef0c864 } from '@payloadcms/richtext-lexical/client'
+import { SuperscriptFeatureClient as SuperscriptFeatureClient_e70f5e05f09f93e00b997edb1ef0c864 } from '@payloadcms/richtext-lexical/client'
+import { InlineCodeFeatureClient as InlineCodeFeatureClient_e70f5e05f09f93e00b997edb1ef0c864 } from '@payloadcms/richtext-lexical/client'
+import { ParagraphFeatureClient as ParagraphFeatureClient_e70f5e05f09f93e00b997edb1ef0c864 } from '@payloadcms/richtext-lexical/client'
+import { HeadingFeatureClient as HeadingFeatureClient_e70f5e05f09f93e00b997edb1ef0c864 } from '@payloadcms/richtext-lexical/client'
+import { AlignFeatureClient as AlignFeatureClient_e70f5e05f09f93e00b997edb1ef0c864 } from '@payloadcms/richtext-lexical/client'
+import { IndentFeatureClient as IndentFeatureClient_e70f5e05f09f93e00b997edb1ef0c864 } from '@payloadcms/richtext-lexical/client'
+import { UnorderedListFeatureClient as UnorderedListFeatureClient_e70f5e05f09f93e00b997edb1ef0c864 } from '@payloadcms/richtext-lexical/client'
+import { OrderedListFeatureClient as OrderedListFeatureClient_e70f5e05f09f93e00b997edb1ef0c864 } from '@payloadcms/richtext-lexical/client'
+import { ChecklistFeatureClient as ChecklistFeatureClient_e70f5e05f09f93e00b997edb1ef0c864 } from '@payloadcms/richtext-lexical/client'
+import { LinkFeatureClient as LinkFeatureClient_e70f5e05f09f93e00b997edb1ef0c864 } from '@payloadcms/richtext-lexical/client'
+import { RelationshipFeatureClient as RelationshipFeatureClient_e70f5e05f09f93e00b997edb1ef0c864 } from '@payloadcms/richtext-lexical/client'
+import { BlockquoteFeatureClient as BlockquoteFeatureClient_e70f5e05f09f93e00b997edb1ef0c864 } from '@payloadcms/richtext-lexical/client'
+import { UploadFeatureClient as UploadFeatureClient_e70f5e05f09f93e00b997edb1ef0c864 } from '@payloadcms/richtext-lexical/client'
+import { HorizontalRuleFeatureClient as HorizontalRuleFeatureClient_e70f5e05f09f93e00b997edb1ef0c864 } from '@payloadcms/richtext-lexical/client'
+import { InlineToolbarFeatureClient as InlineToolbarFeatureClient_e70f5e05f09f93e00b997edb1ef0c864 } from '@payloadcms/richtext-lexical/client'
+import { CollectionCards as CollectionCards_f9c02e79a4aed9a3924487c0cd4cafb1 } from '@payloadcms/next/rsc'
+
+/** @type import('payload').ImportMap */
+export const importMap = {
+  "@payloadcms/richtext-lexical/rsc#RscEntryLexicalCell": RscEntryLexicalCell_44fe37237e0ebf4470c9990d8cb7b07e,
+  "@payloadcms/richtext-lexical/rsc#RscEntryLexicalField": RscEntryLexicalField_44fe37237e0ebf4470c9990d8cb7b07e,
+  "@payloadcms/richtext-lexical/rsc#LexicalDiffComponent": LexicalDiffComponent_44fe37237e0ebf4470c9990d8cb7b07e,
+  "@payloadcms/richtext-lexical/client#BlocksFeatureClient": BlocksFeatureClient_e70f5e05f09f93e00b997edb1ef0c864,
+  "@payloadcms/richtext-lexical/client#BoldFeatureClient": BoldFeatureClient_e70f5e05f09f93e00b997edb1ef0c864,
+  "@payloadcms/richtext-lexical/client#ItalicFeatureClient": ItalicFeatureClient_e70f5e05f09f93e00b997edb1ef0c864,
+  "@payloadcms/richtext-lexical/client#UnderlineFeatureClient": UnderlineFeatureClient_e70f5e05f09f93e00b997edb1ef0c864,
+  "@payloadcms/richtext-lexical/client#StrikethroughFeatureClient": StrikethroughFeatureClient_e70f5e05f09f93e00b997edb1ef0c864,
+  "@payloadcms/richtext-lexical/client#SubscriptFeatureClient": SubscriptFeatureClient_e70f5e05f09f93e00b997edb1ef0c864,
+  "@payloadcms/richtext-lexical/client#SuperscriptFeatureClient": SuperscriptFeatureClient_e70f5e05f09f93e00b997edb1ef0c864,
+  "@payloadcms/richtext-lexical/client#InlineCodeFeatureClient": InlineCodeFeatureClient_e70f5e05f09f93e00b997edb1ef0c864,
+  "@payloadcms/richtext-lexical/client#ParagraphFeatureClient": ParagraphFeatureClient_e70f5e05f09f93e00b997edb1ef0c864,
+  "@payloadcms/richtext-lexical/client#HeadingFeatureClient": HeadingFeatureClient_e70f5e05f09f93e00b997edb1ef0c864,
+  "@payloadcms/richtext-lexical/client#AlignFeatureClient": AlignFeatureClient_e70f5e05f09f93e00b997edb1ef0c864,
+  "@payloadcms/richtext-lexical/client#IndentFeatureClient": IndentFeatureClient_e70f5e05f09f93e00b997edb1ef0c864,
+  "@payloadcms/richtext-lexical/client#UnorderedListFeatureClient": UnorderedListFeatureClient_e70f5e05f09f93e00b997edb1ef0c864,
+  "@payloadcms/richtext-lexical/client#OrderedListFeatureClient": OrderedListFeatureClient_e70f5e05f09f93e00b997edb1ef0c864,
+  "@payloadcms/richtext-lexical/client#ChecklistFeatureClient": ChecklistFeatureClient_e70f5e05f09f93e00b997edb1ef0c864,
+  "@payloadcms/richtext-lexical/client#LinkFeatureClient": LinkFeatureClient_e70f5e05f09f93e00b997edb1ef0c864,
+  "@payloadcms/richtext-lexical/client#RelationshipFeatureClient": RelationshipFeatureClient_e70f5e05f09f93e00b997edb1ef0c864,
+  "@payloadcms/richtext-lexical/client#BlockquoteFeatureClient": BlockquoteFeatureClient_e70f5e05f09f93e00b997edb1ef0c864,
+  "@payloadcms/richtext-lexical/client#UploadFeatureClient": UploadFeatureClient_e70f5e05f09f93e00b997edb1ef0c864,
+  "@payloadcms/richtext-lexical/client#HorizontalRuleFeatureClient": HorizontalRuleFeatureClient_e70f5e05f09f93e00b997edb1ef0c864,
+  "@payloadcms/richtext-lexical/client#InlineToolbarFeatureClient": InlineToolbarFeatureClient_e70f5e05f09f93e00b997edb1ef0c864,
+  "@payloadcms/next/rsc#CollectionCards": CollectionCards_f9c02e79a4aed9a3924487c0cd4cafb1
+}

nextjs-app/app/(payload)/cms-api/[...slug]/route.ts

@@ -0,0 +1,20 @@
+/*
+ * Payload's REST API, mounted at /cms-api rather than /api.
+ * See lib/payloadRoutes.ts — /api is the FastAPI proxy's catch-all.
+ */
+import config from '@payload-config';
+import {
+  REST_DELETE,
+  REST_GET,
+  REST_OPTIONS,
+  REST_PATCH,
+  REST_POST,
+  REST_PUT,
+} from '@payloadcms/next/routes';
+
+export const GET = REST_GET(config);
+export const POST = REST_POST(config);
+export const DELETE = REST_DELETE(config);
+export const PATCH = REST_PATCH(config);
+export const PUT = REST_PUT(config);
+export const OPTIONS = REST_OPTIONS(config);

nextjs-app/app/(payload)/cms-api/graphql-playground/route.ts

@@ -0,0 +1,4 @@
+import config from '@payload-config';
+import { GRAPHQL_PLAYGROUND_GET } from '@payloadcms/next/routes';
+
+export const GET = GRAPHQL_PLAYGROUND_GET(config);

nextjs-app/app/(payload)/cms-api/graphql/route.ts

@@ -0,0 +1,5 @@
+import config from '@payload-config';
+import { GRAPHQL_POST, REST_OPTIONS } from '@payloadcms/next/routes';
+
+export const POST = GRAPHQL_POST(config);
+export const OPTIONS = REST_OPTIONS(config);

nextjs-app/app/(payload)/layout.tsx

@@ -0,0 +1,27 @@
+/**
+ * Root layout for the Payload admin panel.
+ *
+ * This is a SECOND root layout: it renders its own <html>/<body>, as does
+ * app/(frontend)/layout.tsx. Next permits that only while no app/layout.tsx
+ * exists — which is why the site's routes were moved into (frontend). Adding
+ * an app/layout.tsx would nest the admin panel inside the site's nav, footer
+ * and providers and emit nested <html>.
+ */
+import type { ServerFunctionClient } from 'payload';
+import config from '@payload-config';
+import { RootLayout, handleServerFunctions } from '@payloadcms/next/layouts';
+import { importMap } from './admin/importMap.js';
+import '@payloadcms/next/css';
+
+const serverFunction: ServerFunctionClient = async function (args) {
+  'use server';
+  return handleServerFunctions({ ...args, config, importMap });
+};
+
+export default function PayloadLayout({ children }: { children: React.ReactNode }) {
+  return (
+    <RootLayout config={config} importMap={importMap} serverFunction={serverFunction}>
+      {children}
+    </RootLayout>
+  );
+}

nextjs-app/app/robots.ts

@@ -12,9 +12,14 @@ export default function robots(): MetadataRoute.Robots {
       {
         userAgent: '*',
         allow: '/',
-        disallow: ['/api/', '/_next/'],
+        // /admin and /cms-api are also served X-Robots-Tag: noindex by
+        // next.config.mjs. A Disallow alone blocks crawling, not indexing.
+        disallow: ['/api/', '/_next/', '/admin/', '/cms-api/'],
       },
     ],
-    sitemap: absoluteUrl('/sitemap.xml'),
+    // Two sitemaps: /sitemap.xml is proxied from FastAPI and carries the
+    // school corpus; /content-sitemap.xml is Next-owned and carries /about
+    // and the blog. The backend knows nothing about Payload.
+    sitemap: [absoluteUrl('/sitemap.xml'), absoluteUrl('/content-sitemap.xml')],
   };
 }

nextjs-app/blocks/Callout.ts

@@ -0,0 +1,25 @@
+import type { Block } from 'payload';
+
+/**
+ * The house block: "what this number doesn't tell you".
+ *
+ * Blocks are the reason this site runs a CMS rather than flat files — a post
+ * can carry live product components, not screenshots of them. This is the
+ * first and simplest one; a live-chart block follows when a post needs it.
+ */
+export const Callout: Block = {
+  slug: 'callout',
+  labels: { singular: 'Callout', plural: 'Callouts' },
+  fields: [
+    {
+      name: 'tone',
+      type: 'select',
+      defaultValue: 'caveat',
+      options: [
+        { label: 'Caveat: what this does not show', value: 'caveat' },
+        { label: 'Note: useful aside', value: 'note' },
+      ],
+    },
+    { name: 'body', type: 'textarea', required: true },
+  ],
+};

nextjs-app/collections/Media.ts

@@ -0,0 +1,31 @@
+import type { CollectionConfig } from 'payload';
+
+/**
+ * Uploads land on a Docker named volume mounted at /app/media. The path is
+ * absolute because Payload 3 requires it, and it must match the payload_media
+ * mount in docker-compose.portainer.yml exactly — a mismatch writes into the
+ * container's own filesystem, where the next redeploy silently discards it.
+ */
+export const Media: CollectionConfig = {
+  slug: 'media',
+  access: { read: () => true },
+  upload: {
+    staticDir: '/app/media',
+    mimeTypes: ['image/*'],
+    imageSizes: [
+      { name: 'thumbnail', width: 400 },
+      { name: 'hero', width: 1200 },
+    ],
+    adminThumbnail: 'thumbnail',
+  },
+  fields: [
+    {
+      name: 'alt',
+      type: 'text',
+      required: true,
+      // Required rather than optional: a decorative-by-default image is an
+      // accessibility regression on a site parents use under time pressure.
+      admin: { description: 'Describe the image for screen readers.' },
+    },
+  ],
+};

nextjs-app/collections/Posts.ts

@@ -0,0 +1,99 @@
+import type { CollectionConfig } from 'payload';
+import { revalidatePath } from 'next/cache';
+import { lexicalEditor, BlocksFeature } from '@payloadcms/richtext-lexical';
+import { Callout } from '@/blocks/Callout';
+
+/**
+ * Drop the cached copy of a post page when it changes.
+ *
+ * Only the post page needs this. The blog index, the RSS feed and the content
+ * sitemap are force-dynamic — they have no dynamic params, so Next would
+ * prerender them at build time, where CI has no database — which means they
+ * already reflect a change on the next request.
+ *
+ * /blog/[slug] is ISR: generated on first request and cached, so without this
+ * an edit to an already-published post would not appear until the revalidate
+ * window expired — up to an hour of a writer concluding that saving is broken.
+ *
+ * Payload runs in the same process as Next, so this is a direct revalidatePath
+ * call: no webhook, no shared secret, no network hop to get wrong.
+ */
+function revalidatePost(slug: string) {
+  revalidatePath(`/blog/${slug}`);
+}
+
+export const Posts: CollectionConfig = {
+  slug: 'posts',
+  access: {
+    /*
+     * Drafts must be hidden here, not in the pages that query this collection.
+     *
+     * From Payload's own documentation: "The `draft` argument alone does not
+     * restrict documents with `_status: 'draft'` from being returned by the
+     * API." The blog index and post page both filter on `_status`, but that
+     * is a convenience, not a control — a direct GET /cms-api/posts would
+     * hand every unpublished draft to any visitor.
+     *
+     * Returning a query constraint rather than a boolean is the documented
+     * mechanism: Payload merges it into every read for an anonymous caller.
+     */
+    read: ({ req }) => {
+      if (req.user) return true;
+      return { _status: { equals: 'published' } };
+    },
+  },
+  admin: {
+    useAsTitle: 'title',
+    defaultColumns: ['title', 'publishedAt', '_status'],
+  },
+  versions: {
+    // Posts get written across several sittings and previewed before they go
+    // live. Without drafts, saving is publishing.
+    drafts: true,
+  },
+  hooks: {
+    afterChange: [({ doc }) => { revalidatePost(String(doc.slug)); }],
+    afterDelete: [({ doc }) => { revalidatePost(String(doc.slug)); }],
+  },
+  fields: [
+    { name: 'title', type: 'text', required: true },
+    {
+      name: 'slug',
+      type: 'text',
+      required: true,
+      unique: true,
+      index: true,
+      admin: {
+        position: 'sidebar',
+        description: 'The URL segment. Never change it after publishing.',
+      },
+    },
+    {
+      name: 'publishedAt',
+      type: 'date',
+      required: true,
+      admin: { position: 'sidebar', date: { pickerAppearance: 'dayOnly' } },
+    },
+    {
+      name: 'excerpt',
+      type: 'textarea',
+      required: true,
+      maxLength: 200,
+      admin: {
+        description: 'Shown on the index and used as the meta description.',
+      },
+    },
+    { name: 'heroImage', type: 'upload', relationTo: 'media' },
+    {
+      name: 'content',
+      type: 'richText',
+      required: true,
+      editor: lexicalEditor({
+        features: ({ defaultFeatures }) => [
+          ...defaultFeatures,
+          BlocksFeature({ blocks: [Callout] }),
+        ],
+      }),
+    },
+  ],
+};

nextjs-app/collections/Users.ts

@@ -0,0 +1,33 @@
+import type { CollectionConfig } from 'payload';
+
+/**
+ * The site's only authenticated surface. There is one account and no
+ * registration: `create` is closed to everyone, so the first user is seeded
+ * with `payload create-first-user` and no one can add another through the API.
+ */
+export const Users: CollectionConfig = {
+  slug: 'users',
+  auth: {
+    // Slows credential stuffing against a panel that is on the public
+    // internet. Five attempts, then a ten-minute lock.
+    maxLoginAttempts: 5,
+    lockTime: 10 * 60 * 1000,
+  },
+  access: {
+    create: () => false,
+    read: ({ req }) => Boolean(req.user),
+    update: ({ req }) => Boolean(req.user),
+    delete: () => false,
+  },
+  admin: { useAsTitle: 'email' },
+  fields: [
+    {
+      name: 'displayName',
+      type: 'text',
+      required: true,
+      // Rendered as the byline on every post. First name only — the site
+      // publishes no surname and no employer.
+      defaultValue: 'Tudor',
+    },
+  ],
+};

nextjs-app/components/Footer.module.css

@@ -22,7 +22,8 @@
 
 .content {
   display: grid;
-  grid-template-columns: 1.6fr 1fr 1fr;
+  /* Brand column plus three link columns: Product, Resources, About. */
+  grid-template-columns: 1.6fr 1fr 1fr 1fr;
   gap: 2rem;
   margin-bottom: 3rem;
 }
@@ -193,6 +194,14 @@
   color: var(--on-sunken);
 }
 
+/* Four columns crush between the tablet range and the 768px collapse, so
+   pair them up first rather than jumping straight to a single column. */
+@media (max-width: 960px) {
+  .content {
+    grid-template-columns: 1fr 1fr;
+  }
+}
+
 @media (max-width: 768px) {
   .container {
     padding: 2rem 1rem 1.5rem;

nextjs-app/components/Footer.tsx

@@ -93,6 +93,18 @@ export function Footer() {
               </li>
             </ul>
           </div>
+
+          <div className={styles.section}>
+            <h4 className={styles.sectionTitle}>About</h4>
+            <ul className={styles.links}>
+              {/* The only route to a named human. Deliberately not in the nav:
+                  the mobile bottom bar already carries four items, and both of
+                  these are lower intent than any of them. Post bylines link
+                  here too, which is where a reader actually asks the question. */}
+              <li><a href="/about" className={styles.link}>Who&apos;s behind this</a></li>
+              <li><a href="/blog" className={styles.link}>Blog</a></li>
+            </ul>
+          </div>
         </div>
 
         <div className={styles.bottom}>

nextjs-app/components/blog/CalloutBlock.module.css

@@ -0,0 +1,21 @@
+.caveat,
+.note {
+  border-left: 3px solid var(--brand);
+  background: var(--brand-bg);
+  padding: 1rem 1.15rem;
+  margin: 1.75rem 0;
+  border-radius: 0 8px 8px 0;
+}
+
+.note {
+  border-left-color: var(--border-strong);
+  background: var(--bg-secondary);
+}
+
+.body {
+  font-family: var(--font-ui);
+  font-size: 0.95rem;
+  line-height: 1.65;
+  color: var(--text-primary);
+  margin: 0;
+}

nextjs-app/components/blog/CalloutBlock.tsx

@@ -0,0 +1,14 @@
+import styles from './CalloutBlock.module.css';
+
+/**
+ * Renders the Callout block from blocks/Callout.ts. The "caveat" tone is the
+ * one that matters: it is how a post says what a number does not show, in
+ * context, rather than burying it in a closing paragraph.
+ */
+export function CalloutBlock({ tone, body }: { tone: string; body: string }) {
+  return (
+    <aside className={tone === 'caveat' ? styles.caveat : styles.note}>
+      <p className={styles.body}>{body}</p>
+    </aside>
+  );
+}

nextjs-app/docs/PUBLISHING.md

@@ -0,0 +1,86 @@
+# Publishing to the blog
+
+The blog is Payload CMS, running inside the Next.js app. There is no separate
+service and no second deploy. Writing a post is done in the browser and takes
+effect on the live site within seconds.
+
+## Signing in
+
+`https://www.schoolcompare.co.uk/admin`, one account, no registration. If you
+need the account seeded on a fresh environment, run against the container:
+
+```bash
+npx payload create-first-user
+```
+
+Staging has its own admin panel, its own database and its own credentials at
+`https://stx.schoolcompare.co.uk/admin`. Never reuse production's secret or
+password there.
+
+## Writing a post
+
+**Posts → Create New.** The fields:
+
+| Field | Notes |
+|---|---|
+| **Title** | The `<h1>` and the browser tab. |
+| **Slug** | The URL segment, in the sidebar. **Never change it after publishing.** It is the canonical URL, and changing it breaks every existing link and discards the page's accumulated search signal. |
+| **Published at** | The date shown on the post and in the feed. |
+| **Excerpt** | Max 200 characters. Shown on the index *and* used as the meta description, so write it as a standalone sentence rather than a teaser. |
+| **Hero image** | Optional. Becomes the social share image; without one, the site's generated card is used. |
+| **Content** | Rich text. `/` inserts a block. |
+
+**Save as draft** while you're working; drafts are not public. **Publish** when
+it's ready.
+
+### The callout block
+
+One custom block, `Callout`, with two tones:
+
+- **Caveat**: what a number does *not* show. This is the one that matters. It
+  is how a post states a limitation in context rather than burying it in a
+  closing paragraph.
+- **Note**: a useful aside.
+
+### Images
+
+Every image requires alt text; the editor will not let you save without it.
+Uploads go to a Docker volume on the host, which is backed up separately from
+Postgres. An image is not reproducible from the pipeline the way school data
+is.
+
+## How publishing reaches the live site
+
+- `/blog`, `/blog/rss.xml` and `/content-sitemap.xml` are rendered per request,
+  so a new post appears immediately.
+- `/blog/[slug]` is cached after its first request. Publishing or editing fires
+  a `revalidatePath` from the collection's `afterChange` hook, which drops that
+  cached copy, so edits appear immediately too.
+
+If a change doesn't show, it is far more likely the post is still a draft than
+that the cache is stale.
+
+## House style
+
+These rules are why the blog exists. A post that ignores them makes the site
+read more machine-generated, not less.
+
+- **First person singular.** "I built", "I found", never "we provide".
+- **Concrete over general.** "When we were looking at schools in Wandsworth"
+  beats any amount of stated warmth.
+- **State limits before someone else finds them.** Every post that presents a
+  metric says what it does not show. This is the single strongest signal that a
+  human wrote it: generated content does not volunteer its own weaknesses.
+- **No mission statements, no "passionate about", no invented team.** There is
+  one person here.
+- **No em dashes.** They are one of the clearest tells of machine-written
+  prose, which is the whole problem this blog exists to fix. A full stop, a
+  colon, a semicolon or a pair of commas does the job and reads as though a
+  person chose it.
+- **Short sentences.**
+- **Never publish a surname, an employer, or a child's name.** The site's author
+  is "Tudor". See `/about`.
+- **Never invent a figure**, even illustratively. On a site whose whole
+  proposition is official data, a made-up number attached to a real school is
+  the one thing it cannot do, and no illustrative intent survives being
+  screenshotted.

nextjs-app/lib/jsonld.ts

@@ -0,0 +1,78 @@
+import { SITE_URL, absoluteUrl } from '@/lib/site';
+
+/**
+ * The site's author entity.
+ *
+ * First name only, by choice — see /about. That makes this a weaker search
+ * signal than a fully identified author would be, which is why the About page
+ * carries a substantial methodology section: the credibility has to come from
+ * stated provenance rather than from a corroborable identity.
+ *
+ * Everything that needs an author — the About page, every post byline —
+ * references this one shape, so search engines resolve them all to one entity.
+ */
+export function personJsonLd() {
+  return {
+    '@type': 'Person',
+    '@id': `${SITE_URL}/about#tudor`,
+    name: 'Tudor',
+    url: absoluteUrl('/about'),
+    image: absoluteUrl('/brand/tudor.jpg'),
+    description:
+      'Parent in south-west London who built schoolcompare while looking for a primary school.',
+  } as const;
+}
+
+export function organizationJsonLd() {
+  return {
+    '@type': 'Organization',
+    '@id': `${SITE_URL}#organization`,
+    name: 'schoolcompare',
+    url: SITE_URL,
+    logo: absoluteUrl('/icon-512.png'),
+  } as const;
+}
+
+interface PostSummary {
+  title: string;
+  slug: string;
+  excerpt: string;
+  publishedAt: string;
+}
+
+/**
+ * References the Person and Organization by @id rather than repeating them, so
+ * search engines resolve every post and the About page to the one author
+ * entity. Repeating the shape would declare several people with one name.
+ */
+export function blogPostingJsonLd(post: PostSummary) {
+  return {
+    '@type': 'BlogPosting',
+    headline: post.title,
+    description: post.excerpt,
+    url: absoluteUrl(`/blog/${post.slug}`),
+    datePublished: post.publishedAt,
+    author: { '@id': `${SITE_URL}/about#tudor` },
+    publisher: { '@id': `${SITE_URL}#organization` },
+  } as const;
+}
+
+export function breadcrumbJsonLd(post: PostSummary) {
+  return {
+    '@type': 'BreadcrumbList',
+    itemListElement: [
+      {
+        '@type': 'ListItem',
+        position: 1,
+        name: 'Blog',
+        item: absoluteUrl('/blog'),
+      },
+      {
+        '@type': 'ListItem',
+        position: 2,
+        name: post.title,
+        item: absoluteUrl(`/blog/${post.slug}`),
+      },
+    ],
+  } as const;
+}

nextjs-app/lib/payload.ts

@@ -0,0 +1,15 @@
+import { getPayload } from 'payload';
+import config from '@payload-config';
+import type { Payload } from 'payload';
+
+/**
+ * One Payload instance per process. getPayload() is itself memoised by
+ * Payload, but routing every caller through here keeps the config import in a
+ * single place and gives page code one name to mock in tests.
+ *
+ * Never call this at module scope: CI builds the image with no database
+ * reachable, so a build-time connection attempt fails the build.
+ */
+export function getCachedPayload(): Promise<Payload> {
+  return getPayload({ config });
+}

nextjs-app/lib/payloadRoutes.ts

@@ -0,0 +1,23 @@
+/**
+ * Where Payload mounts, defined once.
+ *
+ * These are imported by payload.config.ts and asserted by
+ * __tests__/payload/routes.test.ts. They live in their own module because
+ * payload.config.ts cannot be imported from a Jest test: Payload ships
+ * ESM-only, and next/jest's transformIgnorePatterns skips node_modules — you
+ * cannot un-ignore a package by appending patterns, and forcing it through
+ * `transpilePackages` would change how the production build bundles Payload
+ * to serve a test. Keeping the values here makes them testable without
+ * loading Payload at all.
+ */
+
+/**
+ * Payload's API base. It must NOT be '/api': that path belongs to
+ * app/(frontend)/api/[...path]/route.ts, a catch-all that proxies to FastAPI.
+ * It would swallow every admin API call and forward it to the backend, and
+ * the failure is silent — no error, just wrong responses.
+ */
+export const PAYLOAD_API_ROUTE = '/cms-api';
+
+/** The admin panel. Kept out of the index by robots.txt and X-Robots-Tag. */
+export const PAYLOAD_ADMIN_ROUTE = '/admin';

nextjs-app/migrations/20260902_172826_initial.ts

@@ -0,0 +1,216 @@
+import { MigrateUpArgs, MigrateDownArgs, sql } from '@payloadcms/db-postgres'
+
+export async function up({ db, payload, req }: MigrateUpArgs): Promise<void> {
+  /*
+   * Hand-added, and it must survive any regeneration of this file.
+   *
+   * `schemaName: 'payload'` tells Payload where to put its tables; it does not
+   * create the schema. Every statement below is qualified to "payload", so on
+   * a database that has never run this (staging and production both), the
+   * whole migration fails with `schema "payload" does not exist`. The schema
+   * only existed on the throwaway database used to generate this because it
+   * was created there by hand.
+   */
+  await db.execute(sql`CREATE SCHEMA IF NOT EXISTS "payload";`)
+
+  await db.execute(sql`
+   CREATE TYPE "payload"."enum_posts_status" AS ENUM('draft', 'published');
+  CREATE TYPE "payload"."enum__posts_v_version_status" AS ENUM('draft', 'published');
+  CREATE TABLE "payload"."users_sessions" (
+  	"_order" integer NOT NULL,
+  	"_parent_id" integer NOT NULL,
+  	"id" varchar PRIMARY KEY NOT NULL,
+  	"created_at" timestamp(3) with time zone,
+  	"expires_at" timestamp(3) with time zone NOT NULL
+  );
+  
+  CREATE TABLE "payload"."users" (
+  	"id" serial PRIMARY KEY NOT NULL,
+  	"display_name" varchar DEFAULT 'Tudor' NOT NULL,
+  	"updated_at" timestamp(3) with time zone DEFAULT now() NOT NULL,
+  	"created_at" timestamp(3) with time zone DEFAULT now() NOT NULL,
+  	"email" varchar NOT NULL,
+  	"reset_password_token" varchar,
+  	"reset_password_expiration" timestamp(3) with time zone,
+  	"salt" varchar,
+  	"hash" varchar,
+  	"login_attempts" numeric DEFAULT 0,
+  	"lock_until" timestamp(3) with time zone
+  );
+  
+  CREATE TABLE "payload"."posts" (
+  	"id" serial PRIMARY KEY NOT NULL,
+  	"title" varchar,
+  	"slug" varchar,
+  	"published_at" timestamp(3) with time zone,
+  	"excerpt" varchar,
+  	"hero_image_id" integer,
+  	"content" jsonb,
+  	"updated_at" timestamp(3) with time zone DEFAULT now() NOT NULL,
+  	"created_at" timestamp(3) with time zone DEFAULT now() NOT NULL,
+  	"_status" "payload"."enum_posts_status" DEFAULT 'draft'
+  );
+  
+  CREATE TABLE "payload"."_posts_v" (
+  	"id" serial PRIMARY KEY NOT NULL,
+  	"parent_id" integer,
+  	"version_title" varchar,
+  	"version_slug" varchar,
+  	"version_published_at" timestamp(3) with time zone,
+  	"version_excerpt" varchar,
+  	"version_hero_image_id" integer,
+  	"version_content" jsonb,
+  	"version_updated_at" timestamp(3) with time zone,
+  	"version_created_at" timestamp(3) with time zone,
+  	"version__status" "payload"."enum__posts_v_version_status" DEFAULT 'draft',
+  	"created_at" timestamp(3) with time zone DEFAULT now() NOT NULL,
+  	"updated_at" timestamp(3) with time zone DEFAULT now() NOT NULL,
+  	"latest" boolean
+  );
+  
+  CREATE TABLE "payload"."media" (
+  	"id" serial PRIMARY KEY NOT NULL,
+  	"alt" varchar NOT NULL,
+  	"updated_at" timestamp(3) with time zone DEFAULT now() NOT NULL,
+  	"created_at" timestamp(3) with time zone DEFAULT now() NOT NULL,
+  	"url" varchar,
+  	"thumbnail_u_r_l" varchar,
+  	"filename" varchar,
+  	"mime_type" varchar,
+  	"filesize" numeric,
+  	"width" numeric,
+  	"height" numeric,
+  	"focal_x" numeric,
+  	"focal_y" numeric,
+  	"sizes_thumbnail_url" varchar,
+  	"sizes_thumbnail_width" numeric,
+  	"sizes_thumbnail_height" numeric,
+  	"sizes_thumbnail_mime_type" varchar,
+  	"sizes_thumbnail_filesize" numeric,
+  	"sizes_thumbnail_filename" varchar,
+  	"sizes_hero_url" varchar,
+  	"sizes_hero_width" numeric,
+  	"sizes_hero_height" numeric,
+  	"sizes_hero_mime_type" varchar,
+  	"sizes_hero_filesize" numeric,
+  	"sizes_hero_filename" varchar
+  );
+  
+  CREATE TABLE "payload"."payload_kv" (
+  	"id" serial PRIMARY KEY NOT NULL,
+  	"key" varchar NOT NULL,
+  	"data" jsonb NOT NULL
+  );
+  
+  CREATE TABLE "payload"."payload_locked_documents" (
+  	"id" serial PRIMARY KEY NOT NULL,
+  	"global_slug" varchar,
+  	"updated_at" timestamp(3) with time zone DEFAULT now() NOT NULL,
+  	"created_at" timestamp(3) with time zone DEFAULT now() NOT NULL
+  );
+  
+  CREATE TABLE "payload"."payload_locked_documents_rels" (
+  	"id" serial PRIMARY KEY NOT NULL,
+  	"order" integer,
+  	"parent_id" integer NOT NULL,
+  	"path" varchar NOT NULL,
+  	"users_id" integer,
+  	"posts_id" integer,
+  	"media_id" integer
+  );
+  
+  CREATE TABLE "payload"."payload_preferences" (
+  	"id" serial PRIMARY KEY NOT NULL,
+  	"key" varchar,
+  	"value" jsonb,
+  	"updated_at" timestamp(3) with time zone DEFAULT now() NOT NULL,
+  	"created_at" timestamp(3) with time zone DEFAULT now() NOT NULL
+  );
+  
+  CREATE TABLE "payload"."payload_preferences_rels" (
+  	"id" serial PRIMARY KEY NOT NULL,
+  	"order" integer,
+  	"parent_id" integer NOT NULL,
+  	"path" varchar NOT NULL,
+  	"users_id" integer
+  );
+  
+  CREATE TABLE "payload"."payload_migrations" (
+  	"id" serial PRIMARY KEY NOT NULL,
+  	"name" varchar,
+  	"batch" numeric,
+  	"updated_at" timestamp(3) with time zone DEFAULT now() NOT NULL,
+  	"created_at" timestamp(3) with time zone DEFAULT now() NOT NULL
+  );
+  
+  ALTER TABLE "payload"."users_sessions" ADD CONSTRAINT "users_sessions_parent_id_fk" FOREIGN KEY ("_parent_id") REFERENCES "payload"."users"("id") ON DELETE cascade ON UPDATE no action;
+  ALTER TABLE "payload"."posts" ADD CONSTRAINT "posts_hero_image_id_media_id_fk" FOREIGN KEY ("hero_image_id") REFERENCES "payload"."media"("id") ON DELETE set null ON UPDATE no action;
+  ALTER TABLE "payload"."_posts_v" ADD CONSTRAINT "_posts_v_parent_id_posts_id_fk" FOREIGN KEY ("parent_id") REFERENCES "payload"."posts"("id") ON DELETE set null ON UPDATE no action;
+  ALTER TABLE "payload"."_posts_v" ADD CONSTRAINT "_posts_v_version_hero_image_id_media_id_fk" FOREIGN KEY ("version_hero_image_id") REFERENCES "payload"."media"("id") ON DELETE set null ON UPDATE no action;
+  ALTER TABLE "payload"."payload_locked_documents_rels" ADD CONSTRAINT "payload_locked_documents_rels_parent_fk" FOREIGN KEY ("parent_id") REFERENCES "payload"."payload_locked_documents"("id") ON DELETE cascade ON UPDATE no action;
+  ALTER TABLE "payload"."payload_locked_documents_rels" ADD CONSTRAINT "payload_locked_documents_rels_users_fk" FOREIGN KEY ("users_id") REFERENCES "payload"."users"("id") ON DELETE cascade ON UPDATE no action;
+  ALTER TABLE "payload"."payload_locked_documents_rels" ADD CONSTRAINT "payload_locked_documents_rels_posts_fk" FOREIGN KEY ("posts_id") REFERENCES "payload"."posts"("id") ON DELETE cascade ON UPDATE no action;
+  ALTER TABLE "payload"."payload_locked_documents_rels" ADD CONSTRAINT "payload_locked_documents_rels_media_fk" FOREIGN KEY ("media_id") REFERENCES "payload"."media"("id") ON DELETE cascade ON UPDATE no action;
+  ALTER TABLE "payload"."payload_preferences_rels" ADD CONSTRAINT "payload_preferences_rels_parent_fk" FOREIGN KEY ("parent_id") REFERENCES "payload"."payload_preferences"("id") ON DELETE cascade ON UPDATE no action;
+  ALTER TABLE "payload"."payload_preferences_rels" ADD CONSTRAINT "payload_preferences_rels_users_fk" FOREIGN KEY ("users_id") REFERENCES "payload"."users"("id") ON DELETE cascade ON UPDATE no action;
+  CREATE INDEX "users_sessions_order_idx" ON "payload"."users_sessions" USING btree ("_order");
+  CREATE INDEX "users_sessions_parent_id_idx" ON "payload"."users_sessions" USING btree ("_parent_id");
+  CREATE INDEX "users_updated_at_idx" ON "payload"."users" USING btree ("updated_at");
+  CREATE INDEX "users_created_at_idx" ON "payload"."users" USING btree ("created_at");
+  CREATE UNIQUE INDEX "users_email_idx" ON "payload"."users" USING btree ("email");
+  CREATE UNIQUE INDEX "posts_slug_idx" ON "payload"."posts" USING btree ("slug");
+  CREATE INDEX "posts_hero_image_idx" ON "payload"."posts" USING btree ("hero_image_id");
+  CREATE INDEX "posts_updated_at_idx" ON "payload"."posts" USING btree ("updated_at");
+  CREATE INDEX "posts_created_at_idx" ON "payload"."posts" USING btree ("created_at");
+  CREATE INDEX "posts__status_idx" ON "payload"."posts" USING btree ("_status");
+  CREATE INDEX "_posts_v_parent_idx" ON "payload"."_posts_v" USING btree ("parent_id");
+  CREATE INDEX "_posts_v_version_version_slug_idx" ON "payload"."_posts_v" USING btree ("version_slug");
+  CREATE INDEX "_posts_v_version_version_hero_image_idx" ON "payload"."_posts_v" USING btree ("version_hero_image_id");
+  CREATE INDEX "_posts_v_version_version_updated_at_idx" ON "payload"."_posts_v" USING btree ("version_updated_at");
+  CREATE INDEX "_posts_v_version_version_created_at_idx" ON "payload"."_posts_v" USING btree ("version_created_at");
+  CREATE INDEX "_posts_v_version_version__status_idx" ON "payload"."_posts_v" USING btree ("version__status");
+  CREATE INDEX "_posts_v_created_at_idx" ON "payload"."_posts_v" USING btree ("created_at");
+  CREATE INDEX "_posts_v_updated_at_idx" ON "payload"."_posts_v" USING btree ("updated_at");
+  CREATE INDEX "_posts_v_latest_idx" ON "payload"."_posts_v" USING btree ("latest");
+  CREATE INDEX "media_updated_at_idx" ON "payload"."media" USING btree ("updated_at");
+  CREATE INDEX "media_created_at_idx" ON "payload"."media" USING btree ("created_at");
+  CREATE UNIQUE INDEX "media_filename_idx" ON "payload"."media" USING btree ("filename");
+  CREATE INDEX "media_sizes_thumbnail_sizes_thumbnail_filename_idx" ON "payload"."media" USING btree ("sizes_thumbnail_filename");
+  CREATE INDEX "media_sizes_hero_sizes_hero_filename_idx" ON "payload"."media" USING btree ("sizes_hero_filename");
+  CREATE UNIQUE INDEX "payload_kv_key_idx" ON "payload"."payload_kv" USING btree ("key");
+  CREATE INDEX "payload_locked_documents_global_slug_idx" ON "payload"."payload_locked_documents" USING btree ("global_slug");
+  CREATE INDEX "payload_locked_documents_updated_at_idx" ON "payload"."payload_locked_documents" USING btree ("updated_at");
+  CREATE INDEX "payload_locked_documents_created_at_idx" ON "payload"."payload_locked_documents" USING btree ("created_at");
+  CREATE INDEX "payload_locked_documents_rels_order_idx" ON "payload"."payload_locked_documents_rels" USING btree ("order");
+  CREATE INDEX "payload_locked_documents_rels_parent_idx" ON "payload"."payload_locked_documents_rels" USING btree ("parent_id");
+  CREATE INDEX "payload_locked_documents_rels_path_idx" ON "payload"."payload_locked_documents_rels" USING btree ("path");
+  CREATE INDEX "payload_locked_documents_rels_users_id_idx" ON "payload"."payload_locked_documents_rels" USING btree ("users_id");
+  CREATE INDEX "payload_locked_documents_rels_posts_id_idx" ON "payload"."payload_locked_documents_rels" USING btree ("posts_id");
+  CREATE INDEX "payload_locked_documents_rels_media_id_idx" ON "payload"."payload_locked_documents_rels" USING btree ("media_id");
+  CREATE INDEX "payload_preferences_key_idx" ON "payload"."payload_preferences" USING btree ("key");
+  CREATE INDEX "payload_preferences_updated_at_idx" ON "payload"."payload_preferences" USING btree ("updated_at");
+  CREATE INDEX "payload_preferences_created_at_idx" ON "payload"."payload_preferences" USING btree ("created_at");
+  CREATE INDEX "payload_preferences_rels_order_idx" ON "payload"."payload_preferences_rels" USING btree ("order");
+  CREATE INDEX "payload_preferences_rels_parent_idx" ON "payload"."payload_preferences_rels" USING btree ("parent_id");
+  CREATE INDEX "payload_preferences_rels_path_idx" ON "payload"."payload_preferences_rels" USING btree ("path");
+  CREATE INDEX "payload_preferences_rels_users_id_idx" ON "payload"."payload_preferences_rels" USING btree ("users_id");
+  CREATE INDEX "payload_migrations_updated_at_idx" ON "payload"."payload_migrations" USING btree ("updated_at");
+  CREATE INDEX "payload_migrations_created_at_idx" ON "payload"."payload_migrations" USING btree ("created_at");`)
+}
+
+export async function down({ db, payload, req }: MigrateDownArgs): Promise<void> {
+  await db.execute(sql`
+   DROP TABLE "payload"."users_sessions" CASCADE;
+  DROP TABLE "payload"."users" CASCADE;
+  DROP TABLE "payload"."posts" CASCADE;
+  DROP TABLE "payload"."_posts_v" CASCADE;
+  DROP TABLE "payload"."media" CASCADE;
+  DROP TABLE "payload"."payload_kv" CASCADE;
+  DROP TABLE "payload"."payload_locked_documents" CASCADE;
+  DROP TABLE "payload"."payload_locked_documents_rels" CASCADE;
+  DROP TABLE "payload"."payload_preferences" CASCADE;
+  DROP TABLE "payload"."payload_preferences_rels" CASCADE;
+  DROP TABLE "payload"."payload_migrations" CASCADE;
+  DROP TYPE "payload"."enum_posts_status";
+  DROP TYPE "payload"."enum__posts_v_version_status";`)
+}

nextjs-app/migrations/index.ts

@@ -0,0 +1,9 @@
+import * as migration_20260902_172826_initial from './20260902_172826_initial';
+
+export const migrations = [
+  {
+    up: migration_20260902_172826_initial.up,
+    down: migration_20260902_172826_initial.down,
+    name: '20260902_172826_initial'
+  },
+];

nextjs-app/next.config.mjs

@@ -1,3 +1,5 @@
+import { withPayload } from '@payloadcms/next/withPayload';
+
 /** @type {import('next').NextConfig} */
 const nextConfig = {
   // Enable standalone output for Docker
@@ -86,6 +88,23 @@ const nextConfig = {
           },
         ],
       },
+      {
+        /*
+         * The admin panel and the CMS API must never be indexed.
+         *
+         * X-Robots-Tag, not just the robots.txt Disallow, for the same reason
+         * the staging rule above uses one: a Disallow blocks crawling, which
+         * is not indexing. A disallowed URL found from an external link can
+         * still be indexed without ever being fetched — and worse, blocking
+         * the crawl means the noindex is never seen.
+         */
+        source: '/admin/:path*',
+        headers: [{ key: 'X-Robots-Tag', value: 'noindex, nofollow' }],
+      },
+      {
+        source: '/cms-api/:path*',
+        headers: [{ key: 'X-Robots-Tag', value: 'noindex, nofollow' }],
+      },
       {
         source: '/:path*',
         headers: [
@@ -127,4 +146,4 @@ const nextConfig = {
   },
 };
 
-module.exports = nextConfig;
+export default withPayload(nextConfig);

nextjs-app/package.json

@@ -2,30 +2,38 @@
   "name": "nextjs-app",
   "version": "0.1.0",
   "private": true,
+  "type": "module",
   "description": "SchoolCompare Next.js Application",
   "scripts": {
     "dev": "next dev",
     "build": "next build",
     "start": "next start",
     "typecheck": "tsc --noEmit",
+    "generate:importmap": "payload generate:importmap",
     "test": "jest",
     "test:watch": "jest --watch",
     "test:coverage": "jest --coverage"
   },
   "dependencies": {
     "@floating-ui/react": "^0.27.20",
+    "@payloadcms/db-postgres": "^3.88.0",
+    "@payloadcms/next": "^3.88.0",
+    "@payloadcms/richtext-lexical": "^3.88.0",
     "@types/node": "^25.2.0",
     "@types/react": "^19.2.10",
     "@types/react-dom": "^19.2.3",
     "chart.js": "^4.5.1",
     "eslint": "^9.39.2",
     "eslint-config-next": "^16.1.6",
+    "graphql": "^16.14.2",
     "leaflet": "^1.9.4",
     "next": "^16.1.6",
+    "payload": "^3.88.0",
     "react": "^19.2.4",
     "react-chartjs-2": "^5.3.1",
     "react-dom": "^19.2.4",
     "react-leaflet": "^5.0.0",
+    "sharp": "^0.35.4",
     "typescript": "^5.9.3",
     "zod": "^4.3.6"
   },
@@ -36,7 +44,6 @@
     "@types/jest": "^30.0.0",
     "@types/leaflet": "^1.9.21",
     "jest": "^30.2.0",
-    "jest-environment-jsdom": "^30.2.0",
-    "sharp": "^0.34.5"
+    "jest-environment-jsdom": "^30.2.0"
   }
 }

nextjs-app/payload-types.ts

@@ -0,0 +1,443 @@
+/* tslint:disable */
+/* eslint-disable */
+/**
+ * This file was automatically generated by Payload.
+ * DO NOT MODIFY IT BY HAND. Instead, modify your source Payload config,
+ * and re-run `payload generate:types` to regenerate this file.
+ */
+
+/**
+ * Supported timezones in IANA format.
+ *
+ * This interface was referenced by `Config`'s JSON-Schema
+ * via the `definition` "supportedTimezones".
+ */
+export type SupportedTimezones =
+  | 'Pacific/Midway'
+  | 'Pacific/Niue'
+  | 'Pacific/Honolulu'
+  | 'Pacific/Rarotonga'
+  | 'America/Anchorage'
+  | 'Pacific/Gambier'
+  | 'America/Los_Angeles'
+  | 'America/Tijuana'
+  | 'America/Denver'
+  | 'America/Phoenix'
+  | 'America/Chicago'
+  | 'America/Guatemala'
+  | 'America/New_York'
+  | 'America/Bogota'
+  | 'America/Caracas'
+  | 'America/Santiago'
+  | 'America/Buenos_Aires'
+  | 'America/Sao_Paulo'
+  | 'Atlantic/South_Georgia'
+  | 'Atlantic/Azores'
+  | 'Atlantic/Cape_Verde'
+  | 'Europe/London'
+  | 'Europe/Berlin'
+  | 'Africa/Lagos'
+  | 'Europe/Athens'
+  | 'Africa/Cairo'
+  | 'Europe/Moscow'
+  | 'Asia/Riyadh'
+  | 'Asia/Dubai'
+  | 'Asia/Baku'
+  | 'Asia/Karachi'
+  | 'Asia/Tashkent'
+  | 'Asia/Calcutta'
+  | 'Asia/Dhaka'
+  | 'Asia/Almaty'
+  | 'Asia/Jakarta'
+  | 'Asia/Bangkok'
+  | 'Asia/Shanghai'
+  | 'Asia/Singapore'
+  | 'Asia/Tokyo'
+  | 'Asia/Seoul'
+  | 'Australia/Brisbane'
+  | 'Australia/Sydney'
+  | 'Pacific/Guam'
+  | 'Pacific/Noumea'
+  | 'Pacific/Auckland'
+  | 'Pacific/Fiji';
+
+export interface Config {
+  auth: {
+    users: UserAuthOperations;
+  };
+  blocks: {};
+  collections: {
+    users: User;
+    posts: Post;
+    media: Media;
+    'payload-kv': PayloadKv;
+    'payload-locked-documents': PayloadLockedDocument;
+    'payload-preferences': PayloadPreference;
+    'payload-migrations': PayloadMigration;
+  };
+  collectionsJoins: {};
+  collectionsSelect: {
+    users: UsersSelect<false> | UsersSelect<true>;
+    posts: PostsSelect<false> | PostsSelect<true>;
+    media: MediaSelect<false> | MediaSelect<true>;
+    'payload-kv': PayloadKvSelect<false> | PayloadKvSelect<true>;
+    'payload-locked-documents': PayloadLockedDocumentsSelect<false> | PayloadLockedDocumentsSelect<true>;
+    'payload-preferences': PayloadPreferencesSelect<false> | PayloadPreferencesSelect<true>;
+    'payload-migrations': PayloadMigrationsSelect<false> | PayloadMigrationsSelect<true>;
+  };
+  db: {
+    defaultIDType: number;
+  };
+  fallbackLocale: null;
+  globals: {};
+  globalsSelect: {};
+  locale: null;
+  widgets: {
+    collections: CollectionsWidget;
+  };
+  user: User;
+  jobs: {
+    tasks: unknown;
+    workflows: unknown;
+  };
+}
+export interface UserAuthOperations {
+  forgotPassword: {
+    email: string;
+    password: string;
+  };
+  login: {
+    email: string;
+    password: string;
+  };
+  registerFirstUser: {
+    email: string;
+    password: string;
+  };
+  unlock: {
+    email: string;
+    password: string;
+  };
+}
+/**
+ * This interface was referenced by `Config`'s JSON-Schema
+ * via the `definition` "users".
+ */
+export interface User {
+  id: number;
+  displayName: string;
+  updatedAt: string;
+  createdAt: string;
+  email: string;
+  resetPasswordToken?: string | null;
+  resetPasswordExpiration?: string | null;
+  salt?: string | null;
+  hash?: string | null;
+  loginAttempts?: number | null;
+  lockUntil?: string | null;
+  sessions?:
+    | {
+        id: string;
+        createdAt?: string | null;
+        expiresAt: string;
+      }[]
+    | null;
+  password?: string | null;
+  collection: 'users';
+}
+/**
+ * This interface was referenced by `Config`'s JSON-Schema
+ * via the `definition` "posts".
+ */
+export interface Post {
+  id: number;
+  title: string;
+  /**
+   * The URL segment. Never change it after publishing.
+   */
+  slug: string;
+  publishedAt: string;
+  /**
+   * Shown on the index and used as the meta description.
+   */
+  excerpt: string;
+  heroImage?: (number | null) | Media;
+  content: {
+    root: {
+      type: string;
+      children: {
+        type: any;
+        version: number;
+        [k: string]: unknown;
+      }[];
+      direction: ('ltr' | 'rtl') | null;
+      format: 'left' | 'start' | 'center' | 'right' | 'end' | 'justify' | '';
+      indent: number;
+      version: number;
+    };
+    [k: string]: unknown;
+  };
+  updatedAt: string;
+  createdAt: string;
+  _status?: ('draft' | 'published') | null;
+}
+/**
+ * This interface was referenced by `Config`'s JSON-Schema
+ * via the `definition` "media".
+ */
+export interface Media {
+  id: number;
+  /**
+   * Describe the image for screen readers.
+   */
+  alt: string;
+  updatedAt: string;
+  createdAt: string;
+  url?: string | null;
+  thumbnailURL?: string | null;
+  filename?: string | null;
+  mimeType?: string | null;
+  filesize?: number | null;
+  width?: number | null;
+  height?: number | null;
+  focalX?: number | null;
+  focalY?: number | null;
+  sizes?: {
+    thumbnail?: {
+      url?: string | null;
+      width?: number | null;
+      height?: number | null;
+      mimeType?: string | null;
+      filesize?: number | null;
+      filename?: string | null;
+    };
+    hero?: {
+      url?: string | null;
+      width?: number | null;
+      height?: number | null;
+      mimeType?: string | null;
+      filesize?: number | null;
+      filename?: string | null;
+    };
+  };
+}
+/**
+ * This interface was referenced by `Config`'s JSON-Schema
+ * via the `definition` "payload-kv".
+ */
+export interface PayloadKv {
+  id: number;
+  key: string;
+  data:
+    | {
+        [k: string]: unknown;
+      }
+    | unknown[]
+    | string
+    | number
+    | boolean
+    | null;
+}
+/**
+ * This interface was referenced by `Config`'s JSON-Schema
+ * via the `definition` "payload-locked-documents".
+ */
+export interface PayloadLockedDocument {
+  id: number;
+  document?:
+    | ({
+        relationTo: 'users';
+        value: number | User;
+      } | null)
+    | ({
+        relationTo: 'posts';
+        value: number | Post;
+      } | null)
+    | ({
+        relationTo: 'media';
+        value: number | Media;
+      } | null);
+  globalSlug?: string | null;
+  user: {
+    relationTo: 'users';
+    value: number | User;
+  };
+  updatedAt: string;
+  createdAt: string;
+}
+/**
+ * This interface was referenced by `Config`'s JSON-Schema
+ * via the `definition` "payload-preferences".
+ */
+export interface PayloadPreference {
+  id: number;
+  user: {
+    relationTo: 'users';
+    value: number | User;
+  };
+  key?: string | null;
+  value?:
+    | {
+        [k: string]: unknown;
+      }
+    | unknown[]
+    | string
+    | number
+    | boolean
+    | null;
+  updatedAt: string;
+  createdAt: string;
+}
+/**
+ * This interface was referenced by `Config`'s JSON-Schema
+ * via the `definition` "payload-migrations".
+ */
+export interface PayloadMigration {
+  id: number;
+  name?: string | null;
+  batch?: number | null;
+  updatedAt: string;
+  createdAt: string;
+}
+/**
+ * This interface was referenced by `Config`'s JSON-Schema
+ * via the `definition` "users_select".
+ */
+export interface UsersSelect<T extends boolean = true> {
+  displayName?: T;
+  updatedAt?: T;
+  createdAt?: T;
+  email?: T;
+  resetPasswordToken?: T;
+  resetPasswordExpiration?: T;
+  salt?: T;
+  hash?: T;
+  loginAttempts?: T;
+  lockUntil?: T;
+  sessions?:
+    | T
+    | {
+        id?: T;
+        createdAt?: T;
+        expiresAt?: T;
+      };
+}
+/**
+ * This interface was referenced by `Config`'s JSON-Schema
+ * via the `definition` "posts_select".
+ */
+export interface PostsSelect<T extends boolean = true> {
+  title?: T;
+  slug?: T;
+  publishedAt?: T;
+  excerpt?: T;
+  heroImage?: T;
+  content?: T;
+  updatedAt?: T;
+  createdAt?: T;
+  _status?: T;
+}
+/**
+ * This interface was referenced by `Config`'s JSON-Schema
+ * via the `definition` "media_select".
+ */
+export interface MediaSelect<T extends boolean = true> {
+  alt?: T;
+  updatedAt?: T;
+  createdAt?: T;
+  url?: T;
+  thumbnailURL?: T;
+  filename?: T;
+  mimeType?: T;
+  filesize?: T;
+  width?: T;
+  height?: T;
+  focalX?: T;
+  focalY?: T;
+  sizes?:
+    | T
+    | {
+        thumbnail?:
+          | T
+          | {
+              url?: T;
+              width?: T;
+              height?: T;
+              mimeType?: T;
+              filesize?: T;
+              filename?: T;
+            };
+        hero?:
+          | T
+          | {
+              url?: T;
+              width?: T;
+              height?: T;
+              mimeType?: T;
+              filesize?: T;
+              filename?: T;
+            };
+      };
+}
+/**
+ * This interface was referenced by `Config`'s JSON-Schema
+ * via the `definition` "payload-kv_select".
+ */
+export interface PayloadKvSelect<T extends boolean = true> {
+  key?: T;
+  data?: T;
+}
+/**
+ * This interface was referenced by `Config`'s JSON-Schema
+ * via the `definition` "payload-locked-documents_select".
+ */
+export interface PayloadLockedDocumentsSelect<T extends boolean = true> {
+  document?: T;
+  globalSlug?: T;
+  user?: T;
+  updatedAt?: T;
+  createdAt?: T;
+}
+/**
+ * This interface was referenced by `Config`'s JSON-Schema
+ * via the `definition` "payload-preferences_select".
+ */
+export interface PayloadPreferencesSelect<T extends boolean = true> {
+  user?: T;
+  key?: T;
+  value?: T;
+  updatedAt?: T;
+  createdAt?: T;
+}
+/**
+ * This interface was referenced by `Config`'s JSON-Schema
+ * via the `definition` "payload-migrations_select".
+ */
+export interface PayloadMigrationsSelect<T extends boolean = true> {
+  name?: T;
+  batch?: T;
+  updatedAt?: T;
+  createdAt?: T;
+}
+/**
+ * This interface was referenced by `Config`'s JSON-Schema
+ * via the `definition` "collections_widget".
+ */
+export interface CollectionsWidget {
+  data?: {
+    [k: string]: unknown;
+  };
+  width: 'full';
+}
+/**
+ * This interface was referenced by `Config`'s JSON-Schema
+ * via the `definition` "auth".
+ */
+export interface Auth {
+  [k: string]: unknown;
+}
+
+
+declare module 'payload' {
+  export interface GeneratedTypes extends Config {}
+}

nextjs-app/payload.config.ts

@@ -0,0 +1,40 @@
+import path from 'path';
+import { fileURLToPath } from 'url';
+import { buildConfig } from 'payload';
+import { postgresAdapter } from '@payloadcms/db-postgres';
+import { lexicalEditor } from '@payloadcms/richtext-lexical';
+import sharp from 'sharp';
+import { Users } from '@/collections/Users';
+import { Posts } from '@/collections/Posts';
+import { Media } from '@/collections/Media';
+import { PAYLOAD_API_ROUTE, PAYLOAD_ADMIN_ROUTE } from '@/lib/payloadRoutes';
+import { migrations } from '@/migrations';
+
+const filename = fileURLToPath(import.meta.url);
+const dirname = path.dirname(filename);
+
+export default buildConfig({
+  admin: { user: Users.slug },
+  // Defined in lib/payloadRoutes.ts, which carries the reasoning and is what
+  // the test asserts. Never inline these — /api belongs to the FastAPI proxy.
+  routes: { api: PAYLOAD_API_ROUTE, admin: PAYLOAD_ADMIN_ROUTE },
+  collections: [Users, Posts, Media],
+  editor: lexicalEditor(),
+  secret: process.env.PAYLOAD_SECRET || '',
+  typescript: { outputFile: path.resolve(dirname, 'payload-types.ts') },
+  db: postgresAdapter({
+    pool: { connectionString: process.env.DATABASE_URL },
+    // Its own schema, so no pipeline operation on `public` can reach blog
+    // content. scripts/migrate_csv_to_db.py --drop lives in that blast radius,
+    // as does Airflow's metadata. The schema itself is created by the initial
+    // migration: schemaName says where tables go, it does not create anything.
+    //
+    // prodMigrations runs pending migrations during server init. Without it a
+    // production container connects to an empty schema and fails its first
+    // query with 42P01 — the adapter cannot self-create tables, because
+    // db-postgres/connect.js gates push on NODE_ENV !== 'production'.
+    schemaName: 'payload',
+    prodMigrations: migrations,
+  }),
+  sharp,
+});

nextjs-app/tsconfig.json

@@ -25,6 +25,9 @@
     "paths": {
       "@/*": [
         "./*"
+      ],
+      "@payload-config": [
+        "./payload.config.ts"
       ]
     }
   },

pipeline/dags/school_data_pipeline.py

@@ -190,7 +190,7 @@ with DAG(
 
     dbt_build_ees = BashOperator(
         task_id="dbt_build",
-        bash_command=f"cd {PIPELINE_DIR}/transform && {DBT_BIN} build --profiles-dir . --target production --select stg_ees_ks2+ stg_legacy_ks2+ stg_ees_ks4+ stg_legacy_ks4+ stg_ees_census+ stg_ees_admissions+ stg_ees_ks2_national+ stg_ees_ks4_national+ stg_ees_ks4_destinations+ stg_ees_ks5_destinations+",
+        bash_command=f"cd {PIPELINE_DIR}/transform && {DBT_BIN} build --profiles-dir . --target production --select stg_ees_ks2+ stg_legacy_ks2+ stg_ees_ks4+ stg_legacy_ks4+ stg_ees_census+ stg_ees_admissions+ stg_ees_ks2_national+ stg_ees_ks4_national+ stg_ees_ks4_destinations+ stg_ees_ks5_destinations+ stg_ees_ks4_destinations_national+ stg_ees_ks5_destinations_national+",
     )
 
     sync_typesense_ees = BashOperator(

pipeline/plugins/extractors/tap-uk-ees-destinations/tap_uk_ees_destinations/tap.py

@@ -188,21 +188,25 @@ class DestinationsStream(Stream):
     _national_pinned: list[str] = []
     _indicators: dict[str, str] = {}
 
-    schema = th.PropertiesList(
-        th.Property("urn", th.StringType),
+    # School rows and the England reference are DIFFERENT GRAINS, so they are
+    # different streams. Carrying both in one table meant a null `urn` inside
+    # the primary key, which target-postgres turns into a NOT NULL constraint:
+    # the first national row killed the loader mid-run, and the tap saw only a
+    # BrokenPipeError on its stdout.
+    _MEASURE_PROPERTIES = (
         th.Property("time_period", th.StringType),
         th.Property("pupil_group", th.StringType),
         th.Property("destination_measure", th.StringType),
         th.Property("cohort_pupils", th.StringType),
         th.Property("pupils_raw", th.StringType),
         th.Property("percentage_raw", th.StringType),
-    ).to_dict()
+    )
+    _MEASURE_KEYS = ["time_period", "pupil_group", "destination_measure"]
 
-    primary_keys = ["urn", "time_period", "pupil_group", "destination_measure"]
     replication_key = None
 
     def _query(self, period: str, pinned: list[str], keep_level: str,
-               urn_by_location: dict[str, str]):
+               urn_by_location: dict[str, str]):  # noqa: D401
         """Page one period of one geographic level, yielding Singer records."""
         criteria = [
             {"filters": {"in": list(self._destination_slugs)}},
@@ -248,19 +252,51 @@ class DestinationsStream(Stream):
             "%s: %d school locations, %d time periods",
             self.name, len(urn_by_location), len(periods),
         )
-
-        # Two queries per period, because school rows and the England reference
-        # need different establishment pins — see KS4_NATIONAL_PINNED. The API
-        # cannot filter by geographic level, so each pass keeps its own and
-        # discards the local-authority, district, regional and constituency
-        # rows that come with them.
         for period in periods:
-            yield from self._query(period, self._pinned, "SCH", urn_by_location)
-            yield from self._query(period, self._national_pinned, "NAT", urn_by_location)
+            yield from self._query(
+                period, self._pinned_for_level, self._level, urn_by_location,
+            )
 
 
-class KS4DestinationsStream(DestinationsStream):
-    name = "ees_ks4_destinations"
+class SchoolDestinationsStream(DestinationsStream):
+    """School-level rows. `urn` is part of the key and is never null."""
+
+    _level = "SCH"
+
+    @property
+    def _pinned_for_level(self):
+        return self._pinned
+
+    schema = th.PropertiesList(
+        th.Property("urn", th.StringType),
+        *DestinationsStream._MEASURE_PROPERTIES,
+    ).to_dict()
+
+    primary_keys = ["urn", *DestinationsStream._MEASURE_KEYS]
+
+
+class NationalDestinationsStream(DestinationsStream):
+    """The England reference. No `urn` column at all — a school identifier that
+    is always null is not a column, it is a grain mismatch."""
+
+    _level = "NAT"
+
+    @property
+    def _pinned_for_level(self):
+        return self._national_pinned
+
+    schema = th.PropertiesList(*DestinationsStream._MEASURE_PROPERTIES).to_dict()
+
+    primary_keys = list(DestinationsStream._MEASURE_KEYS)
+
+    def post_process(self, row, context=None):
+        # row_to_record emits urn=None for national rows; drop the key rather
+        # than ship a column that is null in every row.
+        row.pop("urn", None)
+        return row
+
+
+class _KS4Config(DestinationsStream):
     _dataset_id = KS4_DATASET
     _destination_slugs = KS4_DESTINATION_SLUGS
     _pupil_group_slugs = KS4_PUPIL_GROUP_SLUGS
@@ -269,8 +305,7 @@ class KS4DestinationsStream(DestinationsStream):
     _indicators = KS4_INDICATORS
 
 
-class KS5DestinationsStream(DestinationsStream):
-    name = "ees_ks5_destinations"
+class _KS5Config(DestinationsStream):
     _dataset_id = KS5_DATASET
     _destination_slugs = KS5_DESTINATION_SLUGS
     _pupil_group_slugs = KS5_PUPIL_GROUP_SLUGS
@@ -279,12 +314,33 @@ class KS5DestinationsStream(DestinationsStream):
     _indicators = KS5_INDICATORS
 
 
+class KS4DestinationsStream(SchoolDestinationsStream, _KS4Config):
+    name = "ees_ks4_destinations"
+
+
+class KS5DestinationsStream(SchoolDestinationsStream, _KS5Config):
+    name = "ees_ks5_destinations"
+
+
+class KS4NationalDestinationsStream(NationalDestinationsStream, _KS4Config):
+    name = "ees_ks4_destinations_national"
+
+
+class KS5NationalDestinationsStream(NationalDestinationsStream, _KS5Config):
+    name = "ees_ks5_destinations_national"
+
+
 class TapUKEESDestinations(Tap):
     name = "tap-uk-ees-destinations"
     config_jsonschema = th.PropertiesList().to_dict()
 
     def discover_streams(self):
-        return [KS4DestinationsStream(self), KS5DestinationsStream(self)]
+        return [
+            KS4DestinationsStream(self),
+            KS5DestinationsStream(self),
+            KS4NationalDestinationsStream(self),
+            KS5NationalDestinationsStream(self),
+        ]
 
 
 if __name__ == "__main__":

pipeline/plugins/extractors/tap-uk-ees-destinations/tests/test_tap.py

@@ -132,3 +132,68 @@ def test_establishment_dimensions_are_not_pinned():
     establishment_totals = {"4369U", "rgHcN", "EfHQq", "S4ROV"}
     assert not set(KS4_PINNED) & establishment_totals
     assert not set(KS5_PINNED) & establishment_totals
+
+
+# ── Grain separation ────────────────────────────────────────────────────────
+#
+# School rows and the England reference were originally one stream with a
+# nullable `urn` in the primary key. target-postgres turns primary_keys into a
+# NOT NULL constraint, so the first national row killed the loader mid-run and
+# the tap saw only a BrokenPipeError on its stdout — a symptom several frames
+# away from the cause.
+
+def _streams():
+    from tap_uk_ees_destinations.tap import TapUKEESDestinations
+    return TapUKEESDestinations(config={}, validate_config=False).discover_streams()
+
+
+def test_no_stream_has_a_nullable_primary_key_column():
+    for stream in _streams():
+        props = stream.schema["properties"]
+        for key in stream.primary_keys:
+            assert key in props, f"{stream.name}: key {key} is not in the schema"
+        if "urn" in stream.primary_keys:
+            assert stream._level == "SCH", (
+                f"{stream.name} keys on urn but does not emit school rows"
+            )
+
+
+def test_national_streams_carry_no_urn_column_at_all():
+    for stream in _streams():
+        if not stream.name.endswith("_national"):
+            continue
+        assert "urn" not in stream.schema["properties"], (
+            "a school identifier that is null in every row is a grain "
+            "mismatch, not a column"
+        )
+        assert "urn" not in stream.primary_keys
+
+
+def test_national_post_process_drops_the_null_urn():
+    from tap_uk_ees_destinations.tap import KS4NationalDestinationsStream, TapUKEESDestinations
+    tap = TapUKEESDestinations(config={}, validate_config=False)
+    stream = KS4NationalDestinationsStream(tap)
+    row = {"urn": None, "time_period": "202223", "pupil_group": "all",
+           "destination_measure": "school_sixth_form", "cohort_pupils": "1",
+           "pupils_raw": "1", "percentage_raw": "1"}
+    assert "urn" not in stream.post_process(dict(row))
+
+
+def test_school_and_national_streams_exist_for_both_phases():
+    names = {s.name for s in _streams()}
+    assert names == {
+        "ees_ks4_destinations", "ees_ks5_destinations",
+        "ees_ks4_destinations_national", "ees_ks5_destinations_national",
+    }
+
+
+def test_each_stream_queries_its_own_geographic_level_with_its_own_pins():
+    """The national pass needs establishment pinned to Total; the school pass
+    must not pin it at all, or every school returns zero rows."""
+    for stream in _streams():
+        if stream.name.endswith("_national"):
+            assert stream._level == "NAT"
+            assert stream._pinned_for_level is stream._national_pinned
+        else:
+            assert stream._level == "SCH"
+            assert stream._pinned_for_level is stream._pinned

pipeline/transform/models/marts/fact_destination_national.sql

@@ -22,8 +22,7 @@ select
     pupils,
     percentage,
     status
-from {{ ref('stg_ees_ks4_destinations') }}
-where urn is null
+from {{ ref('stg_ees_ks4_destinations_national') }}
 
 union all
 
@@ -36,5 +35,4 @@ select
     pupils,
     percentage,
     status
-from {{ ref('stg_ees_ks5_destinations') }}
-where urn is null
+from {{ ref('stg_ees_ks5_destinations_national') }}

pipeline/transform/models/staging/_stg_sources.yml

@@ -47,6 +47,17 @@ sources:
           16-18 study leavers destinations. Same grain, same suppression
           caveat, and only institutions with post-16 provision appear.
 
+      - name: ees_ks4_destinations_national
+        description: >
+          England KS4 destination measures by pupil group. A separate table
+          from ees_ks4_destinations because it is a separate grain — no school,
+          so no urn column. Same 'c' suppression caveat.
+
+      - name: ees_ks5_destinations_national
+        description: >
+          England 16-18 destination measures by pupil group. Same grain and
+          caveat as ees_ks4_destinations_national.
+
       - name: ees_ks4_performance
         description: KS4 performance tables (long format — one row per school × breakdown × sex)
 

pipeline/transform/models/staging/stg_ees_ks4_destinations.sql

@@ -1,7 +1,9 @@
 {{ config(materialized='table') }}
 
--- Staging model: KS4 leavers destinations, school level plus the England
--- reference (which carries a null urn).
+-- Staging model: KS4 leavers destinations, school level.
+--
+-- School rows only. The England reference is a different grain and lives in
+-- stg_ees_ks4_destinations_national.
 --
 -- DELIBERATELY DOES NOT USE safe_numeric. That macro maps every EES sentinel
 -- (z, c, x, q, u) to NULL, which is right for attainment — there, "suppressed"
@@ -14,13 +16,12 @@
 
 with source as (
     select * from {{ source('raw', 'ees_ks4_destinations') }}
-    -- National rows carry a null urn and feed fact_destination_national.
-    where (urn is null or urn = '' or urn ~ '^[0-9]+$')
+    where urn ~ '^[0-9]+$'
       and time_period ~ '^[0-9]+$'
 )
 
 select
-    case when urn ~ '^[0-9]+$' then cast(trim(urn) as integer) end as urn,
+    cast(trim(urn) as integer) as urn,
     cast(trim(time_period) as integer)                            as year,
     trim(pupil_group)                                             as pupil_group,
     trim(destination_measure)                                     as destination_measure,

pipeline/transform/models/staging/stg_ees_ks4_destinations_national.sql

@@ -0,0 +1,38 @@
+{{ config(materialized='table') }}
+
+-- Staging model: England KS4 destination measures — the national
+-- reference the school sections compare against.
+--
+-- A separate model because it is a separate grain: there is no school here, and
+-- carrying these rows in the school table meant a null urn inside the primary
+-- key, which the Postgres loader rejects.
+--
+-- DELIBERATELY DOES NOT USE safe_numeric, for the same reason as the school
+-- model: 'suppressed' and 'not applicable' are different claims.
+
+with source as (
+    select * from {{ source('raw', 'ees_ks4_destinations_national') }}
+    where time_period ~ '^[0-9]+$'
+)
+
+select
+    cast(trim(time_period) as integer) as year,
+    trim(pupil_group)                  as pupil_group,
+    trim(destination_measure)          as destination_measure,
+
+    case when cohort_pupils ~ '^[0-9]+$'
+         then cast(cohort_pupils as integer) end as cohort_pupils,
+
+    case when pupils_raw ~ '^[0-9]+$'
+         then cast(pupils_raw as integer) end as pupils,
+
+    case when percentage_raw ~ '^-?[0-9]+(\.[0-9]+)?$'
+         then cast(percentage_raw as numeric) end as percentage,
+
+    case
+        when pupils_raw ~ '^[0-9]+$'        then 'published'
+        when lower(trim(pupils_raw)) = 'c'  then 'suppressed'
+        else 'not_applicable'
+    end as status
+
+from source

pipeline/transform/models/staging/stg_ees_ks5_destinations.sql

@@ -1,8 +1,10 @@
 {{ config(materialized='table') }}
 
--- Staging model: 16-18 study leavers destinations, institution level plus the
--- England reference (which carries a null urn). Only sixth forms and colleges
--- appear here, so a secondary with no post-16 provision has no rows at all.
+-- Staging model: 16-18 study leavers destinations, institution level.
+--
+-- Only sixth forms and colleges appear, so a secondary with no post-16
+-- provision has no rows at all. The England reference is a different grain and
+-- lives in stg_ees_ks5_destinations_national.
 --
 -- DELIBERATELY DOES NOT USE safe_numeric. That macro maps every EES sentinel
 -- (z, c, x, q, u) to NULL, which is right for attainment — there, "suppressed"
@@ -15,13 +17,12 @@
 
 with source as (
     select * from {{ source('raw', 'ees_ks5_destinations') }}
-    -- National rows carry a null urn and feed fact_destination_national.
-    where (urn is null or urn = '' or urn ~ '^[0-9]+$')
+    where urn ~ '^[0-9]+$'
       and time_period ~ '^[0-9]+$'
 )
 
 select
-    case when urn ~ '^[0-9]+$' then cast(trim(urn) as integer) end as urn,
+    cast(trim(urn) as integer) as urn,
     cast(trim(time_period) as integer)                            as year,
     trim(pupil_group)                                             as pupil_group,
     trim(destination_measure)                                     as destination_measure,

pipeline/transform/models/staging/stg_ees_ks5_destinations_national.sql

@@ -0,0 +1,38 @@
+{{ config(materialized='table') }}
+
+-- Staging model: England KS5 destination measures — the national
+-- reference the school sections compare against.
+--
+-- A separate model because it is a separate grain: there is no school here, and
+-- carrying these rows in the school table meant a null urn inside the primary
+-- key, which the Postgres loader rejects.
+--
+-- DELIBERATELY DOES NOT USE safe_numeric, for the same reason as the school
+-- model: 'suppressed' and 'not applicable' are different claims.
+
+with source as (
+    select * from {{ source('raw', 'ees_ks5_destinations_national') }}
+    where time_period ~ '^[0-9]+$'
+)
+
+select
+    cast(trim(time_period) as integer) as year,
+    trim(pupil_group)                  as pupil_group,
+    trim(destination_measure)          as destination_measure,
+
+    case when cohort_pupils ~ '^[0-9]+$'
+         then cast(cohort_pupils as integer) end as cohort_pupils,
+
+    case when pupils_raw ~ '^[0-9]+$'
+         then cast(pupils_raw as integer) end as pupils,
+
+    case when percentage_raw ~ '^-?[0-9]+(\.[0-9]+)?$'
+         then cast(percentage_raw as numeric) end as percentage,
+
+    case
+        when pupils_raw ~ '^[0-9]+$'        then 'published'
+        when lower(trim(pupils_raw)) = 'c'  then 'suppressed'
+        else 'not_applicable'
+    end as status
+
+from source