fix(api): bound what a forged CF-Connecting-IP can buy

Code review, both findings valid.

The design doc claimed Cloudflare "replaces the header, so a browser
cannot forge it", and that only the X-Forwarded-For fallback was
forgeable. That is true only for traffic that actually passed through
Cloudflare, and nothing in this process can verify that it did. Reaching
the origin directly, both headers are equally attacker-controlled — and
rotating CF-Connecting-IP mints a fresh rate-limit bucket per request,
defeating per-client limits on every endpoint including the
DataFrame-heavy /api/schools. Against abuse that is worse than the
shared bucket it replaced, which at least capped everyone together.

So the ceiling comes back. I dropped it earlier arguing it belonged at
Cloudflare; that argument assumed the keying was sound, and it is not.
GlobalRateLimitMiddleware counts all /api/ traffic in a fixed window
against a total, independent of client identity, outermost so it refuses
before any work happens. Written by hand because slowapi cannot express
a global cap: default_limits and application_limits are both keyed by
key_func, and the latter needs middleware this app does not install.

It does not make the header trustworthy — it makes trusting it
survivable. The real fix is Authenticated Origin Pulls or an origin
firewall, now documented in DEPLOY.md as the open gap it is.

127.0.0.1 is exempt: the healthcheck curls localhost from inside the
container, and starving it would restart the container and turn a load
spike into an outage loop. Keyed on the peer address, never the Host
header, which the caller sets.

Second finding: suggest_schools_typesense promised "never raises" while
the parsing loop sat outside the try, so int(None) on a malformed
document would have made a keystroke a 500. The loop now skips bad rows
rather than dropping the whole list — and a hit with no document no
longer becomes a suggestion pointing at /school/0.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015mWQnpye9F299NVRCCSRvj

docs/superpowers/specs/2026-08-26-school-autosuggest-design.md

@@ -58,42 +58,80 @@ def client_key(request: Request) -> str:
 except `host` and `connection`, so `CF-Connecting-IP` reaches the backend with
 no proxy change.
 
-Two things make this safe: Cloudflare replaces the header, so a browser cannot
-forge it; and the backend is unreachable from outside the Docker network, so
-nothing can reach it without passing through the proxy. The `X-Forwarded-For`
-fallback *is* forgeable, but only by a caller already inside that network.
+**This header is trustworthy only for traffic that actually passed through
+Cloudflare, and nothing in the application can verify that it did.** An earlier
+draft of this section claimed Cloudflare "replaces the header, so a browser
+cannot forge it", and that only the `X-Forwarded-For` fallback was forgeable.
+That was wrong. Cloudflare does overwrite the header *on requests it handles* —
+but a caller reaching the origin directly sets whatever it likes, and this
+process cannot distinguish an edge-set header from an attacker-set one. Both
+headers are equally forgeable in that scenario.
 
-### The part that is not free
+The consequence is sharper than a weakened defence. An attacker rotating
+`CF-Connecting-IP` per request mints a fresh rate-limit bucket every time and
+evades per-client limits entirely — including on the DataFrame-heavy
+`/api/schools`. Against abuse that is *worse* than the shared bucket it
+replaced, which at least capped everyone at 60/minute together.
 
-The shared bucket has been acting as an accidental global throttle on a
+Two mitigations, and they are not interchangeable:
+
+1. **The real fix is at Cloudflare** — Authenticated Origin Pulls, or an origin
+   firewall that refuses connections not from Cloudflare's ranges. Only the
+   edge can vouch for its own header. This is infrastructure work and is not
+   part of this change; it is the thing that makes the header mean anything.
+2. **The ceiling in §1.1 bounds what evading the keying can achieve** while
+   that remains open. It does not make the header trustworthy — it makes
+   trusting it survivable.
+
+The backend being unreachable from outside the Docker network is a real second
+layer, but it depends on the ingress path in front of the frontend, which this
+design does not control and should not assume.
+
+### 1.1 The ceiling, which is back
+
+The shared bucket was acting as an accidental global throttle on a
 single-process uvicorn backend that filters a 25,000-row DataFrame in-process.
-Correct per-user keying removes that throttle: the origin becomes reachable at
-60/min *per user* rather than 60/min in total.
+Correct per-user keying removes it: the origin becomes reachable at 60/min *per
+user* rather than 60/min in total, and — per above — at an unbounded rate by
+anyone willing to rotate a header.
 
-Per-user fairness and origin protection are different jobs. Conflating them
-is what produced the current behaviour, and the fix must not quietly do it
-again in the other direction.
+An earlier draft dropped the in-app ceiling, arguing it belonged at Cloudflare.
+That argument assumed the keying was sound. It is not, so the ceiling is
+load-bearing rather than redundant, and it ships here:
 
-**The global ceiling does not go in this app.** An earlier draft of this
-section specified one via slowapi's `default_limits`. Reading the library
-shows that would not have worked, twice over: `default_limits` and
-`application_limits` are both evaluated with the same `key_func`, so they are
-per-client across all routes rather than global; and `application_limits` are
-only applied `if in_middleware`, while this app installs no `SlowAPIMiddleware`
-at all. Expressing a genuine global cap would take a second `Limiter` with a
-constant key plus that middleware — two mechanisms to keep correct, for a
-protection this layer is the wrong place for.
+`GlobalRateLimitMiddleware` counts all `/api/` requests in a fixed 60-second
+window against `global_rate_limit_per_minute` (3000), independent of any client
+identity, and refuses with a 429 that names capacity rather than the client —
+an operator has to be able to tell "one noisy client" from "the origin is
+saturated". It is registered last so it is outermost: a ceiling that applies
+after the expensive work has run is not a ceiling.
 
-Cloudflare is already in the request path on both environments and does
-edge-level rate limiting properly, before traffic reaches a single-process
-origin at all. That is where a global ceiling belongs, and it is a dashboard
-change rather than code. Flagged as a follow-up, deliberately not built here.
+slowapi cannot express this. `default_limits` and `application_limits` are both
+evaluated with the same `key_func`, making them per-client rather than global,
+and `application_limits` only apply with `SlowAPIMiddleware` installed, which
+this app does not use. Hence the explicit middleware — about thirty lines, and
+obviously correct, which is what a backstop needs to be.
 
-What ships instead is conservative per-user limits: the existing 60/minute
-default is unchanged, and `/api/suggest` gets 120/minute. Both are estimates
-rather than measurements, and they are a starting point to revisit once the
-keying is correct enough for real per-user traffic to be visible — which it
-is not today, because everyone shares one bucket.
+Requests from `127.0.0.1` are exempt. The container healthcheck runs
+`curl http://localhost:80/api/data-info` from inside the container, and
+starving it would fail the check, restart the container, and turn a load spike
+into an outage loop. The exemption keys on the peer address, never the `Host`
+header, which the caller sets.
+
+3000/minute is an estimate, not a measurement, and worth revisiting against
+real traffic.
+
+### Per-user limits
+
+Per-user fairness and origin protection are different jobs, and this design now
+does both separately: the ceiling above for the origin, and per-route limits
+for fairness. Conflating them is what produced the original behaviour, where
+one bucket served the whole internet.
+
+The existing 60/minute default is unchanged, and `/api/suggest` gets
+120/minute. Both are estimates rather than measurements, and are a starting
+point to revisit once the keying is correct enough for real per-user traffic to
+be visible — which it was not before, because everyone shared one bucket.
 
 ## 2. `GET /api/suggest`
 
@@ -228,11 +266,14 @@ with many source addresses can put more load on a single-process origin than
 they can today. Against this site's traffic that is a theoretical risk rather
 than a live one, but it is a real one and it is the price of the fix.
 
-**Cloudflare bypass.** If the origin is reachable without passing through
-Cloudflare, `CF-Connecting-IP` is absent and the `X-Forwarded-For` fallback is
-forgeable, so limits could be evaded per-request. Closing that properly means
-Authenticated Origin Pulls or an origin firewall, which is infrastructure work
-outside this change. Worth doing separately.
+**Cloudflare bypass — the open one.** If the origin is reachable without
+passing through Cloudflare, `CF-Connecting-IP` is attacker-controlled, and
+rotating it per request defeats per-client limits on every endpoint. The
+ceiling in §1.1 bounds the damage to the origin's total capacity; it does not
+restore per-client fairness under attack, and it cannot. Closing this properly
+means Authenticated Origin Pulls or an origin firewall restricted to
+Cloudflare's published ranges — infrastructure work, outside this change, and
+the single most valuable follow-up here.
 
 **Typesense becomes user-visible.** Today a Typesense outage degrades search to
 a slow substring match. With autosuggest it also means the dropdown silently