Uploads go to a named volume at /app/media. The directory is created in
the image before the mount and covered by the existing chown, because
Docker seeds a fresh named volume from the image path — a missing or
root-owned directory there fails every upload with EACCES at runtime,
long after the build passed.
PAYLOAD_SECRET uses the same :? form as AIRFLOW_ADMIN_PASSWORD: refuse
to start rather than boot with an empty secret and accept forged
sessions. Staging's must differ from production's, which the header
comment now says explicitly. Portainer prefixes volume names per stack,
so payload_media isolates itself.
prodMigrations is not wired yet — generating the initial migration needs
a reachable Postgres. Follows in its own commit.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017YmbBhr8s7GusjDE12hrZM
The simple auth manager generates a random password on first start and
writes it to a file, so every restart of the api-server invalidated the
last one and the password had to be dug out of the container logs again.
The stack now writes that file itself from AIRFLOW_ADMIN_PASSWORD before
exec'ing the api-server. Airflow generates nothing when the file already
exists, so the login is whatever the stack environment says it is.
Written with python rather than echo, so json.dumps escapes a password
containing quotes, backslashes or non-ASCII correctly — verified against
`p@ss "wo\rd' £5`, which round-trips intact.
An unset AIRFLOW_ADMIN_PASSWORD raises KeyError and the container exits.
Falling back to a generated password would silently undo the point of the
change, and a compose-level `:?` gives the same refusal a readable reason.
This does mean the variable MUST be set in Portainer before the next
deploy of either stack.
Not affected by the two Docker gotchas in the upstream docs: this image
has no USER directive so it runs as root, and the file is rewritten from
the environment on every start rather than persisted on a volume.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BvdDKvFFSZuMVDH5fEyTob
Both variables default to empty, so an environment without Unleash has
every flag off — the correct dark state rather than a boot failure.
The cache volume is the mitigation for the one real regression risk in
this design: the SDK evaluates everything False until it syncs, so a
backend cold-starting with an empty cache while Unleash is unreachable
would make a *released* feature disappear. On a named volume the disk
cache survives a restart.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015mWQnpye9F299NVRCCSRvj
- pr-checks.yml: PR gate — frontend typecheck+jest, backend import smoke,
image builds (no push), Claude AI review posted as PR comment (severe
findings block merge)
- deploy.yml (replaces build-and-push.yml): merge to main builds+pushes
images tagged sha-<sha>/staging, deploys the staging Portainer stack via
webhook, runs Playwright E2E journeys against staging, then retags the
verified images :prod (previous kept as :prod-previous) and deploys prod
- docker-compose.portainer.staging.yml: second Portainer stack — :staging
images, sc_staging_* names, own macvlan IPs, Airflow on 8081; data
bootstrapped from source via the staging Airflow DAGs
- prod compose now pins :prod instead of :latest (only the promotion step
moves it; :latest is no longer published)
- e2e/: 6 Playwright journeys (search, postcode, detail, compare, rankings)
driven by BASE_URL — the promotion gate
- scripts/ci/ai_review.py: Claude review with structured JSON findings
- docs/DEPLOY.md: full SDLC doc incl. one-time setup checklist and rollback
- replaced removed 'next lint' with tsc typecheck; fixed stale jest tests
(slug URLs, N/A formatting, stable trend, fake-timer setup)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PqGhF93UrpDNvXBLMjJENL
The postgis/postgis image auto-enables PostGIS on fresh database creation.
No need to do it from airflow-init.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When tasks are removed from a DAG, old serialized metadata in the DB
causes 'Task not found' errors. Delete all DAGs before reserializing
on each deploy to ensure a clean state.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace Airflow 2.x env vars (CORE__SECRET_KEY, CORE__INTERNAL_API_URL) with
correct Airflow 3.x equivalents (API_AUTH__JWT_SECRET, API_AUTH__JWT_ISSUER,
CORE__EXECUTION_API_SERVER_URL) on all three Airflow services.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
With separate containers, task workers in the scheduler need the
api-server's address for the Execution API. Defaults to localhost:8080
which fails across containers. Set INTERNAL_API_URL to the api-server's
Docker service name.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Running both in one container caused JWT secret key race conditions.
Separate containers with the same AIRFLOW__CORE__SECRET_KEY env var
ensures both processes use identical JWT signing keys. Shared
airflow_logs volume allows the api-server to read task logs written
by the scheduler.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The init container and airflow container have separate filesystems, so
airflow.cfg generated by db migrate is not available to the scheduler/
api-server. Without a config file, both processes race to generate
their own with different random JWT secret keys.
Fix by:
1. Running `airflow config list` first to generate airflow.cfg once
2. Setting a fixed SECRET_KEY via env var (>= 64 bytes for SHA512)
3. Adding sleep 3 so scheduler writes config before api-server starts
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Deleting airflow.cfg at container start caused the scheduler and
api-server to each generate their own random JWT secret key, leading
to 'Signature verification failed' when task workers communicated
with the api-server. Let both processes share the config file
generated by db migrate (env vars still override where needed).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When scheduler and api-server run in the same container, both generate
independent JWT signing keys on startup. The scheduler's task workers
then fail with 'Invalid auth token: Signature verification failed'
when communicating with the api-server. Fix by setting a shared
INTERNAL_API_SECRET_KEY via env var.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
With LocalExecutor, tasks run in the scheduler process and logs are
written locally. Running api-server and scheduler in separate containers
meant the api-server couldn't read task logs (empty hostname in log
fetch URL). Combining them into one container eliminates the issue —
logs are always on the local filesystem.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
airflow db migrate generates airflow.cfg with default values that
shadow our env vars (DAGS_FOLDER, WORKER_LOG_SERVER_HOST, etc).
Delete the generated config file before starting each service so
Airflow falls through to env var configuration exclusively.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The scheduler's log server binds to [::]:8793 but doesn't advertise a
hostname, so the api-server gets 'http://:8793/...' (no host) when
fetching task logs. Fix by setting the scheduler's hostname and
configuring WORKER_LOG_SERVER_HOST so the api-server can reach it.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The api-server couldn't fetch task logs because LocalExecutor runs tasks
in the scheduler process, writing logs to its local filesystem. The
api-server tried to fetch via HTTP but the scheduler's log server had
no hostname set. Fix by sharing a named volume for logs between both
containers so the api-server reads logs directly from the filesystem.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add AIRFLOW__CORE__DAGS_FOLDER env var in Dockerfile so it's always set
- Run `airflow dags reserialize` after `db migrate` in init container so
DAGs appear immediately without waiting for scheduler scan interval
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The named volume was shadowing the DAGs built into the pipeline image
with an empty directory. DAGs now served directly from the image and
update on each CI build.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Typesense image has neither curl nor wget. Use bash /dev/tcp for a
simple port connectivity check instead.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Airflow 3 replaced `airflow webserver` with `airflow api-server` and
removed the `airflow users` CLI. Auth is now via SimpleAuthManager
configured through AIRFLOW__CORE__SIMPLE_AUTH_MANAGER_USERS env var.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
tudorandClaude Sonnet 4.6