When tasks are removed from a DAG, old serialized metadata in the DB
causes 'Task not found' errors. Delete all DAGs before reserializing
on each deploy to ensure a clean state.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace Airflow 2.x env vars (CORE__SECRET_KEY, CORE__INTERNAL_API_URL) with
correct Airflow 3.x equivalents (API_AUTH__JWT_SECRET, API_AUTH__JWT_ISSUER,
CORE__EXECUTION_API_SERVER_URL) on all three Airflow services.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
With separate containers, task workers in the scheduler need the
api-server's address for the Execution API. Defaults to localhost:8080
which fails across containers. Set INTERNAL_API_URL to the api-server's
Docker service name.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Running both in one container caused JWT secret key race conditions.
Separate containers with the same AIRFLOW__CORE__SECRET_KEY env var
ensures both processes use identical JWT signing keys. Shared
airflow_logs volume allows the api-server to read task logs written
by the scheduler.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The init container and airflow container have separate filesystems, so
airflow.cfg generated by db migrate is not available to the scheduler/
api-server. Without a config file, both processes race to generate
their own with different random JWT secret keys.
Fix by:
1. Running `airflow config list` first to generate airflow.cfg once
2. Setting a fixed SECRET_KEY via env var (>= 64 bytes for SHA512)
3. Adding sleep 3 so scheduler writes config before api-server starts
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Deleting airflow.cfg at container start caused the scheduler and
api-server to each generate their own random JWT secret key, leading
to 'Signature verification failed' when task workers communicated
with the api-server. Let both processes share the config file
generated by db migrate (env vars still override where needed).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When scheduler and api-server run in the same container, both generate
independent JWT signing keys on startup. The scheduler's task workers
then fail with 'Invalid auth token: Signature verification failed'
when communicating with the api-server. Fix by setting a shared
INTERNAL_API_SECRET_KEY via env var.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
With LocalExecutor, tasks run in the scheduler process and logs are
written locally. Running api-server and scheduler in separate containers
meant the api-server couldn't read task logs (empty hostname in log
fetch URL). Combining them into one container eliminates the issue —
logs are always on the local filesystem.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
airflow db migrate generates airflow.cfg with default values that
shadow our env vars (DAGS_FOLDER, WORKER_LOG_SERVER_HOST, etc).
Delete the generated config file before starting each service so
Airflow falls through to env var configuration exclusively.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The scheduler's log server binds to [::]:8793 but doesn't advertise a
hostname, so the api-server gets 'http://:8793/...' (no host) when
fetching task logs. Fix by setting the scheduler's hostname and
configuring WORKER_LOG_SERVER_HOST so the api-server can reach it.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The api-server couldn't fetch task logs because LocalExecutor runs tasks
in the scheduler process, writing logs to its local filesystem. The
api-server tried to fetch via HTTP but the scheduler's log server had
no hostname set. Fix by sharing a named volume for logs between both
containers so the api-server reads logs directly from the filesystem.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add AIRFLOW__CORE__DAGS_FOLDER env var in Dockerfile so it's always set
- Run `airflow dags reserialize` after `db migrate` in init container so
DAGs appear immediately without waiting for scheduler scan interval
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The named volume was shadowing the DAGs built into the pipeline image
with an empty directory. DAGs now served directly from the image and
update on each CI build.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Typesense image has neither curl nor wget. Use bash /dev/tcp for a
simple port connectivity check instead.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Airflow 3 replaced `airflow webserver` with `airflow api-server` and
removed the `airflow users` CLI. Auth is now via SimpleAuthManager
configured through AIRFLOW__CORE__SIMPLE_AUTH_MANAGER_USERS env var.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
