tudor/school_compare
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(airflow): generate config before starting processes, set fixed secret key
Some checks failed
Build and Push Docker Images / Build Backend (FastAPI) (push) Successful in 31s
Details
Build and Push Docker Images / Build Frontend (Next.js) (push) Successful in 1m3s
Details
Build and Push Docker Images / Build Integrator (push) Successful in 54s
Details
Build and Push Docker Images / Build Pipeline (Meltano + dbt + Airflow) (push) Has been cancelled
Details
Build and Push Docker Images / Trigger Portainer Update (push) Has been cancelled
Details
Build and Push Docker Images / Build Kestra Init (push) Has been cancelled
Details

Browse Source 
The init container and airflow container have separate filesystems, so
airflow.cfg generated by db migrate is not available to the scheduler/
api-server. Without a config file, both processes race to generate
their own with different random JWT secret keys.

Fix by:
1. Running `airflow config list` first to generate airflow.cfg once
2. Setting a fixed SECRET_KEY via env var (>= 64 bytes for SHA512)
3. Adding sleep 3 so scheduler writes config before api-server starts

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Tudor
2026-03-26 16:57:22 +00:00
parent 1dbcc24434
commit 677e80ad70
1 changed files with 2 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
docker-compose.portainer.yml
View File

@@ -188,7 +188,7 @@ services:
airflow: airflow:
image: privaterepo.sitaru.org/tudor/school_compare-pipeline:latest image: privaterepo.sitaru.org/tudor/school_compare-pipeline:latest
container_name: schoolcompare_airflow container_name: schoolcompare_airflow
command: bash -c "airflow scheduler & exec airflow api-server --port 8080" command: bash -c "airflow config list >/dev/null 2>&1 && airflow scheduler & sleep 3 && exec airflow api-server --port 8080"
ports: ports:
- "8080:8080" - "8080:8080"
environment: environment:
@@ -196,6 +196,7 @@ services:
AIRFLOW__DATABASE__SQL_ALCHEMY_CONN: postgresql+psycopg2://${DB_USERNAME}:${DB_PASSWORD}@sc_database:5432/${DB_DATABASE_NAME} AIRFLOW__DATABASE__SQL_ALCHEMY_CONN: postgresql+psycopg2://${DB_USERNAME}:${DB_PASSWORD}@sc_database:5432/${DB_DATABASE_NAME}
AIRFLOW__CORE__DAGS_FOLDER: /opt/pipeline/dags AIRFLOW__CORE__DAGS_FOLDER: /opt/pipeline/dags
AIRFLOW__CORE__LOAD_EXAMPLES: "false" AIRFLOW__CORE__LOAD_EXAMPLES: "false"
AIRFLOW__CORE__SECRET_KEY: "school-compare-airflow-secret-key-that-is-long-enough-for-sha512-jwt-signing"
AIRFLOW__CORE__SIMPLE_AUTH_MANAGER_USERS: "${AIRFLOW_ADMIN_USER:-admin}:admin" AIRFLOW__CORE__SIMPLE_AUTH_MANAGER_USERS: "${AIRFLOW_ADMIN_USER:-admin}:admin"
AIRFLOW__LOGGING__BASE_LOG_FOLDER: /opt/airflow/logs AIRFLOW__LOGGING__BASE_LOG_FOLDER: /opt/airflow/logs
PG_HOST: sc_database PG_HOST: sc_database