fix(pipeline): cache invalidation for IDACI DAG too; curl timeouts
PR Checks / Frontend Typecheck + Tests (pull_request) Successful in 9m37s
PR Checks / Backend Smoke (pull_request) Successful in 6s
PR Checks / Build Backend (no push) (pull_request) Successful in 18s
PR Checks / Build Frontend (no push) (pull_request) Successful in 48s
PR Checks / Build Pipeline (no push) (pull_request) Successful in 35s
PR Checks / AI Code Review (Claude) (pull_request) Successful in 1m40s
PR Checks / Frontend Typecheck + Tests (pull_request) Successful in 9m37s
PR Checks / Backend Smoke (pull_request) Successful in 6s
PR Checks / Build Backend (no push) (pull_request) Successful in 18s
PR Checks / Build Frontend (no push) (pull_request) Successful in 48s
PR Checks / Build Pipeline (no push) (pull_request) Successful in 35s
PR Checks / AI Code Review (Claude) (pull_request) Successful in 1m40s
Addresses AI-review findings: the annual IDACI DAG also rebuilds a mart (fact_deprivation) and needs the reload; curl gets connect/max timeouts so an unreachable backend fails fast instead of hanging the task. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -49,6 +49,7 @@ ADMIN_KEY="${ADMIN_API_KEY:-changeme}"
|
||||
echo "Calling $BACKEND_URL/api/admin/reload ..."
|
||||
|
||||
response=$(curl -s -o /tmp/reload_response.json -w "%{http_code}" \\
|
||||
--connect-timeout 10 --max-time 120 \\
|
||||
-X POST "$BACKEND_URL/api/admin/reload" \\
|
||||
-H "X-API-Key: $ADMIN_KEY" \\
|
||||
-H "Content-Type: application/json")
|
||||
@@ -217,4 +218,9 @@ with DAG(
|
||||
bash_command=f"cd {PIPELINE_DIR}/transform && {DBT_BIN} build --profiles-dir . --target production --select stg_idaci+ fact_deprivation+",
|
||||
)
|
||||
|
||||
extract_idaci >> dbt_build_idaci
|
||||
invalidate_cache_idaci = BashOperator(
|
||||
task_id="invalidate_cache",
|
||||
bash_command=INVALIDATE_CACHE_CMD,
|
||||
)
|
||||
|
||||
extract_idaci >> dbt_build_idaci >> invalidate_cache_idaci
|
||||
|
||||
Reference in New Issue
Block a user