fix(meltano): remove catalog capability and switch elt to run
All checks were successful
Build and Push Docker Images / Build Backend (FastAPI) (push) Successful in 34s
Build and Push Docker Images / Build Frontend (Next.js) (push) Successful in 1m9s
Build and Push Docker Images / Build Integrator (push) Successful in 57s
Build and Push Docker Images / Build Kestra Init (push) Successful in 31s
Build and Push Docker Images / Build Pipeline (Meltano + dbt + Airflow) (push) Successful in 1m26s
Build and Push Docker Images / Trigger Portainer Update (push) Successful in 0s
All checks were successful
Build and Push Docker Images / Build Backend (FastAPI) (push) Successful in 34s
Build and Push Docker Images / Build Frontend (Next.js) (push) Successful in 1m9s
Build and Push Docker Images / Build Integrator (push) Successful in 57s
Build and Push Docker Images / Build Kestra Init (push) Successful in 31s
Build and Push Docker Images / Build Pipeline (Meltano + dbt + Airflow) (push) Successful in 1m26s
Build and Push Docker Images / Trigger Portainer Update (push) Successful in 0s
The `catalog` capability forced Meltano to run --discover and generate a catalog file (tap.properties.json) before each extraction. This fails because our Singer SDK taps emit schemas inline and don't need external catalog files. Removing the capability makes Meltano invoke taps directly without catalog generation. Also switch from deprecated `meltano elt` to `meltano run` for Meltano 4.x compatibility. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -7,22 +7,15 @@ plugins:
|
||||
namespace: uk_gias
|
||||
pip_url: ./plugins/extractors/tap-uk-gias
|
||||
executable: tap-uk-gias
|
||||
capabilities:
|
||||
- catalog
|
||||
- state
|
||||
settings:
|
||||
- name: download_url
|
||||
kind: string
|
||||
description: GIAS bulk CSV download URL
|
||||
value: https://ea-edubase-api-prod.azurewebsites.net/edubase/downloads/public/edubasealldata.csv
|
||||
|
||||
- name: tap-uk-ees
|
||||
namespace: uk_ees
|
||||
pip_url: ./plugins/extractors/tap-uk-ees
|
||||
executable: tap-uk-ees
|
||||
capabilities:
|
||||
- catalog
|
||||
- state
|
||||
settings:
|
||||
- name: base_url
|
||||
kind: string
|
||||
@@ -35,9 +28,6 @@ plugins:
|
||||
namespace: uk_ofsted
|
||||
pip_url: ./plugins/extractors/tap-uk-ofsted
|
||||
executable: tap-uk-ofsted
|
||||
capabilities:
|
||||
- catalog
|
||||
- state
|
||||
settings:
|
||||
- name: mi_url
|
||||
kind: string
|
||||
@@ -47,16 +37,11 @@ plugins:
|
||||
namespace: uk_parent_view
|
||||
pip_url: ./plugins/extractors/tap-uk-parent-view
|
||||
executable: tap-uk-parent-view
|
||||
capabilities:
|
||||
- catalog
|
||||
|
||||
- name: tap-uk-fbit
|
||||
namespace: uk_fbit
|
||||
pip_url: ./plugins/extractors/tap-uk-fbit
|
||||
executable: tap-uk-fbit
|
||||
capabilities:
|
||||
- catalog
|
||||
- state
|
||||
settings:
|
||||
- name: base_url
|
||||
kind: string
|
||||
@@ -66,8 +51,6 @@ plugins:
|
||||
namespace: uk_idaci
|
||||
pip_url: ./plugins/extractors/tap-uk-idaci
|
||||
executable: tap-uk-idaci
|
||||
capabilities:
|
||||
- catalog
|
||||
|
||||
loaders:
|
||||
- name: target-postgres
|
||||
|
||||
Reference in New Issue
Block a user