airflow

DBT 1.0.0 on AWS MWAA

Initially I thought that it sounds too good to be true: https://docs.aws.amazon.com/mwaa/latest/userguide/samples-dbt.html

Then I decided to give it a try – only to realise that dbt-core=0.21.1 ?!

Then I went for dbt-core=1.3.0 – it didn’t work -> collision with apache-ariflow=2.2.2

Then I tried dbt-core=1.0.0 – that one worked. It will work for you too, just use the following requirements.txt

apache-airflow[postgres,mysql,google]==2.2.2

json-rpc==1.13.0
minimal-snowplow-tracker==0.0.2
packaging==20.9
networkx==2.6.3 
mashumaro==2.9
sqlparse==0.4.2

logbook==1.5.3
agate==1.6.1
dbt-extractor==0.4.0

pyparsing==2.4.7 
msgpack==1.0.4
parsedatetime==2.6
pytimeparse==1.1.8
leather==0.3.4
pyyaml==6.0

# Airflow constraints are jsonschema==3.2.0
jsonschema==3.1.1
hologram==0.0.14
dbt-core==1.0.0

psycopg2-binary==2.8.6
dbt-postgres==1.0.0
dbt-redshift==1.0.0
0
Tags: , , ,

About: user


One thought on “DBT 1.0.0 on AWS MWAA”

  1. I just like the valuable information you provide on your articles. I will bookmark your weblog and check again right here regularly. I’m reasonably certain I will learn many new stuff proper right here! Best of luck for the next!

Leave a Reply

Your email address will not be published. Required fields are marked *