Skip to content Skip to sidebar Skip to footer
Showing posts with the label Airflow

How To Use Non-installable Modules From Dag Code?

I have a Git repository which (among other things) holds Airflow DAGs in airflow directory. I have … Read more How To Use Non-installable Modules From Dag Code?

Python - Attributeerror: 'nonetype' Object Has No Attribute 'execute'

I am trying to run a python script that logs into Amazon Redshift DB and then execute a SQL command… Read more Python - Attributeerror: 'nonetype' Object Has No Attribute 'execute'

Apache Airflow Dag Cannot Import Local Module

I do not seem to understand how to import modules into an apache airflow DAG definition file. I wou… Read more Apache Airflow Dag Cannot Import Local Module

Unexpected Airflow Behaviour In Dynamic Task Generation

For reasons acceptable to me, I am trying to dynamically generate ExternalTaskSensor tasks with dif… Read more Unexpected Airflow Behaviour In Dynamic Task Generation

Paperboy Throwing Error: Typeerror: The Json Object Must Be Str, Not 'bytes'?

I have installed paperboy using steps given at https://github.com/timkpaine/paperboy.git and execut… Read more Paperboy Throwing Error: Typeerror: The Json Object Must Be Str, Not 'bytes'?

Run Airflow Dag For Each File

So I have this quite nice DAG in airflow which basically runs several analysis steps (implemented a… Read more Run Airflow Dag For Each File

Retrieve Full Connection Uri From Airflow Postgres Hook

Is there a neater way to get the complete URI from a Postgres hook? .get_uri() doesn't include … Read more Retrieve Full Connection Uri From Airflow Postgres Hook

How Do I Restart Airflow Webserver?

I am using airflow for my data pipeline project. I have configured my project in airflow and start … Read more How Do I Restart Airflow Webserver?

Composer On Google Platform Not Available For Python 3

According to the documentation here: https://cloud.google.com/composer/docs/release-notes Composer … Read more Composer On Google Platform Not Available For Python 3

"handling Signal: Ttou" Message While Running Dag In Airflow

I have created sample DAG, where I had DAG config as below. default_args = { 'owner': &… Read more "handling Signal: Ttou" Message While Running Dag In Airflow

Templatesyntaxerror: Unexpected Char U'\\' At 205 Airflow Macros

I am getting the following error : File ' ', line 1, in template TemplateSyntaxError: unex… Read more Templatesyntaxerror: Unexpected Char U'\\' At 205 Airflow Macros

Airflow - Initiation Of DB Stuck In SQL Server

Trying to setup airflow using SQL Server as the backend, but get stuck during the initdb command: u… Read more Airflow - Initiation Of DB Stuck In SQL Server

Airflow 1.9.0 ExternalTaskSensor Retry_delay=30 Yields TypeError: Can't Pickle _thread.RLock Objects

As the titles says; in Airflow 1.9.0 if you use the retry_delay=30 (or any other number) parameter … Read more Airflow 1.9.0 ExternalTaskSensor Retry_delay=30 Yields TypeError: Can't Pickle _thread.RLock Objects

Airflow BashOperator Can't Find Bash

I'm using Airflow in Centos 7, using Python 3.7. When I run a Bash command through BashOperator… Read more Airflow BashOperator Can't Find Bash