[
https://issues.apache.org/jira/browse/AIRFLOW-3000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ash Berlin-Taylor closed AIRFLOW-3000.
--------------------------------------
Resolution: Won't Do
The example given in the stack-over flow answer (of sub-classing the Operator)
is one way to do to this.
However this is an anti-pattern: this is going to get printed _every time
Airflow parses the DAG_ - not just when it runs, but every time Airflow goes
around it's parsing loop. This is going to be noisy and create possibly many GB
of logs per day.
> Allow to print into the log from operators (ability to Base Operator)
> ---------------------------------------------------------------------
>
> Key: AIRFLOW-3000
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3000
> Project: Apache Airflow
> Issue Type: Task
> Affects Versions: 1.10.0
> Reporter: jack
> Priority: Minor
>
> As described on stack overflow:
> [https://stackoverflow.com/questions/52144108/how-to-print-a-unique-message-in-airflow-operator]
>
> Any print in the code will be shown on the log file. However it a problem
> when creating operators dinamicly
>
> assume this code:
>
> {code:java}
> for i in range(5, 0, -1):
> print("My name is load_ads_to_BigQuery-{}".format{i))
> update_bigquery = GoogleCloudStorageToBigQueryOperator
> (task_id='load_ads_to_BigQuery-{}'.format(I),…){code}
>
> This creates 5 operators.
> The print will be executed 5 times per each operator.
> meaning that if you go to the log of
> {code:java}
> load_ads_to_BigQuery-1 {code}
> you will see:
>
>
> {code:java}
> My name is load_ads_to_BigQuery-1
> My name is load_ads_to_BigQuery-2
> My name is load_ads_to_BigQuery-3
> My name is load_ads_to_BigQuery-4
> My name is load_ads_to_BigQuery-5
> {code}
>
>
> This is a problem because it logs messages of the other operators.
>
> Each operator is unique only with-in the operator itself. meaning that the
> print should be inside the operator as:
> for i in range(5, 0, -1):
>
>
> {code:java}
> update_bigquery = GoogleCloudStorageToBigQueryOperator
> (task_id='load_ads_to_BigQuery-{}'.format(I) , print("My name is
> load_ads_to_BigQuery-{}".format{i)) , …){code}
>
> or something like it. However Airflow does not support printing inside
> operators. It's not one of the allowed arguments.
> Add optional parameter called
> {code:java}
> msg_log {code}
> that if assigned with value it will print the value to the log when the
> operator is executed.
>
> Please add argument on the Base Operator for printing and extend it as
> optional ability to all operators.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
