[
https://issues.apache.org/jira/browse/AIRFLOW-4336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siddharth Anand closed AIRFLOW-4336.
------------------------------------
> GoogleCloudStorageDownloadOperator allow not to save file to log
> ----------------------------------------------------------------
>
> Key: AIRFLOW-4336
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4336
> Project: Apache Airflow
> Issue Type: Wish
> Components: gcp, operators
> Reporter: jack
> Assignee: Kaxil Naik
> Priority: Major
> Fix For: 1.10.4
>
>
> The operator has
> {code:java}
> store_to_xcom_key {code}
> flag which allow the user to deiced if to store the file in Xcom or not but
> there is no flag to specify if the file should be saved to the task log.
> Currently the operator download the file & auto dump the file into the log.
> We use the operator in production. It dump enormousness amount of files of
> GBs which spam the log with millions of rows.
> I fail to see the point of this auto dump to the log. At the end if there is
> a place where something shouldn't be saved to the log it's here. The operator
> gives a copy of the file. There is no point in having it in the log as well.
>
> There are two approaches to fix this:
> # Add flag and allow the user to deiced.
> # Make it default not to save to the log.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
