[ https://issues.apache.org/jira/browse/AIRFLOW-5072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16923561#comment-16923561 ]
ASF subversion and git services commented on AIRFLOW-5072: ---------------------------------------------------------- Commit 4ab6982f0b1d42f20c7d6d62734636445f7e0104 in airflow's branch refs/heads/master from Tobias Kaymak [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=4ab6982 ] [AIRFLOW-5072] gcs_hook should download once (#5685) When a user supplied a filename the expected behaviour is that airflow downloads the file and does not return it's content as a string. > gcs_hook causes out-of-memory error when downloading huge files > --------------------------------------------------------------- > > Key: AIRFLOW-5072 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5072 > Project: Apache Airflow > Issue Type: Bug > Components: gcp > Affects Versions: 1.10.3 > Reporter: Tobias Kaymak > Assignee: Tobias Kaymak > Priority: Major > > Possibly there is an "else" missing here, but the gcs_hook's `download` > method *always* downloads a blob as a string even when a filename was > supplied. This causes the method to take twice as long when a filename is > supplied and for huge blobs it can even cause out-of-memory errors. > I think that there is an else missing? > [https://github.com/apache/airflow/blob/05c01a97497e992c7d8b05a39a7855343dee1603/airflow/contrib/hooks/gcs_hook.py#L176] -- This message was sent by Atlassian Jira (v8.3.2#803003)