Kengo Seki created AIRFLOW-2525: ----------------------------------- Summary: Fix PostgresHook.copy_expert to work with "COPY FROM" Key: AIRFLOW-2525 URL: https://issues.apache.org/jira/browse/AIRFLOW-2525 Project: Apache Airflow Issue Type: Bug Components: hooks Reporter: Kengo Seki Assignee: Kengo Seki
psycopg2's {{copy_expert}} [works with both "COPY FROM" and "COPY TO"|http://initd.org/psycopg/docs/cursor.html#cursor.copy_expert]. But {{PostgresHook.copy_expert}} fails with "COPY FROM" as follows: {code} In [1]: from airflow.hooks.postgres_hook import PostgresHook In [2]: hook = PostgresHook() In [3]: hook.copy_expert("COPY t FROM STDIN", "/tmp/t") [2018-05-24 23:37:54,767] {base_hook.py:83} INFO - Using connection to: localhost --------------------------------------------------------------------------- QueryCanceledError Traceback (most recent call last) <ipython-input-3-7d9c6c8c57fc> in <module>() ----> 1 hook.copy_expert("COPY t FROM STDIN", "/tmp/t") ~/dev/incubator-airflow/airflow/hooks/postgres_hook.py in copy_expert(self, sql, filename, open) 68 with closing(self.get_conn()) as conn: 69 with closing(conn.cursor()) as cur: ---> 70 cur.copy_expert(sql, f) 71 72 @staticmethod QueryCanceledError: COPY from stdin failed: error in .read() call: UnsupportedOperation not readable CONTEXT: COPY t, line 1 {code} This is because the file used in this method is opened with 'w' mode. -- This message was sent by Atlassian JIRA (v7.6.3#76005)