[ https://issues.apache.org/jira/browse/AIRFLOW-2697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sajid Sajid reassigned AIRFLOW-2697: ------------------------------------ Assignee: Sajid Sajid (was: Julian de Ruiter) > Drop snakebite in favour of pyarrow > ----------------------------------- > > Key: AIRFLOW-2697 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2697 > Project: Apache Airflow > Issue Type: Improvement > Affects Versions: 1.9.0 > Reporter: Julian de Ruiter > Assignee: Sajid Sajid > Priority: Blocker > Labels: gsoc, gsoc2020, mentor > Fix For: 2.0.0 > > > The current HdfsHook relies on the snakebite library, which is unfortunately > not compatible with Python 3. To add Python 3 support for the HdfsHook > requires switching to a different library for interacting with HDFS. The > hdfs3 library is an attractive alternative, as it supports Python 3 and seems > to be stable and relatively well supported. > Update: hdfs3 doesn't get any updates anymore. The best library right now > seems to be pyarrow: https://arrow.apache.org/docs/python/filesystems.html > Therefore I would like to upgrade to pyarrow instead of hdfs3. -- This message was sent by Atlassian Jira (v8.20.1#820001)