Fabian Höring created ARROW-7309:
------------------------------------

             Summary: [Python] Support HDFS federation viewfs:// in 
resolve_filesystem_and_path
                 Key: ARROW-7309
                 URL: https://issues.apache.org/jira/browse/ARROW-7309
             Project: Apache Arrow
          Issue Type: Improvement
          Components: Python
    Affects Versions: 0.15.1
            Reporter: Fabian Höring


- Add viewfs support to pyarrow.filesystem.resolve_filesystem_and_path
-  libhdfs already supports injecting the scheme and will automatically resolve 
federation in
    fs = FileSystem#get(URI, conf, ugi)
-  works with Hadoop 2/3

see:
https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/hdfs.c#L770
https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/hdfs.c#L637



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Reply via email to