Fabian Höring created ARROW-7309:
------------------------------------
Summary: [Python] Support HDFS federation viewfs:// in
resolve_filesystem_and_path
Key: ARROW-7309
URL: https://issues.apache.org/jira/browse/ARROW-7309
Project: Apache Arrow
Issue Type: Improvement
Components: Python
Affects Versions: 0.15.1
Reporter: Fabian Höring
- Add viewfs support to pyarrow.filesystem.resolve_filesystem_and_path
- libhdfs already supports injecting the scheme and will automatically resolve
federation in
fs = FileSystem#get(URI, conf, ugi)
- works with Hadoop 2/3
see:
https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/hdfs.c#L770
https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/hdfs.c#L637
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
