Matt Topol created ARROW-2661:
---------------------------------

             Summary: [Python/C++] Allow passing HDFS Config values via 
map/dict instead of needing an hdfs-site.xml file
                 Key: ARROW-2661
                 URL: https://issues.apache.org/jira/browse/ARROW-2661
             Project: Apache Arrow
          Issue Type: New Feature
          Components: C++, Python
    Affects Versions: 0.10.0
            Reporter: Matt Topol


Currently in order to pass HDFS configurations of values down to the underlying 
libhdfs driver you have to either set HADOOP_HOME (using libhdfs) or 
LIBHDFS3_CONF (for libhdfs3) to point to a location of an hdfs-site.xml file to 
use. However the API provided by them allows for calling hdfsBuilderConfSetStr 
to set arbitrary configuration values on the hdfsBuilder object. This would 
allow for consumers to programmatically set any arbitrary configurations they 
want including allowing for encryption and other complex hdfs configurations 
without needing to provide an hdfs-site.xml file.





--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to