You may try out this implementation in Apache Bahir. - https://github.com/apache/bahir/pull/28.
Here we wrote a custom HDFS file system implementation extending the existing Webhdfs implementation to take care of Knox and SSL related requirements. This works fine while accessing data from Spark. Should work for other clients too. Would be happy to help/extend this solution in case you have any concrete requirement. The latest version has support for csv, parquet, json types. Regards, Sourav On Wed, Jan 17, 2018 at 7:54 AM, Rajesh Chandramohan <[email protected] > wrote: > Hi , > > Looks like its related to this KNOX Jira https://issues.apache. > org/jira/browse/KNOX-829 > > > Thanks > \Rajesh > > > On Tuesday 16 January 2018, 4:27:41 PM GMT-8, Rajesh Chandramohan < > [email protected]> wrote: > > > Hi , > > Why we don't have append operation implemented here https://github.com/ > apache/knox/tree/v0.14.0-release/gateway-shell/src/ > main/java/org/apache/hadoop/gateway/shell/hdfs > > That would be useful if we like to have streaming usecase with KNOX right > ? Si nobody bothered to expand for that case or some other reason for not > having append operation implemented in gateway-shell > > > \Rajesh > > > On Tuesday 16 January 2018, 2:11:05 PM GMT-8, Rajesh Chandramohan < > [email protected]> wrote: > > > Hi KNOX-team, > > Is anybody implemented to write Streaming data to HDFS via Knox webhdfs > ? Is that possible where HDFS cluster is kerberized . > Will Authentication/Delegation token will be refreshed etc .. Any thoughts > ? > > > > \Rajesh >
