[
https://issues.apache.org/jira/browse/HDFS-6994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14302004#comment-14302004
]
Haohui Mai commented on HDFS-6994:
----------------------------------
One of the motivations behind building a native HDFS client is to integrate
HDFS with MPP databases like Impala and Vertica. Note that MPP databases
occasionally access thousands of files concurrently -- unfortunately neither
the Java client nor libhdfs3 can efficiently support these use cases.
Since technically we're building a new client from scratch, IMHO it makes sense
to support the new use cases in the new client. I've put up a proof-of-concept
implementation of RemoteBlockReader on github:
https://github.com/haohui/libhdfspp
The code is licensed in Apache License. I'm also interested in combining it
with the current ongoing effort of libhdfs3. Suggestions are appreciated.
> libhdfs3 - A native C/C++ HDFS client
> -------------------------------------
>
> Key: HDFS-6994
> URL: https://issues.apache.org/jira/browse/HDFS-6994
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: hdfs-client
> Reporter: Zhanwei Wang
> Assignee: Zhanwei Wang
> Attachments: HDFS-6994-rpc-8.patch, HDFS-6994.patch
>
>
> Hi All
> I just got the permission to open source libhdfs3, which is a native C/C++
> HDFS client based on Hadoop RPC protocol and HDFS Data Transfer Protocol.
> libhdfs3 provide the libhdfs style C interface and a C++ interface. Support
> both HADOOP RPC version 8 and 9. Support Namenode HA and Kerberos
> authentication.
> libhdfs3 is currently used by HAWQ of Pivotal
> I'd like to integrate libhdfs3 into HDFS source code to benefit others.
> You can find libhdfs3 code from github
> https://github.com/PivotalRD/libhdfs3
> http://pivotalrd.github.io/libhdfs3/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
