[
https://issues.apache.org/jira/browse/HDFS-6994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14264800#comment-14264800
]
Colin Patrick McCabe commented on HDFS-6994:
--------------------------------------------
I don't think you need to do so much work. Just use the existing native
{{MiniDFSCluster}} code. Then, you can provide a mode where you can pass in
the URL of a real HDFS cluster to the test and have it use that instead of
starting its own JVM. This allows you to avoid JNI, but also run the test in a
self-contained fashion.
The amount of interaction we do with {{MiniDFSCluster}} is pretty minimal and
the JNI code is already written. If we start writing new inter-process
communication protocols, we will have to debug those as well. Since JSON is
weakly typed, we would have all the type coercion sloppiness issues of JNI and
then some. Managing external processes is also annoying, especially on Windows
(which we are trying to support).
> libhdfs3 - A native C/C++ HDFS client
> -------------------------------------
>
> Key: HDFS-6994
> URL: https://issues.apache.org/jira/browse/HDFS-6994
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: hdfs-client
> Reporter: Zhanwei Wang
> Assignee: Zhanwei Wang
> Attachments: HDFS-6994-rpc-8.patch, HDFS-6994.patch
>
>
> Hi All
> I just got the permission to open source libhdfs3, which is a native C/C++
> HDFS client based on Hadoop RPC protocol and HDFS Data Transfer Protocol.
> libhdfs3 provide the libhdfs style C interface and a C++ interface. Support
> both HADOOP RPC version 8 and 9. Support Namenode HA and Kerberos
> authentication.
> libhdfs3 is currently used by HAWQ of Pivotal
> I'd like to integrate libhdfs3 into HDFS source code to benefit others.
> You can find libhdfs3 code from github
> https://github.com/PivotalRD/libhdfs3
> http://pivotalrd.github.io/libhdfs3/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
