[
https://issues.apache.org/jira/browse/HDFS-2656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13430778#comment-13430778
]
Todd Lipcon commented on HDFS-2656:
-----------------------------------
Can you please comment on performance of this implementation vs the libhdfs in
trunk?
I am uncomfortable calling this "libhdfs2" if the performance is not up to par
with the current "libhdfs1". One of the major reasons people use libhdfs is
that they're writing a high-performance app, and I would be really surprised if
this HTTP-based client can match the performance of the JNI-based one,
especially when short-circuit read is enabled.
An additional concern is that webhdfs doesn't seem to return any checksum
information with its transfers. So, a client using this C library no longer has
the benefits of checksumming that HDFS provides.
So while I see there are some uses for this, I don't think it's viable as a
_replacement_. Perhaps better to describe it as "libwebhdfs" or something?
The "libhdfs3" work on the other hand sounds useful for a variety of
applications, and should have better performance and memory footprint while not
giving up features like checksumming, etc. Any word on when that would be
available?
> Implement a pure c client based on webhdfs
> ------------------------------------------
>
> Key: HDFS-2656
> URL: https://issues.apache.org/jira/browse/HDFS-2656
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
> Reporter: Zhanwei.Wang
> Attachments: HDFS-2656.patch, HDFS-2656.unfinished.patch
>
>
> Currently, the implementation of libhdfs is based on JNI. The overhead of JVM
> seems a little big, and libhdfs can also not be used in the environment
> without hdfs.
> It seems a good idea to implement a pure c client by wrapping webhdfs. It
> also can be used to access different version of hdfs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
