[
http://issues.apache.org/jira/browse/HADOOP-538?page=comments#action_12438210 ]
Doug Cutting commented on HADOOP-538:
-------------------------------------
A lot of Hadoop developers might not have the right development environment to
build the native libraries. For example, I use Ubuntu, and had previously
manually installed gcc, make and a few other things in order to get libhdfs to
compile, but to get this to compile I had to install zlib1g-dev. I have not
yet tried to build this under cygwin, and don't even know what packages are
required there.
Do you think it would be possible to commit pre-built versions of libhadoop.so
for common platforms? Minimally we could include 32-bit builds for cygwin and
linux. This will complicate the process of changing these, but the alternative
is to complicate the release process, since we currently don't require
non-developers to have much more than a JRE.
And/or we can fallback (with a warning) to using either Sun's implementation or
jzlib when an appropriate libhadoop.so is not available. Then we could still
decide to ship with a pre-built version (e.g., linux only, which would be easy,
since releases are all built on linux) but things would still work fine
out-of-the-box on other platforms and can be optimized by those who wish to
install the required developer tools.
A corallary to these approaches is that the generated libhadoop file name
should probably include the OS and architecture names.
Thoughts?
> Implement a nio's 'direct buffer' based wrapper over zlib to improve
> performance of java.util.zip.{De|In}flater as a 'custom codec'
> -----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-538
> URL: http://issues.apache.org/jira/browse/HADOOP-538
> Project: Hadoop
> Issue Type: Improvement
> Affects Versions: 0.6.1
> Reporter: Arun C Murthy
> Assigned To: Arun C Murthy
> Fix For: 0.7.0
>
> Attachments: HADOOP-538.patch, HADOOP-538_benchmarks.tgz
>
>
> There has been more than one instance where java.util.zip's {De|In}flater
> classes perform unreliably, a simple wrapper over zlib-1.2.3 (latest stable)
> using java.nio.ByteBuffer (i.e. direct buffers) should go a long way in
> alleviating these woes.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
