[
https://issues.apache.org/jira/browse/HADOOP-11064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14126239#comment-14126239
]
Chris Nauroth commented on HADOOP-11064:
----------------------------------------
bq. ...bundle everything, including the native code...
I think there is a reliance on the Maven jar file dependency as a reliable unit
of versioning. Applications expect that if they bundle their relevant Hadoop
jars all at the same version, then everything is going to work. Unfortunately,
it turns out that hadoop-common.jar is an "incomplete" artifact in terms of
Maven versioning because of the tight coupling between the jar and the native
code.
If we bundled the native libs inside hadoop-common.jar (or some new jar), and
extracted it automatically at runtime, then downstream projects could get a
completely versioned artifact through the Maven dependency. The snappy-java
project is an example of something that does this. It detects the OS and
extracts the corresponding native library at runtime. I believe hadoop-lzo has
just started doing something similar too.
Of course, this opens up a can of worms around release engineering. The
current Apache release process only builds the native components on a single
platform. I can't think of a way to make this work without a lot of
infrastructure work as a pre-requisite.
> UnsatisifedLinkError with hadoop 2.4 JARs on hadoop-2.6 due NativeCRC32
> method changes
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-11064
> URL: https://issues.apache.org/jira/browse/HADOOP-11064
> Project: Hadoop Common
> Issue Type: Bug
> Components: native
> Affects Versions: 2.6.0
> Environment: Hadoop 2.6 cluster, trying to run code containing hadoop
> 2.4 JARs
> Reporter: Steve Loughran
> Priority: Blocker
>
> The private native method names and signatures in {{NativeCrc32}} were
> changed in HDFS-6561 ... as a result hadoop-common-2.4 JARs get unsatisifed
> link errors when they try to perform checksums.
> This essentially stops Hadoop 2.4 applications running on Hadoop 2.6 unless
> rebuilt and repackaged with the hadoop- 2.6 JARs
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
