[
https://issues.apache.org/jira/browse/HADOOP-11064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14130426#comment-14130426
]
Chris Nauroth commented on HADOOP-11064:
----------------------------------------
After further discussion, the following is a summary of potential solutions to
this problem:
1. Freeze the libhadoop.so API forever.
2. Library versioning plus maintaining a library on the servers for each
supported release.
3. Bundle the .dll or .so file inside a jar somehow so that YARN / Slider can
distribute it.
#1. Advantages:
* ???
#1. Disadvantages:
* We're unable to improve libhadoop.so in the future.
* There will be puzzling interactions when mixing and matching versions.
"New" bugs in libhadoop.so will show up with old hadoop releases, causing
confusion in bug trackers.
* We don't have any way of enforcing C API stability. Jenkins doesn't check
for it, most Java programmers don't know how to achieve it.
* There is still no ability for applications using new Hadoop versions to
make use of old libhadoop.so versions, unless we adopt an even worse
compatibility policy that nothing new can be added to libhadoop.so.
* Given all of the above, this option seems to be off the table.
#2. Advantages:
* Simple to implement.
* There's already a patch that implements it.
* We want libhadoop.so library versioning anyway, even if we later adopt
another solution in addition to this
#2. Disadvantages:
* Admins using Slider / YARN will need to ensure that the appropriate
versions of libhadoop are present on the server.
#3. Advantages:
* "Cleanest" solution, since it allows us to reuse YARN's existing
distribution mechanisms.
#3. Disadvantages:
* There are technical challenges to bundling a library in a jar that we
haven't yet tackled.
Short-term, there is now agreement that the scope of this jira is to restore
the 2 function signatures that were removed to unblock downstream projects.
Colin plans to arrange a conference call for further discussion next week. The
details will be posted here for anyone who wants to join, and we'll also call
wider attention to this issue on the mailing lists.
> UnsatisifedLinkError with hadoop 2.4 JARs on hadoop-2.6 due NativeCRC32
> method changes
> --------------------------------------------------------------------------------------
>
> Key: HADOOP-11064
> URL: https://issues.apache.org/jira/browse/HADOOP-11064
> Project: Hadoop Common
> Issue Type: Bug
> Components: native
> Affects Versions: 2.6.0
> Environment: Hadoop 2.6 cluster, trying to run code containing hadoop
> 2.4 JARs
> Reporter: Steve Loughran
> Assignee: Colin Patrick McCabe
> Priority: Blocker
> Attachments: HADOOP-11064.001.patch, HADOOP-11064.002.patch,
> HADOOP-11064.003.patch
>
>
> The private native method names and signatures in {{NativeCrc32}} were
> changed in HDFS-6561 ... as a result hadoop-common-2.4 JARs get unsatisifed
> link errors when they try to perform checksums.
> This essentially stops Hadoop 2.4 applications running on Hadoop 2.6 unless
> rebuilt and repackaged with the hadoop- 2.6 JARs
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
