[
https://issues.apache.org/jira/browse/HDFS-6974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14118668#comment-14118668
]
Colin Patrick McCabe commented on HDFS-6974:
--------------------------------------------
Is this really any different than needing to set {{HADOOP_CLASSPATH}}
correctly? We don't handle mixing old jars into the classpath, so why should
we handle mixing old {{hadoop.dll}} files into the path? It seems
inconsistent. But maybe I'm missing something that makes this case different.
bq. There's another extension too: have a getVersion() call that returns
version info (build info etc), which can be used to help in diags. I'd add
that, but still look for hadoop-2.6.lib so that you could have >1 lib on the
path
We don't make any guarantees that the libhadoop supplied with 2.6 will work
with Hadoop 2.6.1. libhadoop doesn't have a fixed or standardized API; it's
just "the C half" of random bits of Hadoop code.
Think if you were making changes to the JNI code and redeploying. You need to
redeploy with the correct, new JNI code, not the old stuff. This again, the
same as with jar files... you wouldn't mix jar files from Hadoop 2.6 and Hadoop
2.6.1 in the same directory. So I would argue for your solution #1.
We could perhaps give a better error message here. We might be able to inject
the git hash into the library, and error out if it didn't match the git hash in
the jar files. But then that means that partial rebuilds of the source tree no
longer work, so maybe not.
> MiniHDFScluster breaks if there is an out of date hadoop.lib on the lib path
> -----------------------------------------------------------------------------
>
> Key: HDFS-6974
> URL: https://issues.apache.org/jira/browse/HDFS-6974
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
> Affects Versions: 2.6.0
> Environment: Windows with a version of Hadoop (HDP2.1) installed
> somewhere via an MSI
> Reporter: Steve Loughran
> Priority: Minor
>
> SLIDER-377 shows the trace of a MiniHDFSCluster test failing on native
> library calls ... the root cause appears to be the 2.4.1 hadoop lib on the
> path doesn't have all the methods needed by branch-2.
> When this situation arises, MiniHDFS cluster fails to work.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
