[
https://issues.apache.org/jira/browse/HADOOP-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12556802#action_12556802
]
Hadoop QA commented on HADOOP-1660:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12372648/HADOOP-1660_0_20080108.patch
against trunk revision .
@author +1. The patch does not contain any @author tags.
javadoc +1. The javadoc tool did not generate any warning messages.
javac +1. The applied patch does not generate any new compiler warnings.
findbugs +1. The patch does not introduce any new Findbugs warnings.
core tests +1. The patch passed core unit tests.
contrib tests -1. The patch failed contrib unit tests.
Test results:
http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1505/testReport/
Findbugs warnings:
http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1505/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results:
http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1505/artifact/trunk/build/test/checkstyle-errors.html
Console output:
http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1505/console
This message is automatically generated.
> add support for native library toDistributedCache
> --------------------------------------------------
>
> Key: HADOOP-1660
> URL: https://issues.apache.org/jira/browse/HADOOP-1660
> Project: Hadoop
> Issue Type: Improvement
> Components: mapred
> Environment: unix (different handling would be required for windows)
> Reporter: Alejandro Abdelnur
> Assignee: Arun C Murthy
> Fix For: 0.16.0
>
> Attachments: HADOOP-1660_0_20080108.patch
>
>
> Currently if a M/R job depends on JNI based component the dynamic library
> must be available in all the task nodes. This is not possible specially when
> you have not control on the cluster machines, just using it as a service.
> It should be possible to specify using the DistributedCache what are the
> native libraries a job needs.
> For example via a new method 'public void addLibrary(Path libraryPath,
> JobConf conf)'.
> The added libraries would make it to the local FS of the task nodes (same way
> as cached resources) but instead been part of the classpath they would be
> copied to a lib directory and that lib directory would be added t the
> LD_LIBRARY_PATH of the task JVM.
> An alternative would be to set the '-Djava.library.path=' task JVM parameter
> to the lib directory above. However, this would break for libraries that
> depend on other libraries as the dependent one would not be in the
> LD_LIBRARY_PATH and the OS would fail to find it as it is not the JVM the one
> doing the load of the dependent one.
> For uncached usage of native libraries, a special directory in the JAR could
> be used for native libraries. But I'd argue that the DistributedCache
> enhancement would be enough, and if somebody wants to use a native library
> s/he should use the DistributedCached. Or a JobConf addLibrary method that
> uses the DistributedCached under the hood at submission time.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
