[
https://issues.apache.org/jira/browse/TWILL-117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14610792#comment-14610792
]
ASF GitHub Bot commented on TWILL-117:
--------------------------------------
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-twill/pull/49
> MapR extension
> ---------------
>
> Key: TWILL-117
> URL: https://issues.apache.org/jira/browse/TWILL-117
> Project: Apache Twill
> Issue Type: New Feature
> Components: yarn
> Affects Versions: 0.5.0-incubating
> Reporter: Tongqing Qiu
> Priority: Minor
> Fix For: 0.6.0-incubating
>
>
> I made some extensions to make twill working for mapr. Besides of what I
> included in github, there are some other issues. Currently, I have to include
> maprfs library in yarn-example pom file, then related jars will be bundled
> and deployed into yarn. These libraries, however, are included in the
> HADOOP_CLASS, I don't know why it is not been loading when twill/yarn
> application runs. So if we don't are not using bundle jar application while
> let twill to scan dependent classes (like CDAP startup services did), the
> jars are missing. The way to make it work right now is a hacking:
> 1. add new MapRFileSystem() in the code of TwillContainerMain and
> Hadoop21YarnAppClient class, so that the dependency will be included, even
> though it does nothing. BTW, I think it works for other hadoop distribution,
> because there two classes already depends the hdfs related stuff and jars are
> packaged into container.jar and application jar (which I think is not
> necessary since very yarn node should already contain these jars)
> 2. Add twillRunner.setJVMOptions("-Djava.library.path=/opt/mapr/lib") when
> run the twill application.
> I think it is really not the right way to do, but not sure how to avoid that,
> just let yarn/twill know to include HADOOP_CLASSPATH when run the application?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
