[ https://issues.apache.org/jira/browse/OOZIE-2606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15531478#comment-15531478 ]
Hadoop QA commented on OOZIE-2606: ---------------------------------- Testing JIRA OOZIE-2606 Cleaning local git workspace ---------------------------- {color:green}+1 PATCH_APPLIES{color} {color:green}+1 CLEAN{color} {color:green}+1 RAW_PATCH_ANALYSIS{color} . {color:green}+1{color} the patch does not introduce any @author tags . {color:green}+1{color} the patch does not introduce any tabs . {color:green}+1{color} the patch does not introduce any trailing spaces . {color:green}+1{color} the patch does not introduce any line longer than 132 . {color:green}+1{color} the patch does adds/modifies 1 testcase(s) {color:green}+1 RAT{color} . {color:green}+1{color} the patch does not seem to introduce new RAT warnings {color:green}+1 JAVADOC{color} . {color:green}+1{color} the patch does not seem to introduce new Javadoc warnings {color:green}+1 COMPILE{color} . {color:green}+1{color} HEAD compiles . {color:green}+1{color} patch compiles . {color:green}+1{color} the patch does not seem to introduce new javac warnings {color:green}+1 BACKWARDS_COMPATIBILITY{color} . {color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations . {color:green}+1{color} the patch does not modify JPA files {color:red}-1 TESTS{color} . Tests run: 1813 . Tests failed: 0 . Tests errors: 1 . The patch failed the following testcases: . {color:green}+1 DISTRO{color} . {color:green}+1{color} distro tarball builds with the patch ---------------------------- {color:red}*-1 Overall result, please check the reported -1(s)*{color} The full output of the test-patch run is available at . https://builds.apache.org/job/oozie-trunk-precommit-build/3339/ > Set spark.yarn.jars to fix Spark 2.0 with Oozie > ----------------------------------------------- > > Key: OOZIE-2606 > URL: https://issues.apache.org/jira/browse/OOZIE-2606 > Project: Oozie > Issue Type: Bug > Components: core > Affects Versions: 4.2.0 > Reporter: Jonathan Kelly > Assignee: Satish Subhashrao Saley > Labels: spark, spark2.0.0 > Fix For: 4.3.0 > > Attachments: OOZIE-2606-2.patch, OOZIE-2606-3.patch, OOZIE-2606.patch > > > Oozie adds all of the jars in the Oozie Spark sharelib to the > DistributedCache such that all jars will be present in the current working > directory of the YARN container (as well as in the container classpath). > However, this is not quite enough to make Spark 2.0 work, since Spark 2.0 by > default looks for the jars in assembly/target/scala-2.11/jars [1] (as if it > is a locally built distribution for development) and will not find them in > the current working directory. > To fix this, we can set spark.yarn.jars to *.jar so that it finds the jars in > the current working directory rather than looking in the wrong place. [2] > [1] > https://github.com/apache/spark/blob/v2.0.0-rc2/launcher/src/main/java/org/apache/spark/launcher/CommandBuilderUtils.java#L357 > [2] > https://github.com/apache/spark/blob/v2.0.0-rc2/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala#L476 > Note: This property will be ignored by Spark 1.x. -- This message was sent by Atlassian JIRA (v6.3.4#6332)