[ https://issues.apache.org/jira/browse/OOZIE-3062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16203429#comment-16203429 ]
Hadoop QA commented on OOZIE-3062: ---------------------------------- Testing JIRA OOZIE-3062 Cleaning local git workspace ---------------------------- {color:green}+1 PATCH_APPLIES{color} {color:green}+1 CLEAN{color} {color:green}+1 RAW_PATCH_ANALYSIS{color} . {color:green}+1{color} the patch does not introduce any @author tags . {color:green}+1{color} the patch does not introduce any tabs . {color:green}+1{color} the patch does not introduce any trailing spaces . {color:green}+1{color} the patch does not introduce any line longer than 132 . {color:green}+1{color} the patch does adds/modifies 1 testcase(s) {color:green}+1 RAT{color} . {color:green}+1{color} the patch does not seem to introduce new RAT warnings {color:green}+1 JAVADOC{color} . {color:green}+1{color} the patch does not seem to introduce new Javadoc warnings . {color:red}WARNING{color}: the current HEAD has 77 Javadoc warning(s) {color:green}+1 COMPILE{color} . {color:green}+1{color} HEAD compiles . {color:green}+1{color} patch compiles . {color:green}+1{color} the patch does not seem to introduce new javac warnings {color:green}+1{color} There are no new bugs found in total. . {color:green}+1{color} There are no new bugs found in [examples]. . {color:green}+1{color} There are no new bugs found in [core]. . {color:green}+1{color} There are no new bugs found in [sharelib/distcp]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive]. . {color:green}+1{color} There are no new bugs found in [sharelib/pig]. . {color:green}+1{color} There are no new bugs found in [sharelib/spark]. . {color:green}+1{color} There are no new bugs found in [sharelib/hive2]. . {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog]. . {color:green}+1{color} There are no new bugs found in [sharelib/sqoop]. . {color:green}+1{color} There are no new bugs found in [sharelib/oozie]. . {color:green}+1{color} There are no new bugs found in [sharelib/streaming]. . {color:green}+1{color} There are no new bugs found in [tools]. . {color:green}+1{color} There are no new bugs found in [docs]. . {color:green}+1{color} There are no new bugs found in [server]. . {color:green}+1{color} There are no new bugs found in [client]. {color:green}+1 BACKWARDS_COMPATIBILITY{color} . {color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations . {color:green}+1{color} the patch does not modify JPA files {color:green}+1 TESTS{color} . Tests run: 2039 . Tests rerun: 65 . Tests failed at first run: org.apache.oozie.command.coord.TestCoordPushDependencyCheckXCommand,org.apache.oozie.action.hadoop.TestJavaActionExecutor, {color:green}+1 DISTRO{color} . {color:green}+1{color} distro tarball builds with the patch ---------------------------- {color:green}*+1 Overall result, good!, no -1s*{color} {color:red}. There is at least one warning, please check{color} The full output of the test-patch run is available at . https://builds.apache.org/job/PreCommit-OOZIE-Build/108/ > Set HADOOP_CONF_DIR for spark action > ------------------------------------ > > Key: OOZIE-3062 > URL: https://issues.apache.org/jira/browse/OOZIE-3062 > Project: Oozie > Issue Type: Bug > Reporter: Satish Subhashrao Saley > Assignee: Satish Subhashrao Saley > Attachments: OOZIE-3062-1.patch, OOZIE-3062-2.patch, > OOZIE-3062-3.patch, OOZIE-3062-4.patch, OOZIE-3062-5.patch, OOZIE-3062-6.patch > > > OOZIE-2569 created core-site.xml, hdfs-site.xml, mapred-site.xml, > yarn-site.xml for spark action. But for spark to consider these files ([spark > code > ref|https://github.com/apache/spark/blob/83fe3b5e10f8dc62245ea37143abb96be0f39805/core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala#L284]), > we need to set HADOOP_CONF_DIR environmental variable, otherwise it will > point to default HADOOP_CONF_DIR configured on the node. Therefore, Oozie > should set HADOOP_CONF_DIR. > Due to a bug YARN-4727, yarn was not considering user specified value for > whiltelisted environment variables including HADOOP_CONF_DIR. So, this will > work on specified hadoop fix versions in YARN-4727. > This fix in Oozie will be no-op on Hadoop without YARN-4727 fix and Oozie > will behave as it is. > To verify this on one node machine. > 1. In hadoop mapred-site.xml, set > {code} > <property> > <name>mapreduce.fileoutputcommitter.marksuccessfuljobs</name> > <value>false</value> > </property> > {code} > 2. Run any spark file copy example from Oozie. Check output directory, > _SUCCESS flag is missing. It should be there because we explicitly set > conf.set("mapreduce.fileoutputcommitter.marksuccessfuljobs", "true"); in > https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L293 -- This message was sent by Atlassian JIRA (v6.4.14#64029)