[
https://issues.apache.org/jira/browse/OOZIE-3062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16645466#comment-16645466
]
Hadoop QA commented on OOZIE-3062:
----------------------------------
Testing JIRA OOZIE-3062
Cleaning local git workspace
----------------------------
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
. {color:green}+1{color} the patch does not introduce any @author tags
. {color:green}+1{color} the patch does not introduce any tabs
. {color:green}+1{color} the patch does not introduce any trailing spaces
. {color:green}+1{color} the patch does not introduce any star imports
. {color:green}+1{color} the patch does not introduce any line longer than
132
. {color:green}+1{color} the patch adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
. {color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
. {color:green}+1{color} Javadoc generation succeeded with the patch
. {color:green}+1{color} the patch does not seem to introduce new Javadoc
warning(s)
. {color:orange}WARNING{color}: the current HEAD has 100 Javadoc warning(s)
{color:green}+1 COMPILE{color}
. {color:green}+1{color} HEAD compiles
. {color:green}+1{color} patch compiles
. {color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in
[fluent-job/fluent-job-api].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/git].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [webapp].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
. {color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
. {color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
. Tests run: 3056
. {color:orange}Tests failed at first run:{color}
TestSubWorkflowActionExecutor#testSubWorkflowRerun
. For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
{color:green}+1 DISTRO{color}
. {color:green}+1{color} distro tarball builds with the patch
----------------------------
{color:green}*+1 Overall result, good!, no -1s*{color}
{color:red}. There is at least one warning, please check{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/882/
> Set HADOOP_CONF_DIR for spark action
> ------------------------------------
>
> Key: OOZIE-3062
> URL: https://issues.apache.org/jira/browse/OOZIE-3062
> Project: Oozie
> Issue Type: Bug
> Reporter: Satish Subhashrao Saley
> Assignee: Satish Subhashrao Saley
> Priority: Major
> Attachments: OOZIE-3062-1.patch, OOZIE-3062-2.patch,
> OOZIE-3062-3.patch, OOZIE-3062-4.patch, OOZIE-3062-5.patch,
> OOZIE-3062-6.patch, OOZIE-3062-7.patch
>
>
> OOZIE-2569 created core-site.xml, hdfs-site.xml, mapred-site.xml,
> yarn-site.xml for spark action. But for spark to consider these files ([spark
> code
> ref|https://github.com/apache/spark/blob/83fe3b5e10f8dc62245ea37143abb96be0f39805/core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala#L284]),
> we need to set HADOOP_CONF_DIR environmental variable, otherwise it will
> point to default HADOOP_CONF_DIR configured on the node. Therefore, Oozie
> should set HADOOP_CONF_DIR.
> Due to a bug YARN-4727, yarn was not considering user specified value for
> whiltelisted environment variables including HADOOP_CONF_DIR. So, this will
> work on specified hadoop fix versions in YARN-4727.
> This fix in Oozie will be no-op on Hadoop without YARN-4727 fix and Oozie
> will behave as it is.
> To verify this on one node machine.
> 1. In hadoop mapred-site.xml, set
> {code}
> <property>
> <name>mapreduce.fileoutputcommitter.marksuccessfuljobs</name>
> <value>false</value>
> </property>
> {code}
> 2. Run any spark file copy example from Oozie. Check output directory,
> _SUCCESS flag is missing. It should be there because we explicitly set
> conf.set("mapreduce.fileoutputcommitter.marksuccessfuljobs", "true"); in
> https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L293
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
