[jira] [Comment Edited] (OOZIE-3057) Custom Partitioner not working in Oozie Mapreduce action

[Raghavi Ravi (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16363563#comment-16363563
 ] 

Raghavi Ravi edited comment on OOZIE-3057 at 2/14/18 7:28 AM:
--

[~gezapeti]
 * I read data from three different sources and merge them based on natural 
key. I have created a composite key for sorting and a natural key based on 
which partitioner distributes data among reducer.
 * My sample input data is in such a way that only one natural key is present 
and i have written custom partitioner to group by natural key so i should get 
only one input group for reducer. When I try running the MR job (*written using 
old API's*) as a Mapreduce action from oozie (using HUE) give me 11 input 
groups instead of 1.
 ** Reduce Input Groups 0 11 11
Reduce Input Records 0 11 11
 * Running the same code using Java action from oozie (using HUE) sends only 
one input group to reducer
 ** Reduce Input Groups 0 1 1
Reduce Input Records 0 11 11
 * The MR job (using old API) run using mapreduce action runs successfully and 
there are no errors found. The metadata has details about partitioner class but 
the class is never evaluated. Tried printing SysOut's from partitioner class 
but they don't get printed.
 * Attaching logs, metaData and Partitioner code for oozie MR action (using old 
API).

Attaching oozie logs, Partitioner class and workflow.xml. The workflow.xml has 
only one mapreduce action that reads RCFiles and creates text files.

[^Logs.zip] [^PonRankPartitioner.java] [^workflow.xml]


was (Author: raghaviravi92):
[~gezapeti]

Attaching oozie logs, Partitioner class and workflow.xml. The workflow.xml has 
only one mapreduce action that reads RCFiles and creates text files.

[^Logs.zip] [^PonRankPartitioner.java] [^workflow.xml]

> Custom Partitioner not working in Oozie Mapreduce action
> 
>
> Key: OOZIE-3057
> URL: https://issues.apache.org/jira/browse/OOZIE-3057
> Project: Oozie
>  Issue Type: Bug
>  Components: action, workflow
>Affects Versions: 4.1.0
> Environment: Red Hat Enterprise Linux Server release 7.2 (Maipo)
> Linux version 3.10.0-327.10.1.el7.x86_64 
> (mockbu...@x86-021.build.eng.bos.redhat.com) (gcc version 4.8.5 20150623 (Red 
> Hat 4.8.5-4) (GCC) ) #1 SMP Sat Jan 23 04:54:55 EST 2016
> oozie version - 4.1.0
> cdh version - 5.10.1
> Hue™ 3.11 - The Hadoop UI
>Reporter: Raghavi Ravi
>Priority: Critical
> Attachments: Logs.zip, PonRankPartitioner.java, workflow.xml
>
>
> I implemented secondary sort in mapreduce using old API 
> (org.apache.hadoop.mapred.*) and trying to execute it using Oozie (From Hue).
> Though I have set the partitioner class in the properties, the partitioner is 
> not being executed. So, I'm not getting output as expected.
> The same code runs fine when run using hadoop command from CLI.
> And here is my workflow.xml
> 
> 
> 
> Action failed, error 
> message[${wf:errorMessage(wf:lastErrorNode())}]
> 
> 
> 
> ${jobTracker}
> ${nameNode}
> 
> 
> mapred.output.dir
> /test_1109_3
> 
> 
> mapred.input.dir
> 
> /apps/hive/warehouse/7360_0609_rx/day=06-09-2017/hour=13/quarter=2/,/apps/hive/warehouse/7360_0609_tx/day=06-09-2017/hour=13/quarter=2/,/apps/hive/warehouse/7360_0509_util/day=05-09-2017/hour=16/quarter=1/
> 
> 
> mapred.input.format.class
> org.apache.hadoop.hive.ql.io.RCFileInputFormat
> 
> 
> mapred.mapper.class
> PonRankMapper
> 
> 
> mapred.reducer.class
> PonRankReducer
> 
> 
> mapred.output.value.comparator.class
> PonRankGroupingComparator
> 
> 
> mapred.mapoutput.key.class
> PonRankPair
> 
> 
> mapred.mapoutput.value.class
> org.apache.hadoop.io.Text
> 
> 
> mapred.reduce.output.key.class
> org.apache.hadoop.io.NullWritable
> 
> 
> mapred.reduce.output.value.class
> org.apache.hadoop.io.Text
> 
> 
> mapred.reduce.tasks
> 1
> 
> 
> mapred.partitioner.class
> PonRankPartitioner
> 
> 
> mapred.mapper.new-api
> False
> 
> 
> 
> 
> 
> 
> 
> When running using hadoop jar command, I set the partitioner class using 
> 

[jira] [Commented] (OOZIE-3182) Oozie components fail with checkstyle errors

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16363575#comment-16363575
 ] 

Hadoop QA commented on OOZIE-3182:
--


Testing JIRA OOZIE-3182

Cleaning local git workspace



{color:red}-1{color} Patch failed to apply to head of branch




> Oozie components fail with checkstyle errors
> 
>
> Key: OOZIE-3182
> URL: https://issues.apache.org/jira/browse/OOZIE-3182
> Project: Oozie
>  Issue Type: Bug
>  Components: client, examples
>Affects Versions: 5.0.0b1
> Environment: uname -a
> Linux pts00607-vm1 4.4.0-31-generic #50-Ubuntu SMP Wed Jul 13 00:05:18 UTC 
> 2016 ppc64le ppc64le ppc64le GNU/Linux
>Reporter: Alisha Prabhu
>Assignee: Alisha Prabhu
>Priority: Major
> Attachments: OOZIE-3182-001.patch, OOZIE-3182-002.patch, 
> error-log.txt, log-OOZIE-3182-002.txt
>
>
> There are multiple checkstyle violations in the oozie-client, oozie-examples, 
> oozie-mini, oozie-sharelib-oozie, oozie-sharelib module.
> Maven command used is mvn clean install -DskipTests



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Failed: OOZIE-3182 PreCommit Build #397

[Apache Jenkins Server]

Jira: https://issues.apache.org/jira/browse/OOZIE-3182
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/397/

###
## LAST 100 LINES OF THE CONSOLE 
###
[...truncated 977.13 KB...]
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ 
oozie-zookeeper-security-tests ---
[INFO] 
[INFO] --- maven-resources-plugin:3.0.2:resources (default-resources) @ 
oozie-zookeeper-security-tests ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 
/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/zookeeper-security-tests/src/main/resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.7.0:compile (default-compile) @ 
oozie-zookeeper-security-tests ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ 
oozie-zookeeper-security-tests ---
[INFO] 
[INFO] --- maven-resources-plugin:3.0.2:resources (default-resources) @ 
oozie-zookeeper-security-tests ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 
/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/zookeeper-security-tests/src/main/resources
[INFO] Copying 3 resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.7.0:compile (default-compile) @ 
oozie-zookeeper-security-tests ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:3.0.2:testResources (default-testResources) @ 
oozie-zookeeper-security-tests ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 
/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/zookeeper-security-tests/src/test/resources
[INFO] Copying 3 resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.7.0:testCompile (default-testCompile) @ 
oozie-zookeeper-security-tests ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 2 source files to 
/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/zookeeper-security-tests/target/test-classes
[INFO] 
/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/zookeeper-security-tests/src/test/java/org/apache/oozie/util/TestZKUtilsWithSecurity.java:
 Some input files use or override a deprecated API.
[INFO] 
/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/zookeeper-security-tests/src/test/java/org/apache/oozie/util/TestZKUtilsWithSecurity.java:
 Recompile with -Xlint:deprecation for details.
[INFO] 
[INFO] >>> findbugs-maven-plugin:3.0.5:check (default-cli) > :findbugs @ 
oozie-zookeeper-security-tests >>>
[INFO] 
[INFO] --- findbugs-maven-plugin:3.0.5:findbugs (findbugs) @ 
oozie-zookeeper-security-tests ---
[INFO] 
[INFO] <<< findbugs-maven-plugin:3.0.5:check (default-cli) < :findbugs @ 
oozie-zookeeper-security-tests <<<
[INFO] 
[INFO] 
[INFO] --- findbugs-maven-plugin:3.0.5:check (default-cli) @ 
oozie-zookeeper-security-tests ---
[INFO] 
[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Oozie Main .. SUCCESS [ 17.360 s]
[INFO] Apache Oozie Client  SUCCESS [ 20.489 s]
[INFO] Apache Oozie Share Lib Oozie ... SUCCESS [ 14.681 s]
[INFO] Apache Oozie Share Lib HCatalog  SUCCESS [  7.610 s]
[INFO] Apache Oozie Share Lib Distcp .. SUCCESS [  6.098 s]
[INFO] Apache Oozie Core .. SUCCESS [01:35 min]
[INFO] Apache Oozie Share Lib Streaming ... SUCCESS [  7.953 s]
[INFO] Apache Oozie Share Lib Pig . SUCCESS [  8.757 s]
[INFO] Apache Oozie Share Lib Hive  SUCCESS [  9.078 s]
[INFO] Apache Oozie Share Lib Hive 2 .. SUCCESS [ 11.278 s]
[INFO] Apache Oozie Share Lib Sqoop ... SUCCESS [  6.630 s]
[INFO] Apache Oozie Examples .. SUCCESS [ 11.751 s]
[INFO] Apache Oozie Share Lib Spark ... SUCCESS [ 12.091 s]
[INFO] Apache Oozie Share Lib . SUCCESS [  0.022 s]
[INFO] Apache Oozie Docs .. SUCCESS [  3.658 s]
[INFO] Apache Oozie WebApp  SUCCESS [ 10.865 s]
[INFO] Apache Oozie Tools . SUCCESS [ 15.960 s]
[INFO] Apache Oozie MiniOozie . SUCCESS [  1.887 s]
[INFO] Apache Oozie Server  SUCCESS [ 12.693 s]
[INFO] Apache Oozie Distro  SUCCESS [  2.522 s]
[INFO] Apache Oozie ZooKeeper Security Tests .. SUCCESS [  3.212 s]
[INFO] 
[INFO] BUILD SUCCESS

[jira] [Commented] (OOZIE-3182) Oozie components fail with checkstyle errors

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16363570#comment-16363570
 ] 

Hadoop QA commented on OOZIE-3182:
--

PreCommit-OOZIE-Build started


> Oozie components fail with checkstyle errors
> 
>
> Key: OOZIE-3182
> URL: https://issues.apache.org/jira/browse/OOZIE-3182
> Project: Oozie
>  Issue Type: Bug
>  Components: client, examples
>Affects Versions: 5.0.0b1
> Environment: uname -a
> Linux pts00607-vm1 4.4.0-31-generic #50-Ubuntu SMP Wed Jul 13 00:05:18 UTC 
> 2016 ppc64le ppc64le ppc64le GNU/Linux
>Reporter: Alisha Prabhu
>Assignee: Alisha Prabhu
>Priority: Major
> Attachments: OOZIE-3182-001.patch, OOZIE-3182-002.patch, 
> error-log.txt, log-OOZIE-3182-002.txt
>
>
> There are multiple checkstyle violations in the oozie-client, oozie-examples, 
> oozie-mini, oozie-sharelib-oozie, oozie-sharelib module.
> Maven command used is mvn clean install -DskipTests



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (OOZIE-3057) Custom Partitioner not working in Oozie Mapreduce action

[Raghavi Ravi (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16363563#comment-16363563
 ] 

Raghavi Ravi edited comment on OOZIE-3057 at 2/14/18 7:00 AM:
--

[~gezapeti]

Attaching oozie logs, Partitioner class and workflow.xml. The workflow.xml has 
only one mapreduce action that reads RCFiles and creates text files.

[^Logs.zip] [^PonRankPartitioner.java] [^workflow.xml]


was (Author: raghaviravi92):
[~gezapeti]

Attaching oozie logs, Partitioner class and workflow.xml. The workflow.xml has 
only one mapreduce action that reads RCFiles and creates text files.

[^Logs.zip] [^PonRankPartitioner.java]

> Custom Partitioner not working in Oozie Mapreduce action
> 
>
> Key: OOZIE-3057
> URL: https://issues.apache.org/jira/browse/OOZIE-3057
> Project: Oozie
>  Issue Type: Bug
>  Components: action, workflow
>Affects Versions: 4.1.0
> Environment: Red Hat Enterprise Linux Server release 7.2 (Maipo)
> Linux version 3.10.0-327.10.1.el7.x86_64 
> (mockbu...@x86-021.build.eng.bos.redhat.com) (gcc version 4.8.5 20150623 (Red 
> Hat 4.8.5-4) (GCC) ) #1 SMP Sat Jan 23 04:54:55 EST 2016
> oozie version - 4.1.0
> cdh version - 5.10.1
> Hue™ 3.11 - The Hadoop UI
>Reporter: Raghavi Ravi
>Priority: Critical
> Attachments: Logs.zip, PonRankPartitioner.java, workflow.xml
>
>
> I implemented secondary sort in mapreduce using old API 
> (org.apache.hadoop.mapred.*) and trying to execute it using Oozie (From Hue).
> Though I have set the partitioner class in the properties, the partitioner is 
> not being executed. So, I'm not getting output as expected.
> The same code runs fine when run using hadoop command from CLI.
> And here is my workflow.xml
> 
> 
> 
> Action failed, error 
> message[${wf:errorMessage(wf:lastErrorNode())}]
> 
> 
> 
> ${jobTracker}
> ${nameNode}
> 
> 
> mapred.output.dir
> /test_1109_3
> 
> 
> mapred.input.dir
> 
> /apps/hive/warehouse/7360_0609_rx/day=06-09-2017/hour=13/quarter=2/,/apps/hive/warehouse/7360_0609_tx/day=06-09-2017/hour=13/quarter=2/,/apps/hive/warehouse/7360_0509_util/day=05-09-2017/hour=16/quarter=1/
> 
> 
> mapred.input.format.class
> org.apache.hadoop.hive.ql.io.RCFileInputFormat
> 
> 
> mapred.mapper.class
> PonRankMapper
> 
> 
> mapred.reducer.class
> PonRankReducer
> 
> 
> mapred.output.value.comparator.class
> PonRankGroupingComparator
> 
> 
> mapred.mapoutput.key.class
> PonRankPair
> 
> 
> mapred.mapoutput.value.class
> org.apache.hadoop.io.Text
> 
> 
> mapred.reduce.output.key.class
> org.apache.hadoop.io.NullWritable
> 
> 
> mapred.reduce.output.value.class
> org.apache.hadoop.io.Text
> 
> 
> mapred.reduce.tasks
> 1
> 
> 
> mapred.partitioner.class
> PonRankPartitioner
> 
> 
> mapred.mapper.new-api
> False
> 
> 
> 
> 
> 
> 
> 
> When running using hadoop jar command, I set the partitioner class using 
> JobConf.setPartitionerClass API.
>  Partitioner is not executed when using old API . Inspite of adding the 
> property.
> 
> mapred.partitioner.class
> PonRankPartitioner
> 
> Executed the same logic using new API's (org.apache.hadoop.mapreduce) and 
> added mapreduce.partitioner.class property in workflow.
> Partitioner was executed and desired outcome was seen.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] Subscription: Oozie Patch Available

[jira]

Issue Subscription
Filter: Oozie Patch Available (107 issues)

Subscriber: ooziedaily

Key Summary
OOZIE-3183  Better logging for SshActionExecutor and extended HA capability 
when calling to remote host
https://issues.apache.org/jira/browse/OOZIE-3183
OOZIE-3182  Oozie components fail with checkstyle errors
https://issues.apache.org/jira/browse/OOZIE-3182
OOZIE-3179  Adding a configurable config-default.xml location to a workflow
https://issues.apache.org/jira/browse/OOZIE-3179
OOZIE-3176  Oozie-core fails with checkstyle errors.
https://issues.apache.org/jira/browse/OOZIE-3176
OOZIE-3172  Upgrade non-transitive Jackson dependencies from 
org.codehaus.jackson to com.fasterxml.jackson
https://issues.apache.org/jira/browse/OOZIE-3172
OOZIE-3135  Configure log4j2 in SqoopMain
https://issues.apache.org/jira/browse/OOZIE-3135
OOZIE-3134  Potential inconsistency between the in-memory SLA map and the Oozie 
database
https://issues.apache.org/jira/browse/OOZIE-3134
OOZIE-3109  Escape log-streaming's HTML-specific characters
https://issues.apache.org/jira/browse/OOZIE-3109
OOZIE-3105  testJMXInstrumentation from the 
org.apache.oozie.util.TestMetricsInstrumentation class is flaky
https://issues.apache.org/jira/browse/OOZIE-3105
OOZIE-3094  fix for grammar mistake
https://issues.apache.org/jira/browse/OOZIE-3094
OOZIE-3091  Oozie Sqoop Avro Import fails with "java.lang.NoClassDefFoundError: 
org/apache/avro/mapred/AvroWrapper"
https://issues.apache.org/jira/browse/OOZIE-3091
OOZIE-3071  Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 
than Spark 2.2.0
https://issues.apache.org/jira/browse/OOZIE-3071
OOZIE-3063  Sanitizing variables that are part of openjpa.ConnectionProperties
https://issues.apache.org/jira/browse/OOZIE-3063
OOZIE-3062  Set HADOOP_CONF_DIR for spark action
https://issues.apache.org/jira/browse/OOZIE-3062
OOZIE-3061  Kill only those child jobs which are not already killed
https://issues.apache.org/jira/browse/OOZIE-3061
OOZIE-3002  address findbugs errors in client lib
https://issues.apache.org/jira/browse/OOZIE-3002
OOZIE-2975  code clean up in pig sharelib, replace Exception with more 
explicit, add try with resources, StringBuilder instead of StringBuffer
https://issues.apache.org/jira/browse/OOZIE-2975
OOZIE-2957  Documentation states that starting a coordinator is possible
https://issues.apache.org/jira/browse/OOZIE-2957
OOZIE-2956  Fix Findbugs warnings related to reliance on default encoding in 
oozie-core
https://issues.apache.org/jira/browse/OOZIE-2956
OOZIE-2955  Fix Findbugs warnings related to reliance on default encoding in 
oozie-client
https://issues.apache.org/jira/browse/OOZIE-2955
OOZIE-2954  Fix Checkstyle issues in oozie-client
https://issues.apache.org/jira/browse/OOZIE-2954
OOZIE-2953  Fix Checkstyle issues in oozie-tools
https://issues.apache.org/jira/browse/OOZIE-2953
OOZIE-2952  Fix Findbugs warnings in oozie-sharelib-oozie
https://issues.apache.org/jira/browse/OOZIE-2952
OOZIE-2949  Escape quotes whitespaces in Sqoop  field
https://issues.apache.org/jira/browse/OOZIE-2949
OOZIE-2942  Fix Findbugs warnings in oozie-examples
https://issues.apache.org/jira/browse/OOZIE-2942
OOZIE-2937  Remove redundant groupId from the child pom's
https://issues.apache.org/jira/browse/OOZIE-2937
OOZIE-2934  Fix "Exceptional return value of java.io.File.mkdirs() ignored" 
Findbugs error in oozie-sharelib-spark
https://issues.apache.org/jira/browse/OOZIE-2934
OOZIE-2927  Append new line character for Hive2 query using query tag
https://issues.apache.org/jira/browse/OOZIE-2927
OOZIE-2914  Consolidate Trim 
https://issues.apache.org/jira/browse/OOZIE-2914
OOZIE-2883  OOZIE throw the error "Missing 
[oozie.service.ProxyUserService.proxyuser.oozie.service.ProxyUserService.proxyuser.mr.groups]
 property"
https://issues.apache.org/jira/browse/OOZIE-2883
OOZIE-2877  Oozie Git Action
https://issues.apache.org/jira/browse/OOZIE-2877
OOZIE-2867  Timezone handling for Coordinators: emphasize "Continent/City" 
format
https://issues.apache.org/jira/browse/OOZIE-2867
OOZIE-2834  ParameterVerifier logging non-useful warning for workflow definition
https://issues.apache.org/jira/browse/OOZIE-2834
OOZIE-2833  when using uber mode the regex pattern used in the 
extractHeapSizeMB method does not allow heap sizes specified in bytes.
https://issues.apache.org/jira/browse/OOZIE-2833
OOZIE-2829  Improve sharelib upload to accept multiple source folders
https://issues.apache.org/jira/browse/OOZIE-2829
OOZIE-2826  Falcon feed fails to aws s3; Oozie joda time version does not meet 

[jira] [Commented] (OOZIE-3057) Custom Partitioner not working in Oozie Mapreduce action

[Raghavi Ravi (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16363563#comment-16363563
 ] 

Raghavi Ravi commented on OOZIE-3057:
-

[~gezapeti]

Attaching oozie logs, Partitioner class and workflow.xml. The workflow.xml has 
only one mapreduce action that reads RCFiles and creates text files.

[^Logs.zip] [^PonRankPartitioner.java]

> Custom Partitioner not working in Oozie Mapreduce action
> 
>
> Key: OOZIE-3057
> URL: https://issues.apache.org/jira/browse/OOZIE-3057
> Project: Oozie
>  Issue Type: Bug
>  Components: action, workflow
>Affects Versions: 4.1.0
> Environment: Red Hat Enterprise Linux Server release 7.2 (Maipo)
> Linux version 3.10.0-327.10.1.el7.x86_64 
> (mockbu...@x86-021.build.eng.bos.redhat.com) (gcc version 4.8.5 20150623 (Red 
> Hat 4.8.5-4) (GCC) ) #1 SMP Sat Jan 23 04:54:55 EST 2016
> oozie version - 4.1.0
> cdh version - 5.10.1
> Hue™ 3.11 - The Hadoop UI
>Reporter: Raghavi Ravi
>Priority: Critical
> Attachments: Logs.zip, PonRankPartitioner.java
>
>
> I implemented secondary sort in mapreduce using old API 
> (org.apache.hadoop.mapred.*) and trying to execute it using Oozie (From Hue).
> Though I have set the partitioner class in the properties, the partitioner is 
> not being executed. So, I'm not getting output as expected.
> The same code runs fine when run using hadoop command from CLI.
> And here is my workflow.xml
> 
> 
> 
> Action failed, error 
> message[${wf:errorMessage(wf:lastErrorNode())}]
> 
> 
> 
> ${jobTracker}
> ${nameNode}
> 
> 
> mapred.output.dir
> /test_1109_3
> 
> 
> mapred.input.dir
> 
> /apps/hive/warehouse/7360_0609_rx/day=06-09-2017/hour=13/quarter=2/,/apps/hive/warehouse/7360_0609_tx/day=06-09-2017/hour=13/quarter=2/,/apps/hive/warehouse/7360_0509_util/day=05-09-2017/hour=16/quarter=1/
> 
> 
> mapred.input.format.class
> org.apache.hadoop.hive.ql.io.RCFileInputFormat
> 
> 
> mapred.mapper.class
> PonRankMapper
> 
> 
> mapred.reducer.class
> PonRankReducer
> 
> 
> mapred.output.value.comparator.class
> PonRankGroupingComparator
> 
> 
> mapred.mapoutput.key.class
> PonRankPair
> 
> 
> mapred.mapoutput.value.class
> org.apache.hadoop.io.Text
> 
> 
> mapred.reduce.output.key.class
> org.apache.hadoop.io.NullWritable
> 
> 
> mapred.reduce.output.value.class
> org.apache.hadoop.io.Text
> 
> 
> mapred.reduce.tasks
> 1
> 
> 
> mapred.partitioner.class
> PonRankPartitioner
> 
> 
> mapred.mapper.new-api
> False
> 
> 
> 
> 
> 
> 
> 
> When running using hadoop jar command, I set the partitioner class using 
> JobConf.setPartitionerClass API.
>  Partitioner is not executed when using old API . Inspite of adding the 
> property.
> 
> mapred.partitioner.class
> PonRankPartitioner
> 
> Executed the same logic using new API's (org.apache.hadoop.mapreduce) and 
> added mapreduce.partitioner.class property in workflow.
> Partitioner was executed and desired outcome was seen.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (OOZIE-3057) Custom Partitioner not working in Oozie Mapreduce action

[Raghavi Ravi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-3057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Raghavi Ravi updated OOZIE-3057:

Attachment: PonRankPartitioner.java

> Custom Partitioner not working in Oozie Mapreduce action
> 
>
> Key: OOZIE-3057
> URL: https://issues.apache.org/jira/browse/OOZIE-3057
> Project: Oozie
>  Issue Type: Bug
>  Components: action, workflow
>Affects Versions: 4.1.0
> Environment: Red Hat Enterprise Linux Server release 7.2 (Maipo)
> Linux version 3.10.0-327.10.1.el7.x86_64 
> (mockbu...@x86-021.build.eng.bos.redhat.com) (gcc version 4.8.5 20150623 (Red 
> Hat 4.8.5-4) (GCC) ) #1 SMP Sat Jan 23 04:54:55 EST 2016
> oozie version - 4.1.0
> cdh version - 5.10.1
> Hue™ 3.11 - The Hadoop UI
>Reporter: Raghavi Ravi
>Priority: Critical
> Attachments: Logs.zip, PonRankPartitioner.java
>
>
> I implemented secondary sort in mapreduce using old API 
> (org.apache.hadoop.mapred.*) and trying to execute it using Oozie (From Hue).
> Though I have set the partitioner class in the properties, the partitioner is 
> not being executed. So, I'm not getting output as expected.
> The same code runs fine when run using hadoop command from CLI.
> And here is my workflow.xml
> 
> 
> 
> Action failed, error 
> message[${wf:errorMessage(wf:lastErrorNode())}]
> 
> 
> 
> ${jobTracker}
> ${nameNode}
> 
> 
> mapred.output.dir
> /test_1109_3
> 
> 
> mapred.input.dir
> 
> /apps/hive/warehouse/7360_0609_rx/day=06-09-2017/hour=13/quarter=2/,/apps/hive/warehouse/7360_0609_tx/day=06-09-2017/hour=13/quarter=2/,/apps/hive/warehouse/7360_0509_util/day=05-09-2017/hour=16/quarter=1/
> 
> 
> mapred.input.format.class
> org.apache.hadoop.hive.ql.io.RCFileInputFormat
> 
> 
> mapred.mapper.class
> PonRankMapper
> 
> 
> mapred.reducer.class
> PonRankReducer
> 
> 
> mapred.output.value.comparator.class
> PonRankGroupingComparator
> 
> 
> mapred.mapoutput.key.class
> PonRankPair
> 
> 
> mapred.mapoutput.value.class
> org.apache.hadoop.io.Text
> 
> 
> mapred.reduce.output.key.class
> org.apache.hadoop.io.NullWritable
> 
> 
> mapred.reduce.output.value.class
> org.apache.hadoop.io.Text
> 
> 
> mapred.reduce.tasks
> 1
> 
> 
> mapred.partitioner.class
> PonRankPartitioner
> 
> 
> mapred.mapper.new-api
> False
> 
> 
> 
> 
> 
> 
> 
> When running using hadoop jar command, I set the partitioner class using 
> JobConf.setPartitionerClass API.
>  Partitioner is not executed when using old API . Inspite of adding the 
> property.
> 
> mapred.partitioner.class
> PonRankPartitioner
> 
> Executed the same logic using new API's (org.apache.hadoop.mapreduce) and 
> added mapreduce.partitioner.class property in workflow.
> Partitioner was executed and desired outcome was seen.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (OOZIE-3057) Custom Partitioner not working in Oozie Mapreduce action

[Raghavi Ravi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-3057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Raghavi Ravi updated OOZIE-3057:

Attachment: Logs.zip

> Custom Partitioner not working in Oozie Mapreduce action
> 
>
> Key: OOZIE-3057
> URL: https://issues.apache.org/jira/browse/OOZIE-3057
> Project: Oozie
>  Issue Type: Bug
>  Components: action, workflow
>Affects Versions: 4.1.0
> Environment: Red Hat Enterprise Linux Server release 7.2 (Maipo)
> Linux version 3.10.0-327.10.1.el7.x86_64 
> (mockbu...@x86-021.build.eng.bos.redhat.com) (gcc version 4.8.5 20150623 (Red 
> Hat 4.8.5-4) (GCC) ) #1 SMP Sat Jan 23 04:54:55 EST 2016
> oozie version - 4.1.0
> cdh version - 5.10.1
> Hue™ 3.11 - The Hadoop UI
>Reporter: Raghavi Ravi
>Priority: Critical
> Attachments: Logs.zip
>
>
> I implemented secondary sort in mapreduce using old API 
> (org.apache.hadoop.mapred.*) and trying to execute it using Oozie (From Hue).
> Though I have set the partitioner class in the properties, the partitioner is 
> not being executed. So, I'm not getting output as expected.
> The same code runs fine when run using hadoop command from CLI.
> And here is my workflow.xml
> 
> 
> 
> Action failed, error 
> message[${wf:errorMessage(wf:lastErrorNode())}]
> 
> 
> 
> ${jobTracker}
> ${nameNode}
> 
> 
> mapred.output.dir
> /test_1109_3
> 
> 
> mapred.input.dir
> 
> /apps/hive/warehouse/7360_0609_rx/day=06-09-2017/hour=13/quarter=2/,/apps/hive/warehouse/7360_0609_tx/day=06-09-2017/hour=13/quarter=2/,/apps/hive/warehouse/7360_0509_util/day=05-09-2017/hour=16/quarter=1/
> 
> 
> mapred.input.format.class
> org.apache.hadoop.hive.ql.io.RCFileInputFormat
> 
> 
> mapred.mapper.class
> PonRankMapper
> 
> 
> mapred.reducer.class
> PonRankReducer
> 
> 
> mapred.output.value.comparator.class
> PonRankGroupingComparator
> 
> 
> mapred.mapoutput.key.class
> PonRankPair
> 
> 
> mapred.mapoutput.value.class
> org.apache.hadoop.io.Text
> 
> 
> mapred.reduce.output.key.class
> org.apache.hadoop.io.NullWritable
> 
> 
> mapred.reduce.output.value.class
> org.apache.hadoop.io.Text
> 
> 
> mapred.reduce.tasks
> 1
> 
> 
> mapred.partitioner.class
> PonRankPartitioner
> 
> 
> mapred.mapper.new-api
> False
> 
> 
> 
> 
> 
> 
> 
> When running using hadoop jar command, I set the partitioner class using 
> JobConf.setPartitionerClass API.
>  Partitioner is not executed when using old API . Inspite of adding the 
> property.
> 
> mapred.partitioner.class
> PonRankPartitioner
> 
> Executed the same logic using new API's (org.apache.hadoop.mapreduce) and 
> added mapreduce.partitioner.class property in workflow.
> Partitioner was executed and desired outcome was seen.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (OOZIE-3182) Oozie components fail with checkstyle errors

[Alisha Prabhu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-3182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alisha Prabhu updated OOZIE-3182:
-
Attachment: log-OOZIE-3182-002.txt

> Oozie components fail with checkstyle errors
> 
>
> Key: OOZIE-3182
> URL: https://issues.apache.org/jira/browse/OOZIE-3182
> Project: Oozie
>  Issue Type: Bug
>  Components: client, examples
>Affects Versions: 5.0.0b1
> Environment: uname -a
> Linux pts00607-vm1 4.4.0-31-generic #50-Ubuntu SMP Wed Jul 13 00:05:18 UTC 
> 2016 ppc64le ppc64le ppc64le GNU/Linux
>Reporter: Alisha Prabhu
>Assignee: Alisha Prabhu
>Priority: Major
> Attachments: OOZIE-3182-001.patch, OOZIE-3182-002.patch, 
> error-log.txt, log-OOZIE-3182-002.txt
>
>
> There are multiple checkstyle violations in the oozie-client, oozie-examples, 
> oozie-mini, oozie-sharelib-oozie, oozie-sharelib module.
> Maven command used is mvn clean install -DskipTests



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (OOZIE-3182) Oozie components fail with checkstyle errors

[Alisha Prabhu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-3182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alisha Prabhu updated OOZIE-3182:
-
Attachment: OOZIE-3182-002.patch

> Oozie components fail with checkstyle errors
> 
>
> Key: OOZIE-3182
> URL: https://issues.apache.org/jira/browse/OOZIE-3182
> Project: Oozie
>  Issue Type: Bug
>  Components: client, examples
>Affects Versions: 5.0.0b1
> Environment: uname -a
> Linux pts00607-vm1 4.4.0-31-generic #50-Ubuntu SMP Wed Jul 13 00:05:18 UTC 
> 2016 ppc64le ppc64le ppc64le GNU/Linux
>Reporter: Alisha Prabhu
>Assignee: Alisha Prabhu
>Priority: Major
> Attachments: OOZIE-3182-001.patch, OOZIE-3182-002.patch
>
>
> There are multiple checkstyle violations in the oozie-client, oozie-examples, 
> oozie-mini, oozie-sharelib-oozie, oozie-sharelib module.
> Maven command used is mvn clean install -DskipTests



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (OOZIE-3182) Oozie components fail with checkstyle errors

[Alisha Prabhu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-3182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alisha Prabhu updated OOZIE-3182:
-
Attachment: error-log.txt

> Oozie components fail with checkstyle errors
> 
>
> Key: OOZIE-3182
> URL: https://issues.apache.org/jira/browse/OOZIE-3182
> Project: Oozie
>  Issue Type: Bug
>  Components: client, examples
>Affects Versions: 5.0.0b1
> Environment: uname -a
> Linux pts00607-vm1 4.4.0-31-generic #50-Ubuntu SMP Wed Jul 13 00:05:18 UTC 
> 2016 ppc64le ppc64le ppc64le GNU/Linux
>Reporter: Alisha Prabhu
>Assignee: Alisha Prabhu
>Priority: Major
> Attachments: OOZIE-3182-001.patch, OOZIE-3182-002.patch, error-log.txt
>
>
> There are multiple checkstyle violations in the oozie-client, oozie-examples, 
> oozie-mini, oozie-sharelib-oozie, oozie-sharelib module.
> Maven command used is mvn clean install -DskipTests



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Failed: OOZIE-3183 PreCommit Build #396

[Apache Jenkins Server]

Jira: https://issues.apache.org/jira/browse/OOZIE-3183
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/396/

###
## LAST 100 LINES OF THE CONSOLE 
###
[...truncated 1.70 MB...]
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar]
 removed
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5sum]
 removed
  Running test-patch task BACKWARDS_COMPATIBILITY
  Running test-patch task TESTS
  Running test-patch task DISTRO


Testing JIRA OOZIE-3183

Cleaning local git workspace



+1 PATCH_APPLIES
+1 CLEAN
-1 RAW_PATCH_ANALYSIS
+1 the patch does not introduce any @author tags
+1 the patch does not introduce any tabs
+1 the patch does not introduce any trailing spaces
+1 the patch does not introduce any line longer than 132
-1 the patch does not add/modify any testcase
+1 RAT
+1 the patch does not seem to introduce new RAT warnings
+1 JAVADOC
+1 the patch does not seem to introduce new Javadoc warnings
WARNING: the current HEAD has 100 Javadoc warning(s)
+1 COMPILE
+1 HEAD compiles
+1 patch compiles
+1 the patch does not seem to introduce new javac warnings
-1 There are [4] new bugs found below threshold in total that must be fixed.
 +1 There are no new bugs found in [webapp].
 -1 There are [4] new bugs found below threshold in [core] that must be fixed.
 You can find the FindBugs diff here (look for the red and orange ones): 
core/findbugs-new.html
 The most important FindBugs errors are:
 At SshActionExecutor.java:[line 136]: This usage of 
java/lang/Runtime.exec([Ljava/lang/String;)Ljava/lang/Process; can be 
vulnerable to Command Injection
 At SshActionExecutor.java:[line 133]: At SshActionExecutor.java:[line 132]
 Dead store to log in 
org.apache.oozie.action.ssh.SshActionExecutor.doExecute(String, String, String, 
String[], boolean, WorkflowAction, String, boolean): At 
SshActionExecutor.java:[line 136]
 java/io/File.init(Ljava/lang/String;)V reads a file whose location 
might be specified by user input: At SshActionExecutor.java:[line 418]
 At SshActionExecutor.java:[line 376]: At SshActionExecutor.java:[line 380]
 +1 There are no new bugs found in [tools].
 +1 There are no new bugs found in [sharelib/hive2].
 +1 There are no new bugs found in [sharelib/distcp].
 +1 There are no new bugs found in [sharelib/hcatalog].
 +1 There are no new bugs found in [sharelib/streaming].
 +1 There are no new bugs found in [sharelib/sqoop].
 +1 There are no new bugs found in [sharelib/oozie].
 +1 There are no new bugs found in [sharelib/pig].
 +1 There are no new bugs found in [sharelib/hive].
 +1 There are no new bugs found in [sharelib/spark].
 +1 There are no new bugs found in [client].
 +1 There are no new bugs found in [examples].
 +1 There are no new bugs found in [docs].
 +1 There are no new bugs found in [server].
+1 BACKWARDS_COMPATIBILITY
+1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient 
annotations
+1 the patch does not modify JPA files
-1 TESTS
Tests run: 2092
Tests failed: 1
Tests errors: 1

The patch failed the following testcases:

testHiveAction(org.apache.oozie.action.hadoop.TestHiveActionExecutor)

Tests failing with errors:
testNewUsingACLs(org.apache.oozie.util.TestZKUtilsWithSecurity)

Tests failed at first run:
TestJavaActionExecutor#testCredentialsSkip
TestCoordActionsKillXCommand#testActionKillCommandActionNumbers
For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
+1 DISTRO
+1 distro tarball builds with the patch 


-1 Overall result, please check the reported -1(s)

 There is at least one warning, please check

The full output of the test-patch run is available at

 https://builds.apache.org/job/PreCommit-OOZIE-Build/396/

Adding comment to JIRA
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0  
0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 
0{"self":"https://issues.apache.org/jira/rest/api/2/issue/13138394/comment/16363447","id":"16363447","author":{"self":"https://issues.apache.org/jira/rest/api/2/user?username=hadoopqa","name":"hadoopqa","key":"hadoopqa","emailAddress":"blackhole
 at hadoop dot apache dot 

[jira] [Commented] (OOZIE-3183) Better logging for SshActionExecutor and extended HA capability when calling to remote host

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16363447#comment-16363447
 ] 

Hadoop QA commented on OOZIE-3183:
--


Testing JIRA OOZIE-3183

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
.{color:red}WARNING{color}: the current HEAD has 100 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:red}-1{color} There are [4] new bugs found below threshold in total that 
must be fixed.
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:red}-1{color} There are [4] new bugs found below threshold in [core] 
that must be fixed.
. You can find the FindBugs diff here (look for the red and orange ones): 
core/findbugs-new.html
. The most important FindBugs errors are:
. At SshActionExecutor.java:[line 136]: This usage of 
java/lang/Runtime.exec([Ljava/lang/String;)Ljava/lang/Process; can be 
vulnerable to Command Injection
. At SshActionExecutor.java:[line 133]: At SshActionExecutor.java:[line 132]
. Dead store to log in 
org.apache.oozie.action.ssh.SshActionExecutor.doExecute(String, String, String, 
String[], boolean, WorkflowAction, String, boolean): At 
SshActionExecutor.java:[line 136]
. java/io/File.init(Ljava/lang/String;)V reads a file whose location 
might be specified by user input: At SshActionExecutor.java:[line 418]
. At SshActionExecutor.java:[line 376]: At SshActionExecutor.java:[line 380]
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [server].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 2092
.Tests failed: 1
.Tests errors: 1

.The patch failed the following testcases:

testHiveAction(org.apache.oozie.action.hadoop.TestHiveActionExecutor)

.Tests failing with errors:
testNewUsingACLs(org.apache.oozie.util.TestZKUtilsWithSecurity)

.{color:orange}Tests failed at first run:{color}
TestJavaActionExecutor#testCredentialsSkip
TestCoordActionsKillXCommand#testActionKillCommandActionNumbers
.For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}

{color:red}. There is at least one warning, please check{color}

The full output of the test-patch run is available at

. https://builds.apache.org/job/PreCommit-OOZIE-Build/396/



> Better logging for SshActionExecutor and extended HA capability when calling 
> to remote host
> ---
>
> Key: OOZIE-3183
> URL: https://issues.apache.org/jira/browse/OOZIE-3183
> Project: Oozie
>  Issue Type: Improvement
>  Components: action
>Affects Versions: 5.0.0b1
>Reporter: Andras Piros
>Assignee: Andras Piros
>

[jira] [Commented] (OOZIE-3183) Better logging for SshActionExecutor and extended HA capability when calling to remote host

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16363362#comment-16363362
 ] 

Hadoop QA commented on OOZIE-3183:
--

PreCommit-OOZIE-Build started


> Better logging for SshActionExecutor and extended HA capability when calling 
> to remote host
> ---
>
> Key: OOZIE-3183
> URL: https://issues.apache.org/jira/browse/OOZIE-3183
> Project: Oozie
>  Issue Type: Improvement
>  Components: action
>Affects Versions: 5.0.0b1
>Reporter: Andras Piros
>Assignee: Andras Piros
>Priority: Major
> Fix For: 5.0.0
>
> Attachments: OOZIE-3183.001.patch
>
>
> In some High Availability environments, when an SSH action's 
> {{SshActionExecutor#start()}} runs on another node than 
> {{SshActionExecutor#check()}}, it can happen that due to network bandwidth 
> reasons calling {{Process#exitValue()}} is not enough - but calling 
> {{Process#waitFor()}} is better.
> Also, for better error root cause analysis, enhance logging along 
> {{SshActionExecutor}} for the critical parts.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (OOZIE-3183) Better logging for SshActionExecutor and extended HA capability when calling to remote host

[Andras Piros (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-3183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Piros updated OOZIE-3183:

Attachment: OOZIE-3183.001.patch

> Better logging for SshActionExecutor and extended HA capability when calling 
> to remote host
> ---
>
> Key: OOZIE-3183
> URL: https://issues.apache.org/jira/browse/OOZIE-3183
> Project: Oozie
>  Issue Type: Improvement
>  Components: action
>Affects Versions: 5.0.0b1
>Reporter: Andras Piros
>Assignee: Andras Piros
>Priority: Major
> Fix For: 5.0.0
>
> Attachments: OOZIE-3183.001.patch
>
>
> In some High Availability environments, when an SSH action's 
> {{SshActionExecutor#start()}} runs on another node than 
> {{SshActionExecutor#check()}}, it can happen that due to network bandwidth 
> reasons calling {{Process#exitValue()}} is not enough - but calling 
> {{Process#waitFor()}} is better.
> Also, for better error root cause analysis, enhance logging along 
> {{SshActionExecutor}} for the critical parts.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (OOZIE-3183) Better logging for SshActionExecutor and extended HA capability when calling to remote host

[Andras Piros (JIRA)]

Andras Piros created OOZIE-3183:
---

 Summary: Better logging for SshActionExecutor and extended HA 
capability when calling to remote host
 Key: OOZIE-3183
 URL: https://issues.apache.org/jira/browse/OOZIE-3183
 Project: Oozie
  Issue Type: Improvement
  Components: action
Affects Versions: 5.0.0b1
Reporter: Andras Piros
Assignee: Andras Piros
 Fix For: 5.0.0


In some High Availability environments, when an SSH action's 
{{SshActionExecutor#start()}} runs on another node than 
{{SshActionExecutor#check()}}, it can happen that due to network bandwidth 
reasons calling {{Process#exitValue()}} is not enough - but calling 
{{Process#waitFor()}} is better.

Also, for better error root cause analysis, enhance logging along 
{{SshActionExecutor}} for the critical parts.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (OOZIE-2814) OYA: Update example workflows to newest schemas

[Andras Piros (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16363090#comment-16363090
 ] 

Andras Piros commented on OOZIE-2814:
-

[~asasvari] thanks for the contribution, +1

> OYA: Update example workflows to newest schemas
> ---
>
> Key: OOZIE-2814
> URL: https://issues.apache.org/jira/browse/OOZIE-2814
> Project: Oozie
>  Issue Type: Sub-task
>Affects Versions: 5.0.0
>Reporter: Robert Kanter
>Assignee: Attila Sasvari
>Priority: Blocker
> Attachments: OOZIE-2814-00.patch, OOZIE-2814-01.patch, 
> OOZIE-2814-02.patch, OOZIE-2814-03.patch, OOZIE-2814-04.patch, 
> OOZIE-2814-05.patch
>
>
> OOZIE-2687 will add in a whole set of newer schemas that are more 
> Yarn-centric.  Most, if not all, examples currently use pretty old versions 
> of the schemas.  We should take this opportunity to update the examples to 
> the newest schemas added by OOZIE-2687, which should help demonstrate them.
> It may make sense to keep the older schemas too, to show that older workflows 
> still work.  We have some examples where we have two workflow XML files that 
> do the same thing in different ways.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 65606: OOZIE-2814 OYA: Update example workflows to newest schemas

[András Piros via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65606/#review197436
---


Ship it!




Ship It!

- András Piros


On Feb. 12, 2018, 4:19 p.m., Attila Sasvari wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65606/
> ---
> 
> (Updated Feb. 12, 2018, 4:19 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> OOZIE-2687 will add in a whole set of newer schemas that are more 
> Yarn-centric. Most, if not all, examples currently use pretty old versions of 
> the schemas. We should take this opportunity to update the examples to the 
> newest schemas added by OOZIE-2687, which should help demonstrate them.
> 
> It may make sense to keep the older schemas too, to show that older workflows 
> still work. We have some examples where we have two workflow XML files that 
> do the same thing in different ways.
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
> bc0f4050bb3788988d3a5352a15220fbc4fa219d 
>   
> core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 
> c236dafa5965403c33e1a400509e7c374fe2637c 
>   core/src/main/resources/oozie-default.xml 
> e53b17b3cf201b4cf7bc5371503e66327eae2126 
>   
> core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 
> 1cc569d8d7a0e61905ad6c4e940cefb00da1343d 
>   
> core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java
>  2cdbde4d2fc86a07b35beadc56012c438baf8d54 
>   examples/src/main/apps/aggregator/coordinator-with-offset.xml 
> 4c351677f4afc6e85f807106d426bb2f922a28b1 
>   examples/src/main/apps/aggregator/coordinator.xml 
> 948edc1adbe09dfb6f7bffb5eebc82f3670658cc 
>   examples/src/main/apps/aggregator/job-with-offset.properties 
> 9a28ae13e50c40f70db4b0f583a63eb42f97388d 
>   examples/src/main/apps/aggregator/job.properties 
> aaf4edf7b9d030b5044610698b27a564b008f77b 
>   examples/src/main/apps/aggregator/workflow.xml 
> 4a801090e31acba23ea49da1c59a8ce4b951c654 
>   examples/src/main/apps/bundle/job.properties 
> c06edeb3f42e2eba3367b9486982dabb4caf6436 
>   examples/src/main/apps/coord-input-logic/job.properties 
> 2cd137b94999c7e1f68bb54928f1c3ce2838d59d 
>   examples/src/main/apps/coord-input-logic/workflow.xml 
> 80d31457eff6e78710ad1fb4d4f1b372c0d1e43c 
>   examples/src/main/apps/cron-schedule/coordinator.xml 
> 48a7de76245e02397b9736a95fa0132504067f9a 
>   examples/src/main/apps/cron-schedule/job.properties 
> 5b5eb43ca2bf9d8cd1b263ca3b94a552b562a20d 
>   examples/src/main/apps/cron-schedule/workflow.xml 
> b53922c8f40e691cb637b91cb182172794296a04 
>   examples/src/main/apps/cron/coordinator.xml 
> 744d14e870b59d0b89f65a78fea63fffa3ff4b8e 
>   examples/src/main/apps/cron/job.properties 
> 42875fd5a5ce393cd11717bcc55fc1bb91e0fcd3 
>   examples/src/main/apps/cron/workflow.xml 
> 63dff976f9e35a26b0ad09078778748d3d02bc3b 
>   examples/src/main/apps/custom-main/job.properties 
> b897512a1c6246308ca17b019ce400f3ec06cd7b 
>   examples/src/main/apps/custom-main/workflow.xml 
> 828dfaa0594aac0539c4e08ae8e37cb89d17eb54 
>   examples/src/main/apps/datelist-java-main/job.properties 
> e8356e3a4a2e1ae41e09f5ea124fb5360bd6e8b6 
>   examples/src/main/apps/datelist-java-main/workflow.xml 
> b59a12ebd1fc21c9e26f6a5b0348eb9c76375c03 
>   examples/src/main/apps/demo/job.properties 
> d70d3208df77e0b612005196907cecd97d7c5d21 
>   examples/src/main/apps/demo/job.windows.properties 
> ea948b20009685e1286053549fc1974669819682 
>   examples/src/main/apps/demo/workflow.xml 
> 0ceca507adc68c0674b794420c9eb733cb320879 
>   examples/src/main/apps/distcp/job.properties 
> ec9422caddcdecfa6ee9a58868cf68d7457ad4db 
>   examples/src/main/apps/distcp/workflow.xml 
> 6deecb377901613a5bb28fc18bf5beff2d30f252 
>   examples/src/main/apps/hadoop-el/job.properties 
> 1d13c27056ac58ad359b5298f12ec046bb4cc61d 
>   examples/src/main/apps/hadoop-el/workflow.xml 
> d4094d44136a7de2f85f5c9be042d5aab24b90be 
>   examples/src/main/apps/hcatalog/job.properties 
> e0aa18c264a2fa92b6c3d501d18fa39733d863ff 
>   examples/src/main/apps/hcatalog/job.properties.security 
> 2fbcc6ee9e5d94d3bba63810e6e788607747940b 
>   examples/src/main/apps/hcatalog/workflow.xml 
> d6593df1b9104179d79262d663135d73de8b55ec 
>   examples/src/main/apps/hcatalog/workflow.xml.security 
> aa3522ae4d5b6cb178f80c2e72e45c4c3de9aab5 
>   examples/src/main/apps/hive/job.properties 
> dbc7ad2a0b135b8c9f0b6e575086ee8260072756 
>   examples/src/main/apps/hive/workflow.xml 
> 449d62d35f2e69d01d9e59f35a0cc9ad057342d2 
>   examples/src/main/apps/hive/workflow.xml.security 
> 35321319dcbb0576c5870952e6e805209b8dc240 
>   

Re: Review Request 65606: OOZIE-2814 OYA: Update example workflows to newest schemas

[András Piros via Review Board]

> On Feb. 12, 2018, 8:05 p.m., András Piros wrote:
> > core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java
> > Line 460 (original), 465-467 (patched)
> > 
> >
> > This is pretty hard to understand. I'd either separate 
> > `resourceManager` and `jobTracker` into two variables / fields, or have 
> > some more clean, generic labelling for the same attribute.
> 
> Attila Sasvari wrote:
> If the user is not using  (meaning 
> is specified), then ``isResourceManagerUsed`` will be 
> true. Then we save this as a flag in ``gdata`` that is reused later, when an 
> action is missing both  and  - we need 
> to make sure we reuse the same tag. 
>   
> I am not sure I totally understand. Can you please expand your ideas a 
> bit? Do you want to refactor the ``GlobalSectionData`` class or subclass it 
> or these should be the fields of the ``LiteWorkflowAppParser`` (in that case 
> we might need to also adjust ``GlobalSectionData`` because of its 
> serialization methods)? Anyway, at the end, it will result in doing a similar 
> check that we have now (if ``resourceManager`` is not null, then use 
>  otherwise ).

As chatted on the subject offline, the proposed refactoring is not worth the 
effort.


> On Feb. 12, 2018, 8:05 p.m., András Piros wrote:
> > core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java
> > Lines 587-594 (patched)
> > 
> >
> > I'd either separate `resourceManager` and `jobTracker` into two 
> > variables / fields, or have some more clean, generic labelling for the same 
> > attribute.

As chatted on the subject offline, the proposed refactoring is not worth the 
effort.


- András


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65606/#review197301
---


On Feb. 12, 2018, 4:19 p.m., Attila Sasvari wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65606/
> ---
> 
> (Updated Feb. 12, 2018, 4:19 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> OOZIE-2687 will add in a whole set of newer schemas that are more 
> Yarn-centric. Most, if not all, examples currently use pretty old versions of 
> the schemas. We should take this opportunity to update the examples to the 
> newest schemas added by OOZIE-2687, which should help demonstrate them.
> 
> It may make sense to keep the older schemas too, to show that older workflows 
> still work. We have some examples where we have two workflow XML files that 
> do the same thing in different ways.
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
> bc0f4050bb3788988d3a5352a15220fbc4fa219d 
>   
> core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 
> c236dafa5965403c33e1a400509e7c374fe2637c 
>   core/src/main/resources/oozie-default.xml 
> e53b17b3cf201b4cf7bc5371503e66327eae2126 
>   
> core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 
> 1cc569d8d7a0e61905ad6c4e940cefb00da1343d 
>   
> core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java
>  2cdbde4d2fc86a07b35beadc56012c438baf8d54 
>   examples/src/main/apps/aggregator/coordinator-with-offset.xml 
> 4c351677f4afc6e85f807106d426bb2f922a28b1 
>   examples/src/main/apps/aggregator/coordinator.xml 
> 948edc1adbe09dfb6f7bffb5eebc82f3670658cc 
>   examples/src/main/apps/aggregator/job-with-offset.properties 
> 9a28ae13e50c40f70db4b0f583a63eb42f97388d 
>   examples/src/main/apps/aggregator/job.properties 
> aaf4edf7b9d030b5044610698b27a564b008f77b 
>   examples/src/main/apps/aggregator/workflow.xml 
> 4a801090e31acba23ea49da1c59a8ce4b951c654 
>   examples/src/main/apps/bundle/job.properties 
> c06edeb3f42e2eba3367b9486982dabb4caf6436 
>   examples/src/main/apps/coord-input-logic/job.properties 
> 2cd137b94999c7e1f68bb54928f1c3ce2838d59d 
>   examples/src/main/apps/coord-input-logic/workflow.xml 
> 80d31457eff6e78710ad1fb4d4f1b372c0d1e43c 
>   examples/src/main/apps/cron-schedule/coordinator.xml 
> 48a7de76245e02397b9736a95fa0132504067f9a 
>   examples/src/main/apps/cron-schedule/job.properties 
> 5b5eb43ca2bf9d8cd1b263ca3b94a552b562a20d 
>   examples/src/main/apps/cron-schedule/workflow.xml 
> b53922c8f40e691cb637b91cb182172794296a04 
>   examples/src/main/apps/cron/coordinator.xml 
> 744d14e870b59d0b89f65a78fea63fffa3ff4b8e 
>   examples/src/main/apps/cron/job.properties 
> 42875fd5a5ce393cd11717bcc55fc1bb91e0fcd3 
>   examples/src/main/apps/cron/workflow.xml 
> 

[jira] [Commented] (OOZIE-2814) OYA: Update example workflows to newest schemas

[Attila Sasvari (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16363064#comment-16363064
 ] 

Attila Sasvari commented on OOZIE-2814:
---

Can someone review this please?

> OYA: Update example workflows to newest schemas
> ---
>
> Key: OOZIE-2814
> URL: https://issues.apache.org/jira/browse/OOZIE-2814
> Project: Oozie
>  Issue Type: Sub-task
>Affects Versions: 5.0.0
>Reporter: Robert Kanter
>Assignee: Attila Sasvari
>Priority: Blocker
> Attachments: OOZIE-2814-00.patch, OOZIE-2814-01.patch, 
> OOZIE-2814-02.patch, OOZIE-2814-03.patch, OOZIE-2814-04.patch, 
> OOZIE-2814-05.patch
>
>
> OOZIE-2687 will add in a whole set of newer schemas that are more 
> Yarn-centric.  Most, if not all, examples currently use pretty old versions 
> of the schemas.  We should take this opportunity to update the examples to 
> the newest schemas added by OOZIE-2687, which should help demonstrate them.
> It may make sense to keep the older schemas too, to show that older workflows 
> still work.  We have some examples where we have two workflow XML files that 
> do the same thing in different ways.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (OOZIE-3134) Potential inconsistency between the in-memory SLA map and the Oozie database

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362942#comment-16362942
 ] 

Hadoop QA commented on OOZIE-3134:
--


Testing JIRA OOZIE-3134

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:red}-1{color} the patch contains 1 line(s) longer than 132 
characters
.{color:green}+1{color} the patch adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
.{color:red}WARNING{color}: the current HEAD has 100 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:red}-1{color} There are [2] new bugs found below threshold in total that 
must be fixed.
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:red}-1{color} There are [2] new bugs found below threshold in [core] 
that must be fixed.
. You can find the FindBugs diff here (look for the red and orange ones): 
core/findbugs-new.html
. The most important FindBugs errors are:
. At FailingConnectionWrapper.java:[line 175]: This use of 
java/sql/Connection.prepareStatement(Ljava/lang/String;II)Ljava/sql/PreparedStatement;
 can be vulnerable to SQL injection
. org.apache.oozie.util.db.FailingDBHelperForTest.dbPredicate should be package 
protected: At FailingConnectionWrapper.java:[line 169]
. At FailingDBHelperForTest.java:[line 30]
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [server].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 2096
.{color:orange}Tests failed at first run:{color}
TestJavaActionExecutor#testCredentialsSkip
.For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}

{color:red}. There is at least one warning, please check{color}

The full output of the test-patch run is available at

. https://builds.apache.org/job/PreCommit-OOZIE-Build/395/



> Potential inconsistency between the in-memory SLA map and the Oozie database
> 
>
> Key: OOZIE-3134
> URL: https://issues.apache.org/jira/browse/OOZIE-3134
> Project: Oozie
>  Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.0.0
>
> Attachments: OOZIE-3134-001.patch, OOZIE-3134-002.patch, 
> OOZIE-3134-003.patch
>
>
> Upon {{SLACalculatorMemory.addRegistration}}, Oozie puts an entry into an 
> in-memory concurrent hashmap ("slaMap"), and later [persists 
> it|https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L380]
>  in the Oozie database. 
> However, if there is a failure during the database operation, a 
> {{JPAExecutorException}} is thrown, and the entry [is not removed from the 
> SLA map| 
> 

Failed: OOZIE-3134 PreCommit Build #395

[Apache Jenkins Server]

Jira: https://issues.apache.org/jira/browse/OOZIE-3134
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/395/

###
## LAST 100 LINES OF THE CONSOLE 
###
[...truncated 1.70 MB...]
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [client].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[ERROR] There are [2] new bugs found below threshold in [core].
[DEBUG] You can find the FindBugs diff here (look for the red and orange ones): 
core/findbugs-new.html
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [tools].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [server].
[ERROR] There are [2] new bugs found below threshold in total that must be 
fixed.
[TRACE] FindBugs diffs checked and reports created
[TRACE] Summary file size is 2816 bytes
[TRACE] Full summary file size is 1880 bytes
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar]
 removed
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5sum]
 removed
  Running test-patch task BACKWARDS_COMPATIBILITY
  Running test-patch task TESTS
  Running test-patch task DISTRO


Testing JIRA OOZIE-3134

Cleaning local git workspace



+1 PATCH_APPLIES
+1 CLEAN
-1 RAW_PATCH_ANALYSIS
+1 the patch does not introduce any @author tags
+1 the patch does not introduce any tabs
+1 the patch does not introduce any trailing spaces
-1 the patch contains 1 line(s) longer than 132 characters
+1 the patch adds/modifies 1 testcase(s)
+1 RAT
+1 the patch does not seem to introduce new RAT warnings
+1 JAVADOC
+1 the patch does not seem to introduce new Javadoc warnings
WARNING: the current HEAD has 100 Javadoc warning(s)
+1 COMPILE
+1 HEAD compiles
+1 patch compiles
+1 the patch does not seem to introduce new javac warnings
-1 There are [2] new bugs found below threshold in total that must be fixed.
 +1 There are no new bugs found in [docs].
 +1 There are no new bugs found in [webapp].
 +1 There are no new bugs found in [sharelib/distcp].
 +1 There are no new bugs found in [sharelib/hive].
 +1 There are no new bugs found in [sharelib/spark].
 +1 There are no new bugs found in [sharelib/hive2].
 +1 There are no new bugs found in [sharelib/hcatalog].
 +1 There are no new bugs found in [sharelib/streaming].
 +1 There are no new bugs found in [sharelib/pig].
 +1 There are no new bugs found in [sharelib/sqoop].
 +1 There are no new bugs found in [sharelib/oozie].
 +1 There are no new bugs found in [examples].
 +1 There are no new bugs found in [client].
 -1 There are [2] new bugs found below threshold in [core] that must be fixed.
 You can find the FindBugs diff here (look for the red and orange ones): 
core/findbugs-new.html
 The most important FindBugs errors are:
 At FailingConnectionWrapper.java:[line 175]: This use of 
java/sql/Connection.prepareStatement(Ljava/lang/String;II)Ljava/sql/PreparedStatement;
 can be vulnerable to SQL injection
 org.apache.oozie.util.db.FailingDBHelperForTest.dbPredicate should be package 
protected: At FailingConnectionWrapper.java:[line 169]
 At FailingDBHelperForTest.java:[line 30]
 +1 There are no new bugs found in [tools].
 +1 There are no new bugs found in [server].
+1 BACKWARDS_COMPATIBILITY
+1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient 
annotations
+1 the patch does not modify JPA files
+1 TESTS
Tests run: 2096
Tests failed at first run:
TestJavaActionExecutor#testCredentialsSkip
For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
+1 DISTRO
+1 distro tarball builds with the patch 


-1 Overall result, please check the reported -1(s)

 There is at least one warning, please check

The full output of the test-patch run is available at

 https://builds.apache.org/job/PreCommit-OOZIE-Build/395/

Adding comment to JIRA
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 
0100  37080 0  100  3708  0   3783 --:--:-- --:--:-- --:--:--  
3783{"self":"https://issues.apache.org/jira/rest/api/2/issue/13120523/comment/16362942","id":"16362942","author":{"self":"https://issues.apache.org/jira/rest/api/2/user?username=hadoopqa","name":"hadoopqa","key":"hadoopqa","emailAddress":"blackhole
 at hadoop dot apache dot 

[jira] [Commented] (OOZIE-3134) Potential inconsistency between the in-memory SLA map and the Oozie database

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362778#comment-16362778
 ] 

Hadoop QA commented on OOZIE-3134:
--

PreCommit-OOZIE-Build started


> Potential inconsistency between the in-memory SLA map and the Oozie database
> 
>
> Key: OOZIE-3134
> URL: https://issues.apache.org/jira/browse/OOZIE-3134
> Project: Oozie
>  Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.0.0
>
> Attachments: OOZIE-3134-001.patch, OOZIE-3134-002.patch, 
> OOZIE-3134-003.patch
>
>
> Upon {{SLACalculatorMemory.addRegistration}}, Oozie puts an entry into an 
> in-memory concurrent hashmap ("slaMap"), and later [persists 
> it|https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L380]
>  in the Oozie database. 
> However, if there is a failure during the database operation, a 
> {{JPAExecutorException}} is thrown, and the entry [is not removed from the 
> SLA map| 
> https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L393].
> It may introduce inconsistency between the Oozie database and the SLA map.
> To prevent this, a rollback mechanism (with proper logging) should be 
> implemented.  It would also make sense to do more sanity/consistency check in 
> the Oozie server.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (OOZIE-2814) OYA: Update example workflows to newest schemas

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362775#comment-16362775
 ] 

Hadoop QA commented on OOZIE-2814:
--


Testing JIRA OOZIE-2814

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:red}-1{color} the patch contains 2 line(s) longer than 132 
characters
.{color:green}+1{color} the patch adds/modifies 2 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
.{color:red}WARNING{color}: the current HEAD has 100 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:green}+1{color} There are no new bugs found in [tools].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 2097
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}

{color:red}. There is at least one warning, please check{color}

The full output of the test-patch run is available at

. https://builds.apache.org/job/PreCommit-OOZIE-Build/394/



> OYA: Update example workflows to newest schemas
> ---
>
> Key: OOZIE-2814
> URL: https://issues.apache.org/jira/browse/OOZIE-2814
> Project: Oozie
>  Issue Type: Sub-task
>Affects Versions: 5.0.0
>Reporter: Robert Kanter
>Assignee: Attila Sasvari
>Priority: Blocker
> Attachments: OOZIE-2814-00.patch, OOZIE-2814-01.patch, 
> OOZIE-2814-02.patch, OOZIE-2814-03.patch, OOZIE-2814-04.patch, 
> OOZIE-2814-05.patch
>
>
> OOZIE-2687 will add in a whole set of newer schemas that are more 
> Yarn-centric.  Most, if not all, examples currently use pretty old versions 
> of the schemas.  We should take this opportunity to update the examples to 
> the newest schemas added by OOZIE-2687, which should help demonstrate them.
> It may make sense to keep the older schemas too, to show that older workflows 
> still work.  We have some examples where we have two workflow XML files that 
> do the same thing in different ways.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Failed: OOZIE-2814 PreCommit Build #394

[Apache Jenkins Server]

Jira: https://issues.apache.org/jira/browse/OOZIE-2814
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/394/

###
## LAST 100 LINES OF THE CONSOLE 
###
[...truncated 1.89 MB...]
[DEBUG] There are no new bugs found in [sharelib/hcatalog].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/pig].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/distcp].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [docs].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [server].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [core].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [examples].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [webapp].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [tools].
[INFO] There are no new bugs found totally].
[TRACE] FindBugs diffs checked and reports created
[TRACE] Summary file size is 2250 bytes
[TRACE] Full summary file size is 1314 bytes
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar]
 removed
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5sum]
 removed
  Running test-patch task BACKWARDS_COMPATIBILITY
  Running test-patch task TESTS
  Running test-patch task DISTRO


Testing JIRA OOZIE-2814

Cleaning local git workspace



+1 PATCH_APPLIES
+1 CLEAN
-1 RAW_PATCH_ANALYSIS
+1 the patch does not introduce any @author tags
+1 the patch does not introduce any tabs
+1 the patch does not introduce any trailing spaces
-1 the patch contains 2 line(s) longer than 132 characters
+1 the patch adds/modifies 2 testcase(s)
+1 RAT
+1 the patch does not seem to introduce new RAT warnings
+1 JAVADOC
+1 the patch does not seem to introduce new Javadoc warnings
WARNING: the current HEAD has 100 Javadoc warning(s)
+1 COMPILE
+1 HEAD compiles
+1 patch compiles
+1 the patch does not seem to introduce new javac warnings
+1 There are no new bugs found in total.
 +1 There are no new bugs found in [client].
 +1 There are no new bugs found in [sharelib/hive].
 +1 There are no new bugs found in [sharelib/sqoop].
 +1 There are no new bugs found in [sharelib/spark].
 +1 There are no new bugs found in [sharelib/streaming].
 +1 There are no new bugs found in [sharelib/hive2].
 +1 There are no new bugs found in [sharelib/oozie].
 +1 There are no new bugs found in [sharelib/hcatalog].
 +1 There are no new bugs found in [sharelib/pig].
 +1 There are no new bugs found in [sharelib/distcp].
 +1 There are no new bugs found in [docs].
 +1 There are no new bugs found in [server].
 +1 There are no new bugs found in [core].
 +1 There are no new bugs found in [examples].
 +1 There are no new bugs found in [webapp].
 +1 There are no new bugs found in [tools].
+1 BACKWARDS_COMPATIBILITY
+1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient 
annotations
+1 the patch does not modify JPA files
+1 TESTS
Tests run: 2097
+1 DISTRO
+1 distro tarball builds with the patch 


-1 Overall result, please check the reported -1(s)

 There is at least one warning, please check

The full output of the test-patch run is available at

 https://builds.apache.org/job/PreCommit-OOZIE-Build/394/

Adding comment to JIRA
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0  
0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 
0{"self":"https://issues.apache.org/jira/rest/api/2/issue/13049362/comment/16362775","id":"16362775","author":{"self":"https://issues.apache.org/jira/rest/api/2/user?username=hadoopqa","name":"hadoopqa","key":"hadoopqa","emailAddress":"blackhole
 at hadoop dot apache dot 

[jira] [Commented] (OOZIE-2814) OYA: Update example workflows to newest schemas

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362604#comment-16362604
 ] 

Hadoop QA commented on OOZIE-2814:
--

PreCommit-OOZIE-Build started


> OYA: Update example workflows to newest schemas
> ---
>
> Key: OOZIE-2814
> URL: https://issues.apache.org/jira/browse/OOZIE-2814
> Project: Oozie
>  Issue Type: Sub-task
>Affects Versions: 5.0.0
>Reporter: Robert Kanter
>Assignee: Attila Sasvari
>Priority: Blocker
> Attachments: OOZIE-2814-00.patch, OOZIE-2814-01.patch, 
> OOZIE-2814-02.patch, OOZIE-2814-03.patch, OOZIE-2814-04.patch, 
> OOZIE-2814-05.patch
>
>
> OOZIE-2687 will add in a whole set of newer schemas that are more 
> Yarn-centric.  Most, if not all, examples currently use pretty old versions 
> of the schemas.  We should take this opportunity to update the examples to 
> the newest schemas added by OOZIE-2687, which should help demonstrate them.
> It may make sense to keep the older schemas too, to show that older workflows 
> still work.  We have some examples where we have two workflow XML files that 
> do the same thing in different ways.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Failed: OOZIE-3179 PreCommit Build #393

[Apache Jenkins Server]

Jira: https://issues.apache.org/jira/browse/OOZIE-3179
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/393/

###
## LAST 100 LINES OF THE CONSOLE 
###
[...truncated 1.70 MB...]
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [server].
[INFO] There are no new bugs found totally].
[TRACE] FindBugs diffs checked and reports created
[TRACE] Summary file size is 2252 bytes
[TRACE] Full summary file size is 1314 bytes
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar]
 removed
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5sum]
 removed
  Running test-patch task BACKWARDS_COMPATIBILITY
  Running test-patch task TESTS
  Running test-patch task DISTRO


Testing JIRA OOZIE-3179

Cleaning local git workspace



+1 PATCH_APPLIES
+1 CLEAN
+1 RAW_PATCH_ANALYSIS
+1 the patch does not introduce any @author tags
+1 the patch does not introduce any tabs
+1 the patch does not introduce any trailing spaces
+1 the patch does not introduce any line longer than 132
+1 the patch adds/modifies 1 testcase(s)
+1 RAT
+1 the patch does not seem to introduce new RAT warnings
+1 JAVADOC
+1 the patch does not seem to introduce new Javadoc warnings
WARNING: the current HEAD has 100 Javadoc warning(s)
+1 COMPILE
+1 HEAD compiles
+1 patch compiles
+1 the patch does not seem to introduce new javac warnings
+1 There are no new bugs found in total.
 +1 There are no new bugs found in [docs].
 +1 There are no new bugs found in [webapp].
 +1 There are no new bugs found in [sharelib/distcp].
 +1 There are no new bugs found in [sharelib/hive].
 +1 There are no new bugs found in [sharelib/spark].
 +1 There are no new bugs found in [sharelib/hive2].
 +1 There are no new bugs found in [sharelib/hcatalog].
 +1 There are no new bugs found in [sharelib/streaming].
 +1 There are no new bugs found in [sharelib/pig].
 +1 There are no new bugs found in [sharelib/sqoop].
 +1 There are no new bugs found in [sharelib/oozie].
 +1 There are no new bugs found in [examples].
 +1 There are no new bugs found in [client].
 +1 There are no new bugs found in [core].
 +1 There are no new bugs found in [tools].
 +1 There are no new bugs found in [server].
+1 BACKWARDS_COMPATIBILITY
+1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient 
annotations
+1 the patch does not modify JPA files
-1 TESTS
Tests run: 2093
Tests failed: 3
Tests errors: 1

The patch failed the following testcases:

testPartitionDependency(org.apache.oozie.service.TestPartitionDependencyManagerService)
testCoordActionRecoveryServiceForWaitingRegisterPartition(org.apache.oozie.service.TestRecoveryService)
testPartitionDependency(org.apache.oozie.service.TestPartitionDependencyManagerEhcache)

Tests failing with errors:
testConnectionRetry(org.apache.oozie.service.TestJMSAccessorService)

Tests failed at first run:
TestJavaActionExecutor#testCredentialsSkip
For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
+1 DISTRO
+1 distro tarball builds with the patch 


-1 Overall result, please check the reported -1(s)

 There is at least one warning, please check

The full output of the test-patch run is available at

 https://builds.apache.org/job/PreCommit-OOZIE-Build/393/

Adding comment to JIRA
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 
0100  36230 0  100  3623  0   4432 --:--:-- --:--:-- --:--:--  
4429{"self":"https://issues.apache.org/jira/rest/api/2/issue/13137525/comment/16362596","id":"16362596","author":{"self":"https://issues.apache.org/jira/rest/api/2/user?username=hadoopqa","name":"hadoopqa","key":"hadoopqa","emailAddress":"blackhole
 at hadoop dot apache dot 
org","avatarUrls":{"48x48":"https://issues.apache.org/jira/secure/useravatar?ownerId=hadoopqa=10393","24x24":"https://issues.apache.org/jira/secure/useravatar?size=small=hadoopqa=10393","16x16":"https://issues.apache.org/jira/secure/useravatar?size=xsmall=hadoopqa=10393","32x32":"https://issues.apache.org/jira/secure/useravatar?size=medium=hadoopqa=10393"},"displayName":"Hadoop
 QA","active":true,"timeZone":"Etc/UTC"},"body":"\nTesting JIRA 
OOZIE-3179\n\nCleaning local git 
workspace\n\n\n\n{color:green}+1 
PATCH_APPLIES{color}\n{color:green}+1 CLEAN{color}\n{color:green}+1 
RAW_PATCH_ANALYSIS{color}\n.{color:green}+1{color} the patch does not 
introduce 

[jira] [Commented] (OOZIE-3179) Adding a configurable config-default.xml location to a workflow

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362596#comment-16362596
 ] 

Hadoop QA commented on OOZIE-3179:
--


Testing JIRA OOZIE-3179

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:green}+1{color} the patch adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
.{color:red}WARNING{color}: the current HEAD has 100 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [server].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 2093
.Tests failed: 3
.Tests errors: 1

.The patch failed the following testcases:

testPartitionDependency(org.apache.oozie.service.TestPartitionDependencyManagerService)
testCoordActionRecoveryServiceForWaitingRegisterPartition(org.apache.oozie.service.TestRecoveryService)
testPartitionDependency(org.apache.oozie.service.TestPartitionDependencyManagerEhcache)

.Tests failing with errors:
testConnectionRetry(org.apache.oozie.service.TestJMSAccessorService)

.{color:orange}Tests failed at first run:{color}
TestJavaActionExecutor#testCredentialsSkip
.For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}

{color:red}. There is at least one warning, please check{color}

The full output of the test-patch run is available at

. https://builds.apache.org/job/PreCommit-OOZIE-Build/393/



> Adding a configurable config-default.xml location to a workflow
> ---
>
> Key: OOZIE-3179
> URL: https://issues.apache.org/jira/browse/OOZIE-3179
> Project: Oozie
>  Issue Type: Improvement
>Reporter: Jason Phelps
>Assignee: Jason Phelps
>Priority: Major
> Attachments: OOZIE-3179-1.patch, OOZIE-3179-2.patch
>
>
> Recently a customer requested a way to include several job.properties 
> variables to every workflow in their cluster.
> Normally the config-default.xml file would be the best way to include a set 
> of variables, but it does not scale very well if that file needs to be 
> updated, as it would require it to be updated in every workflow application 
> path.
> What I am requesting is another job.properties variable that can point to a 
> config-default.xml file, or a comma separated list of config-default.xml 
> files that will all be applied to the workflow.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (OOZIE-3134) Potential inconsistency between the in-memory SLA map and the Oozie database

[Julia Kinga Marton (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362484#comment-16362484
 ] 

Julia Kinga Marton commented on OOZIE-3134:
---

Junit results with dist-test: 
[http://dist-test.cloudera.org/job?job_id=oozie2.martonjuliakinga.1518534651.31839#|http://dist-test.cloudera.org/job?job_id=oozie2.martonjuliakinga.1518534651.31839]

 

> Potential inconsistency between the in-memory SLA map and the Oozie database
> 
>
> Key: OOZIE-3134
> URL: https://issues.apache.org/jira/browse/OOZIE-3134
> Project: Oozie
>  Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.0.0
>
> Attachments: OOZIE-3134-001.patch, OOZIE-3134-002.patch, 
> OOZIE-3134-003.patch
>
>
> Upon {{SLACalculatorMemory.addRegistration}}, Oozie puts an entry into an 
> in-memory concurrent hashmap ("slaMap"), and later [persists 
> it|https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L380]
>  in the Oozie database. 
> However, if there is a failure during the database operation, a 
> {{JPAExecutorException}} is thrown, and the entry [is not removed from the 
> SLA map| 
> https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L393].
> It may introduce inconsistency between the Oozie database and the SLA map.
> To prevent this, a rollback mechanism (with proper logging) should be 
> implemented.  It would also make sense to do more sanity/consistency check in 
> the Oozie server.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (OOZIE-3134) Potential inconsistency between the in-memory SLA map and the Oozie database

[Julia Kinga Marton (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-3134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Julia Kinga Marton updated OOZIE-3134:
--
Attachment: OOZIE-3134-003.patch

> Potential inconsistency between the in-memory SLA map and the Oozie database
> 
>
> Key: OOZIE-3134
> URL: https://issues.apache.org/jira/browse/OOZIE-3134
> Project: Oozie
>  Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.0.0
>
> Attachments: OOZIE-3134-001.patch, OOZIE-3134-002.patch, 
> OOZIE-3134-003.patch
>
>
> Upon {{SLACalculatorMemory.addRegistration}}, Oozie puts an entry into an 
> in-memory concurrent hashmap ("slaMap"), and later [persists 
> it|https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L380]
>  in the Oozie database. 
> However, if there is a failure during the database operation, a 
> {{JPAExecutorException}} is thrown, and the entry [is not removed from the 
> SLA map| 
> https://github.com/apache/oozie/blob/50f4b5984832941f1341586e43fd832c293b3275/core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java#L393].
> It may introduce inconsistency between the Oozie database and the SLA map.
> To prevent this, a rollback mechanism (with proper logging) should be 
> implemented.  It would also make sense to do more sanity/consistency check in 
> the Oozie server.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 65481: OOZIE-3134 - Potential inconsistency between the in-memory SLA map and the Oozie database

[Kinga Marton via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65481/
---

(Updated Feb. 13, 2018, 3:19 p.m.)


Review request for oozie, András Piros and Attila Sasvari.


Repository: oozie-git


Description
---

Upon SLACalculatorMemory.addRegistration, Oozie puts an entry into an in-memory 
concurrent hashmap ("slaMap"), and later persists it in the Oozie database.
However, if there is a failure during the database operation, a 
JPAExecutorException is thrown, and the entry is not removed from the SLA map.
It may introduce inconsistency between the Oozie database and the SLA map.
To prevent this, a rollback mechanism (with proper logging) should be 
implemented. It would also make sense to do more sanity/consistency check in 
the Oozie server.


Diffs (updated)
-

  core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java ef019e73 
  
core/src/main/java/org/apache/oozie/util/db/AlwaysFailingHSQLDriverMapper.java 
PRE-CREATION 
  core/src/main/java/org/apache/oozie/util/db/FailingConnectionWrapper.java 
0e310253 
  core/src/main/java/org/apache/oozie/util/db/FailingDBHelperForTest.java 
PRE-CREATION 
  core/src/main/java/org/apache/oozie/util/db/FailingHSQLDBDriverWrapper.java 
fe9f08b1 
  core/src/main/java/org/apache/oozie/util/db/FailingMySQLDriverWrapper.java 
f0e2b181 
  core/src/test/java/org/apache/oozie/sla/TestSLACalculatorMemory.java ee906f45 


Diff: https://reviews.apache.org/r/65481/diff/3/

Changes: https://reviews.apache.org/r/65481/diff/2-3/


Testing
---


Thanks,

Kinga Marton

Review Request 65636: OOZIE-3179 - Adding a configurable config-default.xml location to a workflow

[Jason Phelps via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65636/
---

Review request for oozie.


Repository: oozie-git


Description
---

Added another client property: `oozie.default.configuration.path` which allows 
for a comma seperated list of HDFS XML property files to be applied to the job. 
This is useful if multiple workflows have shared properties (i.e. database 
names, table names for Sqoop or Hive, spark arguments, etc). This is similar to 
the `oozie.libpath` property which allows user's to define common paths 
workflows to load jars.


Diffs
-

  client/src/main/java/org/apache/oozie/client/OozieClient.java e581e50 
  core/src/main/java/org/apache/oozie/command/wf/SubmitXCommand.java a4d3e2f 
  core/src/test/java/org/apache/oozie/command/wf/TestSubmitXCommand.java 
e71a662 


Diff: https://reviews.apache.org/r/65636/diff/1/


Testing
---


Thanks,

Jason Phelps

[jira] [Commented] (OOZIE-3179) Adding a configurable config-default.xml location to a workflow

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362444#comment-16362444
 ] 

Hadoop QA commented on OOZIE-3179:
--

PreCommit-OOZIE-Build started


> Adding a configurable config-default.xml location to a workflow
> ---
>
> Key: OOZIE-3179
> URL: https://issues.apache.org/jira/browse/OOZIE-3179
> Project: Oozie
>  Issue Type: Improvement
>Reporter: Jason Phelps
>Assignee: Jason Phelps
>Priority: Major
> Attachments: OOZIE-3179-1.patch, OOZIE-3179-2.patch
>
>
> Recently a customer requested a way to include several job.properties 
> variables to every workflow in their cluster.
> Normally the config-default.xml file would be the best way to include a set 
> of variables, but it does not scale very well if that file needs to be 
> updated, as it would require it to be updated in every workflow application 
> path.
> What I am requesting is another job.properties variable that can point to a 
> config-default.xml file, or a comma separated list of config-default.xml 
> files that will all be applied to the workflow.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (OOZIE-3179) Adding a configurable config-default.xml location to a workflow

[Jason Phelps (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362440#comment-16362440
 ] 

Jason Phelps commented on OOZIE-3179:
-

Attached another patch to fix the "longer than 132 characters" warning. 

> Adding a configurable config-default.xml location to a workflow
> ---
>
> Key: OOZIE-3179
> URL: https://issues.apache.org/jira/browse/OOZIE-3179
> Project: Oozie
>  Issue Type: Improvement
>Reporter: Jason Phelps
>Assignee: Jason Phelps
>Priority: Major
> Attachments: OOZIE-3179-1.patch, OOZIE-3179-2.patch
>
>
> Recently a customer requested a way to include several job.properties 
> variables to every workflow in their cluster.
> Normally the config-default.xml file would be the best way to include a set 
> of variables, but it does not scale very well if that file needs to be 
> updated, as it would require it to be updated in every workflow application 
> path.
> What I am requesting is another job.properties variable that can point to a 
> config-default.xml file, or a comma separated list of config-default.xml 
> files that will all be applied to the workflow.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (OOZIE-3179) Adding a configurable config-default.xml location to a workflow

[Jason Phelps (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Phelps updated OOZIE-3179:

Attachment: OOZIE-3179-2.patch

> Adding a configurable config-default.xml location to a workflow
> ---
>
> Key: OOZIE-3179
> URL: https://issues.apache.org/jira/browse/OOZIE-3179
> Project: Oozie
>  Issue Type: Improvement
>Reporter: Jason Phelps
>Assignee: Jason Phelps
>Priority: Major
> Attachments: OOZIE-3179-1.patch, OOZIE-3179-2.patch
>
>
> Recently a customer requested a way to include several job.properties 
> variables to every workflow in their cluster.
> Normally the config-default.xml file would be the best way to include a set 
> of variables, but it does not scale very well if that file needs to be 
> updated, as it would require it to be updated in every workflow application 
> path.
> What I am requesting is another job.properties variable that can point to a 
> config-default.xml file, or a comma separated list of config-default.xml 
> files that will all be applied to the workflow.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (OOZIE-2814) OYA: Update example workflows to newest schemas

[Attila Sasvari (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Sasvari updated OOZIE-2814:
--
Attachment: OOZIE-2814-05.patch

> OYA: Update example workflows to newest schemas
> ---
>
> Key: OOZIE-2814
> URL: https://issues.apache.org/jira/browse/OOZIE-2814
> Project: Oozie
>  Issue Type: Sub-task
>Affects Versions: 5.0.0
>Reporter: Robert Kanter
>Assignee: Attila Sasvari
>Priority: Blocker
> Attachments: OOZIE-2814-00.patch, OOZIE-2814-01.patch, 
> OOZIE-2814-02.patch, OOZIE-2814-03.patch, OOZIE-2814-04.patch, 
> OOZIE-2814-05.patch
>
>
> OOZIE-2687 will add in a whole set of newer schemas that are more 
> Yarn-centric.  Most, if not all, examples currently use pretty old versions 
> of the schemas.  We should take this opportunity to update the examples to 
> the newest schemas added by OOZIE-2687, which should help demonstrate them.
> It may make sense to keep the older schemas too, to show that older workflows 
> still work.  We have some examples where we have two workflow XML files that 
> do the same thing in different ways.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 65481: OOZIE-3134 - Potential inconsistency between the in-memory SLA map and the Oozie database

[Kinga Marton via Review Board]

> On Feb. 6, 2018, 4:15 p.m., András Piros wrote:
> > core/src/main/java/org/apache/oozie/util/db/ThreadVariables.java
> > Lines 23-37 (patched)
> > 
> >
> > I don't see calling `setDbPredicate()` or `unsetDbPredicate()` from 
> > anywhere else than test code.
> > 
> > When using `ThreadLocal` we have to be very careful to unset after 
> > every use to avoid resource / memory leaks.
> > 
> > Generally, I think it would be a better idea to inject a new DB 
> > predicate by other means - I would only go for `ThreadLocal` usage when 
> > most of the time I need different `dbPredicate` to every `Thread` - which 
> > is not the case here.
> > 
> > Let's think a bit more on `Predicate` injecting scenarios apart from 
> > using `ThreadLocal`.

My next ideea is what Peti suggested: a helper class with a static field. Is 
not a very elegant solution. Do you have some better ideeas?


- Kinga


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65481/#review196897
---


On Feb. 6, 2018, 2:43 p.m., Kinga Marton wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65481/
> ---
> 
> (Updated Feb. 6, 2018, 2:43 p.m.)
> 
> 
> Review request for oozie, András Piros and Attila Sasvari.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> Upon SLACalculatorMemory.addRegistration, Oozie puts an entry into an 
> in-memory concurrent hashmap ("slaMap"), and later persists it in the Oozie 
> database.
> However, if there is a failure during the database operation, a 
> JPAExecutorException is thrown, and the entry is not removed from the SLA map.
> It may introduce inconsistency between the Oozie database and the SLA map.
> To prevent this, a rollback mechanism (with proper logging) should be 
> implemented. It would also make sense to do more sanity/consistency check in 
> the Oozie server.
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/sla/SLACalculatorMemory.java ef019e73 
>   
> core/src/main/java/org/apache/oozie/util/db/AlwaysFailingHSQLDriverMapper.java
>  PRE-CREATION 
>   core/src/main/java/org/apache/oozie/util/db/FailingConnectionWrapper.java 
> 0e310253 
>   core/src/main/java/org/apache/oozie/util/db/FailingHSQLDBDriverWrapper.java 
> fe9f08b1 
>   core/src/main/java/org/apache/oozie/util/db/FailingMySQLDriverWrapper.java 
> f0e2b181 
>   core/src/main/java/org/apache/oozie/util/db/ThreadVariables.java 
> PRE-CREATION 
>   core/src/test/java/org/apache/oozie/sla/TestSLACalculatorMemory.java 
> ee906f45 
> 
> 
> Diff: https://reviews.apache.org/r/65481/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Kinga Marton
> 
>

[jira] [Commented] (OOZIE-2814) OYA: Update example workflows to newest schemas

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362419#comment-16362419
 ] 

Hadoop QA commented on OOZIE-2814:
--

PreCommit-OOZIE-Build started


> OYA: Update example workflows to newest schemas
> ---
>
> Key: OOZIE-2814
> URL: https://issues.apache.org/jira/browse/OOZIE-2814
> Project: Oozie
>  Issue Type: Sub-task
>Affects Versions: 5.0.0
>Reporter: Robert Kanter
>Assignee: Attila Sasvari
>Priority: Blocker
> Attachments: OOZIE-2814-00.patch, OOZIE-2814-01.patch, 
> OOZIE-2814-02.patch, OOZIE-2814-03.patch, OOZIE-2814-04.patch
>
>
> OOZIE-2687 will add in a whole set of newer schemas that are more 
> Yarn-centric.  Most, if not all, examples currently use pretty old versions 
> of the schemas.  We should take this opportunity to update the examples to 
> the newest schemas added by OOZIE-2687, which should help demonstrate them.
> It may make sense to keep the older schemas too, to show that older workflows 
> still work.  We have some examples where we have two workflow XML files that 
> do the same thing in different ways.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 65606: OOZIE-2814 OYA: Update example workflows to newest schemas

[Attila Sasvari via Review Board]

> On Feb. 12, 2018, 8:05 p.m., András Piros wrote:
> > core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
> > Lines 266-271 (patched)
> > 
> >
> > What happens if both `resource-manager` and `job-tracker` are filled? 
> > I'd either separate `resourceManager` and `jobTracker` into two variables / 
> > fields, or have some more clean, generic labelling for the same attribute.

Cannot happen. Look at the new XML schemas (it is a choice, XML validation 
fails if both elements are present).


> On Feb. 12, 2018, 8:05 p.m., András Piros wrote:
> > core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java
> > Line 510 (original), 510 (patched)
> > 
> >
> > Isn't it rather an XML validation error? The schema itself is quite 
> > correct.

Please look at the existing code, "E0701: XML schema error, No job-tracker 
defined" is what is orignally defined. I just extended the error message.


> On Feb. 12, 2018, 8:05 p.m., András Piros wrote:
> > core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java
> > Lines 1680 (patched)
> > 
> >
> > Would give a catchier name and remove comment.
> > 
> > What about more test cases:
> > 
> > * both `resource-manager` and `job-tracker` present at the same level
> > * both `resource-manager` and `job-tracker` present at different levels
> > * none of these are present
> 
> Peter Cseh wrote:
> I think it's better to keep both RM and JT as a single variable in the 
> code as they have the same meaning. JT is only there for compatiblity 
> reasons, but it should overwrite RM values of lower levels.

- I believe XML validation is enough to protect against both resource-manager 
and job-tracker present at the same level.
- There are existing tests that check if no job-tracker and resource-manager 
present. It is also validated that default job-tracker is used from oozie-site.
- It is a good idea to test if action level resourcemanager address is prefered 
over the one defined in the global section. Can be added to 
TestJavaActionExecutor.


- Attila


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65606/#review197301
---


On Feb. 12, 2018, 4:19 p.m., Attila Sasvari wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65606/
> ---
> 
> (Updated Feb. 12, 2018, 4:19 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> OOZIE-2687 will add in a whole set of newer schemas that are more 
> Yarn-centric. Most, if not all, examples currently use pretty old versions of 
> the schemas. We should take this opportunity to update the examples to the 
> newest schemas added by OOZIE-2687, which should help demonstrate them.
> 
> It may make sense to keep the older schemas too, to show that older workflows 
> still work. We have some examples where we have two workflow XML files that 
> do the same thing in different ways.
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
> bc0f4050bb3788988d3a5352a15220fbc4fa219d 
>   
> core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 
> c236dafa5965403c33e1a400509e7c374fe2637c 
>   
> core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 
> 1cc569d8d7a0e61905ad6c4e940cefb00da1343d 
>   
> core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java
>  2cdbde4d2fc86a07b35beadc56012c438baf8d54 
>   examples/src/main/apps/aggregator/coordinator-with-offset.xml 
> 4c351677f4afc6e85f807106d426bb2f922a28b1 
>   examples/src/main/apps/aggregator/coordinator.xml 
> 948edc1adbe09dfb6f7bffb5eebc82f3670658cc 
>   examples/src/main/apps/aggregator/job-with-offset.properties 
> 9a28ae13e50c40f70db4b0f583a63eb42f97388d 
>   examples/src/main/apps/aggregator/job.properties 
> aaf4edf7b9d030b5044610698b27a564b008f77b 
>   examples/src/main/apps/aggregator/workflow.xml 
> 4a801090e31acba23ea49da1c59a8ce4b951c654 
>   examples/src/main/apps/bundle/job.properties 
> c06edeb3f42e2eba3367b9486982dabb4caf6436 
>   examples/src/main/apps/coord-input-logic/job.properties 
> 2cd137b94999c7e1f68bb54928f1c3ce2838d59d 
>   examples/src/main/apps/coord-input-logic/workflow.xml 
> 80d31457eff6e78710ad1fb4d4f1b372c0d1e43c 
>   examples/src/main/apps/cron-schedule/coordinator.xml 
> 48a7de76245e02397b9736a95fa0132504067f9a 
>   

[jira] [Updated] (OOZIE-2814) OYA: Update example workflows to newest schemas

[Attila Sasvari (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Sasvari updated OOZIE-2814:
--
Attachment: OOZIE-2814-04.patch

> OYA: Update example workflows to newest schemas
> ---
>
> Key: OOZIE-2814
> URL: https://issues.apache.org/jira/browse/OOZIE-2814
> Project: Oozie
>  Issue Type: Sub-task
>Affects Versions: 5.0.0
>Reporter: Robert Kanter
>Assignee: Attila Sasvari
>Priority: Blocker
> Attachments: OOZIE-2814-00.patch, OOZIE-2814-01.patch, 
> OOZIE-2814-02.patch, OOZIE-2814-03.patch, OOZIE-2814-04.patch
>
>
> OOZIE-2687 will add in a whole set of newer schemas that are more 
> Yarn-centric.  Most, if not all, examples currently use pretty old versions 
> of the schemas.  We should take this opportunity to update the examples to 
> the newest schemas added by OOZIE-2687, which should help demonstrate them.
> It may make sense to keep the older schemas too, to show that older workflows 
> still work.  We have some examples where we have two workflow XML files that 
> do the same thing in different ways.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (OOZIE-3179) Adding a configurable config-default.xml location to a workflow

[Andras Piros (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362382#comment-16362382
 ] 

Andras Piros commented on OOZIE-3179:
-

Thanks for the contribution [~jphelps]! Can you please add a [*ReviewBoard 
review request*|https://reviews.apache.org/dashboard/] as well? Thanks!

> Adding a configurable config-default.xml location to a workflow
> ---
>
> Key: OOZIE-3179
> URL: https://issues.apache.org/jira/browse/OOZIE-3179
> Project: Oozie
>  Issue Type: Improvement
>Reporter: Jason Phelps
>Assignee: Jason Phelps
>Priority: Major
> Attachments: OOZIE-3179-1.patch
>
>
> Recently a customer requested a way to include several job.properties 
> variables to every workflow in their cluster.
> Normally the config-default.xml file would be the best way to include a set 
> of variables, but it does not scale very well if that file needs to be 
> updated, as it would require it to be updated in every workflow application 
> path.
> What I am requesting is another job.properties variable that can point to a 
> config-default.xml file, or a comma separated list of config-default.xml 
> files that will all be applied to the workflow.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (OOZIE-3179) Adding a configurable config-default.xml location to a workflow

[Andras Piros (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Piros reassigned OOZIE-3179:
---

Assignee: Jason Phelps

> Adding a configurable config-default.xml location to a workflow
> ---
>
> Key: OOZIE-3179
> URL: https://issues.apache.org/jira/browse/OOZIE-3179
> Project: Oozie
>  Issue Type: Improvement
>Reporter: Jason Phelps
>Assignee: Jason Phelps
>Priority: Major
> Attachments: OOZIE-3179-1.patch
>
>
> Recently a customer requested a way to include several job.properties 
> variables to every workflow in their cluster.
> Normally the config-default.xml file would be the best way to include a set 
> of variables, but it does not scale very well if that file needs to be 
> updated, as it would require it to be updated in every workflow application 
> path.
> What I am requesting is another job.properties variable that can point to a 
> config-default.xml file, or a comma separated list of config-default.xml 
> files that will all be applied to the workflow.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (OOZIE-3182) Oozie components fail with checkstyle errors

[Andras Piros (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362378#comment-16362378
 ] 

Andras Piros commented on OOZIE-3182:
-

Thanks for the contribution [~alishap]! Can we have modifications that:
# preserve [*the {{.properties}} file 
format*|https://en.wikipedia.org/wiki/.properties] regarding comments (lines 
beginning either with {{#}} or {{!}})
# modify the RAT license format [*only in compatible 
ways*|http://creadur.apache.org/rat/apache-rat-plugin/examples/custom-license.html]

Can you please also add checkstyle output before and after the fix?

> Oozie components fail with checkstyle errors
> 
>
> Key: OOZIE-3182
> URL: https://issues.apache.org/jira/browse/OOZIE-3182
> Project: Oozie
>  Issue Type: Bug
>  Components: client, examples
>Affects Versions: 5.0.0b1
> Environment: uname -a
> Linux pts00607-vm1 4.4.0-31-generic #50-Ubuntu SMP Wed Jul 13 00:05:18 UTC 
> 2016 ppc64le ppc64le ppc64le GNU/Linux
>Reporter: Alisha Prabhu
>Assignee: Alisha Prabhu
>Priority: Major
> Attachments: OOZIE-3182-001.patch
>
>
> There are multiple checkstyle violations in the oozie-client, oozie-examples, 
> oozie-mini, oozie-sharelib-oozie, oozie-sharelib module.
> Maven command used is mvn clean install -DskipTests



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (OOZIE-3182) Oozie components fail with checkstyle errors

[Andras Piros (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-3182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Piros reassigned OOZIE-3182:
---

Assignee: Alisha Prabhu

> Oozie components fail with checkstyle errors
> 
>
> Key: OOZIE-3182
> URL: https://issues.apache.org/jira/browse/OOZIE-3182
> Project: Oozie
>  Issue Type: Bug
>  Components: client, examples
>Affects Versions: 5.0.0b1
> Environment: uname -a
> Linux pts00607-vm1 4.4.0-31-generic #50-Ubuntu SMP Wed Jul 13 00:05:18 UTC 
> 2016 ppc64le ppc64le ppc64le GNU/Linux
>Reporter: Alisha Prabhu
>Assignee: Alisha Prabhu
>Priority: Major
> Attachments: OOZIE-3182-001.patch
>
>
> There are multiple checkstyle violations in the oozie-client, oozie-examples, 
> oozie-mini, oozie-sharelib-oozie, oozie-sharelib module.
> Maven command used is mvn clean install -DskipTests



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[VOTE][RESULT] Oozie 4.3.1(candidate 4)

[Gézapeti]

Hi,

Oozie 4.3.1 RC4 has passed the vote.

The vote tally is:
+1 --> 3 votes from Oozie PMC members (binding), 1 vote from
Contributors (non-binding)
-1 --> 0
0 --> 0


On Tue, Feb 6, 2018 at 1:42 PM, Gézapeti Cseh  wrote:
> Hi,
>
> I have created a build for Oozie 4.3.1, candidate 4.
>
> It includes all the changes in rc3 plus OOZIE-2888 to fix sharelib
> creation with Hadoop 2.7.5. and possibly other newer versions.
>
> Keys to verify the signature of the release artifact are available at
>   http://www.apache.org/dist/oozie/KEYS
>
> Please download, test, and try it out:
>
> http://people.apache.org/~gezapeti/oozie-4.3.1-rc4/
>
> The release, md5 signature, gpg signature, and rat report can be found at
> the above address.
>
> Vote closes on 02/12/2018 11:59 PM CET.

[jira] [Commented] (OOZIE-3182) Oozie components fail with checkstyle errors

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362214#comment-16362214
 ] 

Hadoop QA commented on OOZIE-3182:
--


Testing JIRA OOZIE-3182

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
.{color:red}WARNING{color}: the current HEAD has 100 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:green}+1{color} There are no new bugs found in [tools].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 2092
.{color:orange}Tests failed at first run:{color}
TestJavaActionExecutor#testCredentialsSkip
.For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}

{color:red}. There is at least one warning, please check{color}

The full output of the test-patch run is available at

. https://builds.apache.org/job/PreCommit-OOZIE-Build/391/



> Oozie components fail with checkstyle errors
> 
>
> Key: OOZIE-3182
> URL: https://issues.apache.org/jira/browse/OOZIE-3182
> Project: Oozie
>  Issue Type: Bug
>  Components: client, examples
>Affects Versions: 5.0.0b1
> Environment: uname -a
> Linux pts00607-vm1 4.4.0-31-generic #50-Ubuntu SMP Wed Jul 13 00:05:18 UTC 
> 2016 ppc64le ppc64le ppc64le GNU/Linux
>Reporter: Alisha Prabhu
>Priority: Major
> Attachments: OOZIE-3182-001.patch
>
>
> There are multiple checkstyle violations in the oozie-client, oozie-examples, 
> oozie-mini, oozie-sharelib-oozie, oozie-sharelib module.
> Maven command used is mvn clean install -DskipTests



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Failed: OOZIE-3182 PreCommit Build #391

[Apache Jenkins Server]

Jira: https://issues.apache.org/jira/browse/OOZIE-3182
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/391/

###
## LAST 100 LINES OF THE CONSOLE 
###
[...truncated 1.72 MB...]
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/spark].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/hcatalog].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/sqoop].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/hive2].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/oozie].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [webapp].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [tools].
[INFO] There are no new bugs found totally].
[TRACE] FindBugs diffs checked and reports created
[TRACE] Summary file size is 2253 bytes
[TRACE] Full summary file size is 1314 bytes
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar]
 removed
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5sum]
 removed
  Running test-patch task BACKWARDS_COMPATIBILITY
  Running test-patch task TESTS
  Running test-patch task DISTRO


Testing JIRA OOZIE-3182

Cleaning local git workspace



+1 PATCH_APPLIES
+1 CLEAN
-1 RAW_PATCH_ANALYSIS
+1 the patch does not introduce any @author tags
+1 the patch does not introduce any tabs
+1 the patch does not introduce any trailing spaces
+1 the patch does not introduce any line longer than 132
-1 the patch does not add/modify any testcase
+1 RAT
+1 the patch does not seem to introduce new RAT warnings
+1 JAVADOC
+1 the patch does not seem to introduce new Javadoc warnings
WARNING: the current HEAD has 100 Javadoc warning(s)
+1 COMPILE
+1 HEAD compiles
+1 patch compiles
+1 the patch does not seem to introduce new javac warnings
+1 There are no new bugs found in total.
 +1 There are no new bugs found in [core].
 +1 There are no new bugs found in [server].
 +1 There are no new bugs found in [client].
 +1 There are no new bugs found in [docs].
 +1 There are no new bugs found in [examples].
 +1 There are no new bugs found in [sharelib/hive].
 +1 There are no new bugs found in [sharelib/pig].
 +1 There are no new bugs found in [sharelib/distcp].
 +1 There are no new bugs found in [sharelib/streaming].
 +1 There are no new bugs found in [sharelib/spark].
 +1 There are no new bugs found in [sharelib/hcatalog].
 +1 There are no new bugs found in [sharelib/sqoop].
 +1 There are no new bugs found in [sharelib/hive2].
 +1 There are no new bugs found in [sharelib/oozie].
 +1 There are no new bugs found in [webapp].
 +1 There are no new bugs found in [tools].
+1 BACKWARDS_COMPATIBILITY
+1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient 
annotations
+1 the patch does not modify JPA files
+1 TESTS
Tests run: 2092
Tests failed at first run:
TestJavaActionExecutor#testCredentialsSkip
For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
+1 DISTRO
+1 distro tarball builds with the patch 


-1 Overall result, please check the reported -1(s)

 There is at least one warning, please check

The full output of the test-patch run is available at

 https://builds.apache.org/job/PreCommit-OOZIE-Build/391/

Adding comment to JIRA
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0  
0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 
0{"self":"https://issues.apache.org/jira/rest/api/2/issue/13138114/comment/16362214","id":"16362214","author":{"self":"https://issues.apache.org/jira/rest/api/2/user?username=hadoopqa","name":"hadoopqa","key":"hadoopqa","emailAddress":"blackhole
 at hadoop dot apache dot 

[jira] [Commented] (OOZIE-3182) Oozie components fail with checkstyle errors

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16362127#comment-16362127
 ] 

Hadoop QA commented on OOZIE-3182:
--

PreCommit-OOZIE-Build started


> Oozie components fail with checkstyle errors
> 
>
> Key: OOZIE-3182
> URL: https://issues.apache.org/jira/browse/OOZIE-3182
> Project: Oozie
>  Issue Type: Bug
>  Components: client, examples
>Affects Versions: 5.0.0b1
> Environment: uname -a
> Linux pts00607-vm1 4.4.0-31-generic #50-Ubuntu SMP Wed Jul 13 00:05:18 UTC 
> 2016 ppc64le ppc64le ppc64le GNU/Linux
>Reporter: Alisha Prabhu
>Priority: Major
> Attachments: OOZIE-3182-001.patch
>
>
> There are multiple checkstyle violations in the oozie-client, oozie-examples, 
> oozie-mini, oozie-sharelib-oozie, oozie-sharelib module.
> Maven command used is mvn clean install -DskipTests



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (OOZIE-3182) Oozie components fail with checkstyle errors

[Alisha Prabhu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-3182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alisha Prabhu updated OOZIE-3182:
-
Attachment: oozie.patch

> Oozie components fail with checkstyle errors
> 
>
> Key: OOZIE-3182
> URL: https://issues.apache.org/jira/browse/OOZIE-3182
> Project: Oozie
>  Issue Type: Bug
>  Components: client, examples
>Affects Versions: 5.0.0b1
> Environment: uname -a
> Linux pts00607-vm1 4.4.0-31-generic #50-Ubuntu SMP Wed Jul 13 00:05:18 UTC 
> 2016 ppc64le ppc64le ppc64le GNU/Linux
>Reporter: Alisha Prabhu
>Priority: Major
>
> There are multiple checkstyle violations in the oozie-client, oozie-examples, 
> oozie-mini, oozie-sharelib-oozie, oozie-sharelib module.
> Maven command used is mvn clean install -DskipTests



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (OOZIE-3182) Oozie components fail with checkstyle errors

[Alisha Prabhu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-3182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alisha Prabhu updated OOZIE-3182:
-
Attachment: (was: oozie.patch)

> Oozie components fail with checkstyle errors
> 
>
> Key: OOZIE-3182
> URL: https://issues.apache.org/jira/browse/OOZIE-3182
> Project: Oozie
>  Issue Type: Bug
>  Components: client, examples
>Affects Versions: 5.0.0b1
> Environment: uname -a
> Linux pts00607-vm1 4.4.0-31-generic #50-Ubuntu SMP Wed Jul 13 00:05:18 UTC 
> 2016 ppc64le ppc64le ppc64le GNU/Linux
>Reporter: Alisha Prabhu
>Priority: Major
>
> There are multiple checkstyle violations in the oozie-client, oozie-examples, 
> oozie-mini, oozie-sharelib-oozie, oozie-sharelib module.
> Maven command used is mvn clean install -DskipTests



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (OOZIE-3182) Oozie components fail with checkstyle errors

[Alisha Prabhu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-3182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alisha Prabhu updated OOZIE-3182:
-
Summary: Oozie components fail with checkstyle errors  (was: Oozie 
components fails with checkstyle errors)

> Oozie components fail with checkstyle errors
> 
>
> Key: OOZIE-3182
> URL: https://issues.apache.org/jira/browse/OOZIE-3182
> Project: Oozie
>  Issue Type: Bug
>  Components: client, examples
>Affects Versions: 5.0.0b1
> Environment: uname -a
> Linux pts00607-vm1 4.4.0-31-generic #50-Ubuntu SMP Wed Jul 13 00:05:18 UTC 
> 2016 ppc64le ppc64le ppc64le GNU/Linux
>Reporter: Alisha Prabhu
>Priority: Major
>
> There are multiple checkstyle violations in the oozie-client, oozie-examples, 
> oozie-mini, oozie-sharelib-oozie, oozie-sharelib module.
> Maven command used is mvn clean install -DskipTests



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (OOZIE-3182) Oozie components fails with checkstyle errors

[Alisha Prabhu (JIRA)]

Alisha Prabhu created OOZIE-3182:


 Summary: Oozie components fails with checkstyle errors
 Key: OOZIE-3182
 URL: https://issues.apache.org/jira/browse/OOZIE-3182
 Project: Oozie
  Issue Type: Bug
  Components: client, examples
Affects Versions: 5.0.0b1
 Environment: uname -a
Linux pts00607-vm1 4.4.0-31-generic #50-Ubuntu SMP Wed Jul 13 00:05:18 UTC 2016 
ppc64le ppc64le ppc64le GNU/Linux
Reporter: Alisha Prabhu


There are multiple checkstyle violations in the oozie-client, oozie-examples, 
oozie-mini, oozie-sharelib-oozie, oozie-sharelib module.

Maven command used is mvn clean install -DskipTests



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 65606: OOZIE-2814 OYA: Update example workflows to newest schemas

[Peter Cseh via Review Board]

> On Feb. 12, 2018, 8:05 p.m., András Piros wrote:
> > core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java
> > Lines 1680 (patched)
> > 
> >
> > Would give a catchier name and remove comment.
> > 
> > What about more test cases:
> > 
> > * both `resource-manager` and `job-tracker` present at the same level
> > * both `resource-manager` and `job-tracker` present at different levels
> > * none of these are present

I think it's better to keep both RM and JT as a single variable in the code as 
they have the same meaning. JT is only there for compatiblity reasons, but it 
should overwrite RM values of lower levels.


- Peter


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65606/#review197301
---


On Feb. 12, 2018, 4:19 p.m., Attila Sasvari wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65606/
> ---
> 
> (Updated Feb. 12, 2018, 4:19 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> OOZIE-2687 will add in a whole set of newer schemas that are more 
> Yarn-centric. Most, if not all, examples currently use pretty old versions of 
> the schemas. We should take this opportunity to update the examples to the 
> newest schemas added by OOZIE-2687, which should help demonstrate them.
> 
> It may make sense to keep the older schemas too, to show that older workflows 
> still work. We have some examples where we have two workflow XML files that 
> do the same thing in different ways.
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
> bc0f4050bb3788988d3a5352a15220fbc4fa219d 
>   
> core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 
> c236dafa5965403c33e1a400509e7c374fe2637c 
>   
> core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 
> 1cc569d8d7a0e61905ad6c4e940cefb00da1343d 
>   
> core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java
>  2cdbde4d2fc86a07b35beadc56012c438baf8d54 
>   examples/src/main/apps/aggregator/coordinator-with-offset.xml 
> 4c351677f4afc6e85f807106d426bb2f922a28b1 
>   examples/src/main/apps/aggregator/coordinator.xml 
> 948edc1adbe09dfb6f7bffb5eebc82f3670658cc 
>   examples/src/main/apps/aggregator/job-with-offset.properties 
> 9a28ae13e50c40f70db4b0f583a63eb42f97388d 
>   examples/src/main/apps/aggregator/job.properties 
> aaf4edf7b9d030b5044610698b27a564b008f77b 
>   examples/src/main/apps/aggregator/workflow.xml 
> 4a801090e31acba23ea49da1c59a8ce4b951c654 
>   examples/src/main/apps/bundle/job.properties 
> c06edeb3f42e2eba3367b9486982dabb4caf6436 
>   examples/src/main/apps/coord-input-logic/job.properties 
> 2cd137b94999c7e1f68bb54928f1c3ce2838d59d 
>   examples/src/main/apps/coord-input-logic/workflow.xml 
> 80d31457eff6e78710ad1fb4d4f1b372c0d1e43c 
>   examples/src/main/apps/cron-schedule/coordinator.xml 
> 48a7de76245e02397b9736a95fa0132504067f9a 
>   examples/src/main/apps/cron-schedule/job.properties 
> 5b5eb43ca2bf9d8cd1b263ca3b94a552b562a20d 
>   examples/src/main/apps/cron-schedule/workflow.xml 
> b53922c8f40e691cb637b91cb182172794296a04 
>   examples/src/main/apps/cron/coordinator.xml 
> 744d14e870b59d0b89f65a78fea63fffa3ff4b8e 
>   examples/src/main/apps/cron/job.properties 
> 42875fd5a5ce393cd11717bcc55fc1bb91e0fcd3 
>   examples/src/main/apps/cron/workflow.xml 
> 63dff976f9e35a26b0ad09078778748d3d02bc3b 
>   examples/src/main/apps/custom-main/job.properties 
> b897512a1c6246308ca17b019ce400f3ec06cd7b 
>   examples/src/main/apps/custom-main/workflow.xml 
> 828dfaa0594aac0539c4e08ae8e37cb89d17eb54 
>   examples/src/main/apps/datelist-java-main/job.properties 
> e8356e3a4a2e1ae41e09f5ea124fb5360bd6e8b6 
>   examples/src/main/apps/datelist-java-main/workflow.xml 
> b59a12ebd1fc21c9e26f6a5b0348eb9c76375c03 
>   examples/src/main/apps/demo/job.properties 
> d70d3208df77e0b612005196907cecd97d7c5d21 
>   examples/src/main/apps/demo/job.windows.properties 
> ea948b20009685e1286053549fc1974669819682 
>   examples/src/main/apps/demo/workflow.xml 
> 0ceca507adc68c0674b794420c9eb733cb320879 
>   examples/src/main/apps/distcp/job.properties 
> ec9422caddcdecfa6ee9a58868cf68d7457ad4db 
>   examples/src/main/apps/distcp/workflow.xml 
> 6deecb377901613a5bb28fc18bf5beff2d30f252 
>   examples/src/main/apps/hadoop-el/job.properties 
> 1d13c27056ac58ad359b5298f12ec046bb4cc61d 
>   examples/src/main/apps/hadoop-el/workflow.xml 
> d4094d44136a7de2f85f5c9be042d5aab24b90be 
>   examples/src/main/apps/hcatalog/job.properties 
>