[jira] [Commented] (OOZIE-3176) Oozie-core fails with checkstyle errors.

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16360450#comment-16360450
 ] 

Hadoop QA commented on OOZIE-3176:
--


Testing JIRA OOZIE-3176

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:green}+1{color} the patch adds/modifies 56 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
.{color:red}WARNING{color}: the current HEAD has 100 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [server].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 2092
.Tests failed: 3
.Tests errors: 1

.The patch failed the following testcases:

testPartitionDependency(org.apache.oozie.service.TestPartitionDependencyManagerService)
testCoordActionRecoveryServiceForWaitingRegisterPartition(org.apache.oozie.service.TestRecoveryService)
testPartitionDependency(org.apache.oozie.service.TestPartitionDependencyManagerEhcache)

.Tests failing with errors:
testConnectionRetry(org.apache.oozie.service.TestJMSAccessorService)

.{color:orange}Tests failed at first run:{color}
TestJavaActionExecutor#testCredentialsSkip
.For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
{color:red}-1 DISTRO{color}
.{color:red}-1{color} distro tarball fails with the patch


{color:red}*-1 Overall result, please check the reported -1(s)*{color}

{color:red}. There is at least one warning, please check{color}

The full output of the test-patch run is available at

. https://builds.apache.org/job/PreCommit-OOZIE-Build/385/



> Oozie-core fails with checkstyle errors.
> 
>
> Key: OOZIE-3176
> URL: https://issues.apache.org/jira/browse/OOZIE-3176
> Project: Oozie
>  Issue Type: Bug
>  Components: core
>Affects Versions: 5.0.0b1
> Environment: # uname -a
> Linux in-ibmibm283.persistent.co.in 3.10.0-229.14.1.el7.x86_64 #1 SMP Tue Aug 
> 25 11:21:22 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux
>Reporter: Alisha Prabhu
>Assignee: Alisha Prabhu
>Priority: Minor
> Attachments: Checkstyle-erros.txt, OOZIE-3176-001.patch
>
>
> There are multiple checkstyle violations in the Oozie-core module.
> Maven command used is mvn clean install -DskipTests
> I have attached the Checkstyle-errors.txt file, that has the errors occurred 
> while building.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Failed: OOZIE-3176 PreCommit Build #385

[Apache Jenkins Server]

Jira: https://issues.apache.org/jira/browse/OOZIE-3176
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/385/

###
## LAST 100 LINES OF THE CONSOLE 
###
[...truncated 1.66 MB...]
[DEBUG] There are no new bugs found in [tools].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [server].
[INFO] There are no new bugs found totally].
[TRACE] FindBugs diffs checked and reports created
[TRACE] Summary file size is 2253 bytes
[TRACE] Full summary file size is 1314 bytes
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar]
 removed
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5sum]
 removed
  Running test-patch task BACKWARDS_COMPATIBILITY
  Running test-patch task TESTS
  Running test-patch task DISTRO


Testing JIRA OOZIE-3176

Cleaning local git workspace



+1 PATCH_APPLIES
+1 CLEAN
+1 RAW_PATCH_ANALYSIS
+1 the patch does not introduce any @author tags
+1 the patch does not introduce any tabs
+1 the patch does not introduce any trailing spaces
+1 the patch does not introduce any line longer than 132
+1 the patch adds/modifies 56 testcase(s)
+1 RAT
+1 the patch does not seem to introduce new RAT warnings
+1 JAVADOC
+1 the patch does not seem to introduce new Javadoc warnings
WARNING: the current HEAD has 100 Javadoc warning(s)
+1 COMPILE
+1 HEAD compiles
+1 patch compiles
+1 the patch does not seem to introduce new javac warnings
+1 There are no new bugs found in total.
 +1 There are no new bugs found in [docs].
 +1 There are no new bugs found in [webapp].
 +1 There are no new bugs found in [sharelib/distcp].
 +1 There are no new bugs found in [sharelib/hive].
 +1 There are no new bugs found in [sharelib/spark].
 +1 There are no new bugs found in [sharelib/hive2].
 +1 There are no new bugs found in [sharelib/hcatalog].
 +1 There are no new bugs found in [sharelib/streaming].
 +1 There are no new bugs found in [sharelib/pig].
 +1 There are no new bugs found in [sharelib/sqoop].
 +1 There are no new bugs found in [sharelib/oozie].
 +1 There are no new bugs found in [examples].
 +1 There are no new bugs found in [client].
 +1 There are no new bugs found in [core].
 +1 There are no new bugs found in [tools].
 +1 There are no new bugs found in [server].
+1 BACKWARDS_COMPATIBILITY
+1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient 
annotations
+1 the patch does not modify JPA files
-1 TESTS
Tests run: 2092
Tests failed: 3
Tests errors: 1

The patch failed the following testcases:

testPartitionDependency(org.apache.oozie.service.TestPartitionDependencyManagerService)
testCoordActionRecoveryServiceForWaitingRegisterPartition(org.apache.oozie.service.TestRecoveryService)
testPartitionDependency(org.apache.oozie.service.TestPartitionDependencyManagerEhcache)

Tests failing with errors:
testConnectionRetry(org.apache.oozie.service.TestJMSAccessorService)

Tests failed at first run:
TestJavaActionExecutor#testCredentialsSkip
For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
-1 DISTRO
-1 distro tarball fails with the patch


-1 Overall result, please check the reported -1(s)

 There is at least one warning, please check

The full output of the test-patch run is available at

 https://builds.apache.org/job/PreCommit-OOZIE-Build/385/

Adding comment to JIRA
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0  
0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 
0{"self":"https://issues.apache.org/jira/rest/api/2/issue/13136797/comment/16360450","id":"16360450","author":{"self":"https://issues.apache.org/jira/rest/api/2/user?username=hadoopqa","name":"hadoopqa","key":"hadoopqa","emailAddress":"blackhole
 at hadoop dot apache dot 
org","avatarUrls":{"48x48":"https://issues.apache.org/jira/secure/useravatar?ownerId=hadoopqa&avatarId=10393","24x24":"https://issues.apache.org/jira/secure/useravatar?size=small&ownerId=hadoopqa&avatarId=10393","16x16":"https://issues.apache.org/jira/secure/useravatar?size=xsmall&ownerId=hadoopqa&avatarId=10393","32x32":"https://issues.apache.org/jira/secure/useravatar?size=medium&ownerId=hadoopqa&avatarId=10393"},"displayName":"Hadoop
 QA","active":true,"timeZone":"Etc/UTC"},"body":"\nTesting JIRA 
OOZIE-3176\n\nCleaning local git 
workspace\n\n\n\n{color:green}+1 
PATCH_APPLIES{color}\n{color:green}+1 CLEAN{color}

[jira] [Updated] (OOZIE-2814) OYA: Update example workflows to newest schemas

[Attila Sasvari (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Sasvari updated OOZIE-2814:
--
Attachment: OOZIE-2814-02.patch

> OYA: Update example workflows to newest schemas
> ---
>
> Key: OOZIE-2814
> URL: https://issues.apache.org/jira/browse/OOZIE-2814
> Project: Oozie
>  Issue Type: Sub-task
>Affects Versions: 5.0.0
>Reporter: Robert Kanter
>Assignee: Attila Sasvari
>Priority: Blocker
> Attachments: OOZIE-2814-00.patch, OOZIE-2814-01.patch, 
> OOZIE-2814-02.patch
>
>
> OOZIE-2687 will add in a whole set of newer schemas that are more 
> Yarn-centric.  Most, if not all, examples currently use pretty old versions 
> of the schemas.  We should take this opportunity to update the examples to 
> the newest schemas added by OOZIE-2687, which should help demonstrate them.
> It may make sense to keep the older schemas too, to show that older workflows 
> still work.  We have some examples where we have two workflow XML files that 
> do the same thing in different ways.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (OOZIE-2814) OYA: Update example workflows to newest schemas

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16360640#comment-16360640
 ] 

Hadoop QA commented on OOZIE-2814:
--

PreCommit-OOZIE-Build started


> OYA: Update example workflows to newest schemas
> ---
>
> Key: OOZIE-2814
> URL: https://issues.apache.org/jira/browse/OOZIE-2814
> Project: Oozie
>  Issue Type: Sub-task
>Affects Versions: 5.0.0
>Reporter: Robert Kanter
>Assignee: Attila Sasvari
>Priority: Blocker
> Attachments: OOZIE-2814-00.patch, OOZIE-2814-01.patch, 
> OOZIE-2814-02.patch
>
>
> OOZIE-2687 will add in a whole set of newer schemas that are more 
> Yarn-centric.  Most, if not all, examples currently use pretty old versions 
> of the schemas.  We should take this opportunity to update the examples to 
> the newest schemas added by OOZIE-2687, which should help demonstrate them.
> It may make sense to keep the older schemas too, to show that older workflows 
> still work.  We have some examples where we have two workflow XML files that 
> do the same thing in different ways.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Review Request 65606: OOZIE-2814 OYA: Update example workflows to newest schemas

[Attila Sasvari via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65606/
---

Review request for oozie.


Repository: oozie-git


Description
---

OOZIE-2687 will add in a whole set of newer schemas that are more Yarn-centric. 
Most, if not all, examples currently use pretty old versions of the schemas. We 
should take this opportunity to update the examples to the newest schemas added 
by OOZIE-2687, which should help demonstrate them.

It may make sense to keep the older schemas too, to show that older workflows 
still work. We have some examples where we have two workflow XML files that do 
the same thing in different ways.


Diffs
-

  core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
bc0f4050bb3788988d3a5352a15220fbc4fa219d 
  core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 
c236dafa5965403c33e1a400509e7c374fe2637c 
  core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 
1cc569d8d7a0e61905ad6c4e940cefb00da1343d 
  
core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java
 2cdbde4d2fc86a07b35beadc56012c438baf8d54 
  examples/src/main/apps/aggregator/coordinator-with-offset.xml 
4c351677f4afc6e85f807106d426bb2f922a28b1 
  examples/src/main/apps/aggregator/coordinator.xml 
948edc1adbe09dfb6f7bffb5eebc82f3670658cc 
  examples/src/main/apps/aggregator/job-with-offset.properties 
9a28ae13e50c40f70db4b0f583a63eb42f97388d 
  examples/src/main/apps/aggregator/job.properties 
aaf4edf7b9d030b5044610698b27a564b008f77b 
  examples/src/main/apps/aggregator/workflow.xml 
4a801090e31acba23ea49da1c59a8ce4b951c654 
  examples/src/main/apps/bundle/job.properties 
c06edeb3f42e2eba3367b9486982dabb4caf6436 
  examples/src/main/apps/coord-input-logic/job.properties 
2cd137b94999c7e1f68bb54928f1c3ce2838d59d 
  examples/src/main/apps/coord-input-logic/workflow.xml 
80d31457eff6e78710ad1fb4d4f1b372c0d1e43c 
  examples/src/main/apps/cron-schedule/coordinator.xml 
48a7de76245e02397b9736a95fa0132504067f9a 
  examples/src/main/apps/cron-schedule/job.properties 
5b5eb43ca2bf9d8cd1b263ca3b94a552b562a20d 
  examples/src/main/apps/cron-schedule/workflow.xml 
b53922c8f40e691cb637b91cb182172794296a04 
  examples/src/main/apps/cron/coordinator.xml 
744d14e870b59d0b89f65a78fea63fffa3ff4b8e 
  examples/src/main/apps/cron/job.properties 
42875fd5a5ce393cd11717bcc55fc1bb91e0fcd3 
  examples/src/main/apps/cron/workflow.xml 
63dff976f9e35a26b0ad09078778748d3d02bc3b 
  examples/src/main/apps/custom-main/job.properties 
b897512a1c6246308ca17b019ce400f3ec06cd7b 
  examples/src/main/apps/custom-main/workflow.xml 
828dfaa0594aac0539c4e08ae8e37cb89d17eb54 
  examples/src/main/apps/datelist-java-main/job.properties 
e8356e3a4a2e1ae41e09f5ea124fb5360bd6e8b6 
  examples/src/main/apps/datelist-java-main/workflow.xml 
b59a12ebd1fc21c9e26f6a5b0348eb9c76375c03 
  examples/src/main/apps/demo/job.properties 
d70d3208df77e0b612005196907cecd97d7c5d21 
  examples/src/main/apps/demo/job.windows.properties 
ea948b20009685e1286053549fc1974669819682 
  examples/src/main/apps/demo/workflow.xml 
0ceca507adc68c0674b794420c9eb733cb320879 
  examples/src/main/apps/distcp/job.properties 
ec9422caddcdecfa6ee9a58868cf68d7457ad4db 
  examples/src/main/apps/distcp/workflow.xml 
6deecb377901613a5bb28fc18bf5beff2d30f252 
  examples/src/main/apps/hadoop-el/job.properties 
1d13c27056ac58ad359b5298f12ec046bb4cc61d 
  examples/src/main/apps/hadoop-el/workflow.xml 
d4094d44136a7de2f85f5c9be042d5aab24b90be 
  examples/src/main/apps/hcatalog/job.properties 
e0aa18c264a2fa92b6c3d501d18fa39733d863ff 
  examples/src/main/apps/hcatalog/job.properties.security 
2fbcc6ee9e5d94d3bba63810e6e788607747940b 
  examples/src/main/apps/hcatalog/workflow.xml 
d6593df1b9104179d79262d663135d73de8b55ec 
  examples/src/main/apps/hcatalog/workflow.xml.security 
aa3522ae4d5b6cb178f80c2e72e45c4c3de9aab5 
  examples/src/main/apps/hive/job.properties 
dbc7ad2a0b135b8c9f0b6e575086ee8260072756 
  examples/src/main/apps/hive/workflow.xml 
449d62d35f2e69d01d9e59f35a0cc9ad057342d2 
  examples/src/main/apps/hive/workflow.xml.security 
35321319dcbb0576c5870952e6e805209b8dc240 
  examples/src/main/apps/hive2/job.properties 
2640402472d3b2120941834e4653d363fb49914b 
  examples/src/main/apps/hive2/job.properties.security 
ac980cd3cc37e3b3b359c2e3fefe180cf88b56bc 
  examples/src/main/apps/hive2/workflow.xml 
5f1ccc4fd0791f850c0942452d637048acce13d1 
  examples/src/main/apps/hive2/workflow.xml.security 
6f7304def80317429e58cecddc63fe4c1d3edaa6 
  examples/src/main/apps/java-main/job.properties 
519c53e7313b6a5b54ab54cb2dd31f123819a0f9 
  examples/src/main/apps/java-main/workflow.xml 
98e01ca92187e1c567686f6c2b4f689bb2a5ef6a 
  examples/src/main/apps/map-reduce/job-with-config-class.properties 
0b14cb749b82fcd165f8db9b1832c650e6970f61 
  examples/src/main/apps/map-reduce/jo

[jira] [Created] (OOZIE-3181) High frequency coord. job LAST_ONLY with many past-time actions kills Oozie server

[Oleksandr Kalinin (JIRA)]

Oleksandr Kalinin created OOZIE-3181:


 Summary: High frequency coord. job LAST_ONLY with many past-time 
actions kills Oozie server
 Key: OOZIE-3181
 URL: https://issues.apache.org/jira/browse/OOZIE-3181
 Project: Oozie
  Issue Type: Bug
Affects Versions: 4.3.0
Reporter: Oleksandr Kalinin


User submitting high frequency coordinator job for past time (intentionally or 
by mistake) triggers enormous materialization loop for that job and potentially 
OOM condition even with high heap settings.

Simplest example is:

coordStarts=2017-02-12T09:00Z
coordEnds=2019-02-12T09:00Z
coordFrequency=*/1 * * * *

LAST_ONLY

This triggers non throttled materialization of more than 500K actions lying in 
the past which causes severe memory pressure and eventual GC overhead lockout.

At the same time by definition all past actions will be skipped anyway, thus it 
seems that the only value in materializing them is ability to view SKIPPED 
status later. Is it really worth the risk?

Note : additional severity of this problem in terms of stability is that it's 
not trivial to recover it on ZK-coordinated clusters. Write lock will persist 
which will prevent kill command from taking desired effect, and that lock will 
persist also after restart. To recover, write lock has to be manually removed.

Looking at materialization loop code, I believe there is potential for 
algorithm and throttling improvement to prevent this issue.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (OOZIE-3181) High frequency coord. job LAST_ONLY with many past-time actions kills Oozie server

[Oleksandr Kalinin (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleksandr Kalinin updated OOZIE-3181:
-
Description: 
User submitting high frequency coordinator job for past time (intentionally or 
by mistake) triggers enormous materialization loop for that job and potentially 
OOM condition even with high heap settings.

Simplest example is:

coordStarts=2017-02-12T09:00Z
 coordEnds=2019-02-12T09:00Z
 coordFrequency=*/1 * * * *

LAST_ONLY

This triggers non throttled materialization of more than 500K actions lying in 
the past which causes severe memory pressure and eventual GC overhead lockout.

At the same time by definition all past actions will be skipped anyway, thus it 
seems that the only value in materializing them is ability to view SKIPPED 
status later. Is it really worth the risk?

Note : additional severity of this problem in terms of stability is that it's 
not trivial to recover it on ZK-coordinated clusters. Write lock will persist 
which will prevent kill command from taking desired effect, and that lock will 
persist also after restart. To recover, write lock has to be manually removed.

Looking at materialization loop code, I believe there is potential for 
algorithm improvement to prevent this issue.

  was:
User submitting high frequency coordinator job for past time (intentionally or 
by mistake) triggers enormous materialization loop for that job and potentially 
OOM condition even with high heap settings.

Simplest example is:

coordStarts=2017-02-12T09:00Z
coordEnds=2019-02-12T09:00Z
coordFrequency=*/1 * * * *

LAST_ONLY

This triggers non throttled materialization of more than 500K actions lying in 
the past which causes severe memory pressure and eventual GC overhead lockout.

At the same time by definition all past actions will be skipped anyway, thus it 
seems that the only value in materializing them is ability to view SKIPPED 
status later. Is it really worth the risk?

Note : additional severity of this problem in terms of stability is that it's 
not trivial to recover it on ZK-coordinated clusters. Write lock will persist 
which will prevent kill command from taking desired effect, and that lock will 
persist also after restart. To recover, write lock has to be manually removed.

Looking at materialization loop code, I believe there is potential for 
algorithm and throttling improvement to prevent this issue.


> High frequency coord. job LAST_ONLY with many past-time actions kills Oozie 
> server
> --
>
> Key: OOZIE-3181
> URL: https://issues.apache.org/jira/browse/OOZIE-3181
> Project: Oozie
>  Issue Type: Bug
>Affects Versions: 4.3.0
>Reporter: Oleksandr Kalinin
>Priority: Major
>
> User submitting high frequency coordinator job for past time (intentionally 
> or by mistake) triggers enormous materialization loop for that job and 
> potentially OOM condition even with high heap settings.
> Simplest example is:
> coordStarts=2017-02-12T09:00Z
>  coordEnds=2019-02-12T09:00Z
>  coordFrequency=*/1 * * * *
> LAST_ONLY
> This triggers non throttled materialization of more than 500K actions lying 
> in the past which causes severe memory pressure and eventual GC overhead 
> lockout.
> At the same time by definition all past actions will be skipped anyway, thus 
> it seems that the only value in materializing them is ability to view SKIPPED 
> status later. Is it really worth the risk?
> Note : additional severity of this problem in terms of stability is that it's 
> not trivial to recover it on ZK-coordinated clusters. Write lock will persist 
> which will prevent kill command from taking desired effect, and that lock 
> will persist also after restart. To recover, write lock has to be manually 
> removed.
> Looking at materialization loop code, I believe there is potential for 
> algorithm improvement to prevent this issue.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (OOZIE-3181) High frequency coord. job LAST_ONLY with many past-time actions kills Oozie server

[Oleksandr Kalinin (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleksandr Kalinin updated OOZIE-3181:
-
Description: 
User submitting high frequency LAST_ONLY coordinator job for past time 
(intentionally or by mistake) triggers enormous materialization loop for that 
job and potentially OOM condition even with high heap settings.

Simplest example is:

coordStarts=2017-02-12T09:00Z
 coordEnds=2019-02-12T09:00Z
 coordFrequency=*/1 * * * *

LAST_ONLY

This triggers non throttled materialization of more than 500K actions lying in 
the past which causes severe memory pressure and eventual GC overhead lockout.

At the same time by definition all past actions will be skipped anyway, thus it 
seems that the only value in materializing them is ability to view SKIPPED 
status later. Is it really worth the risk?

Note : additional severity of this problem in terms of stability is that it's 
not trivial to recover it on ZK-coordinated clusters. Write lock will persist 
which will prevent kill command from taking desired effect, and that lock will 
persist also after restart. To recover, write lock has to be manually removed.

Looking at materialization loop code, I believe there is potential for 
algorithm improvement to prevent this issue.

  was:
User submitting high frequency coordinator job for past time (intentionally or 
by mistake) triggers enormous materialization loop for that job and potentially 
OOM condition even with high heap settings.

Simplest example is:

coordStarts=2017-02-12T09:00Z
 coordEnds=2019-02-12T09:00Z
 coordFrequency=*/1 * * * *

LAST_ONLY

This triggers non throttled materialization of more than 500K actions lying in 
the past which causes severe memory pressure and eventual GC overhead lockout.

At the same time by definition all past actions will be skipped anyway, thus it 
seems that the only value in materializing them is ability to view SKIPPED 
status later. Is it really worth the risk?

Note : additional severity of this problem in terms of stability is that it's 
not trivial to recover it on ZK-coordinated clusters. Write lock will persist 
which will prevent kill command from taking desired effect, and that lock will 
persist also after restart. To recover, write lock has to be manually removed.

Looking at materialization loop code, I believe there is potential for 
algorithm improvement to prevent this issue.


> High frequency coord. job LAST_ONLY with many past-time actions kills Oozie 
> server
> --
>
> Key: OOZIE-3181
> URL: https://issues.apache.org/jira/browse/OOZIE-3181
> Project: Oozie
>  Issue Type: Bug
>Affects Versions: 4.3.0
>Reporter: Oleksandr Kalinin
>Priority: Major
>
> User submitting high frequency LAST_ONLY coordinator job for past time 
> (intentionally or by mistake) triggers enormous materialization loop for that 
> job and potentially OOM condition even with high heap settings.
> Simplest example is:
> coordStarts=2017-02-12T09:00Z
>  coordEnds=2019-02-12T09:00Z
>  coordFrequency=*/1 * * * *
> LAST_ONLY
> This triggers non throttled materialization of more than 500K actions lying 
> in the past which causes severe memory pressure and eventual GC overhead 
> lockout.
> At the same time by definition all past actions will be skipped anyway, thus 
> it seems that the only value in materializing them is ability to view SKIPPED 
> status later. Is it really worth the risk?
> Note : additional severity of this problem in terms of stability is that it's 
> not trivial to recover it on ZK-coordinated clusters. Write lock will persist 
> which will prevent kill command from taking desired effect, and that lock 
> will persist also after restart. To recover, write lock has to be manually 
> removed.
> Looking at materialization loop code, I believe there is potential for 
> algorithm improvement to prevent this issue.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (OOZIE-3181) High frequency LAST_ONLY coord. job with many past-time actions kills Oozie server

[Oleksandr Kalinin (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleksandr Kalinin updated OOZIE-3181:
-
Summary: High frequency LAST_ONLY coord. job with many past-time actions 
kills Oozie server  (was: High frequency coord. job LAST_ONLY with many 
past-time actions kills Oozie server)

> High frequency LAST_ONLY coord. job with many past-time actions kills Oozie 
> server
> --
>
> Key: OOZIE-3181
> URL: https://issues.apache.org/jira/browse/OOZIE-3181
> Project: Oozie
>  Issue Type: Bug
>Affects Versions: 4.3.0
>Reporter: Oleksandr Kalinin
>Priority: Major
>
> User submitting high frequency LAST_ONLY coordinator job for past time 
> (intentionally or by mistake) triggers enormous materialization loop for that 
> job and potentially OOM condition even with high heap settings.
> Simplest example is:
> coordStarts=2017-02-12T09:00Z
>  coordEnds=2019-02-12T09:00Z
>  coordFrequency=*/1 * * * *
> LAST_ONLY
> This triggers non throttled materialization of more than 500K actions lying 
> in the past which causes severe memory pressure and eventual GC overhead 
> lockout.
> At the same time by definition all past actions will be skipped anyway, thus 
> it seems that the only value in materializing them is ability to view SKIPPED 
> status later. Is it really worth the risk?
> Note : additional severity of this problem in terms of stability is that it's 
> not trivial to recover it on ZK-coordinated clusters. Write lock will persist 
> which will prevent kill command from taking desired effect, and that lock 
> will persist also after restart. To recover, write lock has to be manually 
> removed.
> Looking at materialization loop code, I believe there is potential for 
> algorithm improvement to prevent this issue.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (OOZIE-3181) High frequency LAST_ONLY coord. job with many past time actions kills Oozie server

[Oleksandr Kalinin (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleksandr Kalinin updated OOZIE-3181:
-
Summary: High frequency LAST_ONLY coord. job with many past time actions 
kills Oozie server  (was: High frequency LAST_ONLY coord. job with many 
past-time actions kills Oozie server)

> High frequency LAST_ONLY coord. job with many past time actions kills Oozie 
> server
> --
>
> Key: OOZIE-3181
> URL: https://issues.apache.org/jira/browse/OOZIE-3181
> Project: Oozie
>  Issue Type: Bug
>Affects Versions: 4.3.0
>Reporter: Oleksandr Kalinin
>Priority: Major
>
> User submitting high frequency LAST_ONLY coordinator job for past time 
> (intentionally or by mistake) triggers enormous materialization loop for that 
> job and potentially OOM condition even with high heap settings.
> Simplest example is:
> coordStarts=2017-02-12T09:00Z
>  coordEnds=2019-02-12T09:00Z
>  coordFrequency=*/1 * * * *
> LAST_ONLY
> This triggers non throttled materialization of more than 500K actions lying 
> in the past which causes severe memory pressure and eventual GC overhead 
> lockout.
> At the same time by definition all past actions will be skipped anyway, thus 
> it seems that the only value in materializing them is ability to view SKIPPED 
> status later. Is it really worth the risk?
> Note : additional severity of this problem in terms of stability is that it's 
> not trivial to recover it on ZK-coordinated clusters. Write lock will persist 
> which will prevent kill command from taking desired effect, and that lock 
> will persist also after restart. To recover, write lock has to be manually 
> removed.
> Looking at materialization loop code, I believe there is potential for 
> algorithm improvement to prevent this issue.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (OOZIE-3181) High frequency LAST_ONLY coord. job with many past time actions kills Oozie server

[Oleksandr Kalinin (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleksandr Kalinin updated OOZIE-3181:
-
Description: 
User submitting high frequency LAST_ONLY coordinator job for past time 
(intentionally or by mistake) triggers enormous materialization loop for that 
job and potentially OOM condition even with high heap settings.

Simplest example is:

coordStarts=2017-02-12T09:00Z
 coordEnds=2019-02-12T09:00Z
 coordFrequency=*/1 * * * *

LAST_ONLY

This triggers non throttled materialization of more than 500K actions lying in 
the past which causes severe memory pressure and eventual GC overhead lockout.

At the same time by definition all past actions will be skipped anyway, thus it 
seems that the only value in materializing them is ability to view SKIPPED 
status later. Is it really worth the risk?

Note : additional severity of this problem is that it's not trivial to recover 
it on ZK-coordinated clusters. Write lock will persist which will prevent kill 
command from taking desired effect, and that lock will persist also after 
restart. To recover, write lock has to be manually removed.

Looking at materialization loop code, I believe there is potential for 
algorithm improvement to prevent this issue.

  was:
User submitting high frequency LAST_ONLY coordinator job for past time 
(intentionally or by mistake) triggers enormous materialization loop for that 
job and potentially OOM condition even with high heap settings.

Simplest example is:

coordStarts=2017-02-12T09:00Z
 coordEnds=2019-02-12T09:00Z
 coordFrequency=*/1 * * * *

LAST_ONLY

This triggers non throttled materialization of more than 500K actions lying in 
the past which causes severe memory pressure and eventual GC overhead lockout.

At the same time by definition all past actions will be skipped anyway, thus it 
seems that the only value in materializing them is ability to view SKIPPED 
status later. Is it really worth the risk?

Note : additional severity of this problem in terms of stability is that it's 
not trivial to recover it on ZK-coordinated clusters. Write lock will persist 
which will prevent kill command from taking desired effect, and that lock will 
persist also after restart. To recover, write lock has to be manually removed.

Looking at materialization loop code, I believe there is potential for 
algorithm improvement to prevent this issue.


> High frequency LAST_ONLY coord. job with many past time actions kills Oozie 
> server
> --
>
> Key: OOZIE-3181
> URL: https://issues.apache.org/jira/browse/OOZIE-3181
> Project: Oozie
>  Issue Type: Bug
>Affects Versions: 4.3.0
>Reporter: Oleksandr Kalinin
>Priority: Major
>
> User submitting high frequency LAST_ONLY coordinator job for past time 
> (intentionally or by mistake) triggers enormous materialization loop for that 
> job and potentially OOM condition even with high heap settings.
> Simplest example is:
> coordStarts=2017-02-12T09:00Z
>  coordEnds=2019-02-12T09:00Z
>  coordFrequency=*/1 * * * *
> LAST_ONLY
> This triggers non throttled materialization of more than 500K actions lying 
> in the past which causes severe memory pressure and eventual GC overhead 
> lockout.
> At the same time by definition all past actions will be skipped anyway, thus 
> it seems that the only value in materializing them is ability to view SKIPPED 
> status later. Is it really worth the risk?
> Note : additional severity of this problem is that it's not trivial to 
> recover it on ZK-coordinated clusters. Write lock will persist which will 
> prevent kill command from taking desired effect, and that lock will persist 
> also after restart. To recover, write lock has to be manually removed.
> Looking at materialization loop code, I believe there is potential for 
> algorithm improvement to prevent this issue.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (OOZIE-3181) High frequency LAST_ONLY coord. job with many past time actions kills Oozie server

[Oleksandr Kalinin (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleksandr Kalinin updated OOZIE-3181:
-
Description: 
User submitting high frequency LAST_ONLY coordinator job for past time 
(intentionally or by mistake) triggers enormous materialization loop for that 
job and potentially OOM condition even with high heap settings.

Simplest example is:

coordStarts=2017-02-12T09:00Z
 coordEnds=2019-02-12T09:00Z
 coordFrequency=*/1 * * * *

LAST_ONLY

Since throttling parameters are ignored on LAST_ONLY jobs, this triggers non 
throttled materialization of more than 500K actions lying in the past which 
causes severe memory pressure and eventual GC overhead lockout.

At the same time by definition all past actions will be skipped anyway, thus it 
seems that the only value in materializing them is ability to view SKIPPED 
status later. Is it really worth the risk?

Note : additional severity of this problem is that it's not trivial to recover 
it on ZK-coordinated clusters. Write lock will persist which will prevent kill 
command from taking desired effect, and that lock will persist also after 
restart. To recover, write lock has to be manually removed.

Looking at materialization loop code, I believe there is potential for 
algorithm improvement to prevent this issue.

  was:
User submitting high frequency LAST_ONLY coordinator job for past time 
(intentionally or by mistake) triggers enormous materialization loop for that 
job and potentially OOM condition even with high heap settings.

Simplest example is:

coordStarts=2017-02-12T09:00Z
 coordEnds=2019-02-12T09:00Z
 coordFrequency=*/1 * * * *

LAST_ONLY

This triggers non throttled materialization of more than 500K actions lying in 
the past which causes severe memory pressure and eventual GC overhead lockout.

At the same time by definition all past actions will be skipped anyway, thus it 
seems that the only value in materializing them is ability to view SKIPPED 
status later. Is it really worth the risk?

Note : additional severity of this problem is that it's not trivial to recover 
it on ZK-coordinated clusters. Write lock will persist which will prevent kill 
command from taking desired effect, and that lock will persist also after 
restart. To recover, write lock has to be manually removed.

Looking at materialization loop code, I believe there is potential for 
algorithm improvement to prevent this issue.


> High frequency LAST_ONLY coord. job with many past time actions kills Oozie 
> server
> --
>
> Key: OOZIE-3181
> URL: https://issues.apache.org/jira/browse/OOZIE-3181
> Project: Oozie
>  Issue Type: Bug
>Affects Versions: 4.3.0
>Reporter: Oleksandr Kalinin
>Priority: Major
>
> User submitting high frequency LAST_ONLY coordinator job for past time 
> (intentionally or by mistake) triggers enormous materialization loop for that 
> job and potentially OOM condition even with high heap settings.
> Simplest example is:
> coordStarts=2017-02-12T09:00Z
>  coordEnds=2019-02-12T09:00Z
>  coordFrequency=*/1 * * * *
> LAST_ONLY
> Since throttling parameters are ignored on LAST_ONLY jobs, this triggers non 
> throttled materialization of more than 500K actions lying in the past which 
> causes severe memory pressure and eventual GC overhead lockout.
> At the same time by definition all past actions will be skipped anyway, thus 
> it seems that the only value in materializing them is ability to view SKIPPED 
> status later. Is it really worth the risk?
> Note : additional severity of this problem is that it's not trivial to 
> recover it on ZK-coordinated clusters. Write lock will persist which will 
> prevent kill command from taking desired effect, and that lock will persist 
> also after restart. To recover, write lock has to be manually removed.
> Looking at materialization loop code, I believe there is potential for 
> algorithm improvement to prevent this issue.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (OOZIE-3181) High frequency LAST_ONLY coord. job with many past time actions kills Oozie server

[Oleksandr Kalinin (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Oleksandr Kalinin updated OOZIE-3181:
-
Description: 
User submitting high frequency LAST_ONLY coordinator job for past time 
(intentionally or by mistake) triggers enormous materialization loop for that 
job and potentially OOM condition even with high heap settings.

Simplest example is:

coordStarts=2017-02-12T09:00Z
 coordEnds=2019-02-12T09:00Z
 coordFrequency=*/1 * * * *

LAST_ONLY

Since throttling parameters are ignored on LAST_ONLY past actions, this 
triggers non throttled materialization of more than 500K actions lying in the 
past which causes severe memory pressure and eventual GC overhead lockout.

At the same time by definition all past actions will be skipped anyway, thus it 
seems that the only value in materializing them is ability to view SKIPPED 
status later. Is it really worth the risk?

Note : additional severity of this problem is that it's not trivial to recover 
it on ZK-coordinated clusters. Write lock will persist which will prevent kill 
command from taking desired effect, and that lock will persist also after 
restart. To recover, write lock has to be manually removed.

Looking at materialization loop code, I believe there is potential for 
algorithm improvement to prevent this issue.

  was:
User submitting high frequency LAST_ONLY coordinator job for past time 
(intentionally or by mistake) triggers enormous materialization loop for that 
job and potentially OOM condition even with high heap settings.

Simplest example is:

coordStarts=2017-02-12T09:00Z
 coordEnds=2019-02-12T09:00Z
 coordFrequency=*/1 * * * *

LAST_ONLY

Since throttling parameters are ignored on LAST_ONLY jobs, this triggers non 
throttled materialization of more than 500K actions lying in the past which 
causes severe memory pressure and eventual GC overhead lockout.

At the same time by definition all past actions will be skipped anyway, thus it 
seems that the only value in materializing them is ability to view SKIPPED 
status later. Is it really worth the risk?

Note : additional severity of this problem is that it's not trivial to recover 
it on ZK-coordinated clusters. Write lock will persist which will prevent kill 
command from taking desired effect, and that lock will persist also after 
restart. To recover, write lock has to be manually removed.

Looking at materialization loop code, I believe there is potential for 
algorithm improvement to prevent this issue.


> High frequency LAST_ONLY coord. job with many past time actions kills Oozie 
> server
> --
>
> Key: OOZIE-3181
> URL: https://issues.apache.org/jira/browse/OOZIE-3181
> Project: Oozie
>  Issue Type: Bug
>Affects Versions: 4.3.0
>Reporter: Oleksandr Kalinin
>Priority: Major
>
> User submitting high frequency LAST_ONLY coordinator job for past time 
> (intentionally or by mistake) triggers enormous materialization loop for that 
> job and potentially OOM condition even with high heap settings.
> Simplest example is:
> coordStarts=2017-02-12T09:00Z
>  coordEnds=2019-02-12T09:00Z
>  coordFrequency=*/1 * * * *
> LAST_ONLY
> Since throttling parameters are ignored on LAST_ONLY past actions, this 
> triggers non throttled materialization of more than 500K actions lying in the 
> past which causes severe memory pressure and eventual GC overhead lockout.
> At the same time by definition all past actions will be skipped anyway, thus 
> it seems that the only value in materializing them is ability to view SKIPPED 
> status later. Is it really worth the risk?
> Note : additional severity of this problem is that it's not trivial to 
> recover it on ZK-coordinated clusters. Write lock will persist which will 
> prevent kill command from taking desired effect, and that lock will persist 
> also after restart. To recover, write lock has to be manually removed.
> Looking at materialization loop code, I believe there is potential for 
> algorithm improvement to prevent this issue.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (OOZIE-3181) High frequency LAST_ONLY coord. job with many past time actions kills Oozie server

[Oleksandr Kalinin (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16360723#comment-16360723
 ] 

Oleksandr Kalinin commented on OOZIE-3181:
--

Alternatively, it should be possible to implement configuration parameter that 
controls maximum number of past actions that can be materialized in one shot, 
and fail the workflow if the actual number exceeds the maximum.

> High frequency LAST_ONLY coord. job with many past time actions kills Oozie 
> server
> --
>
> Key: OOZIE-3181
> URL: https://issues.apache.org/jira/browse/OOZIE-3181
> Project: Oozie
>  Issue Type: Bug
>Affects Versions: 4.3.0
>Reporter: Oleksandr Kalinin
>Priority: Major
>
> User submitting high frequency LAST_ONLY coordinator job for past time 
> (intentionally or by mistake) triggers enormous materialization loop for that 
> job and potentially OOM condition even with high heap settings.
> Simplest example is:
> coordStarts=2017-02-12T09:00Z
>  coordEnds=2019-02-12T09:00Z
>  coordFrequency=*/1 * * * *
> LAST_ONLY
> Since throttling parameters are ignored on LAST_ONLY past actions, this 
> triggers non throttled materialization of more than 500K actions lying in the 
> past which causes severe memory pressure and eventual GC overhead lockout.
> At the same time by definition all past actions will be skipped anyway, thus 
> it seems that the only value in materializing them is ability to view SKIPPED 
> status later. Is it really worth the risk?
> Note : additional severity of this problem is that it's not trivial to 
> recover it on ZK-coordinated clusters. Write lock will persist which will 
> prevent kill command from taking desired effect, and that lock will persist 
> also after restart. To recover, write lock has to be manually removed.
> Looking at materialization loop code, I believe there is potential for 
> algorithm improvement to prevent this issue.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Failed: OOZIE-2814 PreCommit Build #386

[Apache Jenkins Server]

Jira: https://issues.apache.org/jira/browse/OOZIE-2814
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/386/

###
## LAST 100 LINES OF THE CONSOLE 
###
[...truncated 1.74 MB...]
  Running test-patch task BACKWARDS_COMPATIBILITY
  Running test-patch task TESTS
  Running test-patch task DISTRO


Testing JIRA OOZIE-2814

Cleaning local git workspace



+1 PATCH_APPLIES
+1 CLEAN
+1 RAW_PATCH_ANALYSIS
+1 the patch does not introduce any @author tags
+1 the patch does not introduce any tabs
+1 the patch does not introduce any trailing spaces
+1 the patch does not introduce any line longer than 132
+1 the patch adds/modifies 2 testcase(s)
+1 RAT
+1 the patch does not seem to introduce new RAT warnings
+1 JAVADOC
+1 the patch does not seem to introduce new Javadoc warnings
WARNING: the current HEAD has 100 Javadoc warning(s)
+1 COMPILE
+1 HEAD compiles
+1 patch compiles
+1 the patch does not seem to introduce new javac warnings
+1 There are no new bugs found in total.
 +1 There are no new bugs found in [docs].
 +1 There are no new bugs found in [webapp].
 +1 There are no new bugs found in [sharelib/distcp].
 +1 There are no new bugs found in [sharelib/hive].
 +1 There are no new bugs found in [sharelib/spark].
 +1 There are no new bugs found in [sharelib/hive2].
 +1 There are no new bugs found in [sharelib/hcatalog].
 +1 There are no new bugs found in [sharelib/streaming].
 +1 There are no new bugs found in [sharelib/pig].
 +1 There are no new bugs found in [sharelib/sqoop].
 +1 There are no new bugs found in [sharelib/oozie].
 +1 There are no new bugs found in [examples].
 +1 There are no new bugs found in [client].
 +1 There are no new bugs found in [core].
 +1 There are no new bugs found in [tools].
 +1 There are no new bugs found in [server].
+1 BACKWARDS_COMPATIBILITY
+1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient 
annotations
+1 the patch does not modify JPA files
-1 TESTS
Tests run: 2095
Tests failed: 11
Tests errors: 2

The patch failed the following testcases:

testParserDefaultJobTrackerFail(org.apache.oozie.workflow.lite.TestLiteWorkflowAppParser)
testParserGlobalExtensionActions(org.apache.oozie.workflow.lite.TestLiteWorkflowAppParser)
testParserDefaultJobTrackerWithGlobal(org.apache.oozie.workflow.lite.TestLiteWorkflowAppParser)
testParserGlobalLocalAlreadyExists(org.apache.oozie.workflow.lite.TestLiteWorkflowAppParser)
testParserDefaultJobTracker(org.apache.oozie.workflow.lite.TestLiteWorkflowAppParser)
testParserGlobalJobXML(org.apache.oozie.workflow.lite.TestLiteWorkflowAppParser)
testParserGlobal(org.apache.oozie.workflow.lite.TestLiteWorkflowAppParser)
testPartitionDependency(org.apache.oozie.service.TestPartitionDependencyManagerService)
testCoordActionRecoveryServiceForWaitingRegisterPartition(org.apache.oozie.service.TestRecoveryService)
testConnectionDrop(org.apache.oozie.jms.TestJMSJobEventListener)
testPartitionDependency(org.apache.oozie.service.TestPartitionDependencyManagerEhcache)

Tests failing with errors:
testConfigDefaultPropsToAction(org.apache.oozie.action.hadoop.TestMapReduceActionExecutor)
testConnectionRetry(org.apache.oozie.service.TestJMSAccessorService)

Tests failed at first run:
TestJavaActionExecutor#testCredentialsSkip
For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
+1 DISTRO
+1 distro tarball builds with the patch 


-1 Overall result, please check the reported -1(s)

 There is at least one warning, please check

The full output of the test-patch run is available at

 https://builds.apache.org/job/PreCommit-OOZIE-Build/386/

Adding comment to JIRA
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 
0100  44010 0  100  4401  0   3110  0:00:01  0:00:01 --:--:--  
3112{"self":"https://issues.apache.org/jira/rest/api/2/issue/13049362/comment/16360733","id":"16360733","author":{"self":"https://issues.apache.org/jira/rest/api/2/user?username=hadoopqa","name":"hadoopqa","key":"hadoopqa","emailAddress":"blackhole
 at hadoop dot apache dot 
org","avatarUrls":{"48x48":"https://issues.apache.org/jira/secure/useravatar?ownerId=hadoopqa&avatarId=10393","24x24":"https://issues.apache.org/jira/secure/useravatar?size=small&ownerId=hadoopqa&avatarId=10393","16x16":"https://issues.apache.org/jira/secure/useravatar?size=xsmall&ownerId=hadoopqa&avatarId=10393","32x32":"https://issues.apache.org/jira/secure/useravatar?size=medium&ownerId=hadoopqa&avatarId=10393"},"displayName":"Hadoop
 QA","active":true,"timeZone":"Etc/UTC"},"body":"\nTesting JIRA 
OOZIE-2814\n\nCleaning local git 
wo

[jira] [Commented] (OOZIE-2814) OYA: Update example workflows to newest schemas

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16360733#comment-16360733
 ] 

Hadoop QA commented on OOZIE-2814:
--


Testing JIRA OOZIE-2814

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:green}+1{color} the patch adds/modifies 2 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
.{color:red}WARNING{color}: the current HEAD has 100 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [server].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 2095
.Tests failed: 11
.Tests errors: 2

.The patch failed the following testcases:

testParserDefaultJobTrackerFail(org.apache.oozie.workflow.lite.TestLiteWorkflowAppParser)
testParserGlobalExtensionActions(org.apache.oozie.workflow.lite.TestLiteWorkflowAppParser)
testParserDefaultJobTrackerWithGlobal(org.apache.oozie.workflow.lite.TestLiteWorkflowAppParser)
testParserGlobalLocalAlreadyExists(org.apache.oozie.workflow.lite.TestLiteWorkflowAppParser)
testParserDefaultJobTracker(org.apache.oozie.workflow.lite.TestLiteWorkflowAppParser)
testParserGlobalJobXML(org.apache.oozie.workflow.lite.TestLiteWorkflowAppParser)
testParserGlobal(org.apache.oozie.workflow.lite.TestLiteWorkflowAppParser)
testPartitionDependency(org.apache.oozie.service.TestPartitionDependencyManagerService)
testCoordActionRecoveryServiceForWaitingRegisterPartition(org.apache.oozie.service.TestRecoveryService)
testConnectionDrop(org.apache.oozie.jms.TestJMSJobEventListener)
testPartitionDependency(org.apache.oozie.service.TestPartitionDependencyManagerEhcache)

.Tests failing with errors:
testConfigDefaultPropsToAction(org.apache.oozie.action.hadoop.TestMapReduceActionExecutor)
testConnectionRetry(org.apache.oozie.service.TestJMSAccessorService)

.{color:orange}Tests failed at first run:{color}
TestJavaActionExecutor#testCredentialsSkip
.For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}

{color:red}. There is at least one warning, please check{color}

The full output of the test-patch run is available at

. https://builds.apache.org/job/PreCommit-OOZIE-Build/386/



> OYA: Update example workflows to newest schemas
> ---
>
> Key: OOZIE-2814
> URL: https://issues.apache.org/jira/browse/OOZIE-2814
> Project: Oozie
>  Issue Type: Sub-task
>Affects Versions: 5.0.0
>Reporter: Robert Kanter
>Assignee: Attila Sasvari
>Priority: Blocker
> Attachments: OOZIE-2814-0

[jira] [Commented] (OOZIE-3180) Oozie Coordinator does not allow us to override hadoop credential provider factory at run time

[Steve Loughran (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16360741#comment-16360741
 ] 

Steve Loughran commented on OOZIE-3180:
---

HADOOP-14507 might help here

> Oozie Coordinator does not allow us to override hadoop credential provider 
> factory at run time
> --
>
> Key: OOZIE-3180
> URL: https://issues.apache.org/jira/browse/OOZIE-3180
> Project: Oozie
>  Issue Type: Improvement
>  Components: coordinator
>Affects Versions: 4.2.0, 4.3.0
>Reporter: Mahadevan Krishnan
>Priority: Major
>
> It looks like we cannot override 
> _hadoop_.security._credential_._provider_._path_ defined in oozie-site.xml at 
> run time per application basis without making changes to oozie-site.xml and 
> restarting oozie service?
>   
>   I am interested in defining my coordinator like this where I need to 
> monitor for availability of file in the S3 bucket which is controlled by 
> access key and secret access key. This access key and secret access key is 
> maintained in application specific JCEK file and hence I cannot define 
> _hadoop_.security._credential_._provider_._path_ in core-site.xml or 
> oozie-site.xml and I will need to pass this property dynamically for each 
> coordinator. I believe this flexibility is not there. It will be good to add 
> this feature
>   
>  
>   initial-instance="2015-05-10T15:25Z" timezone="Europe/Zurich">
>  
> s3a://mybucket/a/b/${YEAR}/${MONTH}/${DAY}/Success.txt
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (OOZIE-2814) OYA: Update example workflows to newest schemas

[Andras Piros (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16360766#comment-16360766
 ] 

Andras Piros commented on OOZIE-2814:
-

[~asasvari] the failing tests within {{TestLiteWorkflowAppParser}} seem to be 
related.

> OYA: Update example workflows to newest schemas
> ---
>
> Key: OOZIE-2814
> URL: https://issues.apache.org/jira/browse/OOZIE-2814
> Project: Oozie
>  Issue Type: Sub-task
>Affects Versions: 5.0.0
>Reporter: Robert Kanter
>Assignee: Attila Sasvari
>Priority: Blocker
> Attachments: OOZIE-2814-00.patch, OOZIE-2814-01.patch, 
> OOZIE-2814-02.patch
>
>
> OOZIE-2687 will add in a whole set of newer schemas that are more 
> Yarn-centric.  Most, if not all, examples currently use pretty old versions 
> of the schemas.  We should take this opportunity to update the examples to 
> the newest schemas added by OOZIE-2687, which should help demonstrate them.
> It may make sense to keep the older schemas too, to show that older workflows 
> still work.  We have some examples where we have two workflow XML files that 
> do the same thing in different ways.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (OOZIE-3061) Kill only those child jobs which are not already killed

[Mate Juhasz (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-3061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mate Juhasz updated OOZIE-3061:
---
Attachment: OOZIE-3061-004.patch

> Kill only those child jobs which are not already killed
> ---
>
> Key: OOZIE-3061
> URL: https://issues.apache.org/jira/browse/OOZIE-3061
> Project: Oozie
>  Issue Type: Bug
>Affects Versions: 4.3.0
>Reporter: Satish Subhashrao Saley
>Assignee: Mate Juhasz
>Priority: Trivial
>  Labels: newbie, newbiee
> Fix For: 5.0.0
>
> Attachments: OOZIE-3061-001.patch, OOZIE-3061-002.patch, 
> OOZIE-3061-003.patch, OOZIE-3061-004.patch
>
>
> Here we kill all child jobs. 
> https://github.com/apache/oozie/blob/master/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java#L265
> We should check before killing for already killed application.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (OOZIE-3061) Kill only those child jobs which are not already killed

[Mate Juhasz (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16360807#comment-16360807
 ] 

Mate Juhasz commented on OOZIE-3061:


That makes sense, thanks for pointing out! I have attached a patch for both 
classes.

> Kill only those child jobs which are not already killed
> ---
>
> Key: OOZIE-3061
> URL: https://issues.apache.org/jira/browse/OOZIE-3061
> Project: Oozie
>  Issue Type: Bug
>Affects Versions: 4.3.0
>Reporter: Satish Subhashrao Saley
>Assignee: Mate Juhasz
>Priority: Trivial
>  Labels: newbie, newbiee
> Fix For: 5.0.0
>
> Attachments: OOZIE-3061-001.patch, OOZIE-3061-002.patch, 
> OOZIE-3061-003.patch, OOZIE-3061-004.patch
>
>
> Here we kill all child jobs. 
> https://github.com/apache/oozie/blob/master/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java#L265
> We should check before killing for already killed application.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (OOZIE-3061) Kill only those child jobs which are not already killed

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16360811#comment-16360811
 ] 

Hadoop QA commented on OOZIE-3061:
--

PreCommit-OOZIE-Build started


> Kill only those child jobs which are not already killed
> ---
>
> Key: OOZIE-3061
> URL: https://issues.apache.org/jira/browse/OOZIE-3061
> Project: Oozie
>  Issue Type: Bug
>Affects Versions: 4.3.0
>Reporter: Satish Subhashrao Saley
>Assignee: Mate Juhasz
>Priority: Trivial
>  Labels: newbie, newbiee
> Fix For: 5.0.0
>
> Attachments: OOZIE-3061-001.patch, OOZIE-3061-002.patch, 
> OOZIE-3061-003.patch, OOZIE-3061-004.patch
>
>
> Here we kill all child jobs. 
> https://github.com/apache/oozie/blob/master/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java#L265
> We should check before killing for already killed application.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (OOZIE-3172) Upgrade non-transitive Jackson dependencies from org.codehaus.jackson to com.fasterxml.jackson

[Andras Piros (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-3172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Piros updated OOZIE-3172:

Attachment: OOZIE-3172.003.patch

> Upgrade non-transitive Jackson dependencies from org.codehaus.jackson to 
> com.fasterxml.jackson
> --
>
> Key: OOZIE-3172
> URL: https://issues.apache.org/jira/browse/OOZIE-3172
> Project: Oozie
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 5.0.0b1
>Reporter: Andras Piros
>Assignee: Andras Piros
>Priority: Major
> Attachments: OOZIE-3172.001.patch, OOZIE-3172.002.patch, 
> OOZIE-3172.003.patch
>
>
> Jackson 1.9.3 is way too old, and has several security vulnerabilities as 
> well. Jackson 2.9.2 covers most of these.
> Let's switch from {{org.codehaus.jackson}} to {{com.fasterxml.jackson}} in 
> Oozie's direct (non-transitive) dependencies.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (OOZIE-3172) Upgrade non-transitive Jackson dependencies from org.codehaus.jackson to com.fasterxml.jackson

[Attila Sasvari (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16360817#comment-16360817
 ] 

Attila Sasvari commented on OOZIE-3172:
---

+1

> Upgrade non-transitive Jackson dependencies from org.codehaus.jackson to 
> com.fasterxml.jackson
> --
>
> Key: OOZIE-3172
> URL: https://issues.apache.org/jira/browse/OOZIE-3172
> Project: Oozie
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 5.0.0b1
>Reporter: Andras Piros
>Assignee: Andras Piros
>Priority: Major
> Attachments: OOZIE-3172.001.patch, OOZIE-3172.002.patch, 
> OOZIE-3172.003.patch
>
>
> Jackson 1.9.3 is way too old, and has several security vulnerabilities as 
> well. Jackson 2.9.2 covers most of these.
> Let's switch from {{org.codehaus.jackson}} to {{com.fasterxml.jackson}} in 
> Oozie's direct (non-transitive) dependencies.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Failed: OOZIE-3061 PreCommit Build #387

[Apache Jenkins Server]

Jira: https://issues.apache.org/jira/browse/OOZIE-3061
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/387/

###
## LAST 100 LINES OF THE CONSOLE 
###
[...truncated 1.70 MB...]
[DEBUG] There are no new bugs found in [tools].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [server].
[INFO] There are no new bugs found totally].
[TRACE] FindBugs diffs checked and reports created
[TRACE] Summary file size is 2253 bytes
[TRACE] Full summary file size is 1314 bytes
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar]
 removed
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5sum]
 removed
  Running test-patch task BACKWARDS_COMPATIBILITY
  Running test-patch task TESTS
  Running test-patch task DISTRO


Testing JIRA OOZIE-3061

Cleaning local git workspace



+1 PATCH_APPLIES
+1 CLEAN
-1 RAW_PATCH_ANALYSIS
+1 the patch does not introduce any @author tags
+1 the patch does not introduce any tabs
+1 the patch does not introduce any trailing spaces
-1 the patch contains 1 line(s) longer than 132 characters
-1 the patch does not add/modify any testcase
+1 RAT
+1 the patch does not seem to introduce new RAT warnings
+1 JAVADOC
+1 the patch does not seem to introduce new Javadoc warnings
WARNING: the current HEAD has 100 Javadoc warning(s)
+1 COMPILE
+1 HEAD compiles
+1 patch compiles
+1 the patch does not seem to introduce new javac warnings
+1 There are no new bugs found in total.
 +1 There are no new bugs found in [docs].
 +1 There are no new bugs found in [webapp].
 +1 There are no new bugs found in [sharelib/distcp].
 +1 There are no new bugs found in [sharelib/hive].
 +1 There are no new bugs found in [sharelib/spark].
 +1 There are no new bugs found in [sharelib/hive2].
 +1 There are no new bugs found in [sharelib/hcatalog].
 +1 There are no new bugs found in [sharelib/streaming].
 +1 There are no new bugs found in [sharelib/pig].
 +1 There are no new bugs found in [sharelib/sqoop].
 +1 There are no new bugs found in [sharelib/oozie].
 +1 There are no new bugs found in [examples].
 +1 There are no new bugs found in [client].
 +1 There are no new bugs found in [core].
 +1 There are no new bugs found in [tools].
 +1 There are no new bugs found in [server].
+1 BACKWARDS_COMPATIBILITY
+1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient 
annotations
+1 the patch does not modify JPA files
-1 TESTS
Tests run: 2092
Tests failed: 3
Tests errors: 1

The patch failed the following testcases:

testPartitionDependency(org.apache.oozie.service.TestPartitionDependencyManagerService)
testCoordActionRecoveryServiceForWaitingRegisterPartition(org.apache.oozie.service.TestRecoveryService)
testPartitionDependency(org.apache.oozie.service.TestPartitionDependencyManagerEhcache)

Tests failing with errors:
testConnectionRetry(org.apache.oozie.service.TestJMSAccessorService)

Tests failed at first run:
TestJavaActionExecutor#testCredentialsSkip
For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
+1 DISTRO
+1 distro tarball builds with the patch 


-1 Overall result, please check the reported -1(s)

 There is at least one warning, please check

The full output of the test-patch run is available at

 https://builds.apache.org/job/PreCommit-OOZIE-Build/387/

Adding comment to JIRA
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0  
0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 
0{"self":"https://issues.apache.org/jira/rest/api/2/issue/13103093/comment/16360979","id":"16360979","author":{"self":"https://issues.apache.org/jira/rest/api/2/user?username=hadoopqa","name":"hadoopqa","key":"hadoopqa","emailAddress":"blackhole
 at hadoop dot apache dot 
org","avatarUrls":{"48x48":"https://issues.apache.org/jira/secure/useravatar?ownerId=hadoopqa&avatarId=10393","24x24":"https://issues.apache.org/jira/secure/useravatar?size=small&ownerId=hadoopqa&avatarId=10393","16x16":"https://issues.apache.org/jira/secure/useravatar?size=xsmall&ownerId=hadoopqa&avatarId=10393","32x32":"https://issues.apache.org/jira/secure/useravatar?size=medium&ownerId=hadoopqa&avatarId=10393"},"displayName":"Hadoop
 QA","active":true,"timeZone":"Etc/UTC"},"body":"\nTesting JIRA 
OOZIE-3061\n\nCleaning local git 
workspace\n\n\n\n{color:green}+1 
PATCH_APPLIES{color}\n{color:green}+1 CLEA

[jira] [Commented] (OOZIE-3061) Kill only those child jobs which are not already killed

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16360979#comment-16360979
 ] 

Hadoop QA commented on OOZIE-3061:
--


Testing JIRA OOZIE-3061

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:red}-1{color} the patch contains 1 line(s) longer than 132 
characters
.{color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
.{color:red}WARNING{color}: the current HEAD has 100 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [server].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 2092
.Tests failed: 3
.Tests errors: 1

.The patch failed the following testcases:

testPartitionDependency(org.apache.oozie.service.TestPartitionDependencyManagerService)
testCoordActionRecoveryServiceForWaitingRegisterPartition(org.apache.oozie.service.TestRecoveryService)
testPartitionDependency(org.apache.oozie.service.TestPartitionDependencyManagerEhcache)

.Tests failing with errors:
testConnectionRetry(org.apache.oozie.service.TestJMSAccessorService)

.{color:orange}Tests failed at first run:{color}
TestJavaActionExecutor#testCredentialsSkip
.For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}

{color:red}. There is at least one warning, please check{color}

The full output of the test-patch run is available at

. https://builds.apache.org/job/PreCommit-OOZIE-Build/387/



> Kill only those child jobs which are not already killed
> ---
>
> Key: OOZIE-3061
> URL: https://issues.apache.org/jira/browse/OOZIE-3061
> Project: Oozie
>  Issue Type: Bug
>Affects Versions: 4.3.0
>Reporter: Satish Subhashrao Saley
>Assignee: Mate Juhasz
>Priority: Trivial
>  Labels: newbie, newbiee
> Fix For: 5.0.0
>
> Attachments: OOZIE-3061-001.patch, OOZIE-3061-002.patch, 
> OOZIE-3061-003.patch, OOZIE-3061-004.patch
>
>
> Here we kill all child jobs. 
> https://github.com/apache/oozie/blob/master/sharelib/oozie/src/main/java/org/apache/oozie/action/hadoop/LauncherMain.java#L265
> We should check before killing for already killed application.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (OOZIE-3172) Upgrade non-transitive Jackson dependencies from org.codehaus.jackson to com.fasterxml.jackson

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16360981#comment-16360981
 ] 

Hadoop QA commented on OOZIE-3172:
--

PreCommit-OOZIE-Build started


> Upgrade non-transitive Jackson dependencies from org.codehaus.jackson to 
> com.fasterxml.jackson
> --
>
> Key: OOZIE-3172
> URL: https://issues.apache.org/jira/browse/OOZIE-3172
> Project: Oozie
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 5.0.0b1
>Reporter: Andras Piros
>Assignee: Andras Piros
>Priority: Major
> Attachments: OOZIE-3172.001.patch, OOZIE-3172.002.patch, 
> OOZIE-3172.003.patch
>
>
> Jackson 1.9.3 is way too old, and has several security vulnerabilities as 
> well. Jackson 2.9.2 covers most of these.
> Let's switch from {{org.codehaus.jackson}} to {{com.fasterxml.jackson}} in 
> Oozie's direct (non-transitive) dependencies.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 65606: OOZIE-2814 OYA: Update example workflows to newest schemas

[Attila Sasvari via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65606/
---

(Updated Feb. 12, 2018, 4:19 p.m.)


Review request for oozie.


Repository: oozie-git


Description
---

OOZIE-2687 will add in a whole set of newer schemas that are more Yarn-centric. 
Most, if not all, examples currently use pretty old versions of the schemas. We 
should take this opportunity to update the examples to the newest schemas added 
by OOZIE-2687, which should help demonstrate them.

It may make sense to keep the older schemas too, to show that older workflows 
still work. We have some examples where we have two workflow XML files that do 
the same thing in different ways.


Diffs (updated)
-

  core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
bc0f4050bb3788988d3a5352a15220fbc4fa219d 
  core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 
c236dafa5965403c33e1a400509e7c374fe2637c 
  core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 
1cc569d8d7a0e61905ad6c4e940cefb00da1343d 
  
core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java
 2cdbde4d2fc86a07b35beadc56012c438baf8d54 
  examples/src/main/apps/aggregator/coordinator-with-offset.xml 
4c351677f4afc6e85f807106d426bb2f922a28b1 
  examples/src/main/apps/aggregator/coordinator.xml 
948edc1adbe09dfb6f7bffb5eebc82f3670658cc 
  examples/src/main/apps/aggregator/job-with-offset.properties 
9a28ae13e50c40f70db4b0f583a63eb42f97388d 
  examples/src/main/apps/aggregator/job.properties 
aaf4edf7b9d030b5044610698b27a564b008f77b 
  examples/src/main/apps/aggregator/workflow.xml 
4a801090e31acba23ea49da1c59a8ce4b951c654 
  examples/src/main/apps/bundle/job.properties 
c06edeb3f42e2eba3367b9486982dabb4caf6436 
  examples/src/main/apps/coord-input-logic/job.properties 
2cd137b94999c7e1f68bb54928f1c3ce2838d59d 
  examples/src/main/apps/coord-input-logic/workflow.xml 
80d31457eff6e78710ad1fb4d4f1b372c0d1e43c 
  examples/src/main/apps/cron-schedule/coordinator.xml 
48a7de76245e02397b9736a95fa0132504067f9a 
  examples/src/main/apps/cron-schedule/job.properties 
5b5eb43ca2bf9d8cd1b263ca3b94a552b562a20d 
  examples/src/main/apps/cron-schedule/workflow.xml 
b53922c8f40e691cb637b91cb182172794296a04 
  examples/src/main/apps/cron/coordinator.xml 
744d14e870b59d0b89f65a78fea63fffa3ff4b8e 
  examples/src/main/apps/cron/job.properties 
42875fd5a5ce393cd11717bcc55fc1bb91e0fcd3 
  examples/src/main/apps/cron/workflow.xml 
63dff976f9e35a26b0ad09078778748d3d02bc3b 
  examples/src/main/apps/custom-main/job.properties 
b897512a1c6246308ca17b019ce400f3ec06cd7b 
  examples/src/main/apps/custom-main/workflow.xml 
828dfaa0594aac0539c4e08ae8e37cb89d17eb54 
  examples/src/main/apps/datelist-java-main/job.properties 
e8356e3a4a2e1ae41e09f5ea124fb5360bd6e8b6 
  examples/src/main/apps/datelist-java-main/workflow.xml 
b59a12ebd1fc21c9e26f6a5b0348eb9c76375c03 
  examples/src/main/apps/demo/job.properties 
d70d3208df77e0b612005196907cecd97d7c5d21 
  examples/src/main/apps/demo/job.windows.properties 
ea948b20009685e1286053549fc1974669819682 
  examples/src/main/apps/demo/workflow.xml 
0ceca507adc68c0674b794420c9eb733cb320879 
  examples/src/main/apps/distcp/job.properties 
ec9422caddcdecfa6ee9a58868cf68d7457ad4db 
  examples/src/main/apps/distcp/workflow.xml 
6deecb377901613a5bb28fc18bf5beff2d30f252 
  examples/src/main/apps/hadoop-el/job.properties 
1d13c27056ac58ad359b5298f12ec046bb4cc61d 
  examples/src/main/apps/hadoop-el/workflow.xml 
d4094d44136a7de2f85f5c9be042d5aab24b90be 
  examples/src/main/apps/hcatalog/job.properties 
e0aa18c264a2fa92b6c3d501d18fa39733d863ff 
  examples/src/main/apps/hcatalog/job.properties.security 
2fbcc6ee9e5d94d3bba63810e6e788607747940b 
  examples/src/main/apps/hcatalog/workflow.xml 
d6593df1b9104179d79262d663135d73de8b55ec 
  examples/src/main/apps/hcatalog/workflow.xml.security 
aa3522ae4d5b6cb178f80c2e72e45c4c3de9aab5 
  examples/src/main/apps/hive/job.properties 
dbc7ad2a0b135b8c9f0b6e575086ee8260072756 
  examples/src/main/apps/hive/workflow.xml 
449d62d35f2e69d01d9e59f35a0cc9ad057342d2 
  examples/src/main/apps/hive/workflow.xml.security 
35321319dcbb0576c5870952e6e805209b8dc240 
  examples/src/main/apps/hive2/job.properties 
2640402472d3b2120941834e4653d363fb49914b 
  examples/src/main/apps/hive2/job.properties.security 
ac980cd3cc37e3b3b359c2e3fefe180cf88b56bc 
  examples/src/main/apps/hive2/workflow.xml 
5f1ccc4fd0791f850c0942452d637048acce13d1 
  examples/src/main/apps/hive2/workflow.xml.security 
6f7304def80317429e58cecddc63fe4c1d3edaa6 
  examples/src/main/apps/java-main/job.properties 
519c53e7313b6a5b54ab54cb2dd31f123819a0f9 
  examples/src/main/apps/java-main/workflow.xml 
98e01ca92187e1c567686f6c2b4f689bb2a5ef6a 
  examples/src/main/apps/map-reduce/job-with-config-class.properties 
0b14cb749b82fcd165f8db9b1832c650e

[jira] [Updated] (OOZIE-2814) OYA: Update example workflows to newest schemas

[Attila Sasvari (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Attila Sasvari updated OOZIE-2814:
--
Attachment: OOZIE-2814-03.patch

> OYA: Update example workflows to newest schemas
> ---
>
> Key: OOZIE-2814
> URL: https://issues.apache.org/jira/browse/OOZIE-2814
> Project: Oozie
>  Issue Type: Sub-task
>Affects Versions: 5.0.0
>Reporter: Robert Kanter
>Assignee: Attila Sasvari
>Priority: Blocker
> Attachments: OOZIE-2814-00.patch, OOZIE-2814-01.patch, 
> OOZIE-2814-02.patch, OOZIE-2814-03.patch
>
>
> OOZIE-2687 will add in a whole set of newer schemas that are more 
> Yarn-centric.  Most, if not all, examples currently use pretty old versions 
> of the schemas.  We should take this opportunity to update the examples to 
> the newest schemas added by OOZIE-2687, which should help demonstrate them.
> It may make sense to keep the older schemas too, to show that older workflows 
> still work.  We have some examples where we have two workflow XML files that 
> do the same thing in different ways.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Success: OOZIE-3172 PreCommit Build #388

[Apache Jenkins Server]

Jira: https://issues.apache.org/jira/browse/OOZIE-3172
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/388/

###
## LAST 100 LINES OF THE CONSOLE 
###
[...truncated 1.70 MB...]
[DEBUG] There are no new bugs found in [sharelib/pig].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/sqoop].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/oozie].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [examples].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [client].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [core].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [tools].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [server].
[INFO] There are no new bugs found totally].
[TRACE] FindBugs diffs checked and reports created
[TRACE] Summary file size is 2252 bytes
[TRACE] Full summary file size is 1314 bytes
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar]
 removed
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5sum]
 removed
  Running test-patch task BACKWARDS_COMPATIBILITY
  Running test-patch task TESTS
  Running test-patch task DISTRO


Testing JIRA OOZIE-3172

Cleaning local git workspace



+1 PATCH_APPLIES
+1 CLEAN
+1 RAW_PATCH_ANALYSIS
+1 the patch does not introduce any @author tags
+1 the patch does not introduce any tabs
+1 the patch does not introduce any trailing spaces
+1 the patch does not introduce any line longer than 132
+1 the patch adds/modifies 2 testcase(s)
+1 RAT
+1 the patch does not seem to introduce new RAT warnings
+1 JAVADOC
+1 the patch does not seem to introduce new Javadoc warnings
WARNING: the current HEAD has 100 Javadoc warning(s)
+1 COMPILE
+1 HEAD compiles
+1 patch compiles
+1 the patch does not seem to introduce new javac warnings
+1 There are no new bugs found in total.
 +1 There are no new bugs found in [docs].
 +1 There are no new bugs found in [webapp].
 +1 There are no new bugs found in [sharelib/distcp].
 +1 There are no new bugs found in [sharelib/hive].
 +1 There are no new bugs found in [sharelib/spark].
 +1 There are no new bugs found in [sharelib/hive2].
 +1 There are no new bugs found in [sharelib/hcatalog].
 +1 There are no new bugs found in [sharelib/streaming].
 +1 There are no new bugs found in [sharelib/pig].
 +1 There are no new bugs found in [sharelib/sqoop].
 +1 There are no new bugs found in [sharelib/oozie].
 +1 There are no new bugs found in [examples].
 +1 There are no new bugs found in [client].
 +1 There are no new bugs found in [core].
 +1 There are no new bugs found in [tools].
 +1 There are no new bugs found in [server].
+1 BACKWARDS_COMPATIBILITY
+1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient 
annotations
+1 the patch does not modify JPA files
+1 TESTS
Tests run: 2092
Tests failed at first run:
TestJavaActionExecutor#testCredentialsSkip
TestJMSJobEventListener#testConnectionDrop
For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
+1 DISTRO
+1 distro tarball builds with the patch 


+1 Overall result, good!, no -1s

 There is at least one warning, please check

The full output of the test-patch run is available at

 https://builds.apache.org/job/PreCommit-OOZIE-Build/388/

Adding comment to JIRA
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 
0100  31670 0  100  3167  0   3459 --:--:-- --:--:-- --:--:--  
3457{"self":"https://issues.apache.org/jira/rest/api/2/issue/13134446/comment/16361149","id":"16361149","author":{"self":"https://issues.apache.org/jira/rest/api/2/user?username=hadoopqa","name":"hadoopqa","key":"hadoopqa","emailAddress":"blackhole
 at hadoop dot apache dot 
org","avatarUrls":{"48x48":"https://issues.apache.org/jira/secure/useravatar?ownerId=hadoopqa&avatarId=10393","24x24":"https://issues.apache.org/jira/secure/useravatar?size=small&ownerId=hadoopqa&avatarId=10393","16x16":"https://issues.apache.org/jira/secure/useravatar?size=xsmall&ownerId=hado

[jira] [Commented] (OOZIE-3172) Upgrade non-transitive Jackson dependencies from org.codehaus.jackson to com.fasterxml.jackson

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16361149#comment-16361149
 ] 

Hadoop QA commented on OOZIE-3172:
--


Testing JIRA OOZIE-3172

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:green}+1{color} the patch adds/modifies 2 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
.{color:red}WARNING{color}: the current HEAD has 100 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [server].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 2092
.{color:orange}Tests failed at first run:{color}
TestJavaActionExecutor#testCredentialsSkip
TestJMSJobEventListener#testConnectionDrop
.For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:green}*+1 Overall result, good!, no -1s*{color}

{color:red}. There is at least one warning, please check{color}

The full output of the test-patch run is available at

. https://builds.apache.org/job/PreCommit-OOZIE-Build/388/



> Upgrade non-transitive Jackson dependencies from org.codehaus.jackson to 
> com.fasterxml.jackson
> --
>
> Key: OOZIE-3172
> URL: https://issues.apache.org/jira/browse/OOZIE-3172
> Project: Oozie
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 5.0.0b1
>Reporter: Andras Piros
>Assignee: Andras Piros
>Priority: Major
> Attachments: OOZIE-3172.001.patch, OOZIE-3172.002.patch, 
> OOZIE-3172.003.patch
>
>
> Jackson 1.9.3 is way too old, and has several security vulnerabilities as 
> well. Jackson 2.9.2 covers most of these.
> Let's switch from {{org.codehaus.jackson}} to {{com.fasterxml.jackson}} in 
> Oozie's direct (non-transitive) dependencies.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (OOZIE-2814) OYA: Update example workflows to newest schemas

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16361150#comment-16361150
 ] 

Hadoop QA commented on OOZIE-2814:
--

PreCommit-OOZIE-Build started


> OYA: Update example workflows to newest schemas
> ---
>
> Key: OOZIE-2814
> URL: https://issues.apache.org/jira/browse/OOZIE-2814
> Project: Oozie
>  Issue Type: Sub-task
>Affects Versions: 5.0.0
>Reporter: Robert Kanter
>Assignee: Attila Sasvari
>Priority: Blocker
> Attachments: OOZIE-2814-00.patch, OOZIE-2814-01.patch, 
> OOZIE-2814-02.patch, OOZIE-2814-03.patch
>
>
> OOZIE-2687 will add in a whole set of newer schemas that are more 
> Yarn-centric.  Most, if not all, examples currently use pretty old versions 
> of the schemas.  We should take this opportunity to update the examples to 
> the newest schemas added by OOZIE-2687, which should help demonstrate them.
> It may make sense to keep the older schemas too, to show that older workflows 
> still work.  We have some examples where we have two workflow XML files that 
> do the same thing in different ways.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (OOZIE-3172) Upgrade non-transitive Jackson dependencies from org.codehaus.jackson to com.fasterxml.jackson

[Andras Piros (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16361175#comment-16361175
 ] 

Andras Piros commented on OOZIE-3172:
-

Thanks [~asasvari] for the review! [~gezapeti] can you please have another 
review?

> Upgrade non-transitive Jackson dependencies from org.codehaus.jackson to 
> com.fasterxml.jackson
> --
>
> Key: OOZIE-3172
> URL: https://issues.apache.org/jira/browse/OOZIE-3172
> Project: Oozie
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 5.0.0b1
>Reporter: Andras Piros
>Assignee: Andras Piros
>Priority: Major
> Attachments: OOZIE-3172.001.patch, OOZIE-3172.002.patch, 
> OOZIE-3172.003.patch
>
>
> Jackson 1.9.3 is way too old, and has several security vulnerabilities as 
> well. Jackson 2.9.2 covers most of these.
> Let's switch from {{org.codehaus.jackson}} to {{com.fasterxml.jackson}} in 
> Oozie's direct (non-transitive) dependencies.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Success: OOZIE-2814 PreCommit Build #389

[Apache Jenkins Server]

Jira: https://issues.apache.org/jira/browse/OOZIE-2814
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/389/

###
## LAST 100 LINES OF THE CONSOLE 
###
[...truncated 1.69 MB...]
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/streaming].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/pig].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/sqoop].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/oozie].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [examples].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [client].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [core].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [tools].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [server].
[INFO] There are no new bugs found totally].
[TRACE] FindBugs diffs checked and reports created
[TRACE] Summary file size is 2252 bytes
[TRACE] Full summary file size is 1314 bytes
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar]
 removed
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5sum]
 removed
  Running test-patch task BACKWARDS_COMPATIBILITY
  Running test-patch task TESTS
  Running test-patch task DISTRO


Testing JIRA OOZIE-2814

Cleaning local git workspace



+1 PATCH_APPLIES
+1 CLEAN
+1 RAW_PATCH_ANALYSIS
+1 the patch does not introduce any @author tags
+1 the patch does not introduce any tabs
+1 the patch does not introduce any trailing spaces
+1 the patch does not introduce any line longer than 132
+1 the patch adds/modifies 2 testcase(s)
+1 RAT
+1 the patch does not seem to introduce new RAT warnings
+1 JAVADOC
+1 the patch does not seem to introduce new Javadoc warnings
WARNING: the current HEAD has 100 Javadoc warning(s)
+1 COMPILE
+1 HEAD compiles
+1 patch compiles
+1 the patch does not seem to introduce new javac warnings
+1 There are no new bugs found in total.
 +1 There are no new bugs found in [docs].
 +1 There are no new bugs found in [webapp].
 +1 There are no new bugs found in [sharelib/distcp].
 +1 There are no new bugs found in [sharelib/hive].
 +1 There are no new bugs found in [sharelib/spark].
 +1 There are no new bugs found in [sharelib/hive2].
 +1 There are no new bugs found in [sharelib/hcatalog].
 +1 There are no new bugs found in [sharelib/streaming].
 +1 There are no new bugs found in [sharelib/pig].
 +1 There are no new bugs found in [sharelib/sqoop].
 +1 There are no new bugs found in [sharelib/oozie].
 +1 There are no new bugs found in [examples].
 +1 There are no new bugs found in [client].
 +1 There are no new bugs found in [core].
 +1 There are no new bugs found in [tools].
 +1 There are no new bugs found in [server].
+1 BACKWARDS_COMPATIBILITY
+1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient 
annotations
+1 the patch does not modify JPA files
+1 TESTS
Tests run: 1241
+1 DISTRO
+1 distro tarball builds with the patch 


+1 Overall result, good!, no -1s

 There is at least one warning, please check

The full output of the test-patch run is available at

 https://builds.apache.org/job/PreCommit-OOZIE-Build/389/

Adding comment to JIRA
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 0  
0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 
0{"self":"https://issues.apache.org/jira/rest/api/2/issue/13049362/comment/16361254","id":"16361254","author":{"self":"https://issues.apache.org/jira/rest/api/2/user?username=hadoopqa","name":"hadoopqa","key":"hadoopqa","emailAddress":"blackhole
 at hadoop dot apache dot 
org","avatarUrls":{"48x48":"https://issues.apache.org/jira/secure/useravatar?ownerId=hadoopqa&avatarId=10393","24x24":"https://issues.apache.org/jira/secure/useravatar?size=small&ownerId=hadoopqa&avatarId=10393","16x16":"https://issues.apache.org/jira/secure/

[jira] [Commented] (OOZIE-2814) OYA: Update example workflows to newest schemas

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16361254#comment-16361254
 ] 

Hadoop QA commented on OOZIE-2814:
--


Testing JIRA OOZIE-2814

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than 
132
.{color:green}+1{color} the patch adds/modifies 2 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
.{color:red}WARNING{color}: the current HEAD has 100 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [server].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 1241
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:green}*+1 Overall result, good!, no -1s*{color}

{color:red}. There is at least one warning, please check{color}

The full output of the test-patch run is available at

. https://builds.apache.org/job/PreCommit-OOZIE-Build/389/



> OYA: Update example workflows to newest schemas
> ---
>
> Key: OOZIE-2814
> URL: https://issues.apache.org/jira/browse/OOZIE-2814
> Project: Oozie
>  Issue Type: Sub-task
>Affects Versions: 5.0.0
>Reporter: Robert Kanter
>Assignee: Attila Sasvari
>Priority: Blocker
> Attachments: OOZIE-2814-00.patch, OOZIE-2814-01.patch, 
> OOZIE-2814-02.patch, OOZIE-2814-03.patch
>
>
> OOZIE-2687 will add in a whole set of newer schemas that are more 
> Yarn-centric.  Most, if not all, examples currently use pretty old versions 
> of the schemas.  We should take this opportunity to update the examples to 
> the newest schemas added by OOZIE-2687, which should help demonstrate them.
> It may make sense to keep the older schemas too, to show that older workflows 
> still work.  We have some examples where we have two workflow XML files that 
> do the same thing in different ways.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 65606: OOZIE-2814 OYA: Update example workflows to newest schemas

[András Piros via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65606/#review197301
---




core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
Lines 266-271 (patched)


What happens if both `resource-manager` and `job-tracker` are filled? I'd 
either separate `resourceManager` and `jobTracker` into two variables / fields, 
or have some more clean, generic labelling for the same attribute.



core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java
Line 460 (original), 465-467 (patched)


This is pretty hard to understand. I'd either separate `resourceManager` 
and `jobTracker` into two variables / fields, or have some more clean, generic 
labelling for the same attribute.



core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java
Lines 472-478 (patched)


I'd either separate `resourceManager` and `jobTracker` into two variables / 
fields, or have some more clean, generic labelling for the same attribute.



core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java
Lines 587-594 (patched)


I'd either separate `resourceManager` and `jobTracker` into two variables / 
fields, or have some more clean, generic labelling for the same attribute.



core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java
Lines 390 (patched)


Typo: `testSimplestSubmitWithResourceManagerOK()`



core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java
Line 510 (original), 510 (patched)


Isn't it rather an XML validation error? The schema itself is quite correct.



core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java
Lines 1680 (patched)


Would give a catchier name and remove comment.

What about more test cases:

* both `resource-manager` and `job-tracker` present at the same level
* both `resource-manager` and `job-tracker` present at different levels
* none of these are present



core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java
Lines 1681 (patched)


`testWorkflowWithGlobalLevelResourceManager()`



core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java
Lines 1681-1730 (patched)


In fact, the only difference between both methods seem the workflow XML to 
me. Extract to a base method and feed w/ two different workflow XMLs seem a 
viable alternative to me.



core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java
Lines 1682-1684 (patched)


Extract method `newLiteWorkflowAppParser()`



core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java
Lines 1708 (patched)


`testWorkflowWithActionLevelResourceManager()`



core/src/test/java/org/apache/oozie/workflow/lite/TestLiteWorkflowAppParser.java
Lines 1709-1711 (patched)


Extract method `newLiteWorkflowAppParser()`


- András Piros


On Feb. 12, 2018, 4:19 p.m., Attila Sasvari wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65606/
> ---
> 
> (Updated Feb. 12, 2018, 4:19 p.m.)
> 
> 
> Review request for oozie.
> 
> 
> Repository: oozie-git
> 
> 
> Description
> ---
> 
> OOZIE-2687 will add in a whole set of newer schemas that are more 
> Yarn-centric. Most, if not all, examples currently use pretty old versions of 
> the schemas. We should take this opportunity to update the examples to the 
> newest schemas added by OOZIE-2687, which should help demonstrate them.
> 
> It may make sense to keep the older schemas too, to show that older workflows 
> still work. We have some examples where we have two workflow XML files that 
> do the same thing in different ways.
> 
> 
> Diffs
> -
> 
>   core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java 
> bc0f4050bb3788988d3a5352a15220fbc4fa219d 
>   
> core/src/main/java/org/apache/oozie/workflow/lite/LiteWorkflowAppParser.java 
> c236dafa5965403c33e1a400509e7c374fe2637c 
>   
> core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutor.java 
> 1cc569d8d7a0e6

[jira] [Updated] (OOZIE-3179) Adding a configurable config-default.xml location to a workflow

[Jason Phelps (JIRA)]

 [ 
https://issues.apache.org/jira/browse/OOZIE-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Phelps updated OOZIE-3179:

Attachment: OOZIE-3179-1.patch

> Adding a configurable config-default.xml location to a workflow
> ---
>
> Key: OOZIE-3179
> URL: https://issues.apache.org/jira/browse/OOZIE-3179
> Project: Oozie
>  Issue Type: Improvement
>Reporter: Jason Phelps
>Priority: Major
> Attachments: OOZIE-3179-1.patch
>
>
> Recently a customer requested a way to include several job.properties 
> variables to every workflow in their cluster.
> Normally the config-default.xml file would be the best way to include a set 
> of variables, but it does not scale very well if that file needs to be 
> updated, as it would require it to be updated in every workflow application 
> path.
> What I am requesting is another job.properties variable that can point to a 
> config-default.xml file, or a comma separated list of config-default.xml 
> files that will all be applied to the workflow.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (OOZIE-3179) Adding a configurable config-default.xml location to a workflow

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16361559#comment-16361559
 ] 

Hadoop QA commented on OOZIE-3179:
--

PreCommit-OOZIE-Build started


> Adding a configurable config-default.xml location to a workflow
> ---
>
> Key: OOZIE-3179
> URL: https://issues.apache.org/jira/browse/OOZIE-3179
> Project: Oozie
>  Issue Type: Improvement
>Reporter: Jason Phelps
>Priority: Major
> Attachments: OOZIE-3179-1.patch
>
>
> Recently a customer requested a way to include several job.properties 
> variables to every workflow in their cluster.
> Normally the config-default.xml file would be the best way to include a set 
> of variables, but it does not scale very well if that file needs to be 
> updated, as it would require it to be updated in every workflow application 
> path.
> What I am requesting is another job.properties variable that can point to a 
> config-default.xml file, or a comma separated list of config-default.xml 
> files that will all be applied to the workflow.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Failed: OOZIE-3179 PreCommit Build #390

[Apache Jenkins Server]

Jira: https://issues.apache.org/jira/browse/OOZIE-3179
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/390/

###
## LAST 100 LINES OF THE CONSOLE 
###
[...truncated 1.70 MB...]
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [server].
[INFO] There are no new bugs found totally].
[TRACE] FindBugs diffs checked and reports created
[TRACE] Summary file size is 2250 bytes
[TRACE] Full summary file size is 1314 bytes
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar]
 removed
[TRACE] File 
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5sum]
 removed
  Running test-patch task BACKWARDS_COMPATIBILITY
  Running test-patch task TESTS
  Running test-patch task DISTRO


Testing JIRA OOZIE-3179

Cleaning local git workspace



+1 PATCH_APPLIES
+1 CLEAN
-1 RAW_PATCH_ANALYSIS
+1 the patch does not introduce any @author tags
+1 the patch does not introduce any tabs
+1 the patch does not introduce any trailing spaces
-1 the patch contains 1 line(s) longer than 132 characters
+1 the patch adds/modifies 1 testcase(s)
+1 RAT
+1 the patch does not seem to introduce new RAT warnings
+1 JAVADOC
+1 the patch does not seem to introduce new Javadoc warnings
WARNING: the current HEAD has 100 Javadoc warning(s)
+1 COMPILE
+1 HEAD compiles
+1 patch compiles
+1 the patch does not seem to introduce new javac warnings
+1 There are no new bugs found in total.
 +1 There are no new bugs found in [docs].
 +1 There are no new bugs found in [webapp].
 +1 There are no new bugs found in [sharelib/distcp].
 +1 There are no new bugs found in [sharelib/hive].
 +1 There are no new bugs found in [sharelib/spark].
 +1 There are no new bugs found in [sharelib/hive2].
 +1 There are no new bugs found in [sharelib/hcatalog].
 +1 There are no new bugs found in [sharelib/streaming].
 +1 There are no new bugs found in [sharelib/pig].
 +1 There are no new bugs found in [sharelib/sqoop].
 +1 There are no new bugs found in [sharelib/oozie].
 +1 There are no new bugs found in [examples].
 +1 There are no new bugs found in [client].
 +1 There are no new bugs found in [core].
 +1 There are no new bugs found in [tools].
 +1 There are no new bugs found in [server].
+1 BACKWARDS_COMPATIBILITY
+1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient 
annotations
+1 the patch does not modify JPA files
-1 TESTS
Tests run: 2093
Tests failed: 3
Tests errors: 1

The patch failed the following testcases:

testPartitionDependency(org.apache.oozie.service.TestPartitionDependencyManagerService)
testCoordActionRecoveryServiceForWaitingRegisterPartition(org.apache.oozie.service.TestRecoveryService)
testPartitionDependency(org.apache.oozie.service.TestPartitionDependencyManagerEhcache)

Tests failing with errors:
testConnectionRetry(org.apache.oozie.service.TestJMSAccessorService)

Tests failed at first run:
TestJavaActionExecutor#testCredentialsSkip
For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
+1 DISTRO
+1 distro tarball builds with the patch 


-1 Overall result, please check the reported -1(s)

 There is at least one warning, please check

The full output of the test-patch run is available at

 https://builds.apache.org/job/PreCommit-OOZIE-Build/390/

Adding comment to JIRA
  % Total% Received % Xferd  Average Speed   TimeTime Time  Current
 Dload  Upload   Total   SpentLeft  Speed
  0 00 00 0  0  0 --:--:-- --:--:-- --:--:-- 
0100  36210 0  100  3621  0   4300 --:--:-- --:--:-- --:--:--  
4300{"self":"https://issues.apache.org/jira/rest/api/2/issue/13137525/comment/16361638","id":"16361638","author":{"self":"https://issues.apache.org/jira/rest/api/2/user?username=hadoopqa","name":"hadoopqa","key":"hadoopqa","emailAddress":"blackhole
 at hadoop dot apache dot 
org","avatarUrls":{"48x48":"https://issues.apache.org/jira/secure/useravatar?ownerId=hadoopqa&avatarId=10393","24x24":"https://issues.apache.org/jira/secure/useravatar?size=small&ownerId=hadoopqa&avatarId=10393","16x16":"https://issues.apache.org/jira/secure/useravatar?size=xsmall&ownerId=hadoopqa&avatarId=10393","32x32":"https://issues.apache.org/jira/secure/useravatar?size=medium&ownerId=hadoopqa&avatarId=10393"},"displayName":"Hadoop
 QA","active":true,"timeZone":"Etc/UTC"},"body":"\nTesting JIRA 
OOZIE-3179\n\nCleaning local git 
workspace\n\n\n\n{color:green}+1 
PATCH_APPLIES{color}\n{color:green}+1 CLEAN{color}\n{color:red}-1 
RAW_PATCH_ANALYSIS{color}\n.

[jira] [Commented] (OOZIE-3179) Adding a configurable config-default.xml location to a workflow

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/OOZIE-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16361638#comment-16361638
 ] 

Hadoop QA commented on OOZIE-3179:
--


Testing JIRA OOZIE-3179

Cleaning local git workspace



{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:red}-1{color} the patch contains 1 line(s) longer than 132 
characters
.{color:green}+1{color} the patch adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT 
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc 
warnings
.{color:red}WARNING{color}: the current HEAD has 100 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac 
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [server].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA 
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 2093
.Tests failed: 3
.Tests errors: 1

.The patch failed the following testcases:

testPartitionDependency(org.apache.oozie.service.TestPartitionDependencyManagerService)
testCoordActionRecoveryServiceForWaitingRegisterPartition(org.apache.oozie.service.TestRecoveryService)
testPartitionDependency(org.apache.oozie.service.TestPartitionDependencyManagerEhcache)

.Tests failing with errors:
testConnectionRetry(org.apache.oozie.service.TestJMSAccessorService)

.{color:orange}Tests failed at first run:{color}
TestJavaActionExecutor#testCredentialsSkip
.For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch 


{color:red}*-1 Overall result, please check the reported -1(s)*{color}

{color:red}. There is at least one warning, please check{color}

The full output of the test-patch run is available at

. https://builds.apache.org/job/PreCommit-OOZIE-Build/390/



> Adding a configurable config-default.xml location to a workflow
> ---
>
> Key: OOZIE-3179
> URL: https://issues.apache.org/jira/browse/OOZIE-3179
> Project: Oozie
>  Issue Type: Improvement
>Reporter: Jason Phelps
>Priority: Major
> Attachments: OOZIE-3179-1.patch
>
>
> Recently a customer requested a way to include several job.properties 
> variables to every workflow in their cluster.
> Normally the config-default.xml file would be the best way to include a set 
> of variables, but it does not scale very well if that file needs to be 
> updated, as it would require it to be updated in every workflow application 
> path.
> What I am requesting is another job.properties variable that can point to a 
> config-default.xml file, or a comma separated list of config-default.xml 
> files that will all be applied to the workflow.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] Subscription: Oozie Patch Available

[jira]

Issue Subscription
Filter: Oozie Patch Available (105 issues)

Subscriber: ooziedaily

Key Summary
OOZIE-3179  Adding a configurable config-default.xml location to a workflow
https://issues.apache.org/jira/browse/OOZIE-3179
OOZIE-3176  Oozie-core fails with checkstyle errors.
https://issues.apache.org/jira/browse/OOZIE-3176
OOZIE-3172  Upgrade non-transitive Jackson dependencies from 
org.codehaus.jackson to com.fasterxml.jackson
https://issues.apache.org/jira/browse/OOZIE-3172
OOZIE-3135  Configure log4j2 in SqoopMain
https://issues.apache.org/jira/browse/OOZIE-3135
OOZIE-3134  Potential inconsistency between the in-memory SLA map and the Oozie 
database
https://issues.apache.org/jira/browse/OOZIE-3134
OOZIE-3109  Escape log-streaming's HTML-specific characters
https://issues.apache.org/jira/browse/OOZIE-3109
OOZIE-3105  testJMXInstrumentation from the 
org.apache.oozie.util.TestMetricsInstrumentation class is flaky
https://issues.apache.org/jira/browse/OOZIE-3105
OOZIE-3094  fix for grammar mistake
https://issues.apache.org/jira/browse/OOZIE-3094
OOZIE-3091  Oozie Sqoop Avro Import fails with "java.lang.NoClassDefFoundError: 
org/apache/avro/mapred/AvroWrapper"
https://issues.apache.org/jira/browse/OOZIE-3091
OOZIE-3071  Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 
than Spark 2.2.0
https://issues.apache.org/jira/browse/OOZIE-3071
OOZIE-3063  Sanitizing variables that are part of openjpa.ConnectionProperties
https://issues.apache.org/jira/browse/OOZIE-3063
OOZIE-3062  Set HADOOP_CONF_DIR for spark action
https://issues.apache.org/jira/browse/OOZIE-3062
OOZIE-3061  Kill only those child jobs which are not already killed
https://issues.apache.org/jira/browse/OOZIE-3061
OOZIE-3002  address findbugs errors in client lib
https://issues.apache.org/jira/browse/OOZIE-3002
OOZIE-2975  code clean up in pig sharelib, replace Exception with more 
explicit, add try with resources, StringBuilder instead of StringBuffer
https://issues.apache.org/jira/browse/OOZIE-2975
OOZIE-2957  Documentation states that starting a coordinator is possible
https://issues.apache.org/jira/browse/OOZIE-2957
OOZIE-2956  Fix Findbugs warnings related to reliance on default encoding in 
oozie-core
https://issues.apache.org/jira/browse/OOZIE-2956
OOZIE-2955  Fix Findbugs warnings related to reliance on default encoding in 
oozie-client
https://issues.apache.org/jira/browse/OOZIE-2955
OOZIE-2954  Fix Checkstyle issues in oozie-client
https://issues.apache.org/jira/browse/OOZIE-2954
OOZIE-2953  Fix Checkstyle issues in oozie-tools
https://issues.apache.org/jira/browse/OOZIE-2953
OOZIE-2952  Fix Findbugs warnings in oozie-sharelib-oozie
https://issues.apache.org/jira/browse/OOZIE-2952
OOZIE-2949  Escape quotes whitespaces in Sqoop  field
https://issues.apache.org/jira/browse/OOZIE-2949
OOZIE-2942  Fix Findbugs warnings in oozie-examples
https://issues.apache.org/jira/browse/OOZIE-2942
OOZIE-2937  Remove redundant groupId from the child pom's
https://issues.apache.org/jira/browse/OOZIE-2937
OOZIE-2934  Fix "Exceptional return value of java.io.File.mkdirs() ignored" 
Findbugs error in oozie-sharelib-spark
https://issues.apache.org/jira/browse/OOZIE-2934
OOZIE-2927  Append new line character for Hive2 query using query tag
https://issues.apache.org/jira/browse/OOZIE-2927
OOZIE-2914  Consolidate Trim 
https://issues.apache.org/jira/browse/OOZIE-2914
OOZIE-2883  OOZIE throw the error "Missing 
[oozie.service.ProxyUserService.proxyuser.oozie.service.ProxyUserService.proxyuser.mr.groups]
 property"
https://issues.apache.org/jira/browse/OOZIE-2883
OOZIE-2877  Oozie Git Action
https://issues.apache.org/jira/browse/OOZIE-2877
OOZIE-2867  Timezone handling for Coordinators: emphasize "Continent/City" 
format
https://issues.apache.org/jira/browse/OOZIE-2867
OOZIE-2834  ParameterVerifier logging non-useful warning for workflow definition
https://issues.apache.org/jira/browse/OOZIE-2834
OOZIE-2833  when using uber mode the regex pattern used in the 
extractHeapSizeMB method does not allow heap sizes specified in bytes.
https://issues.apache.org/jira/browse/OOZIE-2833
OOZIE-2829  Improve sharelib upload to accept multiple source folders
https://issues.apache.org/jira/browse/OOZIE-2829
OOZIE-2826  Falcon feed fails to aws s3; Oozie joda time version does not meet 
required jar version 2.2 or later
https://issues.apache.org/jira/browse/OOZIE-2826
OOZIE-2814  OYA: Update example workflows to newest schemas
https://issues.apache.org/jira/browse/OOZIE-2814
OOZIE-2812  SparkConfigurationService should support loading config