[jira] Subscription: Oozie Patch Available
Issue Subscription Filter: Oozie Patch Available (95 issues) Subscriber: ooziedaily Key Summary OOZIE-3401 TestPySpark failure https://issues.apache.org/jira/browse/OOZIE-3401 OOZIE-3397 Improve logging in NotificationXCommand https://issues.apache.org/jira/browse/OOZIE-3397 OOZIE-3375 Can't use empty in coordinator https://issues.apache.org/jira/browse/OOZIE-3375 OOZIE-3367 Using && in EL expressions in oozie bundle.xml files generates parse errors https://issues.apache.org/jira/browse/OOZIE-3367 OOZIE-3366 Update workflow status and subworkflow status on suspend command https://issues.apache.org/jira/browse/OOZIE-3366 OOZIE-3364 Rerunning Oozie bundle jobs starts the coordinators in indeterminate order https://issues.apache.org/jira/browse/OOZIE-3364 OOZIE-3362 When killed, SSH action should kill the spawned processes on target host https://issues.apache.org/jira/browse/OOZIE-3362 OOZIE-3326 Sqoop Action should support tez delegation tokens for hive-import https://issues.apache.org/jira/browse/OOZIE-3326 OOZIE-3320 Oozie ShellAction should support absolute bash file path https://issues.apache.org/jira/browse/OOZIE-3320 OOZIE-3319 Log SSH action callback error output https://issues.apache.org/jira/browse/OOZIE-3319 OOZIE-3301 Update NOTICE file https://issues.apache.org/jira/browse/OOZIE-3301 OOZIE-3274 Remove slf4j https://issues.apache.org/jira/browse/OOZIE-3274 OOZIE-3266 Coord action rerun support RERUN_SKIP_NODES option https://issues.apache.org/jira/browse/OOZIE-3266 OOZIE-3265 properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear together https://issues.apache.org/jira/browse/OOZIE-3265 OOZIE-3256 refactor OozieCLI class https://issues.apache.org/jira/browse/OOZIE-3256 OOZIE-3249 [tools] Instrumentation log parser https://issues.apache.org/jira/browse/OOZIE-3249 OOZIE-3218 Oozie Sqoop action with command splits the select clause into multiple parts due to delimiter being space https://issues.apache.org/jira/browse/OOZIE-3218 OOZIE-3199 Let system property restriction configurable https://issues.apache.org/jira/browse/OOZIE-3199 OOZIE-3196 Authorization: restrict world readability by user https://issues.apache.org/jira/browse/OOZIE-3196 OOZIE-3194 Oozie should set proper permissions to sharelib after upload https://issues.apache.org/jira/browse/OOZIE-3194 OOZIE-3186 Oozie is unable to use configuration linked using jceks://file/... https://issues.apache.org/jira/browse/OOZIE-3186 OOZIE-3179 Adding a configurable config-default.xml location to a workflow https://issues.apache.org/jira/browse/OOZIE-3179 OOZIE-3170 Oozie Diagnostic Bundle tool fails with NPE due to missing service class https://issues.apache.org/jira/browse/OOZIE-3170 OOZIE-3137 Add support for log4j2 in HiveMain https://issues.apache.org/jira/browse/OOZIE-3137 OOZIE-3135 Configure log4j2 in SqoopMain https://issues.apache.org/jira/browse/OOZIE-3135 OOZIE-3091 Oozie Sqoop Avro Import fails with "java.lang.NoClassDefFoundError: org/apache/avro/mapred/AvroWrapper" https://issues.apache.org/jira/browse/OOZIE-3091 OOZIE-3071 Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than Spark 2.2.0 https://issues.apache.org/jira/browse/OOZIE-3071 OOZIE-3063 Sanitizing variables that are part of openjpa.ConnectionProperties https://issues.apache.org/jira/browse/OOZIE-3063 OOZIE-3062 Set HADOOP_CONF_DIR for spark action https://issues.apache.org/jira/browse/OOZIE-3062 OOZIE-2952 Fix Findbugs warnings in oozie-sharelib-oozie https://issues.apache.org/jira/browse/OOZIE-2952 OOZIE-2949 Escape quotes whitespaces in Sqoop field https://issues.apache.org/jira/browse/OOZIE-2949 OOZIE-2927 Append new line character for Hive2 query using query tag https://issues.apache.org/jira/browse/OOZIE-2927 OOZIE-2834 ParameterVerifier logging non-useful warning for workflow definition https://issues.apache.org/jira/browse/OOZIE-2834 OOZIE-2833 when using uber mode the regex pattern used in the extractHeapSizeMB method does not allow heap sizes specified in bytes. https://issues.apache.org/jira/browse/OOZIE-2833 OOZIE-2812 SparkConfigurationService should support loading configurations from multiple Spark versions https://issues.apache.org/jira/browse/OOZIE-2812 OOZIE-2795 Create lib directory or symlink for Oozie CLI during packaging https://issues.apache.org/jira/browse/OOZIE-2795 OOZIE-2784 Include WEEK as a parameter in the Coordinator Expression Language Evaulator https://issues.apache.org/jira/browse/OOZIE-2784 OOZIE-2779 Mask
[jira] [Commented] (OOZIE-3397) Improve logging in NotificationXCommand
[
https://issues.apache.org/jira/browse/OOZIE-3397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16723190#comment-16723190
]
Hadoop QA commented on OOZIE-3397:
--
Testing JIRA OOZIE-3397
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any star imports
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} Javadoc generation succeeded with the patch
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warning(s)
.{color:orange}WARNING{color}: the current HEAD has 100 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/git].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in
[fluent-job/fluent-job-api].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [core].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 3099
.{color:orange}Tests failed at first run:{color}
TestPurgeXCommand#testPurgeBundleWithCoordChildWithWFChild3MoreThanLimit
.For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
{color:red}. There is at least one warning, please check{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/946/
> Improve logging in NotificationXCommand
> ---
>
> Key: OOZIE-3397
> URL: https://issues.apache.org/jira/browse/OOZIE-3397
> Project: Oozie
> Issue Type: Improvement
> Components: core
>Affects Versions: trunk
>Reporter: Julia Kinga Marton
>Assignee: Andras Salamon
>Priority: Major
> Fix For: 5.2.0
>
> Attachments: OOZIE-3397-amend-01-01.patch, OOZIE-3397.001.patch,
> OOZIE-3397.002.patch
>
>
> Around the notification sending (NotificationXCommand) there is not so much
> logging. For example if the HTTP call fails, the error is suppressed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Failed: OOZIE-3397 PreCommit Build #946
Jira: https://issues.apache.org/jira/browse/OOZIE-3397
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/946/
###
## LAST 100 LINES OF THE CONSOLE
###
[...truncated 1.91 MB...]
[DEBUG] There are no new bugs found in [fluent-job/fluent-job-api].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [server].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [webapp].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [examples].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [core].
[INFO] There are no new bugs found totally].
[TRACE] FindBugs diffs checked and reports created
[TRACE] Summary file size is 2561 bytes
[TRACE] Full summary file size is 1471 bytes
[TRACE] File
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar]
removed
[TRACE] File
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5sum]
removed
Running test-patch task BACKWARDS_COMPATIBILITY
Running test-patch task TESTS
Running test-patch task DISTRO
Testing JIRA OOZIE-3397
Cleaning local git workspace
+1 PATCH_APPLIES
+1 CLEAN
-1 RAW_PATCH_ANALYSIS
+1 the patch does not introduce any @author tags
+1 the patch does not introduce any tabs
+1 the patch does not introduce any trailing spaces
+1 the patch does not introduce any star imports
+1 the patch does not introduce any line longer than 132
-1 the patch does not add/modify any testcase
+1 RAT
+1 the patch does not seem to introduce new RAT warnings
+1 JAVADOC
+1 Javadoc generation succeeded with the patch
+1 the patch does not seem to introduce new Javadoc warning(s)
WARNING: the current HEAD has 100 Javadoc warning(s)
+1 COMPILE
+1 HEAD compiles
+1 patch compiles
+1 the patch does not seem to introduce new javac warnings
+1 There are no new bugs found in total.
+1 There are no new bugs found in [sharelib/hive2].
+1 There are no new bugs found in [sharelib/spark].
+1 There are no new bugs found in [sharelib/oozie].
+1 There are no new bugs found in [sharelib/pig].
+1 There are no new bugs found in [sharelib/streaming].
+1 There are no new bugs found in [sharelib/hive].
+1 There are no new bugs found in [sharelib/distcp].
+1 There are no new bugs found in [sharelib/hcatalog].
+1 There are no new bugs found in [sharelib/sqoop].
+1 There are no new bugs found in [sharelib/git].
+1 There are no new bugs found in [client].
+1 There are no new bugs found in [docs].
+1 There are no new bugs found in [tools].
+1 There are no new bugs found in [fluent-job/fluent-job-api].
+1 There are no new bugs found in [server].
+1 There are no new bugs found in [webapp].
+1 There are no new bugs found in [examples].
+1 There are no new bugs found in [core].
+1 BACKWARDS_COMPATIBILITY
+1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient
annotations
+1 the patch does not modify JPA files
+1 TESTS
Tests run: 3099
Tests failed at first run:
TestPurgeXCommand#testPurgeBundleWithCoordChildWithWFChild3MoreThanLimit
For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
+1 DISTRO
+1 distro tarball builds with the patch
-1 Overall result, please check the reported -1(s)
There is at least one warning, please check
The full output of the test-patch run is available at
https://builds.apache.org/job/PreCommit-OOZIE-Build/946/
Adding comment to JIRA
% Total% Received % Xferd Average Speed TimeTime Time Current
Dload Upload Total SpentLeft Speed
0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0
0 00 00 0 0 0 --:--:-- --:--:-- --:--:--
0{"self":"https://issues.apache.org/jira/rest/api/2/issue/13202009/comment/16723190","id":"16723190","author":{"self":"https://issues.apache.org/jira/rest/api/2/user?username=hadoopqa","name":"hadoopqa","key":"hadoopqa","avatarUrls":{"48x48":"https://issues.apache.org/jira/secure/useravatar?ownerId=hadoopqa=10393","24x24":"https://issues.apache.org/jira/secure/useravatar?size=small=hadoopqa=10393","16x16":"https://issues.apache.org/jira/secure/useravatar?size=xsmall=hadoopqa=10393","32x32":"https://issues.apache.org/jira/secure/useravatar?size=medium=hadoopqa=10393"},"displayName":"Hadoop
QA","active":true,"timeZone":"Etc/UTC"},"body":"\nTesting JIRA
OOZIE-3397\n\nCleaning local git
[jira] [Commented] (OOZIE-3397) Improve logging in NotificationXCommand
[ https://issues.apache.org/jira/browse/OOZIE-3397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16723089#comment-16723089 ] Hadoop QA commented on OOZIE-3397: -- PreCommit-OOZIE-Build started > Improve logging in NotificationXCommand > --- > > Key: OOZIE-3397 > URL: https://issues.apache.org/jira/browse/OOZIE-3397 > Project: Oozie > Issue Type: Improvement > Components: core >Affects Versions: trunk >Reporter: Julia Kinga Marton >Assignee: Andras Salamon >Priority: Major > Fix For: 5.2.0 > > Attachments: OOZIE-3397-amend-01-01.patch, OOZIE-3397.001.patch, > OOZIE-3397.002.patch > > > Around the notification sending (NotificationXCommand) there is not so much > logging. For example if the HTTP call fails, the error is suppressed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (OOZIE-3350) Forkjoin validation does not fail if a node is reachable from two different forks
[
https://issues.apache.org/jira/browse/OOZIE-3350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16723087#comment-16723087
]
Peter Bacsko edited comment on OOZIE-3350 at 12/17/18 3:37 PM:
---
[~andras.piros] basically the idea is what I outlined above: for every {{fork}}
node, we maintain a list of nodes seen so far. So, we introduce a map like
{{Map>}}. Every node which is not a fork should
belong to one and only one {{ForkNodeDef}}. If it belongs to two, then we have
a problem.
was (Author: pbacsko):
[~andras.piros] basically the idea is what I outlined above: for every {{fork}}
node, we maintain a list of nodes seen so far. So, we introduce a map like
{{Map}}. Every node which is not a fork should belong
to one and only one {{ForkNodeDef}}. If it belongs to two, then we have a
problem.
> Forkjoin validation does not fail if a node is reachable from two different
> forks
> -
>
> Key: OOZIE-3350
> URL: https://issues.apache.org/jira/browse/OOZIE-3350
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 4.3.1
>Reporter: wang jinyin
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: trunk
>
> Original Estimate: 120h
> Remaining Estimate: 120h
>
> when "multiple ok to same node" under decision node, forkjoin validation
> error.
>
> such as below example, 'action_C' and 'action_D' both transition to
> 'action_E'.
> But, because they are under same topDecisionParent 'decision_A', validator
> will not throw any exception.
>
> {quote}
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3350) Forkjoin validation does not fail if a node is reachable from two different forks
[
https://issues.apache.org/jira/browse/OOZIE-3350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16723087#comment-16723087
]
Peter Bacsko commented on OOZIE-3350:
-
[~andras.piros] basically the idea is what I outlined above: for every {{fork}}
node, we maintain a list of nodes seen so far. So, we introduce a map like
{{Map}}. Every node which is not a fork should belong
to one and only one {{ForkNodeDef}}. If it belongs to two, then we have a
problem.
> Forkjoin validation does not fail if a node is reachable from two different
> forks
> -
>
> Key: OOZIE-3350
> URL: https://issues.apache.org/jira/browse/OOZIE-3350
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 4.3.1
>Reporter: wang jinyin
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: trunk
>
> Original Estimate: 120h
> Remaining Estimate: 120h
>
> when "multiple ok to same node" under decision node, forkjoin validation
> error.
>
> such as below example, 'action_C' and 'action_D' both transition to
> 'action_E'.
> But, because they are under same topDecisionParent 'decision_A', validator
> will not throw any exception.
>
> {quote}
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3397) Improve logging in NotificationXCommand
[ https://issues.apache.org/jira/browse/OOZIE-3397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16723083#comment-16723083 ] Andras Piros commented on OOZIE-3397: - I'm convinced [~asalamon74] we already have the necessary amount of automatic testing on that part of functionality. +1 (pending Jenkins job for amendment patch 01) > Improve logging in NotificationXCommand > --- > > Key: OOZIE-3397 > URL: https://issues.apache.org/jira/browse/OOZIE-3397 > Project: Oozie > Issue Type: Improvement > Components: core >Affects Versions: trunk >Reporter: Julia Kinga Marton >Assignee: Andras Salamon >Priority: Major > Fix For: 5.2.0 > > Attachments: OOZIE-3397-amend-01-01.patch, OOZIE-3397.001.patch, > OOZIE-3397.002.patch > > > Around the notification sending (NotificationXCommand) there is not so much > logging. For example if the HTTP call fails, the error is suppressed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3397) Improve logging in NotificationXCommand
[
https://issues.apache.org/jira/browse/OOZIE-3397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16722769#comment-16722769
]
Andras Salamon commented on OOZIE-3397:
---
[~andras.piros] I think my tests were really testing {{HttpURLConnection}} and
not {{NotificationXCommand}}. {{NotificationXCommand}} has nothing to do with
certificates, we should not add tests like that.
We might add unit tests for the following scenarios:
# Good and fast connection with response code 200.
# Good and slow ( but faster than timeout ) with response code 200
# Good but slow ( slower than timeout ) with response code 200
# Good connection with response code other than 200.
# Bad connection ( {{IOException}} )
To be honest I don't really see much additional value for these tests. The
first and third cases are already tested in
{{TestWorkflowNotificationXCommand}} and
{{TestCoordActionNotificationXCommand}}. {{sendNotification}} has {{void}}
return value and it does not throw an Exception, so if we want to test it we
need to check it we have to check the content of the log which might not be a
good idea. Or we could extract
[this|https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/command/NotificationXCommand.java#L110-L125]
part of the method into a new method and test that method.
> Improve logging in NotificationXCommand
> ---
>
> Key: OOZIE-3397
> URL: https://issues.apache.org/jira/browse/OOZIE-3397
> Project: Oozie
> Issue Type: Improvement
> Components: core
>Affects Versions: trunk
>Reporter: Julia Kinga Marton
>Assignee: Andras Salamon
>Priority: Major
> Fix For: 5.2.0
>
> Attachments: OOZIE-3397-amend-01-01.patch, OOZIE-3397.001.patch,
> OOZIE-3397.002.patch
>
>
> Around the notification sending (NotificationXCommand) there is not so much
> logging. For example if the HTTP call fails, the error is suppressed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
