[jira] [Commented] (OOZIE-3265) properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear together
[ https://issues.apache.org/jira/browse/OOZIE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16494770#comment-16494770 ] TIAN XING commented on OOZIE-3265: -- [~gezapeti] test case added > properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear > together > -- > > Key: OOZIE-3265 > URL: https://issues.apache.org/jira/browse/OOZIE-3265 > Project: Oozie > Issue Type: Task >Affects Versions: 5.0.0 >Reporter: TIAN XING >Assignee: TIAN XING >Priority: Minor > Attachments: OOZIE-3265-v1.patch, OOZIE-3265-v2.patch, rerun.patch > > > Currently when you re-run a workflow with property "oozie.wf.rerun.failnodes" > set to true, > you can no longer re-run it again with "oozie.wf.rerun.skip.nodes" property > specified, even if you set "oozie.wf.rerun.failnodes" to false. > This kind of limitation is not reasonable. There is only one case where > "oozie.wf.rerun.failnodes" is true and "oozie.wf.rerun.skip.nodes" is not > null or empty, that should be disallowed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3265) properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear together
[ https://issues.apache.org/jira/browse/OOZIE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16494740#comment-16494740 ] Hadoop QA commented on OOZIE-3265: -- PreCommit-OOZIE-Build started > properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear > together > -- > > Key: OOZIE-3265 > URL: https://issues.apache.org/jira/browse/OOZIE-3265 > Project: Oozie > Issue Type: Task >Affects Versions: 5.0.0 >Reporter: TIAN XING >Assignee: TIAN XING >Priority: Minor > Attachments: OOZIE-3265-v1.patch, OOZIE-3265-v2.patch, rerun.patch > > > Currently when you re-run a workflow with property "oozie.wf.rerun.failnodes" > set to true, > you can no longer re-run it again with "oozie.wf.rerun.skip.nodes" property > specified, even if you set "oozie.wf.rerun.failnodes" to false. > This kind of limitation is not reasonable. There is only one case where > "oozie.wf.rerun.failnodes" is true and "oozie.wf.rerun.skip.nodes" is not > null or empty, that should be disallowed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-3265) properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear together
[ https://issues.apache.org/jira/browse/OOZIE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] TIAN XING updated OOZIE-3265: - Attachment: OOZIE-3265-v2.patch > properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear > together > -- > > Key: OOZIE-3265 > URL: https://issues.apache.org/jira/browse/OOZIE-3265 > Project: Oozie > Issue Type: Task >Affects Versions: 5.0.0 >Reporter: TIAN XING >Assignee: TIAN XING >Priority: Minor > Attachments: OOZIE-3265-v1.patch, OOZIE-3265-v2.patch, rerun.patch > > > Currently when you re-run a workflow with property "oozie.wf.rerun.failnodes" > set to true, > you can no longer re-run it again with "oozie.wf.rerun.skip.nodes" property > specified, even if you set "oozie.wf.rerun.failnodes" to false. > This kind of limitation is not reasonable. There is only one case where > "oozie.wf.rerun.failnodes" is true and "oozie.wf.rerun.skip.nodes" is not > null or empty, that should be disallowed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] Subscription: Oozie Patch Available
Issue Subscription Filter: Oozie Patch Available (99 issues) Subscriber: ooziedaily Key Summary OOZIE-3265 properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear together https://issues.apache.org/jira/browse/OOZIE-3265 OOZIE-3256 refactor OozieCLI class https://issues.apache.org/jira/browse/OOZIE-3256 OOZIE-3249 [tools] Instrumentation log parser https://issues.apache.org/jira/browse/OOZIE-3249 OOZIE-3218 Oozie Sqoop action with command splits the select clause into multiple parts due to delimiter being space https://issues.apache.org/jira/browse/OOZIE-3218 OOZIE-3199 Let system property restriction configurable https://issues.apache.org/jira/browse/OOZIE-3199 OOZIE-3196 Authorization: restrict world readability by user https://issues.apache.org/jira/browse/OOZIE-3196 OOZIE-3194 Oozie should set proper permissions to sharelib after upload https://issues.apache.org/jira/browse/OOZIE-3194 OOZIE-3193 Applications are not killed when submitted via subworkflow https://issues.apache.org/jira/browse/OOZIE-3193 OOZIE-3186 Oozie is unable to use configuration linked using jceks://file/... https://issues.apache.org/jira/browse/OOZIE-3186 OOZIE-3179 Adding a configurable config-default.xml location to a workflow https://issues.apache.org/jira/browse/OOZIE-3179 OOZIE-3170 Oozie Diagnostic Bundle tool fails with NPE due to missing service class https://issues.apache.org/jira/browse/OOZIE-3170 OOZIE-3160 PriorityDelayQueue put()/take() can cause significant CPU load due to busy waiting https://issues.apache.org/jira/browse/OOZIE-3160 OOZIE-3156 SSH action status turns OK wrongly when failed to connect to host https://issues.apache.org/jira/browse/OOZIE-3156 OOZIE-3135 Configure log4j2 in SqoopMain https://issues.apache.org/jira/browse/OOZIE-3135 OOZIE-3109 Escape log-streaming's HTML-specific characters https://issues.apache.org/jira/browse/OOZIE-3109 OOZIE-3091 Oozie Sqoop Avro Import fails with "java.lang.NoClassDefFoundError: org/apache/avro/mapred/AvroWrapper" https://issues.apache.org/jira/browse/OOZIE-3091 OOZIE-3071 Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than Spark 2.2.0 https://issues.apache.org/jira/browse/OOZIE-3071 OOZIE-3063 Sanitizing variables that are part of openjpa.ConnectionProperties https://issues.apache.org/jira/browse/OOZIE-3063 OOZIE-3062 Set HADOOP_CONF_DIR for spark action https://issues.apache.org/jira/browse/OOZIE-3062 OOZIE-3061 Kill only those child jobs which are not already killed https://issues.apache.org/jira/browse/OOZIE-3061 OOZIE-2956 Fix Findbugs warnings related to reliance on default encoding in oozie-core https://issues.apache.org/jira/browse/OOZIE-2956 OOZIE-2955 Fix Findbugs warnings related to reliance on default encoding in oozie-client https://issues.apache.org/jira/browse/OOZIE-2955 OOZIE-2954 Fix Checkstyle issues in oozie-client https://issues.apache.org/jira/browse/OOZIE-2954 OOZIE-2953 Fix Checkstyle issues in oozie-tools https://issues.apache.org/jira/browse/OOZIE-2953 OOZIE-2952 Fix Findbugs warnings in oozie-sharelib-oozie https://issues.apache.org/jira/browse/OOZIE-2952 OOZIE-2949 Escape quotes whitespaces in Sqoop field https://issues.apache.org/jira/browse/OOZIE-2949 OOZIE-2942 [examples] Fix Findbugs warnings https://issues.apache.org/jira/browse/OOZIE-2942 OOZIE-2927 Append new line character for Hive2 query using query tag https://issues.apache.org/jira/browse/OOZIE-2927 OOZIE-2877 Oozie Git Action https://issues.apache.org/jira/browse/OOZIE-2877 OOZIE-2834 ParameterVerifier logging non-useful warning for workflow definition https://issues.apache.org/jira/browse/OOZIE-2834 OOZIE-2833 when using uber mode the regex pattern used in the extractHeapSizeMB method does not allow heap sizes specified in bytes. https://issues.apache.org/jira/browse/OOZIE-2833 OOZIE-2829 Improve sharelib upload to accept multiple source folders https://issues.apache.org/jira/browse/OOZIE-2829 OOZIE-2812 SparkConfigurationService should support loading configurations from multiple Spark versions https://issues.apache.org/jira/browse/OOZIE-2812 OOZIE-2795 Create lib directory or symlink for Oozie CLI during packaging https://issues.apache.org/jira/browse/OOZIE-2795 OOZIE-2791 ShareLib installation may fail on busy Hadoop clusters https://issues.apache.org/jira/browse/OOZIE-2791 OOZIE-2784 Include WEEK as a parameter in the Coordinator Expression Language Evaulator https://issues.apache.org/jira/browse/OOZIE-2784 OOZIE-2779 Mask Hive2 action Beeline JDBC passwo
[jira] [Commented] (OOZIE-3265) properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear together
[
https://issues.apache.org/jira/browse/OOZIE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16494635#comment-16494635
]
Hadoop QA commented on OOZIE-3265:
--
Testing JIRA OOZIE-3265
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warning(s)
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
error(s)
.{color:red}ERROR{color}: the current HEAD has 2 Javadoc error(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [client].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 2133
.Tests failed: 0
.Tests errors: 2
.The patch failed the following testcases:
.Tests failing with errors:
testRerunWithExistingConf(org.apache.oozie.command.wf.TestReRunXCommand)
testValidateReRunConfiguration(org.apache.oozie.TestDagEngine)
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/588/
> properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear
> together
> --
>
> Key: OOZIE-3265
> URL: https://issues.apache.org/jira/browse/OOZIE-3265
> Project: Oozie
> Issue Type: Task
>Affects Versions: 5.0.0
>Reporter: TIAN XING
>Assignee: TIAN XING
>Priority: Minor
> Attachments: OOZIE-3265-v1.patch, rerun.patch
>
>
> Currently when you re-run a workflow with property "oozie.wf.rerun.failnodes"
> set to true,
> you can no longer re-run it again with "oozie.wf.rerun.skip.nodes" property
> specified, even if you set "oozie.wf.rerun.failnodes" to false.
> This kind of limitation is not reasonable. There is only one case where
> "oozie.wf.rerun.failnodes" is true and "oozie.wf.rerun.skip.nodes" is not
> null or empty, that should be disallowed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Failed: OOZIE-3265 PreCommit Build #588
Jira: https://issues.apache.org/jira/browse/OOZIE-3265
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/588/
###
## LAST 100 LINES OF THE CONSOLE
###
[...truncated 1.72 MB...]
[DEBUG] There are no new bugs found in [sharelib/distcp].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/spark].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [client].
[INFO] There are no new bugs found totally].
[TRACE] FindBugs diffs checked and reports created
[TRACE] Summary file size is 2365 bytes
[TRACE] Full summary file size is 1314 bytes
[TRACE] File
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar]
removed
[TRACE] File
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5sum]
removed
Running test-patch task BACKWARDS_COMPATIBILITY
Running test-patch task TESTS
Running test-patch task DISTRO
Testing JIRA OOZIE-3265
Cleaning local git workspace
+1 PATCH_APPLIES
+1 CLEAN
+1 RAW_PATCH_ANALYSIS
+1 the patch does not introduce any @author tags
+1 the patch does not introduce any tabs
+1 the patch does not introduce any trailing spaces
+1 the patch does not introduce any line longer than 132
+1 the patch adds/modifies 1 testcase(s)
+1 RAT
+1 the patch does not seem to introduce new RAT warnings
+1 JAVADOC
+1 JAVADOC
+1 the patch does not seem to introduce new Javadoc warning(s)
+1 the patch does not seem to introduce new Javadoc error(s)
ERROR: the current HEAD has 2 Javadoc error(s)
+1 COMPILE
+1 HEAD compiles
+1 patch compiles
+1 the patch does not seem to introduce new javac warnings
+1 There are no new bugs found in total.
+1 There are no new bugs found in [examples].
+1 There are no new bugs found in [webapp].
+1 There are no new bugs found in [core].
+1 There are no new bugs found in [tools].
+1 There are no new bugs found in [server].
+1 There are no new bugs found in [docs].
+1 There are no new bugs found in [sharelib/hive2].
+1 There are no new bugs found in [sharelib/pig].
+1 There are no new bugs found in [sharelib/streaming].
+1 There are no new bugs found in [sharelib/hive].
+1 There are no new bugs found in [sharelib/hcatalog].
+1 There are no new bugs found in [sharelib/sqoop].
+1 There are no new bugs found in [sharelib/oozie].
+1 There are no new bugs found in [sharelib/distcp].
+1 There are no new bugs found in [sharelib/spark].
+1 There are no new bugs found in [client].
+1 BACKWARDS_COMPATIBILITY
+1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient
annotations
+1 the patch does not modify JPA files
-1 TESTS
Tests run: 2133
Tests failed: 0
Tests errors: 2
The patch failed the following testcases:
Tests failing with errors:
testRerunWithExistingConf(org.apache.oozie.command.wf.TestReRunXCommand)
testValidateReRunConfiguration(org.apache.oozie.TestDagEngine)
+1 DISTRO
+1 distro tarball builds with the patch
-1 Overall result, please check the reported -1(s)
The full output of the test-patch run is available at
https://builds.apache.org/job/PreCommit-OOZIE-Build/588/
Adding comment to JIRA
% Total% Received % Xferd Average Speed TimeTime Time Current
Dload Upload Total SpentLeft Speed
0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0
0 00 00 0 0 0 --:--:-- --:--:-- --:--:--
0{"self":"https://issues.apache.org/jira/rest/api/2/issue/13162562/comment/16494635","id":"16494635","author":{"self":"https://issues.apache.org/jira/rest/api/2/user?username=hadoopqa","name":"hadoopqa","key":"hadoopqa","emailAddress":"blackhole
at hadoop dot apache dot
org","avatarUrls":{"48x48":"https://issues.apache.org/jira/secure/useravatar?ownerId=hadoopqa&avatarId=10393","24x24":"https://issues.apache.org/jira/secure/useravatar?size=small&ownerId=hadoopqa&avatarId=10393","16x16":"https://issues.apache.org/jira/secure/useravatar?size=xsmall&ownerId=hadoopqa&avatarId=10393","32x32":"https://issues.apache.org/jira/secure/useravatar?size=medium&ownerId=hadoopqa&avatarId=10393"},"displayName":"Hadoop
QA","active":true,"timeZone":"Etc/UTC"},"body":"\nTesting JIRA
OOZIE-3265\n\nCleaning local git
workspace\n\n\n\n{color:green}+1
PATCH_APPLIES{color}\n{color:green}+1 CLEAN{color}\n{color:green}+1
RAW_PATCH_ANALYSIS{color}\n.{color:green}+1{color} the patch does not
introduce any @author tags\n.{color:green}+1{color} the patch does not
introduce a
[jira] [Commented] (OOZIE-3265) properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear together
[ https://issues.apache.org/jira/browse/OOZIE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16494586#comment-16494586 ] Hadoop QA commented on OOZIE-3265: -- PreCommit-OOZIE-Build started > properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear > together > -- > > Key: OOZIE-3265 > URL: https://issues.apache.org/jira/browse/OOZIE-3265 > Project: Oozie > Issue Type: Task >Affects Versions: 5.0.0 >Reporter: TIAN XING >Assignee: TIAN XING >Priority: Minor > Attachments: OOZIE-3265-v1.patch, rerun.patch > > > Currently when you re-run a workflow with property "oozie.wf.rerun.failnodes" > set to true, > you can no longer re-run it again with "oozie.wf.rerun.skip.nodes" property > specified, even if you set "oozie.wf.rerun.failnodes" to false. > This kind of limitation is not reasonable. There is only one case where > "oozie.wf.rerun.failnodes" is true and "oozie.wf.rerun.skip.nodes" is not > null or empty, that should be disallowed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-3265) properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear together
[ https://issues.apache.org/jira/browse/OOZIE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] TIAN XING updated OOZIE-3265: - Attachment: OOZIE-3265-v1.patch > properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear > together > -- > > Key: OOZIE-3265 > URL: https://issues.apache.org/jira/browse/OOZIE-3265 > Project: Oozie > Issue Type: Task >Affects Versions: 5.0.0 >Reporter: TIAN XING >Assignee: TIAN XING >Priority: Minor > Attachments: OOZIE-3265-v1.patch, rerun.patch > > > Currently when you re-run a workflow with property "oozie.wf.rerun.failnodes" > set to true, > you can no longer re-run it again with "oozie.wf.rerun.skip.nodes" property > specified, even if you set "oozie.wf.rerun.failnodes" to false. > This kind of limitation is not reasonable. There is only one case where > "oozie.wf.rerun.failnodes" is true and "oozie.wf.rerun.skip.nodes" is not > null or empty, that should be disallowed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3268) TestCoordELExtensions should restore the property "oozie.test.config.file"
[
https://issues.apache.org/jira/browse/OOZIE-3268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16493778#comment-16493778
]
Peter Bacsko commented on OOZIE-3268:
-
There's still something wrong. I performed {{lsof}} in a loop to catch the
point when the number of open files becomes too large. It's clearly Derby that
is re-initialized before every test. I also remember seeing and perhaps even
solving this issue, but apparently we haven't solved it yet.
> TestCoordELExtensions should restore the property "oozie.test.config.file"
> --
>
> Key: OOZIE-3268
> URL: https://issues.apache.org/jira/browse/OOZIE-3268
> Project: Oozie
> Issue Type: Sub-task
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
>
> The following {{setUp()}} method can cause serious test problems depending
> the order of test execution:
> {noformat}
>@Override
> protected void setUp() throws Exception {
> setSystemProperty("oozie.test.config.file", new File(OOZIE_SRC_DIR,
>
> "core/src/test/resources/oozie-site-coordel.xml").getAbsolutePath());
> super.setUp();
> services = new Services();
> setClassesToBeExcluded(services.getConf(), excludedServices);
> services.init();
> }
> {noformat}
> The problem is, this causes Oozie to switch to Derby from HSQLDB. However,
> before every testcase, the previous instance of Derby is not closed, leading
> to excessive number of open files. After a certain point, all tests fail with
> "Too many files open" exception.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3268) TestCoordELExtensions should restore the property "oozie.test.config.file"
[
https://issues.apache.org/jira/browse/OOZIE-3268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16493712#comment-16493712
]
Peter Cseh commented on OOZIE-3268:
---
I thought Xtestcase makes sure that these properties are reset:
https://github.com/apache/oozie/blob/master/core/src/test/java/org/apache/oozie/test/XTestCase.java#L751
> TestCoordELExtensions should restore the property "oozie.test.config.file"
> --
>
> Key: OOZIE-3268
> URL: https://issues.apache.org/jira/browse/OOZIE-3268
> Project: Oozie
> Issue Type: Sub-task
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
>
> The following {{setUp()}} method can cause serious test problems depending
> the order of test execution:
> {noformat}
>@Override
> protected void setUp() throws Exception {
> setSystemProperty("oozie.test.config.file", new File(OOZIE_SRC_DIR,
>
> "core/src/test/resources/oozie-site-coordel.xml").getAbsolutePath());
> super.setUp();
> services = new Services();
> setClassesToBeExcluded(services.getConf(), excludedServices);
> services.init();
> }
> {noformat}
> The problem is, this causes Oozie to switch to Derby from HSQLDB. However,
> before every testcase, the previous instance of Derby is not closed, leading
> to excessive number of open files. After a certain point, all tests fail with
> "Too many files open" exception.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (OOZIE-3268) TestCoordELExtensions should restore the property "oozie.test.config.file"
Peter Bacsko created OOZIE-3268:
---
Summary: TestCoordELExtensions should restore the property
"oozie.test.config.file"
Key: OOZIE-3268
URL: https://issues.apache.org/jira/browse/OOZIE-3268
Project: Oozie
Issue Type: Sub-task
Reporter: Peter Bacsko
Assignee: Peter Bacsko
The following {{setUp()}} method can cause serious test problems depending the
order of test execution:
{noformat}
@Override
protected void setUp() throws Exception {
setSystemProperty("oozie.test.config.file", new File(OOZIE_SRC_DIR,
"core/src/test/resources/oozie-site-coordel.xml").getAbsolutePath());
super.setUp();
services = new Services();
setClassesToBeExcluded(services.getConf(), excludedServices);
services.init();
}
{noformat}
The problem is, this causes Oozie to switch to Derby from HSQLDB. However,
before every testcase, the previous instance of Derby is not closed, leading to
excessive number of open files. After a certain point, all tests fail with "Too
many files open" exception.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-3268) TestCoordELExtensions should restore the property "oozie.test.config.file"
[
https://issues.apache.org/jira/browse/OOZIE-3268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Bacsko updated OOZIE-3268:
Description:
The following {{setUp()}} method can cause serious test problems depending the
order of test execution:
{noformat}
@Override
protected void setUp() throws Exception {
setSystemProperty("oozie.test.config.file", new File(OOZIE_SRC_DIR,
"core/src/test/resources/oozie-site-coordel.xml").getAbsolutePath());
super.setUp();
services = new Services();
setClassesToBeExcluded(services.getConf(), excludedServices);
services.init();
}
{noformat}
The problem is, this causes Oozie to switch to Derby from HSQLDB. However,
before every testcase, the previous instance of Derby is not closed, leading to
excessive number of open files. After a certain point, all tests fail with "Too
many files open" exception.
was:
The following {{setUp()}} method can cause serious test problems depending the
order of test execution:
{noformat}
@Override
protected void setUp() throws Exception {
setSystemProperty("oozie.test.config.file", new File(OOZIE_SRC_DIR,
"core/src/test/resources/oozie-site-coordel.xml").getAbsolutePath());
super.setUp();
services = new Services();
setClassesToBeExcluded(services.getConf(), excludedServices);
services.init();
}
{noformat}
The problem is, this causes Oozie to switch to Derby from HSQLDB. However,
before every testcase, the previous instance of Derby is not closed, leading to
excessive number of open files. After a certain point, all tests fail with "Too
many files open" exception.
> TestCoordELExtensions should restore the property "oozie.test.config.file"
> --
>
> Key: OOZIE-3268
> URL: https://issues.apache.org/jira/browse/OOZIE-3268
> Project: Oozie
> Issue Type: Sub-task
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
>
> The following {{setUp()}} method can cause serious test problems depending
> the order of test execution:
> {noformat}
>@Override
> protected void setUp() throws Exception {
> setSystemProperty("oozie.test.config.file", new File(OOZIE_SRC_DIR,
>
> "core/src/test/resources/oozie-site-coordel.xml").getAbsolutePath());
> super.setUp();
> services = new Services();
> setClassesToBeExcluded(services.getConf(), excludedServices);
> services.init();
> }
> {noformat}
> The problem is, this causes Oozie to switch to Derby from HSQLDB. However,
> before every testcase, the previous instance of Derby is not closed, leading
> to excessive number of open files. After a certain point, all tests fail with
> "Too many files open" exception.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3267) Re-run coord action without checking input dependencies
[ https://issues.apache.org/jira/browse/OOZIE-3267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16493591#comment-16493591 ] TIAN XING commented on OOZIE-3267: -- [~gezapeti] Let's assume that the task of coord job is to train a predict model every day 0:01, based on 24 input files of yesterday (1 file per hour). If all the 24 files are available, the predict model can get best accuracy. if some input files are missing, we can get the predict model as well but with lower accuracy. In most cases, we want to wait for all input files, however, let's say, it is 2am now, hour-09-yesterday input file is still missing, and we have to get the predict model for production environment before 3am even with lower accuracy. In such case, we need start the coord action right now without checking inputs. the coordInput OR logic won't help. if use OR logic, we have to know in advance which input files are conditional. What's more, once we fix coord input logic, the coord action will always start without waiting certain inputs. In our example, it is the user that decide whether coordAction wait for certain inputs or not, which is more flexible. > Re-run coord action without checking input dependencies > --- > > Key: OOZIE-3267 > URL: https://issues.apache.org/jira/browse/OOZIE-3267 > Project: Oozie > Issue Type: New Feature >Reporter: TIAN XING >Assignee: TIAN XING >Priority: Major > > A coord action will get stuck in WAITING status if some of its input > dependencies are missing. > However, there are cases where users want to force start the coord action > without checking input dependencies. > For example, the process of training a day model need 24 different hour input > files to get best accuracy ( 23 input files are also OK, but the accuracy may > decrease). > In some cases (e.g., one hour input file get delayed), it is more important > to get the model on time than high accuracy. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3156) SSH action status turns OK wrongly when failed to connect to host
[
https://issues.apache.org/jira/browse/OOZIE-3156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16493556#comment-16493556
]
Andras Piros commented on OOZIE-3156:
-
Thanks for the contribution [~txsing]! Can you please update
{{TestSshActionExecutor}} with a new test case covering retry functionality, as
well as extend {{DG_SshActionExtension.twiki}} to document the fix?
Review comments:
* {{SSH_CONNECT_ERROR_CODE}} could be {{final}}
* {{retriesMax}} should be {{retryCount}}
* in order to actually have a chance that the connection error doesn't reoccur,
we should {{Thread#sleep()}} some time in between, or use an
[*{{ScheduledThreadPoolExecutor}}*|https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ScheduledThreadPoolExecutor.html]
to perform waiting without busy waiting
* the waiting should be based on an exponential backoff like in
[*{{OperationRetryHandler#handleRetry()}}*|https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/util/db/OperationRetryHandler.java#L123-L129]
> SSH action status turns OK wrongly when failed to connect to host
> -
>
> Key: OOZIE-3156
> URL: https://issues.apache.org/jira/browse/OOZIE-3156
> Project: Oozie
> Issue Type: Bug
> Components: action
>Affects Versions: 5.0.0
>Reporter: TIAN XING
>Assignee: TIAN XING
>Priority: Major
> Attachments: ssh-check-bug.patch
>
>
> When {{check()}} method of {{SshActionExecutor}} gets invoked, oozie will ssh
> connect to the host and check whether the pid of the process that ssh action
> started is still there (by checking the returned value of command "{{ssh
> ps -p }}" ) to determine whether ssh action completes or not.
> However, we found cases where oozie fails to connect to host during action
> status check (e.g., the host is under heavy load, or network is bad etc.).
> In such cases, the return value of command "{{ssh ps -p }}"
> will be 255 (ssh command exits with the exit status of the remote command or
> with 255 if an error occurred.).
> According the current logic of method {{getActionStatus()}} in
> {{SshActionExecutor}}, the action status will be determined as OK which may
> not be correct.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Failed: OOZIE-3265 PreCommit Build #587
Jira: https://issues.apache.org/jira/browse/OOZIE-3265
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/587/
###
## LAST 100 LINES OF THE CONSOLE
###
[...truncated 1.71 MB...]
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/streaming].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/hive].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/hcatalog].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/sqoop].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/oozie].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/distcp].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/spark].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [client].
[INFO] There are no new bugs found totally].
[TRACE] FindBugs diffs checked and reports created
[TRACE] Summary file size is 2366 bytes
[TRACE] Full summary file size is 1314 bytes
[TRACE] File
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build@2/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar]
removed
[TRACE] File
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build@2/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5sum]
removed
Running test-patch task BACKWARDS_COMPATIBILITY
Running test-patch task TESTS
Running test-patch task DISTRO
Testing JIRA OOZIE-3265
Cleaning local git workspace
+1 PATCH_APPLIES
+1 CLEAN
-1 RAW_PATCH_ANALYSIS
+1 the patch does not introduce any @author tags
+1 the patch does not introduce any tabs
+1 the patch does not introduce any trailing spaces
+1 the patch does not introduce any line longer than 132
-1 the patch does not add/modify any testcase
+1 RAT
+1 the patch does not seem to introduce new RAT warnings
+1 JAVADOC
+1 JAVADOC
+1 the patch does not seem to introduce new Javadoc warning(s)
+1 the patch does not seem to introduce new Javadoc error(s)
ERROR: the current HEAD has 2 Javadoc error(s)
+1 COMPILE
+1 HEAD compiles
+1 patch compiles
+1 the patch does not seem to introduce new javac warnings
+1 There are no new bugs found in total.
+1 There are no new bugs found in [examples].
+1 There are no new bugs found in [webapp].
+1 There are no new bugs found in [core].
+1 There are no new bugs found in [tools].
+1 There are no new bugs found in [server].
+1 There are no new bugs found in [docs].
+1 There are no new bugs found in [sharelib/hive2].
+1 There are no new bugs found in [sharelib/pig].
+1 There are no new bugs found in [sharelib/streaming].
+1 There are no new bugs found in [sharelib/hive].
+1 There are no new bugs found in [sharelib/hcatalog].
+1 There are no new bugs found in [sharelib/sqoop].
+1 There are no new bugs found in [sharelib/oozie].
+1 There are no new bugs found in [sharelib/distcp].
+1 There are no new bugs found in [sharelib/spark].
+1 There are no new bugs found in [client].
+1 BACKWARDS_COMPATIBILITY
+1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient
annotations
+1 the patch does not modify JPA files
+1 TESTS
Tests run: 2132
+1 DISTRO
+1 distro tarball builds with the patch
-1 Overall result, please check the reported -1(s)
The full output of the test-patch run is available at
https://builds.apache.org/job/PreCommit-OOZIE-Build/587/
Adding comment to JIRA
% Total% Received % Xferd Average Speed TimeTime Time Current
Dload Upload Total SpentLeft Speed
0 00 00 0 0 0 --:--:-- --:--:-- --:--:--
0{"self":"https://issues.apache.org/jira/rest/api/2/issue/13162562/comment/16493537","id":"16493537","author":{"self":"https://issues.apache.org/jira/rest/api/2/user?username=hadoopqa","name":"hadoopqa","key":"hadoopqa","emailAddress":"blackhole
at hadoop dot apache dot
org","avatarUrls":{"48x48":"https://issues.apache.org/jira/secure/useravatar?ownerId=hadoopqa&avatarId=10393","24x24":"https://issues.apache.org/jira/secure/useravatar?size=small&ownerId=hadoopqa&avatarId=10393","16x16":"https://issues.apache.org/jira/secure/useravatar?size=xsmall&ownerId=hadoopqa&avatarId=10393","32x32":"https://issues.apache.org/jira/secure/useravatar?size=medium
[jira] [Commented] (OOZIE-3265) properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear together
[
https://issues.apache.org/jira/browse/OOZIE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16493537#comment-16493537
]
Hadoop QA commented on OOZIE-3265:
--
Testing JIRA OOZIE-3265
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warning(s)
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
error(s)
.{color:red}ERROR{color}: the current HEAD has 2 Javadoc error(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [client].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 2132
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/587/
> properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear
> together
> --
>
> Key: OOZIE-3265
> URL: https://issues.apache.org/jira/browse/OOZIE-3265
> Project: Oozie
> Issue Type: Task
>Affects Versions: 5.0.0
>Reporter: TIAN XING
>Assignee: TIAN XING
>Priority: Minor
> Attachments: rerun.patch
>
>
> Currently when you re-run a workflow with property "oozie.wf.rerun.failnodes"
> set to true,
> you can no longer re-run it again with "oozie.wf.rerun.skip.nodes" property
> specified, even if you set "oozie.wf.rerun.failnodes" to false.
> This kind of limitation is not reasonable. There is only one case where
> "oozie.wf.rerun.failnodes" is true and "oozie.wf.rerun.skip.nodes" is not
> null or empty, that should be disallowed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Failed: OOZIE-3156 PreCommit Build #586
Jira: https://issues.apache.org/jira/browse/OOZIE-3156
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/586/
###
## LAST 100 LINES OF THE CONSOLE
###
[...truncated 1.71 MB...]
[DEBUG] There are no new bugs found in [sharelib/hive].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/hcatalog].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/sqoop].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/oozie].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/distcp].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [sharelib/spark].
[TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts
[DEBUG] There are no new bugs found in [client].
[INFO] There are no new bugs found totally].
[TRACE] FindBugs diffs checked and reports created
[TRACE] Summary file size is 2366 bytes
[TRACE] Full summary file size is 1314 bytes
[TRACE] File
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar]
removed
[TRACE] File
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/FINDBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5sum]
removed
Running test-patch task BACKWARDS_COMPATIBILITY
Running test-patch task TESTS
Running test-patch task DISTRO
Testing JIRA OOZIE-3156
Cleaning local git workspace
+1 PATCH_APPLIES
+1 CLEAN
-1 RAW_PATCH_ANALYSIS
+1 the patch does not introduce any @author tags
+1 the patch does not introduce any tabs
+1 the patch does not introduce any trailing spaces
-1 the patch contains 1 line(s) longer than 132 characters
-1 the patch does not add/modify any testcase
+1 RAT
+1 the patch does not seem to introduce new RAT warnings
+1 JAVADOC
+1 JAVADOC
+1 the patch does not seem to introduce new Javadoc warning(s)
+1 the patch does not seem to introduce new Javadoc error(s)
ERROR: the current HEAD has 2 Javadoc error(s)
+1 COMPILE
+1 HEAD compiles
+1 patch compiles
+1 the patch does not seem to introduce new javac warnings
+1 There are no new bugs found in total.
+1 There are no new bugs found in [examples].
+1 There are no new bugs found in [webapp].
+1 There are no new bugs found in [core].
+1 There are no new bugs found in [tools].
+1 There are no new bugs found in [server].
+1 There are no new bugs found in [docs].
+1 There are no new bugs found in [sharelib/hive2].
+1 There are no new bugs found in [sharelib/pig].
+1 There are no new bugs found in [sharelib/streaming].
+1 There are no new bugs found in [sharelib/hive].
+1 There are no new bugs found in [sharelib/hcatalog].
+1 There are no new bugs found in [sharelib/sqoop].
+1 There are no new bugs found in [sharelib/oozie].
+1 There are no new bugs found in [sharelib/distcp].
+1 There are no new bugs found in [sharelib/spark].
+1 There are no new bugs found in [client].
+1 BACKWARDS_COMPATIBILITY
+1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient
annotations
+1 the patch does not modify JPA files
+1 TESTS
Tests run: 2132
Tests failed at first run:
TestCoordMaterializeTriggerService#testCoordMaterializeTriggerService3
For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
+1 DISTRO
+1 distro tarball builds with the patch
-1 Overall result, please check the reported -1(s)
The full output of the test-patch run is available at
https://builds.apache.org/job/PreCommit-OOZIE-Build/586/
Adding comment to JIRA
% Total% Received % Xferd Average Speed TimeTime Time Current
Dload Upload Total SpentLeft Speed
0 00 00 0 0 0 --:--:-- --:--:-- --:--:--
0{"self":"https://issues.apache.org/jira/rest/api/2/issue/13130096/comment/16493534","id":"16493534","author":{"self":"https://issues.apache.org/jira/rest/api/2/user?username=hadoopqa","name":"hadoopqa","key":"hadoopqa","emailAddress":"blackhole
at hadoop dot apache dot
org","avatarUrls":{"48x48":"https://issues.apache.org/jira/secure/useravatar?ownerId=hadoopqa&avatarId=10393","24x24":"https://issues.apache.org/jira/secure/useravatar?size=small&ownerId=hadoopqa&avatarId=10393","16x16":"https://issues.apache.org/jira/secure/useravatar?size=xsmall&ownerId=hadoopqa&avatarId=10393","32x32":"https://issues.apache.org/jira/secure/useravatar?size=medium&ownerId=hadoopqa&avatarId=10393"},"displayName":"Ha
[jira] [Commented] (OOZIE-3156) SSH action status turns OK wrongly when failed to connect to host
[
https://issues.apache.org/jira/browse/OOZIE-3156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16493534#comment-16493534
]
Hadoop QA commented on OOZIE-3156:
--
Testing JIRA OOZIE-3156
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:red}-1{color} the patch contains 1 line(s) longer than 132
characters
.{color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warning(s)
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
error(s)
.{color:red}ERROR{color}: the current HEAD has 2 Javadoc error(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [client].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 2132
.{color:orange}Tests failed at first run:{color}
TestCoordMaterializeTriggerService#testCoordMaterializeTriggerService3
.For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/586/
> SSH action status turns OK wrongly when failed to connect to host
> -
>
> Key: OOZIE-3156
> URL: https://issues.apache.org/jira/browse/OOZIE-3156
> Project: Oozie
> Issue Type: Bug
> Components: action
>Affects Versions: 5.0.0
>Reporter: TIAN XING
>Assignee: TIAN XING
>Priority: Major
> Attachments: ssh-check-bug.patch
>
>
> When {{check()}} method of {{SshActionExecutor}} gets invoked, oozie will ssh
> connect to the host and check whether the pid of the process that ssh action
> started is still there (by checking the returned value of command "{{ssh
> ps -p }}" ) to determine whether ssh action completes or not.
> However, we found cases where oozie fails to connect to host during action
> status check (e.g., the host is under heavy load, or network is bad etc.).
> In such cases, the return value of command "{{ssh ps -p }}"
> will be 255 (ssh command exits with the exit status of the remote command or
> with 255 if an error occurred.).
> According the current logic of method {{getActionStatus()}} in
> {{SshActionExecutor}}, the action status will be determined as OK which may
> not be correct.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3256) refactor OozieCLI class
[ https://issues.apache.org/jira/browse/OOZIE-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16493497#comment-16493497 ] Peter Bacsko commented on OOZIE-3256: - [~dbist13] can you please add a short description to this ticket? What's the main rationale behind the refactor, what's wrong with the current code, etc. > refactor OozieCLI class > --- > > Key: OOZIE-3256 > URL: https://issues.apache.org/jira/browse/OOZIE-3256 > Project: Oozie > Issue Type: Improvement >Affects Versions: 5.0.0 >Reporter: Artem Ervits >Assignee: Artem Ervits >Priority: Minor > Attachments: OOZIE-3256.00.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3267) Re-run coord action without checking input dependencies
[ https://issues.apache.org/jira/browse/OOZIE-3267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16493489#comment-16493489 ] Peter Cseh commented on OOZIE-3267: --- You can already achieve this using [conditional input logic in coordinators|https://oozie.apache.org/docs/5.0.0/CoordinatorFunctionalSpec.html#a6.10._Conditional_coordinator_input_logic], can't you? > Re-run coord action without checking input dependencies > --- > > Key: OOZIE-3267 > URL: https://issues.apache.org/jira/browse/OOZIE-3267 > Project: Oozie > Issue Type: New Feature >Reporter: TIAN XING >Assignee: TIAN XING >Priority: Major > > A coord action will get stuck in WAITING status if some of its input > dependencies are missing. > However, there are cases where users want to force start the coord action > without checking input dependencies. > For example, the process of training a day model need 24 different hour input > files to get best accuracy ( 23 input files are also OK, but the accuracy may > decrease). > In some cases (e.g., one hour input file get delayed), it is more important > to get the model on time than high accuracy. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (OOZIE-3267) Re-run coord action without checking input dependencies
TIAN XING created OOZIE-3267: Summary: Re-run coord action without checking input dependencies Key: OOZIE-3267 URL: https://issues.apache.org/jira/browse/OOZIE-3267 Project: Oozie Issue Type: New Feature Reporter: TIAN XING Assignee: TIAN XING A coord action will get stuck in WAITING status if some of its input dependencies are missing. However, there are cases where users want to force start the coord action without checking input dependencies. For example, the process of training a day model need 24 different hour input files to get best accuracy ( 23 input files are also OK, but the accuracy may decrease). In some cases (e.g., one hour input file get delayed), it is more important to get the model on time than high accuracy. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3265) properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear together
[ https://issues.apache.org/jira/browse/OOZIE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16493476#comment-16493476 ] Peter Cseh commented on OOZIE-3265: --- This sounds reasonable. Can you add a test case to make sure we don't break it in the future? > properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear > together > -- > > Key: OOZIE-3265 > URL: https://issues.apache.org/jira/browse/OOZIE-3265 > Project: Oozie > Issue Type: Task >Affects Versions: 5.0.0 >Reporter: TIAN XING >Assignee: TIAN XING >Priority: Minor > Attachments: rerun.patch > > > Currently when you re-run a workflow with property "oozie.wf.rerun.failnodes" > set to true, > you can no longer re-run it again with "oozie.wf.rerun.skip.nodes" property > specified, even if you set "oozie.wf.rerun.failnodes" to false. > This kind of limitation is not reasonable. There is only one case where > "oozie.wf.rerun.failnodes" is true and "oozie.wf.rerun.skip.nodes" is not > null or empty, that should be disallowed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (OOZIE-3266) Coord action rerun support RERUN_SKIP_NODES option
TIAN XING created OOZIE-3266: Summary: Coord action rerun support RERUN_SKIP_NODES option Key: OOZIE-3266 URL: https://issues.apache.org/jira/browse/OOZIE-3266 Project: Oozie Issue Type: New Feature Reporter: TIAN XING Assignee: TIAN XING currently, when you re-run a workflow job, you have 3 options # re-run all of its action nodes # re-run failed nodes only # re-run with specified nodes skipped if this workflow job is generated by a coord action. you can re-run this coord action with 2 options # re-run all of the workflow action nodes (generate a new workflow job id) # re-run failed workflow action nodes only (workflow job id not changed) now we want to add a another option - re-run with specified workflow nodes skipped (workflow job id not changed) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3156) SSH action status turns OK wrongly when failed to connect to host
[
https://issues.apache.org/jira/browse/OOZIE-3156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16493449#comment-16493449
]
Hadoop QA commented on OOZIE-3156:
--
PreCommit-OOZIE-Build started
> SSH action status turns OK wrongly when failed to connect to host
> -
>
> Key: OOZIE-3156
> URL: https://issues.apache.org/jira/browse/OOZIE-3156
> Project: Oozie
> Issue Type: Bug
> Components: action
>Affects Versions: 5.0.0
>Reporter: TIAN XING
>Assignee: TIAN XING
>Priority: Major
> Attachments: ssh-check-bug.patch
>
>
> When {{check()}} method of {{SshActionExecutor}} gets invoked, oozie will ssh
> connect to the host and check whether the pid of the process that ssh action
> started is still there (by checking the returned value of command "{{ssh
> ps -p }}" ) to determine whether ssh action completes or not.
> However, we found cases where oozie fails to connect to host during action
> status check (e.g., the host is under heavy load, or network is bad etc.).
> In such cases, the return value of command "{{ssh ps -p }}"
> will be 255 (ssh command exits with the exit status of the remote command or
> with 255 if an error occurred.).
> According the current logic of method {{getActionStatus()}} in
> {{SshActionExecutor}}, the action status will be determined as OK which may
> not be correct.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3265) properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear together
[ https://issues.apache.org/jira/browse/OOZIE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16493450#comment-16493450 ] Hadoop QA commented on OOZIE-3265: -- PreCommit-OOZIE-Build started > properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear > together > -- > > Key: OOZIE-3265 > URL: https://issues.apache.org/jira/browse/OOZIE-3265 > Project: Oozie > Issue Type: Task >Affects Versions: 5.0.0 >Reporter: TIAN XING >Assignee: TIAN XING >Priority: Minor > Attachments: rerun.patch > > > Currently when you re-run a workflow with property "oozie.wf.rerun.failnodes" > set to true, > you can no longer re-run it again with "oozie.wf.rerun.skip.nodes" property > specified, even if you set "oozie.wf.rerun.failnodes" to false. > This kind of limitation is not reasonable. There is only one case where > "oozie.wf.rerun.failnodes" is true and "oozie.wf.rerun.skip.nodes" is not > null or empty, that should be disallowed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-3265) properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear together
[ https://issues.apache.org/jira/browse/OOZIE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] TIAN XING updated OOZIE-3265: - Description: Currently when you re-run a workflow with property "oozie.wf.rerun.failnodes" set to true, you can no longer re-run it again with "oozie.wf.rerun.skip.nodes" property specified, even if you set "oozie.wf.rerun.failnodes" to false. This kind of limitation is not reasonable. There is only one case where "oozie.wf.rerun.failnodes" is true and "oozie.wf.rerun.skip.nodes" is not null or empty, that should be disallowed. was: Currently when you re-run a workflow with property "oozie.wf.rerun.failnodes" set to true, you can no longer re-run it again with "oozie.wf.rerun.skip.nodes" property specified, even if you set "oozie.wf.rerun.failnodes" to false. This kind of limitation is not reasonable. There is only one case where "oozie.wf.rerun.failnodes" is true and "oozie.wf.rerun.failnodes" is not null or empty, that should be disallowed. > properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear > together > -- > > Key: OOZIE-3265 > URL: https://issues.apache.org/jira/browse/OOZIE-3265 > Project: Oozie > Issue Type: Task >Affects Versions: 5.0.0 >Reporter: TIAN XING >Assignee: TIAN XING >Priority: Minor > Attachments: rerun.patch > > > Currently when you re-run a workflow with property "oozie.wf.rerun.failnodes" > set to true, > you can no longer re-run it again with "oozie.wf.rerun.skip.nodes" property > specified, even if you set "oozie.wf.rerun.failnodes" to false. > This kind of limitation is not reasonable. There is only one case where > "oozie.wf.rerun.failnodes" is true and "oozie.wf.rerun.skip.nodes" is not > null or empty, that should be disallowed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (OOZIE-3265) properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear together
[ https://issues.apache.org/jira/browse/OOZIE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Piros reassigned OOZIE-3265: --- Assignee: TIAN XING > properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear > together > -- > > Key: OOZIE-3265 > URL: https://issues.apache.org/jira/browse/OOZIE-3265 > Project: Oozie > Issue Type: Task >Affects Versions: 5.0.0 >Reporter: TIAN XING >Assignee: TIAN XING >Priority: Minor > Attachments: rerun.patch > > > Currently when you re-run a workflow with property "oozie.wf.rerun.failnodes" > set to true, > you can no longer re-run it again with "oozie.wf.rerun.skip.nodes" property > specified, even if you set "oozie.wf.rerun.failnodes" to false. > This kind of limitation is not reasonable. There is only one case where > "oozie.wf.rerun.failnodes" is true and "oozie.wf.rerun.failnodes" is not null > or empty, that should be disallowed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3156) SSH action status turns OK wrongly when failed to connect to host
[
https://issues.apache.org/jira/browse/OOZIE-3156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16493440#comment-16493440
]
TIAN XING commented on OOZIE-3156:
--
[~andras.piros] hey Andras, any news on this patch?
> SSH action status turns OK wrongly when failed to connect to host
> -
>
> Key: OOZIE-3156
> URL: https://issues.apache.org/jira/browse/OOZIE-3156
> Project: Oozie
> Issue Type: Bug
> Components: action
>Reporter: TIAN XING
>Assignee: TIAN XING
>Priority: Major
> Attachments: ssh-check-bug.patch
>
>
> When {{check()}} method of {{SshActionExecutor}} gets invoked, oozie will ssh
> connect to the host and check whether the pid of the process that ssh action
> started is still there (by checking the returned value of command "{{ssh
> ps -p }}" ) to determine whether ssh action completes or not.
> However, we found cases where oozie fails to connect to host during action
> status check (e.g., the host is under heavy load, or network is bad etc.).
> In such cases, the return value of command "{{ssh ps -p }}"
> will be 255 (ssh command exits with the exit status of the remote command or
> with 255 if an error occurred.).
> According the current logic of method {{getActionStatus()}} in
> {{SshActionExecutor}}, the action status will be determined as OK which may
> not be correct.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-3156) SSH action status turns OK wrongly when failed to connect to host
[
https://issues.apache.org/jira/browse/OOZIE-3156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
TIAN XING updated OOZIE-3156:
-
Description:
When {{check()}} method of {{SshActionExecutor}} gets invoked, oozie will ssh
connect to the host and check whether the pid of the process that ssh action
started is still there (by checking the returned value of command "{{ssh
ps -p }}" ) to determine whether ssh action completes or not.
However, we found cases where oozie fails to connect to host during action
status check (e.g., the host is under heavy load, or network is bad etc.).
In such cases, the return value of command "{{ssh ps -p }}" will
be 255 (ssh command exits with the exit status of the remote command or with
255 if an error occurred.).
According the current logic of method {{getActionStatus()}} in
{{SshActionExecutor}}, the action status will be determined as OK which may not
be correct.
was:
When {{check()}} method of {{SshActionExecutor}} gets invoked, oozie will ssh
connect to the host and check whether action shell pid is still there (by
checking the returned value of command {{ssh $hostIp ps -p $pid}} ) to
determine whether the action is running or not.
However, there are cases where oozie fails to connect to the host during action
status check (e.g., the host is under heavy load, or network is bad etc.).
In such cases, the return value of the command {{ssh $hostIp ps -p $pid}} will
be 255 (ssh command exits with the exit status of the remote command or with
255 if an error occurred.).
According the current logic of method {{getActionStatus()}} in
{{SshActionExecutor}}, the action status will be determined as OK which may not
be correct.
> SSH action status turns OK wrongly when failed to connect to host
> -
>
> Key: OOZIE-3156
> URL: https://issues.apache.org/jira/browse/OOZIE-3156
> Project: Oozie
> Issue Type: Bug
> Components: action
>Reporter: TIAN XING
>Assignee: TIAN XING
>Priority: Major
> Attachments: ssh-check-bug.patch
>
>
> When {{check()}} method of {{SshActionExecutor}} gets invoked, oozie will ssh
> connect to the host and check whether the pid of the process that ssh action
> started is still there (by checking the returned value of command "{{ssh
> ps -p }}" ) to determine whether ssh action completes or not.
> However, we found cases where oozie fails to connect to host during action
> status check (e.g., the host is under heavy load, or network is bad etc.).
> In such cases, the return value of command "{{ssh ps -p }}"
> will be 255 (ssh command exits with the exit status of the remote command or
> with 255 if an error occurred.).
> According the current logic of method {{getActionStatus()}} in
> {{SshActionExecutor}}, the action status will be determined as OK which may
> not be correct.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-3156) SSH action status turns OK wrongly when failed to connect to host
[
https://issues.apache.org/jira/browse/OOZIE-3156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
TIAN XING updated OOZIE-3156:
-
Affects Version/s: (was: 4.3.0)
(was: 4.2.0)
(was: 4.1.0)
(was: 4.0.0)
Fix Version/s: (was: 4.3.0)
> SSH action status turns OK wrongly when failed to connect to host
> -
>
> Key: OOZIE-3156
> URL: https://issues.apache.org/jira/browse/OOZIE-3156
> Project: Oozie
> Issue Type: Bug
> Components: action
>Reporter: TIAN XING
>Assignee: TIAN XING
>Priority: Major
> Attachments: ssh-check-bug.patch
>
>
> When {{check()}} method of {{SshActionExecutor}} gets invoked, oozie will ssh
> connect to the host and check whether action shell pid is still there (by
> checking the returned value of command {{ssh $hostIp ps -p $pid}} ) to
> determine whether the action is running or not.
> However, there are cases where oozie fails to connect to the host during
> action status check (e.g., the host is under heavy load, or network is bad
> etc.).
> In such cases, the return value of the command {{ssh $hostIp ps -p $pid}}
> will be 255 (ssh command exits with the exit status of the remote command or
> with 255 if an error occurred.).
> According the current logic of method {{getActionStatus()}} in
> {{SshActionExecutor}}, the action status will be determined as OK which may
> not be correct.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (OOZIE-3265) properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear together
TIAN XING created OOZIE-3265: Summary: properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear together Key: OOZIE-3265 URL: https://issues.apache.org/jira/browse/OOZIE-3265 Project: Oozie Issue Type: Task Reporter: TIAN XING Attachments: rerun.patch Currently when you re-run a workflow with property "oozie.wf.rerun.failnodes" set to true, you can no longer re-run it again with "oozie.wf.rerun.skip.nodes" property specified, even if you set "oozie.wf.rerun.failnodes" to false. This kind of limitation is not reasonable. There is only one case where "oozie.wf.rerun.failnodes" is true and "oozie.wf.rerun.failnodes" is not null or empty, that should be disallowed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
