[jira] [Assigned] (OOZIE-2458) 'oozie-setup.sh sharelib create' should ensure uploaded jars are world readable
[ https://issues.apache.org/jira/browse/OOZIE-2458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Denes reassigned OOZIE-2458: --- Assignee: (was: Ferenc Denes) > 'oozie-setup.sh sharelib create' should ensure uploaded jars are world > readable > --- > > Key: OOZIE-2458 > URL: https://issues.apache.org/jira/browse/OOZIE-2458 > Project: Oozie > Issue Type: Bug > Components: scripts >Affects Versions: 4.1.0 >Reporter: Ferenc Denes >Priority: Critical > Fix For: trunk > > Attachments: OOZIE-2458-1.patch > > > If the default umask of HDFS does not include world read then the Oozie > sharelib gets created accordingly, and then the jars can not be read at > runtime because Oozie workflows impersonate the submitting user. > The 'oozie-setup.sh sharelib create' command should ensure that the installed > jars are world readable. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (OOZIE-2740) oozie help misspelled coordinator (coordiantor) and retrieved (retreived)
[
https://issues.apache.org/jira/browse/OOZIE-2740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15712350#comment-15712350
]
Ferenc Denes edited comment on OOZIE-2740 at 12/1/16 4:09 PM:
--
+1 (nonbinding) (
https://www.freebsd.org/doc/en_US.ISO8859-1/books/faq/misc.html#bikeshed-painting
)
was (Author: fdenes):
+1 (nonbinding)
> oozie help misspelled coordinator (coordiantor) and retrieved (retreived)
> -
>
> Key: OOZIE-2740
> URL: https://issues.apache.org/jira/browse/OOZIE-2740
> Project: Oozie
> Issue Type: Bug
> Components: client
>Affects Versions: 4.1.0
>Reporter: Grant Sohn
>Priority: Trivial
> Attachments: OOZIE-2740.1.patch, OOZIE-2740.2.patch
>
>
> Spelling errors and concatenated words.
> {noformat}
> oozie help
> usage:
> the env variable 'OOZIE_URL' is used as default value for the '-oozie'
> option
> the env variable 'OOZIE_TIMEZONE' is used as default value for the
> '-timezone' option
> the env variable 'OOZIE_AUTH' is used as default value for the '-auth'
> option
> custom headers for Oozie web services can be specified using
> '-Dheader:NAME=VALUE'
> oozie help : display usage for all commands or specified command
> oozie version : show client version
> oozie job : job operations
> -action coordinator rerun/kill on action ids
> (requires -rerun/-kill);
>coordinator log retrieval on action
> ids(requires -log)
> -allruns Get workflow jobs corresponding to a
> coordinator action
>including all the reruns
> -auth select authentication type
> [SIMPLE|KERBEROS]
> -change change a coordinator or bundle job
> -config job configuration file '.xml' or
> '.properties'
> -configcontentjob configuration
> -coordinator bundle rerun on coordinator names
> (requires -rerun)
> -D set/override value for given property
> -date coordinator/bundle rerun on action
> dates (requires -rerun);
>coordinator log retrieval on action
> dates (requires -log)
> -debug Use debug mode to see debugging
> statements on stdout
> -definition job definition
> -diff Show diff of the new coord definition
> and properties with the
>existing one (default true)
> -doas doAs user, impersonates as the
> specified user
> -dryrunDryrun a workflow (since 3.3.2) or
> coordinator (since 2.0)
>job without actually executing it
> -failedruns the failed workflow actions of
> the coordinator actions
>(requires -rerun)
> -filter
> [;]*
>(All Coordinator actions satisfying
> the filters will be
>retreived).
>key: status or nominaltime
>comparator: =, !=, <, <=, >, >=. = is
> used as OR and others
>as AND
>status: values are valid status like
> SUCCEEDED, KILLED etc.
>Only = and != apply for status
>nominaltime: time of format
> -MM-dd'T'HH:mm'Z'
> -ignore change status of a coordinator job or
> action to IGNORED
>(-action required to ignore coord
> actions)
> -info info of a job
> -interval polling interval in minutes (default
> is 5, requires -poll)
> -kill kill a job (coordinator can mention
> -action or -date)
> -len number of actions (default TOTAL
> ACTIONS, requires -info)
> -localtime use local time (same as passing your
> time zone to -timezone).
>Overrides -timezone option
> -log job log
> -logfilterjob log search parameter. Can be
> specified as -logfilter
>opt1=val1;opt2=val1;opt3=val1.
> Supported options are recent,
>start, end, logleve
[jira] [Commented] (OOZIE-2740) oozie help misspelled coordinator (coordiantor) and retrieved (retreived)
[
https://issues.apache.org/jira/browse/OOZIE-2740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15712350#comment-15712350
]
Ferenc Denes commented on OOZIE-2740:
-
+1 (nonbinding)
> oozie help misspelled coordinator (coordiantor) and retrieved (retreived)
> -
>
> Key: OOZIE-2740
> URL: https://issues.apache.org/jira/browse/OOZIE-2740
> Project: Oozie
> Issue Type: Bug
> Components: client
>Affects Versions: 4.1.0
>Reporter: Grant Sohn
>Priority: Trivial
> Attachments: OOZIE-2740.1.patch, OOZIE-2740.2.patch
>
>
> Spelling errors and concatenated words.
> {noformat}
> oozie help
> usage:
> the env variable 'OOZIE_URL' is used as default value for the '-oozie'
> option
> the env variable 'OOZIE_TIMEZONE' is used as default value for the
> '-timezone' option
> the env variable 'OOZIE_AUTH' is used as default value for the '-auth'
> option
> custom headers for Oozie web services can be specified using
> '-Dheader:NAME=VALUE'
> oozie help : display usage for all commands or specified command
> oozie version : show client version
> oozie job : job operations
> -action coordinator rerun/kill on action ids
> (requires -rerun/-kill);
>coordinator log retrieval on action
> ids(requires -log)
> -allruns Get workflow jobs corresponding to a
> coordinator action
>including all the reruns
> -auth select authentication type
> [SIMPLE|KERBEROS]
> -change change a coordinator or bundle job
> -config job configuration file '.xml' or
> '.properties'
> -configcontentjob configuration
> -coordinator bundle rerun on coordinator names
> (requires -rerun)
> -D set/override value for given property
> -date coordinator/bundle rerun on action
> dates (requires -rerun);
>coordinator log retrieval on action
> dates (requires -log)
> -debug Use debug mode to see debugging
> statements on stdout
> -definition job definition
> -diff Show diff of the new coord definition
> and properties with the
>existing one (default true)
> -doas doAs user, impersonates as the
> specified user
> -dryrunDryrun a workflow (since 3.3.2) or
> coordinator (since 2.0)
>job without actually executing it
> -failedruns the failed workflow actions of
> the coordinator actions
>(requires -rerun)
> -filter
> [;]*
>(All Coordinator actions satisfying
> the filters will be
>retreived).
>key: status or nominaltime
>comparator: =, !=, <, <=, >, >=. = is
> used as OR and others
>as AND
>status: values are valid status like
> SUCCEEDED, KILLED etc.
>Only = and != apply for status
>nominaltime: time of format
> -MM-dd'T'HH:mm'Z'
> -ignore change status of a coordinator job or
> action to IGNORED
>(-action required to ignore coord
> actions)
> -info info of a job
> -interval polling interval in minutes (default
> is 5, requires -poll)
> -kill kill a job (coordinator can mention
> -action or -date)
> -len number of actions (default TOTAL
> ACTIONS, requires -info)
> -localtime use local time (same as passing your
> time zone to -timezone).
>Overrides -timezone option
> -log job log
> -logfilterjob log search parameter. Can be
> specified as -logfilter
>opt1=val1;opt2=val1;opt3=val1.
> Supported options are recent,
>start, end, loglevel, text, limit and
> debug
> -nocleanup do not clean up output-events of the
> coordiantor rerun
>actions (
[jira] [Commented] (OOZIE-2598) OYA: Shell Action
[ https://issues.apache.org/jira/browse/OOZIE-2598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15547944#comment-15547944 ] Ferenc Denes commented on OOZIE-2598: - You catch 3 kind of exceptions, and rethrow it with the same message. Can you change the messages to capture the original type instead of the generic "Exception". > OYA: Shell Action > - > > Key: OOZIE-2598 > URL: https://issues.apache.org/jira/browse/OOZIE-2598 > Project: Oozie > Issue Type: Sub-task >Affects Versions: oya >Reporter: Robert Kanter >Assignee: Peter Bacsko > Fix For: oya > > Attachments: OOZIE-2598.01.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2662) DB migration fails if DB is too big
[ https://issues.apache.org/jira/browse/OOZIE-2662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Denes updated OOZIE-2662: Summary: DB migration fails if DB is too big (was: DB migration fails is DB is too big) > DB migration fails if DB is too big > --- > > Key: OOZIE-2662 > URL: https://issues.apache.org/jira/browse/OOZIE-2662 > Project: Oozie > Issue Type: Bug >Reporter: Peter Cseh >Assignee: Andras Piros > Attachments: OOZIE-2662.001.patch > > > The initial version of the DB import tool commits all the workflows, actions > etc. in one huge commit. If it does not fits into the memory, AOOME is thrown. > We should commit every 1k or 10k elements to prevent this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2458) 'oozie-setup.sh sharelib create' should ensure uploaded jars are world readable
[ https://issues.apache.org/jira/browse/OOZIE-2458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15389631#comment-15389631 ] Ferenc Denes commented on OOZIE-2458: - We want to keep the read only, so we mask out all others. > 'oozie-setup.sh sharelib create' should ensure uploaded jars are world > readable > --- > > Key: OOZIE-2458 > URL: https://issues.apache.org/jira/browse/OOZIE-2458 > Project: Oozie > Issue Type: Bug > Components: scripts >Affects Versions: 4.1.0 >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Critical > Fix For: trunk > > Attachments: OOZIE-2458-1.patch > > > If the default umask of HDFS does not include world read then the Oozie > sharelib gets created accordingly, and then the jars can not be read at > runtime because Oozie workflows impersonate the submitting user. > The 'oozie-setup.sh sharelib create' command should ensure that the installed > jars are world readable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2507) Expose monitoring via JMX beans in Oozie
[ https://issues.apache.org/jira/browse/OOZIE-2507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15376826#comment-15376826 ] Ferenc Denes commented on OOZIE-2507: - Thanks [~rkanter] > Expose monitoring via JMX beans in Oozie > > > Key: OOZIE-2507 > URL: https://issues.apache.org/jira/browse/OOZIE-2507 > Project: Oozie > Issue Type: Improvement > Components: monitoring >Affects Versions: trunk >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Minor > Fix For: trunk > > Attachments: OOZIE-2507-4.patch, OOZIE-2507-6.patch, > OOZIE-2507-7.patch, OOZIE-2507-8.patch > > > We should please extend oozie to allow monitoring via JMX beans with Zabbix, > similar to other apps like Jconsole, Nagios, IBM Tivoli, HP OpenVIew, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2507) Expose monitoring via JMX beans in Oozie
[ https://issues.apache.org/jira/browse/OOZIE-2507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15335911#comment-15335911 ] Ferenc Denes commented on OOZIE-2507: - The failed test case is unrelated. > Expose monitoring via JMX beans in Oozie > > > Key: OOZIE-2507 > URL: https://issues.apache.org/jira/browse/OOZIE-2507 > Project: Oozie > Issue Type: Improvement > Components: monitoring >Affects Versions: trunk >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Minor > Fix For: 4.1.0 > > Attachments: OOZIE-2507-4.patch, OOZIE-2507-6.patch, > OOZIE-2507-7.patch, OOZIE-2507-8.patch > > > We should please extend oozie to allow monitoring via JMX beans with Zabbix, > similar to other apps like Jconsole, Nagios, IBM Tivoli, HP OpenVIew, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2507) Expose monitoring via JMX beans in Oozie
[ https://issues.apache.org/jira/browse/OOZIE-2507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Denes updated OOZIE-2507: Attachment: OOZIE-2507-8.patch > Expose monitoring via JMX beans in Oozie > > > Key: OOZIE-2507 > URL: https://issues.apache.org/jira/browse/OOZIE-2507 > Project: Oozie > Issue Type: Improvement > Components: monitoring >Affects Versions: trunk >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Minor > Fix For: 4.1.0 > > Attachments: OOZIE-2507-4.patch, OOZIE-2507-6.patch, > OOZIE-2507-7.patch, OOZIE-2507-8.patch > > > We should please extend oozie to allow monitoring via JMX beans with Zabbix, > similar to other apps like Jconsole, Nagios, IBM Tivoli, HP OpenVIew, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2550) Flaky tests in TestZKUUIDService.java
[
https://issues.apache.org/jira/browse/OOZIE-2550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15322479#comment-15322479
]
Ferenc Denes commented on OOZIE-2550:
-
I think it is a good point to log the thread's name/id to see where a
particular thing happened.
On the other hand it should not be hidden in a flaky test jira.
> Flaky tests in TestZKUUIDService.java
> -
>
> Key: OOZIE-2550
> URL: https://issues.apache.org/jira/browse/OOZIE-2550
> Project: Oozie
> Issue Type: Bug
> Components: tests
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Minor
> Attachments: OOZIE-2550-005.patch
>
>
> 1. Test case testMultipleIDGeneration_withMultiThread in TestZKUUIDService
> uses an ArrayList which is written by two threads simultaneously. This is
> dangerous and the list must be externally synchronized to prevent race
> conditions.
> Another problem is that you cannot put items at arbitrary indexes in an
> ArrayList. For example, if the list is empty, the following code throws
> ArrayIndexOutOfBoundException:
> {code}
> List test = new ArrayList<>(1);
> test.add(22, true);
> {code}
> In an unlucky scheduling event, the following can happen:
> 1. The list has a certain number of elements, the value of "size" inside the
> list is 1000
> 2. Thread-1 retrieves the next ID from ZK UUID service, which is 1001
> 3. Thread-2 retrieves the next ID from ZK UUID service, which is 1002
> 4. Thread-2 runs faster than Thread-1 and tries to call {{list.add(1002,
> true)}} which fails.
> The following error was caught during a test run:
> {code}
> Tests run: 8, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 53.571 sec
> <<< FAILURE!
> testMultipleIDGeneration_withMultiThread(org.apache.oozie.service.TestZKUUIDService)
> Time elapsed: 0.02 sec <<< ERROR!
> java.lang.IndexOutOfBoundsException: Index: 89, Size: 89
> at java.util.ArrayList.rangeCheck(ArrayList.java:653)
> at java.util.ArrayList.get(ArrayList.java:429)
> at
> org.apache.oozie.service.TestZKUUIDService.testMultipleIDGeneration_withMultiThread(TestZKUUIDService.java:134)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at junit.framework.TestCase.runTest(TestCase.java:168)
> at junit.framework.TestCase.runBare(TestCase.java:134)
> at junit.framework.TestResult$1.protect(TestResult.java:110)
> at junit.framework.TestResult.runProtected(TestResult.java:128)
> at junit.framework.TestResult.run(TestResult.java:113)
> at junit.framework.TestCase.run(TestCase.java:124)
> at junit.framework.TestSuite.runTest(TestSuite.java:243)
> at junit.framework.TestSuite.run(TestSuite.java:238)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
> at org.junit.runners.Suite.runChild(Suite.java:128)
> at org.junit.runners.Suite.runChild(Suite.java:24)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> 2. The order in which the tests are executed is random. The problem is that
> testResetSequence sets the maximum sequence number to 900. Because this value
> is stored in a static variable inside ZKUUIDService, it affects
> testIDGeneration and testMultipleIDGeneration if these tests are run after
> testResetSequence.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-2507) Expose monitoring via JMX beans in Oozie
[ https://issues.apache.org/jira/browse/OOZIE-2507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15314122#comment-15314122 ] Ferenc Denes commented on OOZIE-2507: - [~rkanter] thanks for the review, I have implemented all of them, and attached a new patch. > Expose monitoring via JMX beans in Oozie > > > Key: OOZIE-2507 > URL: https://issues.apache.org/jira/browse/OOZIE-2507 > Project: Oozie > Issue Type: Improvement > Components: monitoring >Affects Versions: trunk >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Minor > Fix For: 4.1.0 > > Attachments: OOZIE-2507-4.patch, OOZIE-2507-6.patch, > OOZIE-2507-7.patch > > > We should please extend oozie to allow monitoring via JMX beans with Zabbix, > similar to other apps like Jconsole, Nagios, IBM Tivoli, HP OpenVIew, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2507) Expose monitoring via JMX beans in Oozie
[ https://issues.apache.org/jira/browse/OOZIE-2507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Denes updated OOZIE-2507: Attachment: OOZIE-2507-7.patch > Expose monitoring via JMX beans in Oozie > > > Key: OOZIE-2507 > URL: https://issues.apache.org/jira/browse/OOZIE-2507 > Project: Oozie > Issue Type: Improvement > Components: monitoring >Affects Versions: trunk >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Minor > Fix For: 4.1.0 > > Attachments: OOZIE-2507-4.patch, OOZIE-2507-6.patch, > OOZIE-2507-7.patch > > > We should please extend oozie to allow monitoring via JMX beans with Zabbix, > similar to other apps like Jconsole, Nagios, IBM Tivoli, HP OpenVIew, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2550) Flaky tests in TestZKUUIDService.java
[
https://issues.apache.org/jira/browse/OOZIE-2550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15313948#comment-15313948
]
Ferenc Denes commented on OOZIE-2550:
-
[~pbacsko] Good catch in principal.
What I would change:
1. Destroy the ZKUUIDService created at setup method.
2. The array you have created holds references to Boolean variables null at
default. The assertTrue has a boolean parameter so it will throw a NPE at
unboxing when one is not set. You should fill the AtomicReferenceArray with
False values at startup. (AtomicReferenceArray is fine here, but you might
consider using a native Boolean[] instead of this structure for this usecase as
a more lightweight solution)
> Flaky tests in TestZKUUIDService.java
> -
>
> Key: OOZIE-2550
> URL: https://issues.apache.org/jira/browse/OOZIE-2550
> Project: Oozie
> Issue Type: Bug
> Components: tests
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Minor
> Attachments: OOZIE-2550-001.patch, OOZIE-2550-002.patch
>
>
> 1. Test case testMultipleIDGeneration_withMultiThread in TestZKUUIDService
> uses an ArrayList which is written by two threads simultaneously. This is
> dangerous and the list must be externally synchronized to prevent race
> conditions.
> Another problem is that you cannot put items at arbitrary indexes in an
> ArrayList. For example, if the list is empty, the following code throws
> ArrayIndexOutOfBoundException:
> {code}
> List test = new ArrayList<>(1);
> test.add(22, true);
> {code}
> In an unlucky scheduling event, the following can happen:
> 1. The list has a certain number of elements, the value of "size" inside the
> list is 1000
> 2. Thread-1 retrieves the next ID from ZK UUID service, which is 1001
> 3. Thread-2 retrieves the next ID from ZK UUID service, which is 1002
> 4. Thread-2 runs faster than Thread-1 and tries to call {{list.add(1002,
> true)}} which fails.
> The following error was caught during a test run:
> {code}
> Tests run: 8, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 53.571 sec
> <<< FAILURE!
> testMultipleIDGeneration_withMultiThread(org.apache.oozie.service.TestZKUUIDService)
> Time elapsed: 0.02 sec <<< ERROR!
> java.lang.IndexOutOfBoundsException: Index: 89, Size: 89
> at java.util.ArrayList.rangeCheck(ArrayList.java:653)
> at java.util.ArrayList.get(ArrayList.java:429)
> at
> org.apache.oozie.service.TestZKUUIDService.testMultipleIDGeneration_withMultiThread(TestZKUUIDService.java:134)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at junit.framework.TestCase.runTest(TestCase.java:168)
> at junit.framework.TestCase.runBare(TestCase.java:134)
> at junit.framework.TestResult$1.protect(TestResult.java:110)
> at junit.framework.TestResult.runProtected(TestResult.java:128)
> at junit.framework.TestResult.run(TestResult.java:113)
> at junit.framework.TestCase.run(TestCase.java:124)
> at junit.framework.TestSuite.runTest(TestSuite.java:243)
> at junit.framework.TestSuite.run(TestSuite.java:238)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
> at org.junit.runners.Suite.runChild(Suite.java:128)
> at org.junit.runners.Suite.runChild(Suite.java:24)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> 2. The order in which the tests are executed is random. The problem is that
> testResetSequence sets the maximum sequence number to 900. Because this value
> is stored in a static variable inside ZKUUIDService, it affects
> testIDGeneration and testMultipleIDGeneration if these tests are run after
> testResetSequence.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-2482) Pyspark job fails with Oozie
[
https://issues.apache.org/jira/browse/OOZIE-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15245547#comment-15245547
]
Ferenc Denes commented on OOZIE-2482:
-
Please feel free to work on that, I'm not close to the solution yet, and tied
down with other issues.
> Pyspark job fails with Oozie
>
>
> Key: OOZIE-2482
> URL: https://issues.apache.org/jira/browse/OOZIE-2482
> Project: Oozie
> Issue Type: Bug
> Components: core, workflow
>Affects Versions: 4.2.0
> Environment: Hadoop 2.7.2, Spark 1.6.0 on Yarn, Oozie 4.2.0
> Cluster secured with Kerberos
>Reporter: Alexandre Linte
>Assignee: Ferenc Denes
>
> Hello,
> I'm trying to run pi.py example in a pyspark job with Oozie. Every try I made
> failed for the same reason: key not found: SPARK_HOME.
> Note: A scala job works well in the environment with Oozie.
> The logs on the executors are:
> {noformat}
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/mnt/hd4/hadoop/yarn/local/filecache/145/slf4j-log4j12-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/mnt/hd2/hadoop/yarn/local/filecache/155/spark-assembly-1.6.0-hadoop2.7.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/opt/application/Hadoop/hadoop-2.7.2/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> log4j:ERROR setFile(null,true) call failed.
> java.io.FileNotFoundException:
> /mnt/hd7/hadoop/yarn/log/application_1454673025841_13136/container_1454673025841_13136_01_01
> (Is a directory)
> at java.io.FileOutputStream.open(Native Method)
> at java.io.FileOutputStream.(FileOutputStream.java:221)
> at java.io.FileOutputStream.(FileOutputStream.java:142)
> at org.apache.log4j.FileAppender.setFile(FileAppender.java:294)
> at
> org.apache.log4j.FileAppender.activateOptions(FileAppender.java:165)
> at
> org.apache.hadoop.yarn.ContainerLogAppender.activateOptions(ContainerLogAppender.java:55)
> at
> org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307)
> at
> org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:172)
> at
> org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:104)
> at
> org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:809)
> at
> org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:735)
> at
> org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:615)
> at
> org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:502)
> at
> org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:547)
> at
> org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:483)
> at org.apache.log4j.LogManager.(LogManager.java:127)
> at
> org.slf4j.impl.Log4jLoggerFactory.getLogger(Log4jLoggerFactory.java:64)
> at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:285)
> at
> org.apache.commons.logging.impl.SLF4JLogFactory.getInstance(SLF4JLogFactory.java:155)
> at
> org.apache.commons.logging.impl.SLF4JLogFactory.getInstance(SLF4JLogFactory.java:132)
> at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:275)
> at
> org.apache.hadoop.service.AbstractService.(AbstractService.java:43)
> Using properties file: null
> Parsed arguments:
> master yarn-master
> deployMode cluster
> executorMemory null
> executorCores null
> totalExecutorCores null
> propertiesFile null
> driverMemorynull
> driverCores null
> driverExtraClassPathnull
> driverExtraLibraryPath null
> driverExtraJavaOptions null
> supervise false
> queue null
> numExecutorsnull
> files null
> pyFiles null
> archivesnull
> mainClass null
> primaryResource
> hdfs://hadoopsandbox/User/toto/WORK/Oozie/pyspark/lib/pi.py
> namePysparkpi example
> childArgs [100]
> jarsnull
> packagesnull
> packagesExclusions null
> repositoriesnull
> verbose true
> Spark properties used, including those specified through
> --conf and those from the properties file null:
> spark.e
[jira] [Assigned] (OOZIE-2482) Pyspark job fails with Oozie
[
https://issues.apache.org/jira/browse/OOZIE-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ferenc Denes reassigned OOZIE-2482:
---
Assignee: Ferenc Denes (was: Satish Subhashrao Saley)
> Pyspark job fails with Oozie
>
>
> Key: OOZIE-2482
> URL: https://issues.apache.org/jira/browse/OOZIE-2482
> Project: Oozie
> Issue Type: Bug
> Components: core, workflow
>Affects Versions: 4.2.0
> Environment: Hadoop 2.7.2, Spark 1.6.0 on Yarn, Oozie 4.2.0
> Cluster secured with Kerberos
>Reporter: Alexandre Linte
>Assignee: Ferenc Denes
>
> Hello,
> I'm trying to run pi.py example in a pyspark job with Oozie. Every try I made
> failed for the same reason: key not found: SPARK_HOME.
> Note: A scala job works well in the environment with Oozie.
> The logs on the executors are:
> {noformat}
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/mnt/hd4/hadoop/yarn/local/filecache/145/slf4j-log4j12-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/mnt/hd2/hadoop/yarn/local/filecache/155/spark-assembly-1.6.0-hadoop2.7.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/opt/application/Hadoop/hadoop-2.7.2/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> log4j:ERROR setFile(null,true) call failed.
> java.io.FileNotFoundException:
> /mnt/hd7/hadoop/yarn/log/application_1454673025841_13136/container_1454673025841_13136_01_01
> (Is a directory)
> at java.io.FileOutputStream.open(Native Method)
> at java.io.FileOutputStream.(FileOutputStream.java:221)
> at java.io.FileOutputStream.(FileOutputStream.java:142)
> at org.apache.log4j.FileAppender.setFile(FileAppender.java:294)
> at
> org.apache.log4j.FileAppender.activateOptions(FileAppender.java:165)
> at
> org.apache.hadoop.yarn.ContainerLogAppender.activateOptions(ContainerLogAppender.java:55)
> at
> org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307)
> at
> org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:172)
> at
> org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:104)
> at
> org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:809)
> at
> org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:735)
> at
> org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:615)
> at
> org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:502)
> at
> org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:547)
> at
> org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:483)
> at org.apache.log4j.LogManager.(LogManager.java:127)
> at
> org.slf4j.impl.Log4jLoggerFactory.getLogger(Log4jLoggerFactory.java:64)
> at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:285)
> at
> org.apache.commons.logging.impl.SLF4JLogFactory.getInstance(SLF4JLogFactory.java:155)
> at
> org.apache.commons.logging.impl.SLF4JLogFactory.getInstance(SLF4JLogFactory.java:132)
> at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:275)
> at
> org.apache.hadoop.service.AbstractService.(AbstractService.java:43)
> Using properties file: null
> Parsed arguments:
> master yarn-master
> deployMode cluster
> executorMemory null
> executorCores null
> totalExecutorCores null
> propertiesFile null
> driverMemorynull
> driverCores null
> driverExtraClassPathnull
> driverExtraLibraryPath null
> driverExtraJavaOptions null
> supervise false
> queue null
> numExecutorsnull
> files null
> pyFiles null
> archivesnull
> mainClass null
> primaryResource
> hdfs://hadoopsandbox/User/toto/WORK/Oozie/pyspark/lib/pi.py
> namePysparkpi example
> childArgs [100]
> jarsnull
> packagesnull
> packagesExclusions null
> repositoriesnull
> verbose true
> Spark properties used, including those specified through
> --conf and those from the properties file null:
> spark.executorEnv.SPARK_HOME -> /opt/application/Spark/current
> spark.executorEnv.PYTHONPATH -> /op
[jira] [Commented] (OOZIE-2507) Expose monitoring via JMX beans in Oozie
[ https://issues.apache.org/jira/browse/OOZIE-2507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15240005#comment-15240005 ] Ferenc Denes commented on OOZIE-2507: - [~rkanter] thanks for the review. 1. WE cannot do too much, I don't like to add Sun specific jars as well. 2, 3 valid points. Fixed them, and also added documentation for the setting. > Expose monitoring via JMX beans in Oozie > > > Key: OOZIE-2507 > URL: https://issues.apache.org/jira/browse/OOZIE-2507 > Project: Oozie > Issue Type: Improvement > Components: monitoring >Affects Versions: trunk >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Minor > Fix For: 4.1.0 > > Attachments: OOZIE-2507-4.patch, OOZIE-2507-6.patch > > > We should please extend oozie to allow monitoring via JMX beans with Zabbix, > similar to other apps like Jconsole, Nagios, IBM Tivoli, HP OpenVIew, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2507) Expose monitoring via JMX beans in Oozie
[ https://issues.apache.org/jira/browse/OOZIE-2507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Denes updated OOZIE-2507: Attachment: (was: OOZIE-2507-5.patch) > Expose monitoring via JMX beans in Oozie > > > Key: OOZIE-2507 > URL: https://issues.apache.org/jira/browse/OOZIE-2507 > Project: Oozie > Issue Type: Improvement > Components: monitoring >Affects Versions: trunk >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Minor > Fix For: 4.1.0 > > Attachments: OOZIE-2507-4.patch, OOZIE-2507-6.patch > > > We should please extend oozie to allow monitoring via JMX beans with Zabbix, > similar to other apps like Jconsole, Nagios, IBM Tivoli, HP OpenVIew, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2507) Expose monitoring via JMX beans in Oozie
[ https://issues.apache.org/jira/browse/OOZIE-2507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Denes updated OOZIE-2507: Attachment: OOZIE-2507-6.patch > Expose monitoring via JMX beans in Oozie > > > Key: OOZIE-2507 > URL: https://issues.apache.org/jira/browse/OOZIE-2507 > Project: Oozie > Issue Type: Improvement > Components: monitoring >Affects Versions: trunk >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Minor > Fix For: 4.1.0 > > Attachments: OOZIE-2507-4.patch, OOZIE-2507-6.patch > > > We should please extend oozie to allow monitoring via JMX beans with Zabbix, > similar to other apps like Jconsole, Nagios, IBM Tivoli, HP OpenVIew, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2507) Expose monitoring via JMX beans in Oozie
[ https://issues.apache.org/jira/browse/OOZIE-2507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Denes updated OOZIE-2507: Attachment: OOZIE-2507-5.patch > Expose monitoring via JMX beans in Oozie > > > Key: OOZIE-2507 > URL: https://issues.apache.org/jira/browse/OOZIE-2507 > Project: Oozie > Issue Type: Improvement > Components: monitoring >Affects Versions: trunk >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Minor > Fix For: 4.1.0 > > Attachments: OOZIE-2507-4.patch, OOZIE-2507-5.patch > > > We should please extend oozie to allow monitoring via JMX beans with Zabbix, > similar to other apps like Jconsole, Nagios, IBM Tivoli, HP OpenVIew, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2507) Expose monitoring via JMX beans in Oozie
[ https://issues.apache.org/jira/browse/OOZIE-2507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Denes updated OOZIE-2507: Attachment: OOZIE-2507-4.patch > Expose monitoring via JMX beans in Oozie > > > Key: OOZIE-2507 > URL: https://issues.apache.org/jira/browse/OOZIE-2507 > Project: Oozie > Issue Type: Improvement > Components: monitoring >Affects Versions: trunk >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Minor > Fix For: 4.1.0 > > Attachments: OOZIE-2507-4.patch > > > We should please extend oozie to allow monitoring via JMX beans with Zabbix, > similar to other apps like Jconsole, Nagios, IBM Tivoli, HP OpenVIew, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2507) Expose monitoring via JMX beans in Oozie
[ https://issues.apache.org/jira/browse/OOZIE-2507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Denes updated OOZIE-2507: Attachment: (was: OOZIE-2507-2.patch) > Expose monitoring via JMX beans in Oozie > > > Key: OOZIE-2507 > URL: https://issues.apache.org/jira/browse/OOZIE-2507 > Project: Oozie > Issue Type: Improvement > Components: monitoring >Affects Versions: trunk >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Minor > Fix For: 4.1.0 > > > We should please extend oozie to allow monitoring via JMX beans with Zabbix, > similar to other apps like Jconsole, Nagios, IBM Tivoli, HP OpenVIew, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2507) Expose monitoring via JMX beans in Oozie
[ https://issues.apache.org/jira/browse/OOZIE-2507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Denes updated OOZIE-2507: Attachment: (was: OOZIE-2507-3.patch) > Expose monitoring via JMX beans in Oozie > > > Key: OOZIE-2507 > URL: https://issues.apache.org/jira/browse/OOZIE-2507 > Project: Oozie > Issue Type: Improvement > Components: monitoring >Affects Versions: trunk >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Minor > Fix For: 4.1.0 > > > We should please extend oozie to allow monitoring via JMX beans with Zabbix, > similar to other apps like Jconsole, Nagios, IBM Tivoli, HP OpenVIew, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2507) Expose monitoring via JMX beans in Oozie
[ https://issues.apache.org/jira/browse/OOZIE-2507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Denes updated OOZIE-2507: Attachment: (was: OOZIE-2507-1.patch) > Expose monitoring via JMX beans in Oozie > > > Key: OOZIE-2507 > URL: https://issues.apache.org/jira/browse/OOZIE-2507 > Project: Oozie > Issue Type: Improvement > Components: monitoring >Affects Versions: trunk >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Minor > Fix For: 4.1.0 > > > We should please extend oozie to allow monitoring via JMX beans with Zabbix, > similar to other apps like Jconsole, Nagios, IBM Tivoli, HP OpenVIew, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2507) Expose monitoring via JMX beans in Oozie
[ https://issues.apache.org/jira/browse/OOZIE-2507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Denes updated OOZIE-2507: Attachment: OOZIE-2507-3.patch > Expose monitoring via JMX beans in Oozie > > > Key: OOZIE-2507 > URL: https://issues.apache.org/jira/browse/OOZIE-2507 > Project: Oozie > Issue Type: Improvement > Components: monitoring >Affects Versions: trunk >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Minor > Fix For: 4.1.0 > > Attachments: OOZIE-2507-1.patch, OOZIE-2507-2.patch, > OOZIE-2507-3.patch > > > We should please extend oozie to allow monitoring via JMX beans with Zabbix, > similar to other apps like Jconsole, Nagios, IBM Tivoli, HP OpenVIew, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2506) Add logs into RecoverService for logging information about queued commnads
[ https://issues.apache.org/jira/browse/OOZIE-2506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15235060#comment-15235060 ] Ferenc Denes commented on OOZIE-2506: - +1 (nonbinding) > Add logs into RecoverService for logging information about queued commnads > -- > > Key: OOZIE-2506 > URL: https://issues.apache.org/jira/browse/OOZIE-2506 > Project: Oozie > Issue Type: Bug > Components: core >Reporter: abhishek bafna >Assignee: abhishek bafna > Fix For: 4.3.0 > > Attachments: OOZIE-2506-01.patch, OOZIE-2506-02.patch > > > Currently, RecoveryService does not logs information about workflow actions > commands and coordinator commands. It just logs the counter for different > commands got queued. Logging the different commands after queuing can be > helpful in debugging a system which have a lot work load. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2507) Expose monitoring via JMX beans in Oozie
[ https://issues.apache.org/jira/browse/OOZIE-2507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15234657#comment-15234657 ] Ferenc Denes commented on OOZIE-2507: - Test case is unrelated > Expose monitoring via JMX beans in Oozie > > > Key: OOZIE-2507 > URL: https://issues.apache.org/jira/browse/OOZIE-2507 > Project: Oozie > Issue Type: Improvement > Components: monitoring >Affects Versions: trunk >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Minor > Fix For: 4.1.0 > > Attachments: OOZIE-2507-1.patch, OOZIE-2507-2.patch > > > We should please extend oozie to allow monitoring via JMX beans with Zabbix, > similar to other apps like Jconsole, Nagios, IBM Tivoli, HP OpenVIew, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2489) XML parsing is vulnerable
[ https://issues.apache.org/jira/browse/OOZIE-2489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15234655#comment-15234655 ] Ferenc Denes commented on OOZIE-2489: - Security patch we do not need to modify tests here. > XML parsing is vulnerable > - > > Key: OOZIE-2489 > URL: https://issues.apache.org/jira/browse/OOZIE-2489 > Project: Oozie > Issue Type: Bug >Affects Versions: 4.1.0 >Reporter: Ferenc Denes >Assignee: Ferenc Denes > Labels: security, xml > Fix For: trunk > > Attachments: OOZIE-2489-1.patch, OOZIE-2489-2.patch, > OOZIE-2489-3.patch > > > The XML parsing has some security problems: > XML External Entity attack: > XML External Entities attacks benefit from an XML feature to build documents > dynamically at the time of processing. An XML entity allows inclusion of data > dynamically from a given resource. External entities allow an XML document to > include data from an external URI. Unless configured to do otherwise, > external entities force the XML parser to access the resource specified by > the URI, e.g., a file on the local machine or on a remote system. This > behavior exposes the application to XML External Entity (XXE) attacks, which > can be used to perform denial of service of the local system, gain > unauthorized access to files on the local machine, scan remote machines, and > perform denial of service of remote systems. > The following XML document shows an example of an XXE attack. > > > ]>&xxe; > This example could crash the server (on a UNIX system), if the XML parser > attempts to substitute the entity with the contents of the /dev/random file. > XML Entity Expansion injection also known as XML Bombs are DoS attacks that > benefit from valid and well-formed XML blocks that expand exponentially until > they exhaust the server allocated resources. XML allows to define custom > entities which act as string substitution macros. By nesting recurrent entity > resolutions, an attacker can easily crash the server resources. > The following XML document shows an example of an XML Bomb. > > > > > > > > > > > ]> > &lol9; > Both problems can be solved by setting features and parameters of the XML > parser factories. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2505) space on classpath causes error in spark-job
[
https://issues.apache.org/jira/browse/OOZIE-2505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15232856#comment-15232856
]
Ferenc Denes commented on OOZIE-2505:
-
+1 (nonbinding), test cases are not related, test cases are not attached as it
mends the tests themselves.
> space on classpath causes error in spark-job
>
>
> Key: OOZIE-2505
> URL: https://issues.apache.org/jira/browse/OOZIE-2505
> Project: Oozie
> Issue Type: Bug
>Reporter: Peter Cseh
>Assignee: Peter Cseh
> Attachments: OOZIE-2505-001.patch, OOZIE-2505-002.patch
>
>
> When I tried to run TestSparkMain from IDEA I got the following error:
> {quote}
> java.lang.reflect.UndeclaredThrowableException: Unknown exception in doAs
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1203)
>
> Caused by: java.security.PrivilegedActionException:
> java.net.URISyntaxException: Illegal character in path at index 60:
> /opt/homebrew-cask/Caskroom/intellij-idea-ce/2016.1/IntelliJ IDEA
> CE.app/Contents/lib/idea_rt.jar
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> ...
> at org.apache.spark.util.Utils$.resolveURI(Utils.scala:1343)
> at
> org.apache.spark.util.Utils$$anonfun$resolveURIs$1.apply(Utils.scala:1366)
> at
> org.apache.spark.util.Utils$$anonfun$resolveURIs$1.apply(Utils.scala:1366)
> {quote}
> Spaces in the paths to the jarfiles should be escaped to %20.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-2507) Expose monitoring via JMX beans in Oozie
[ https://issues.apache.org/jira/browse/OOZIE-2507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15232844#comment-15232844 ] Ferenc Denes commented on OOZIE-2507: - Tested it by attaching jconsole as a jmx client as well as with the unit test (which failed before the implementation was in place) > Expose monitoring via JMX beans in Oozie > > > Key: OOZIE-2507 > URL: https://issues.apache.org/jira/browse/OOZIE-2507 > Project: Oozie > Issue Type: Improvement > Components: monitoring >Affects Versions: trunk >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Minor > Fix For: 4.1.0 > > Attachments: OOZIE-2507-1.patch, OOZIE-2507-2.patch > > > We should please extend oozie to allow monitoring via JMX beans with Zabbix, > similar to other apps like Jconsole, Nagios, IBM Tivoli, HP OpenVIew, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2507) Expose monitoring via JMX beans in Oozie
[ https://issues.apache.org/jira/browse/OOZIE-2507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Denes updated OOZIE-2507: Attachment: OOZIE-2507-2.patch > Expose monitoring via JMX beans in Oozie > > > Key: OOZIE-2507 > URL: https://issues.apache.org/jira/browse/OOZIE-2507 > Project: Oozie > Issue Type: Improvement > Components: monitoring >Affects Versions: trunk >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Minor > Fix For: 4.1.0 > > Attachments: OOZIE-2507-1.patch, OOZIE-2507-2.patch > > > We should please extend oozie to allow monitoring via JMX beans with Zabbix, > similar to other apps like Jconsole, Nagios, IBM Tivoli, HP OpenVIew, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2507) Expose monitoring via JMX beans in Oozie
[ https://issues.apache.org/jira/browse/OOZIE-2507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Denes updated OOZIE-2507: Attachment: OOZIE-2507-1.patch > Expose monitoring via JMX beans in Oozie > > > Key: OOZIE-2507 > URL: https://issues.apache.org/jira/browse/OOZIE-2507 > Project: Oozie > Issue Type: Improvement > Components: monitoring >Affects Versions: trunk >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Minor > Fix For: 4.1.0 > > Attachments: OOZIE-2507-1.patch > > > We should please extend oozie to allow monitoring via JMX beans with Zabbix, > similar to other apps like Jconsole, Nagios, IBM Tivoli, HP OpenVIew, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OOZIE-2507) Expose monitoring via JMX beans in Oozie
Ferenc Denes created OOZIE-2507: --- Summary: Expose monitoring via JMX beans in Oozie Key: OOZIE-2507 URL: https://issues.apache.org/jira/browse/OOZIE-2507 Project: Oozie Issue Type: Improvement Components: monitoring Affects Versions: trunk Reporter: Ferenc Denes Assignee: Ferenc Denes Priority: Minor Fix For: 4.1.0 We should please extend oozie to allow monitoring via JMX beans with Zabbix, similar to other apps like Jconsole, Nagios, IBM Tivoli, HP OpenVIew, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2391) spark-opts value in workflow.xml is not parsed properly
[
https://issues.apache.org/jira/browse/OOZIE-2391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15231903#comment-15231903
]
Ferenc Denes commented on OOZIE-2391:
-
+1 (nonbinding). Good catch on the jars parsing error as well.
> spark-opts value in workflow.xml is not parsed properly
> ---
>
> Key: OOZIE-2391
> URL: https://issues.apache.org/jira/browse/OOZIE-2391
> Project: Oozie
> Issue Type: Bug
>Affects Versions: trunk
>Reporter: Praveen Bathala
>Assignee: Peter Cseh
> Labels: newbie
> Attachments: OOZIE-2391-001.patch, OOZIE-2391-002.patch
>
>
> When providing spark-opts in workflow.xml for oozie, the "--conf" property
> values is not parsed properly.
> Example spark opts
> {code:xml}
> --conf
> spark.executor.extraJavaOptions=-Dconfig.url=hdfs:///my.conf
> -Dmy.env=${env}
> {code}
> In this case the value for property "spark.executor.extraJavaOptions" is
> "-Dconfig.url=hdfs:///my.conf -Dmy.env=${env}", but in the oozie code here
> https://github.com/apache/oozie/blob/38e33811bc3f9d3b1a1a024dcab6af760734b696/sharelib/spark/src/main/java/org/apache/oozie/action/hadoop/SparkMain.java#L110
> the string is just split by space, which make the "-Dmy.env=${env}" a
> separate argument to spark and spark fails job as that argument is not valid.
> This doesn't work with enclosing the whole value in quotes too.
> Error from stderr:
> Error: Unrecognized option '-Dmy.env=development"'.
> Run with --help for usage help or --verbose for debug output
> Intercepting System.exit(1)
> Failing Oozie Launcher, Main class
> [org.apache.oozie.action.hadoop.SparkMain], exit code [1]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (OOZIE-2489) XML parsing is vulnerable
[ https://issues.apache.org/jira/browse/OOZIE-2489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Denes updated OOZIE-2489: Attachment: OOZIE-2489-3.patch > XML parsing is vulnerable > - > > Key: OOZIE-2489 > URL: https://issues.apache.org/jira/browse/OOZIE-2489 > Project: Oozie > Issue Type: Bug >Affects Versions: 4.1.0 >Reporter: Ferenc Denes >Assignee: Ferenc Denes > Labels: security, xml > Fix For: trunk > > Attachments: OOZIE-2489-1.patch, OOZIE-2489-2.patch, > OOZIE-2489-3.patch > > > The XML parsing has some security problems: > XML External Entity attack: > XML External Entities attacks benefit from an XML feature to build documents > dynamically at the time of processing. An XML entity allows inclusion of data > dynamically from a given resource. External entities allow an XML document to > include data from an external URI. Unless configured to do otherwise, > external entities force the XML parser to access the resource specified by > the URI, e.g., a file on the local machine or on a remote system. This > behavior exposes the application to XML External Entity (XXE) attacks, which > can be used to perform denial of service of the local system, gain > unauthorized access to files on the local machine, scan remote machines, and > perform denial of service of remote systems. > The following XML document shows an example of an XXE attack. > > > ]>&xxe; > This example could crash the server (on a UNIX system), if the XML parser > attempts to substitute the entity with the contents of the /dev/random file. > XML Entity Expansion injection also known as XML Bombs are DoS attacks that > benefit from valid and well-formed XML blocks that expand exponentially until > they exhaust the server allocated resources. XML allows to define custom > entities which act as string substitution macros. By nesting recurrent entity > resolutions, an attacker can easily crash the server resources. > The following XML document shows an example of an XML Bomb. > > > > > > > > > > > ]> > &lol9; > Both problems can be solved by setting features and parameters of the XML > parser factories. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2492) JSON security issue in js code
[ https://issues.apache.org/jira/browse/OOZIE-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15215748#comment-15215748 ] Ferenc Denes commented on OOZIE-2492: - Test cases are not related. I have checked the GUI manually (as lack of unit tests) and saw it working with the new js. > JSON security issue in js code > -- > > Key: OOZIE-2492 > URL: https://issues.apache.org/jira/browse/OOZIE-2492 > Project: Oozie > Issue Type: Bug > Components: client, security >Affects Versions: 4.1.0 >Reporter: Ferenc Denes >Assignee: Ferenc Denes > Labels: security, web-console > Fix For: trunk > > Attachments: OOZIE-2492-1.patch > > > JSON parsing is done using the eval js method in several places in the > oozie-console.js, which allows code injection. > The project already contains a json parser library, which should be used all > around the code. > We are aware that most of the json documents parsed are from the oozie > server, and not from the user directly. However fixing it all will make the > code most robust and consistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2459) TestCoordActionInputCheckXCommand.testNone is flakey
[ https://issues.apache.org/jira/browse/OOZIE-2459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15215746#comment-15215746 ] Ferenc Denes commented on OOZIE-2459: - Funny, that the testNone failed. Any other ideas who might set the status to SUCCEEDED? > TestCoordActionInputCheckXCommand.testNone is flakey > > > Key: OOZIE-2459 > URL: https://issues.apache.org/jira/browse/OOZIE-2459 > Project: Oozie > Issue Type: Bug > Components: action, tests >Affects Versions: 4.1.0 >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Minor > Fix For: trunk > > Attachments: OOZIE-2459-1.patch > > > TestCoordActionInputCheckXCommand.testNone is flakey. > Also the TestCoordActionInputCheckXCommandNonUTC.testNone which is very > similar. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2489) XML parsing is vulnerable
[ https://issues.apache.org/jira/browse/OOZIE-2489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Denes updated OOZIE-2489: Attachment: OOZIE-2489-2.patch > XML parsing is vulnerable > - > > Key: OOZIE-2489 > URL: https://issues.apache.org/jira/browse/OOZIE-2489 > Project: Oozie > Issue Type: Bug >Affects Versions: 4.1.0 >Reporter: Ferenc Denes >Assignee: Ferenc Denes > Labels: security, xml > Fix For: trunk > > Attachments: OOZIE-2489-1.patch, OOZIE-2489-2.patch > > > The XML parsing has some security problems: > XML External Entity attack: > XML External Entities attacks benefit from an XML feature to build documents > dynamically at the time of processing. An XML entity allows inclusion of data > dynamically from a given resource. External entities allow an XML document to > include data from an external URI. Unless configured to do otherwise, > external entities force the XML parser to access the resource specified by > the URI, e.g., a file on the local machine or on a remote system. This > behavior exposes the application to XML External Entity (XXE) attacks, which > can be used to perform denial of service of the local system, gain > unauthorized access to files on the local machine, scan remote machines, and > perform denial of service of remote systems. > The following XML document shows an example of an XXE attack. > > > ]>&xxe; > This example could crash the server (on a UNIX system), if the XML parser > attempts to substitute the entity with the contents of the /dev/random file. > XML Entity Expansion injection also known as XML Bombs are DoS attacks that > benefit from valid and well-formed XML blocks that expand exponentially until > they exhaust the server allocated resources. XML allows to define custom > entities which act as string substitution macros. By nesting recurrent entity > resolutions, an attacker can easily crash the server resources. > The following XML document shows an example of an XML Bomb. > > > > > > > > > > > ]> > &lol9; > Both problems can be solved by setting features and parameters of the XML > parser factories. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2492) JSON security issue in js code
[ https://issues.apache.org/jira/browse/OOZIE-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15211647#comment-15211647 ] Ferenc Denes commented on OOZIE-2492: - Test cases are not related. > JSON security issue in js code > -- > > Key: OOZIE-2492 > URL: https://issues.apache.org/jira/browse/OOZIE-2492 > Project: Oozie > Issue Type: Bug > Components: client, security >Affects Versions: 4.1.0 >Reporter: Ferenc Denes >Assignee: Ferenc Denes > Labels: security, web-console > Fix For: trunk > > Attachments: OOZIE-2492-1.patch > > > JSON parsing is done using the eval js method in several places in the > oozie-console.js, which allows code injection. > The project already contains a json parser library, which should be used all > around the code. > We are aware that most of the json documents parsed are from the oozie > server, and not from the user directly. However fixing it all will make the > code most robust and consistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2492) JSON security issue in js code
[ https://issues.apache.org/jira/browse/OOZIE-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Denes updated OOZIE-2492: Attachment: OOZIE-2492-1.patch > JSON security issue in js code > -- > > Key: OOZIE-2492 > URL: https://issues.apache.org/jira/browse/OOZIE-2492 > Project: Oozie > Issue Type: Bug > Components: client, security >Affects Versions: 4.1.0 >Reporter: Ferenc Denes >Assignee: Ferenc Denes > Attachments: OOZIE-2492-1.patch > > > JSON parsing is done using the eval js method in several places in the > oozie-console.js, which allows code injection. > The project already contains a json parser library, which should be used all > around the code. > We are aware that most of the json documents parsed are from the oozie > server, and not from the user directly. However fixing it all will make the > code most robust and consistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OOZIE-2492) JSON security issue in js code
Ferenc Denes created OOZIE-2492: --- Summary: JSON security issue in js code Key: OOZIE-2492 URL: https://issues.apache.org/jira/browse/OOZIE-2492 Project: Oozie Issue Type: Bug Components: client, security Affects Versions: 4.1.0 Reporter: Ferenc Denes Assignee: Ferenc Denes JSON parsing is done using the eval js method in several places in the oozie-console.js, which allows code injection. The project already contains a json parser library, which should be used all around the code. We are aware that most of the json documents parsed are from the oozie server, and not from the user directly. However fixing it all will make the code most robust and consistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2429) TestEventGeneration test is flakey
[ https://issues.apache.org/jira/browse/OOZIE-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208226#comment-15208226 ] Ferenc Denes commented on OOZIE-2429: - [~rkanter] Thanks for the review and the fix > TestEventGeneration test is flakey > -- > > Key: OOZIE-2429 > URL: https://issues.apache.org/jira/browse/OOZIE-2429 > Project: Oozie > Issue Type: Bug > Components: action, tests >Affects Versions: trunk >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Minor > Fix For: trunk > > Attachments: OOZIE-2429-1.patch, OOZIE-2429-2.patch, > OOZIE-2429-3.patch, OOZIE-2429-3.patch, OOZIE-2429-addendum-1.patch, > OOZIE-2429-addendum-2.patch, OOZIE-2429-addendum-3.patch > > > TestEventGeneration's testForNoDuplicates fails time to time depending on the > circumstances of the test. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2489) XML parsing is vulnerable
[ https://issues.apache.org/jira/browse/OOZIE-2489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Denes updated OOZIE-2489: Attachment: OOZIE-2489-1.patch > XML parsing is vulnerable > - > > Key: OOZIE-2489 > URL: https://issues.apache.org/jira/browse/OOZIE-2489 > Project: Oozie > Issue Type: Bug >Affects Versions: 4.1.0 >Reporter: Ferenc Denes >Assignee: Ferenc Denes > Labels: security, xml > Fix For: trunk > > Attachments: OOZIE-2489-1.patch > > > The XML parsing has some security problems: > XML External Entity attack: > XML External Entities attacks benefit from an XML feature to build documents > dynamically at the time of processing. An XML entity allows inclusion of data > dynamically from a given resource. External entities allow an XML document to > include data from an external URI. Unless configured to do otherwise, > external entities force the XML parser to access the resource specified by > the URI, e.g., a file on the local machine or on a remote system. This > behavior exposes the application to XML External Entity (XXE) attacks, which > can be used to perform denial of service of the local system, gain > unauthorized access to files on the local machine, scan remote machines, and > perform denial of service of remote systems. > The following XML document shows an example of an XXE attack. > > > ]>&xxe; > This example could crash the server (on a UNIX system), if the XML parser > attempts to substitute the entity with the contents of the /dev/random file. > XML Entity Expansion injection also known as XML Bombs are DoS attacks that > benefit from valid and well-formed XML blocks that expand exponentially until > they exhaust the server allocated resources. XML allows to define custom > entities which act as string substitution macros. By nesting recurrent entity > resolutions, an attacker can easily crash the server resources. > The following XML document shows an example of an XML Bomb. > > > > > > > > > > > ]> > &lol9; > Both problems can be solved by setting features and parameters of the XML > parser factories. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OOZIE-2489) XML parsing is vulnerable
Ferenc Denes created OOZIE-2489: --- Summary: XML parsing is vulnerable Key: OOZIE-2489 URL: https://issues.apache.org/jira/browse/OOZIE-2489 Project: Oozie Issue Type: Bug Affects Versions: 4.1.0 Reporter: Ferenc Denes Assignee: Ferenc Denes Fix For: trunk The XML parsing has some security problems: XML External Entity attack: XML External Entities attacks benefit from an XML feature to build documents dynamically at the time of processing. An XML entity allows inclusion of data dynamically from a given resource. External entities allow an XML document to include data from an external URI. Unless configured to do otherwise, external entities force the XML parser to access the resource specified by the URI, e.g., a file on the local machine or on a remote system. This behavior exposes the application to XML External Entity (XXE) attacks, which can be used to perform denial of service of the local system, gain unauthorized access to files on the local machine, scan remote machines, and perform denial of service of remote systems. The following XML document shows an example of an XXE attack. ]>&xxe; This example could crash the server (on a UNIX system), if the XML parser attempts to substitute the entity with the contents of the /dev/random file. XML Entity Expansion injection also known as XML Bombs are DoS attacks that benefit from valid and well-formed XML blocks that expand exponentially until they exhaust the server allocated resources. XML allows to define custom entities which act as string substitution macros. By nesting recurrent entity resolutions, an attacker can easily crash the server resources. The following XML document shows an example of an XML Bomb. ]> &lol9; Both problems can be solved by setting features and parameters of the XML parser factories. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2429) TestEventGeneration test is flakey
[ https://issues.apache.org/jira/browse/OOZIE-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Denes updated OOZIE-2429: Attachment: OOZIE-2429-addendum-2.patch > TestEventGeneration test is flakey > -- > > Key: OOZIE-2429 > URL: https://issues.apache.org/jira/browse/OOZIE-2429 > Project: Oozie > Issue Type: Bug > Components: action, tests >Affects Versions: trunk >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Minor > Fix For: trunk > > Attachments: OOZIE-2429-1.patch, OOZIE-2429-2.patch, > OOZIE-2429-3.patch, OOZIE-2429-3.patch, OOZIE-2429-addendum-1.patch, > OOZIE-2429-addendum-2.patch > > > TestEventGeneration's testForNoDuplicates fails time to time depending on the > circumstances of the test. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2429) TestEventGeneration test is flakey
[ https://issues.apache.org/jira/browse/OOZIE-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15206147#comment-15206147 ] Ferenc Denes commented on OOZIE-2429: - [~rkanter] Thanks for the review. 1. I have checked the size of the queue in the previous line, just above the code quoted here. I made the change now. The assertEquals is there to catch if there are duplicate elements, in which case the size will be less than 3. 2. The braces are there to ensure that the variables are not mixed up. It is not only about formatting. Like it was previously, when one asserted on an unrelated variable by mistake in this very test. In this way the compiler checks this case for you. I have kept it as is. Attached the new patch. > TestEventGeneration test is flakey > -- > > Key: OOZIE-2429 > URL: https://issues.apache.org/jira/browse/OOZIE-2429 > Project: Oozie > Issue Type: Bug > Components: action, tests >Affects Versions: trunk >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Minor > Fix For: trunk > > Attachments: OOZIE-2429-1.patch, OOZIE-2429-2.patch, > OOZIE-2429-3.patch, OOZIE-2429-3.patch, OOZIE-2429-addendum-1.patch > > > TestEventGeneration's testForNoDuplicates fails time to time depending on the > circumstances of the test. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2429) TestEventGeneration test is flakey
[ https://issues.apache.org/jira/browse/OOZIE-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Denes updated OOZIE-2429: Attachment: OOZIE-2429-addendum-1.patch > TestEventGeneration test is flakey > -- > > Key: OOZIE-2429 > URL: https://issues.apache.org/jira/browse/OOZIE-2429 > Project: Oozie > Issue Type: Bug > Components: action, tests >Affects Versions: trunk >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Minor > Fix For: trunk > > Attachments: OOZIE-2429-1.patch, OOZIE-2429-2.patch, > OOZIE-2429-3.patch, OOZIE-2429-3.patch, OOZIE-2429-addendum-1.patch > > > TestEventGeneration's testForNoDuplicates fails time to time depending on the > circumstances of the test. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2429) TestEventGeneration test is flakey
[ https://issues.apache.org/jira/browse/OOZIE-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Denes updated OOZIE-2429: Attachment: (was: OOZIE-2429-addendum.patch) > TestEventGeneration test is flakey > -- > > Key: OOZIE-2429 > URL: https://issues.apache.org/jira/browse/OOZIE-2429 > Project: Oozie > Issue Type: Bug > Components: action, tests >Affects Versions: trunk >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Minor > Fix For: trunk > > Attachments: OOZIE-2429-1.patch, OOZIE-2429-2.patch, > OOZIE-2429-3.patch, OOZIE-2429-3.patch > > > TestEventGeneration's testForNoDuplicates fails time to time depending on the > circumstances of the test. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2429) TestEventGeneration test is flakey
[ https://issues.apache.org/jira/browse/OOZIE-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Denes updated OOZIE-2429: Attachment: OOZIE-2429-addendum.patch Test checking was in the wrong order. > TestEventGeneration test is flakey > -- > > Key: OOZIE-2429 > URL: https://issues.apache.org/jira/browse/OOZIE-2429 > Project: Oozie > Issue Type: Bug > Components: action, tests >Affects Versions: trunk >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Minor > Fix For: trunk > > Attachments: OOZIE-2429-1.patch, OOZIE-2429-2.patch, > OOZIE-2429-3.patch, OOZIE-2429-3.patch, OOZIE-2429-addendum.patch > > > TestEventGeneration's testForNoDuplicates fails time to time depending on the > circumstances of the test. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (OOZIE-2429) TestEventGeneration test is flakey
[ https://issues.apache.org/jira/browse/OOZIE-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Denes reopened OOZIE-2429: - > TestEventGeneration test is flakey > -- > > Key: OOZIE-2429 > URL: https://issues.apache.org/jira/browse/OOZIE-2429 > Project: Oozie > Issue Type: Bug > Components: action, tests >Affects Versions: trunk >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Minor > Fix For: trunk > > Attachments: OOZIE-2429-1.patch, OOZIE-2429-2.patch, > OOZIE-2429-3.patch, OOZIE-2429-3.patch > > > TestEventGeneration's testForNoDuplicates fails time to time depending on the > circumstances of the test. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2459) TestCoordActionInputCheckXCommand.testNone is flakey
[
https://issues.apache.org/jira/browse/OOZIE-2459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15204521#comment-15204521
]
Ferenc Denes commented on OOZIE-2459:
-
[~abhishekbafna] I could not find the logs I based my research on, probably
they were purged already. The logs we have do not contain the first run of the
StatusTransitService's relevant entries.
The logs I have found now are:
{noformat}
Setting testcase work
dir[/data/jenkins/workspace/CDH5.6.x-Oozie-4.1.0-mr2-JDK8/core/target/test-data/oozietests/org.apache.oozie.command.coord.TestCoordActionInputCheckXCommand/testNone]
18:37:31,362 INFO XLogService:520 - USER[test] GROUP[-] TOKEN[] APP[no-op-wf]
JOB[000-151010183730105-oozie-jenk-C]
ACTION[000-151010183730105-oozie-jenk-C@5]
***
STARTUP MSG: Oozie BUILD_VERSION [4.1.0-cdh5.6.0-SNAPSHOT] compiled by
[jenkins] on [${build.time}]
STARTUP MSG: revision [${vc.revision}]@[${vc.url}]
***
18:37:31,362 INFO XLogService:520 - USER[test] GROUP[-] TOKEN[] APP[no-op-wf]
JOB[000-151010183730105-oozie-jenk-C]
ACTION[000-151010183730105-oozie-jenk-C@5] Log4j configuration file
[oozie-log4j.properties]
18:37:31,362 INFO XLogService:520 - USER[test] GROUP[-] TOKEN[] APP[no-op-wf]
JOB[000-151010183730105-oozie-jenk-C]
ACTION[000-151010183730105-oozie-jenk-C@5] Log4j configuration file loaded
from [CLASSPATH]
18:37:31,362 INFO XLogService:520 - USER[test] GROUP[-] TOKEN[] APP[no-op-wf]
JOB[000-151010183730105-oozie-jenk-C]
ACTION[000-151010183730105-oozie-jenk-C@5] Log4j reload interval [disabled]
18:37:31,363 WARN XLogService:523 - USER[test] GROUP[-] TOKEN[] APP[no-op-wf]
JOB[000-151010183730105-oozie-jenk-C]
ACTION[000-151010183730105-oozie-jenk-C@5] Oozie WS log will be disabled,
missing property 'log4j.appender.oozie.File' for 'oozie' appender
18:37:31,363 INFO ConfigurationService:520 - USER[test] GROUP[-] TOKEN[]
APP[no-op-wf] JOB[000-151010183730105-oozie-jenk-C]
ACTION[000-151010183730105-oozie-jenk-C@5] Oozie home dir
[/data/jenkins/workspace/CDH5.6.x-Oozie-4.1.0-mr2-JDK8/core/target/test-data/oozietests/org.apache.oozie.command.coord.TestCoordActionInputCheckXCommand/testNone]
18:37:31,363 INFO ConfigurationService:520 - USER[test] GROUP[-] TOKEN[]
APP[no-op-wf] JOB[000-151010183730105-oozie-jenk-C]
ACTION[000-151010183730105-oozie-jenk-C@5] Oozie conf dir
[/data/jenkins/workspace/CDH5.6.x-Oozie-4.1.0-mr2-JDK8/core/target/test-data/oozietests/org.apache.oozie.command.coord.TestCoordActionInputCheckXCommand/testNone/conf]
18:37:31,363 INFO ConfigurationService:520 - USER[test] GROUP[-] TOKEN[]
APP[no-op-wf] JOB[000-151010183730105-oozie-jenk-C]
ACTION[000-151010183730105-oozie-jenk-C@5] Oozie conf file [oozie-site.xml]
18:37:31,365 DEBUG ConfigurationService:526 - USER[test] GROUP[-] TOKEN[]
APP[no-op-wf] JOB[000-151010183730105-oozie-jenk-C]
ACTION[000-151010183730105-oozie-jenk-C@5] Overriding configuration with
oozie-site, [oozie.service.JPAService.jdbc.url]
18:37:31,365 DEBUG ConfigurationService:526 - USER[test] GROUP[-] TOKEN[]
APP[no-op-wf] JOB[000-151010183730105-oozie-jenk-C]
ACTION[000-151010183730105-oozie-jenk-C@5] Overriding configuration with
oozie-site, [oozie.service.JPAService.create.db.schema]
18:37:31,365 DEBUG ConfigurationService:526 - USER[test] GROUP[-] TOKEN[]
APP[no-op-wf] JOB[000-151010183730105-oozie-jenk-C]
ACTION[000-151010183730105-oozie-jenk-C@5] Overriding configuration with
oozie-site, [oozie.service.JPAService.jdbc.driver]
18:37:31,365 DEBUG ConfigurationService:526 - USER[test] GROUP[-] TOKEN[]
APP[no-op-wf] JOB[000-151010183730105-oozie-jenk-C]
ACTION[000-151010183730105-oozie-jenk-C@5] Overriding configuration with
oozie-site, [oozie.services]
18:37:31,366 INFO ConfigurationService:520 - USER[test] GROUP[-] TOKEN[]
APP[no-op-wf] JOB[000-151010183730105-oozie-jenk-C]
ACTION[000-151010183730105-oozie-jenk-C@5] Configuration change via System
Property, [oozie.service.HadoopAccessorService.supported.filesystems]=[*]
18:37:31,366 WARN ConfigurationService:523 - USER[test] GROUP[-] TOKEN[]
APP[no-op-wf] JOB[000-151010183730105-oozie-jenk-C]
ACTION[000-151010183730105-oozie-jenk-C@5] System property
[oozie.action.newId] no defined in Oozie configuration, ignored
18:37:31,366 WARN ConfigurationService:523 - USER[test] GROUP[-] TOKEN[]
APP[no-op-wf] JOB[000-151010183730105-oozie-jenk-C]
ACTION[000-151010183730105-oozie-jenk-C@5] System property
[oozie.action.conf.xml] no defined in Oozie configuration, ignored
18:37:31,366 WARN Services:523 - USER[test] GROUP[-] TOKEN[] APP[no-op-wf]
JOB[000-151010183730105-oozie-jenk-C]
ACTION[000-15101
[jira] [Commented] (OOZIE-2459) TestCoordActionInputCheckXCommand.testNone is flakey
[ https://issues.apache.org/jira/browse/OOZIE-2459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15191036#comment-15191036 ] Ferenc Denes commented on OOZIE-2459: - [~abhishekbafna] I did not see anything like that in the logs, however I have noticed that when the test failed I could see the log message: Acquired lock for [org.apache.oozie.service.StatusTransitService] before the actual coordinator actions started. Have you included the log of a failed case? Also now I cannot see other options why this coordinator transition to failed, so the worst thing what can happen by committing this code is to reopen the case after the test case fails again. > TestCoordActionInputCheckXCommand.testNone is flakey > > > Key: OOZIE-2459 > URL: https://issues.apache.org/jira/browse/OOZIE-2459 > Project: Oozie > Issue Type: Bug > Components: action, tests >Affects Versions: 4.1.0 >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Minor > Fix For: trunk > > Attachments: OOZIE-2459-1.patch > > > TestCoordActionInputCheckXCommand.testNone is flakey. > Also the TestCoordActionInputCheckXCommandNonUTC.testNone which is very > similar. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2459) TestCoordActionInputCheckXCommand.testNone is flakey
[ https://issues.apache.org/jira/browse/OOZIE-2459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15191034#comment-15191034 ] Ferenc Denes commented on OOZIE-2459: - [~abhishekbafna] I did not see anything like that in the logs, however I have noticed that when the test failed I could see the log message: Acquired lock for [org.apache.oozie.service.StatusTransitService] before the actual coordinator actions started. Have you included the log of a failed case? Also now I cannot see other options why this coordinator transition to failed, so the worst thing what can happen by committing this code is to reopen the case after the test case fails again. > TestCoordActionInputCheckXCommand.testNone is flakey > > > Key: OOZIE-2459 > URL: https://issues.apache.org/jira/browse/OOZIE-2459 > Project: Oozie > Issue Type: Bug > Components: action, tests >Affects Versions: 4.1.0 >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Minor > Fix For: trunk > > Attachments: OOZIE-2459-1.patch > > > TestCoordActionInputCheckXCommand.testNone is flakey. > Also the TestCoordActionInputCheckXCommandNonUTC.testNone which is very > similar. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2459) TestCoordActionInputCheckXCommand.testNone is flakey
[ https://issues.apache.org/jira/browse/OOZIE-2459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Denes updated OOZIE-2459: Attachment: OOZIE-2459-1.patch > TestCoordActionInputCheckXCommand.testNone is flakey > > > Key: OOZIE-2459 > URL: https://issues.apache.org/jira/browse/OOZIE-2459 > Project: Oozie > Issue Type: Bug > Components: action, tests >Affects Versions: 4.1.0 >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Minor > Fix For: trunk > > Attachments: OOZIE-2459-1.patch > > > TestCoordActionInputCheckXCommand.testNone is flakey. > Also the TestCoordActionInputCheckXCommandNonUTC.testNone which is very > similar. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2459) TestCoordActionInputCheckXCommand.testNone is flakey
[ https://issues.apache.org/jira/browse/OOZIE-2459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15190705#comment-15190705 ] Ferenc Denes commented on OOZIE-2459: - Looks like StatusTransitService is our friend here, as the coordinator is created without an action inside, that service easily puts it to SUCCESS. > TestCoordActionInputCheckXCommand.testNone is flakey > > > Key: OOZIE-2459 > URL: https://issues.apache.org/jira/browse/OOZIE-2459 > Project: Oozie > Issue Type: Bug > Components: action, tests >Affects Versions: 4.1.0 >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Minor > Fix For: trunk > > > TestCoordActionInputCheckXCommand.testNone is flakey. > Also the TestCoordActionInputCheckXCommandNonUTC.testNone which is very > similar. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (OOZIE-2459) TestCoordActionInputCheckXCommand.testNone is flakey
[ https://issues.apache.org/jira/browse/OOZIE-2459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Denes reassigned OOZIE-2459: --- Assignee: Ferenc Denes > TestCoordActionInputCheckXCommand.testNone is flakey > > > Key: OOZIE-2459 > URL: https://issues.apache.org/jira/browse/OOZIE-2459 > Project: Oozie > Issue Type: Bug > Components: action, tests >Affects Versions: 4.1.0 >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Minor > Fix For: trunk > > > TestCoordActionInputCheckXCommand.testNone is flakey. > Also the TestCoordActionInputCheckXCommandNonUTC.testNone which is very > similar. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2459) TestCoordActionInputCheckXCommand.testNone is flakey
[ https://issues.apache.org/jira/browse/OOZIE-2459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15189640#comment-15189640 ] Ferenc Denes commented on OOZIE-2459: - As I can see this test has 4 subtests in there. At some failed occasions' log shows, that the Coordinator action goes to SUCCESS somewhere between or near the 1st and the second subtest. From that on the precondition check fails: ... 08:38:08,753 INFO CoordActionNotificationXCommand:520 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[000-160304083808369-oozie-jenk-C] ACTION[000-160304083808369-oozie-jenk-C@1] No Notification URL is defined. Therefore nothing to notify for job 000-160304083808369-oozie-jenk-C@1 08:38:09,626 DEBUG CoordActionInputCheckXCommand:526 - USER[test] GROUP[-] TOKEN[] APP[no-op-wf] JOB[000-160304083808369-oozie-jenk-C] ACTION[000-160304083808369-oozie-jenk-C@2] Acquired lock for [000-160304083808369-oozie-jenk-C] in [coord_action_input] 08:38:09,628 DEBUG CoordActionInputCheckXCommand:526 - USER[test] GROUP[-] TOKEN[] APP[no-op-wf] JOB[000-160304083808369-oozie-jenk-C] ACTION[000-160304083808369-oozie-jenk-C@2] Released lock for [000-160304083808369-oozie-jenk-C] in [coord_action_input] 08:38:09,628 WARN CoordActionInputCheckXCommand:523 - USER[test] GROUP[-] TOKEN[] APP[no-op-wf] JOB[000-160304083808369-oozie-jenk-C] ACTION[000-160304083808369-oozie-jenk-C@2] E1100: Command precondition does not hold before execution, [[000-160304083808369-oozie-jenk-C@2]::CoordActionInputCheck:: Ignoring action. Coordinator job is not in RUNNING/RUNNINGWITHERROR/PAUSED/PAUSEDWITHERROR state, but state=SUCCEEDED], Error Code: E1100 08:38:09,628 INFO TestCoordActionInputCheckXCommandNonUTC:520 - USER[test] GROUP[-] TOKEN[] APP[no-op-wf] JOB[000-160304083808369-oozie-jenk-C] ACTION[000-160304083808369-oozie-jenk-C@2] Waiting up to [5,000] msec 08:38:09,645 DEBUG CoordActionInputCheckXCommand:526 - USER[test] GROUP[-] TOKEN[] APP[no-op-wf] JOB[000-160304083808369-oozie-jenk-C] ACTION[000-160304083808369-oozie-jenk-C@3] Acquired lock for [000-160304083808369-oozie-jenk-C] in [coord_action_input] 08:38:09,646 DEBUG CoordActionInputCheckXCommand:526 - USER[test] GROUP[-] TOKEN[] APP[no-op-wf] JOB[000-160304083808369-oozie-jenk-C] ACTION[000-160304083808369-oozie-jenk-C@3] Released lock for [000-160304083808369-oozie-jenk-C] in [coord_action_input] 08:38:09,646 WARN CoordActionInputCheckXCommand:523 - USER[test] GROUP[-] TOKEN[] APP[no-op-wf] JOB[000-160304083808369-oozie-jenk-C] ACTION[000-160304083808369-oozie-jenk-C@3] E1100: Command precondition does not hold before execution, [[000-160304083808369-oozie-jenk-C@3]::CoordActionInputCheck:: Ignoring action. Coordinator job is not in RUNNING/RUNNINGWITHERROR/PAUSED/PAUSEDWITHERROR state, but state=SUCCEEDED], Error Code: E1100 08:38:09,646 INFO TestCoordActionInputCheckXCommandNonUTC:520 - USER[test] GROUP[-] TOKEN[] APP[no-op-wf] JOB[000-160304083808369-oozie-jenk-C] ACTION[000-160304083808369-oozie-jenk-C@3] Waiting up to [5,000] msec ... 08:38:14,657 INFO TestCoordActionInputCheckXCommandNonUTC:520 - USER[test] GROUP[-] TOKEN[] APP[no-op-wf] JOB[000-160304083808369-oozie-jenk-C] ACTION[000-160304083808369-oozie-jenk-C@3] Waiting timed out after [5,000] msec 08:38:14,662 INFO Services:520 - USER[test] GROUP[-] TOKEN[] APP[no-op-wf] JOB[000-160304083808369-oozie-jenk-C] ACTION[000-160304083808369-oozie-jenk-C@3] ... This causes the action to remain in WAITING (as initially it is), as nothing checks/deals with a SUCCESS coordinator's action. The reason also might be, that as the precondition fails the coordinator is not processed any more. Either way the action is not touched. I see two options here: 1. Figure out which service puts the coordinator to SUCCESS, and prevent it doing that. This might be tricky, but doable. Any ideas what that service is? 2. Separate the subtests to 4 distinct tests so they do not interfere with each other. This would increase the test time by a few seconds, but would result in a cleaner test. Note: This might be the case in the testLastOnly test as well, however there the last few actions should be in WAITING state, so it never fails even if the coordinator is not in a desired state. As the test never fails probably no human checked it. > TestCoordActionInputCheckXCommand.testNone is flakey > > > Key: OOZIE-2459 > URL: https://issues.apache.org/jira/browse/OOZIE-2459 > Project: Oozie > Issue Type: Bug > Components: action, tests >Affects Versions: 4.1.0 >Reporter: Ferenc Denes >Priority: Minor > Fix For: trunk > > > TestCoordActionInputCheckXCommand.testNon
[jira] [Commented] (OOZIE-2466) Repeated failure of testForNoDuplicates and testSamplers
[ https://issues.apache.org/jira/browse/OOZIE-2466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15178243#comment-15178243 ] Ferenc Denes commented on OOZIE-2466: - The error disappeared, the test failures are unrelated. > Repeated failure of testForNoDuplicates and testSamplers > > > Key: OOZIE-2466 > URL: https://issues.apache.org/jira/browse/OOZIE-2466 > Project: Oozie > Issue Type: Test > Components: build >Affects Versions: 4.1.0 >Reporter: Manjunath Ballur >Assignee: Ferenc Denes >Priority: Trivial > Labels: flakey, test, unit-test > Fix For: trunk > > Attachments: OOZIE-2466-1.patch, OOZIE-2466-1.patch > > > I was looking at, pre-commit builds: > https://builds.apache.org/job/oozie-trunk-precommit-build/ > Most of the times, the build is failing due to failure of following 2 test > cases: > == > org.apache.oozie.event.TestEventGeneration.testForNoDuplicates > org.apache.oozie.util.TestMetricsInstrumentation.testSamplers > Every time, these test cases fail with the same error and I feel, they need > to be fixed. > Details below: > === > org.apache.oozie.event.TestEventGeneration.testForNoDuplicates > == > Error Message > expected:<3> but was:<4> > junit.framework.AssertionFailedError: expected:<3> but was:<4> > at junit.framework.Assert.fail(Assert.java:50) > at junit.framework.Assert.failNotEquals(Assert.java:287) > at junit.framework.Assert.assertEquals(Assert.java:67) > at junit.framework.Assert.assertEquals(Assert.java:199) > at junit.framework.Assert.assertEquals(Assert.java:205) > at > org.apache.oozie.event.TestEventGeneration.testForNoDuplicates(TestEventGeneration.java:559) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at junit.framework.TestCase.runTest(TestCase.java:168) > at junit.framework.TestCase.runBare(TestCase.java:134) > at junit.framework.TestResult$1.protect(TestResult.java:110) > at junit.framework.TestResult.runProtected(TestResult.java:128) > at junit.framework.TestResult.run(TestResult.java:113) > at junit.framework.TestCase.run(TestCase.java:124) > at junit.framework.TestSuite.runTest(TestSuite.java:243) > at junit.framework.TestSuite.run(TestSuite.java:238) > at > org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83) > at org.junit.runners.Suite.runChild(Suite.java:128) > at org.junit.runners.Suite.runChild(Suite.java:24) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > > > > org.apache.oozie.util.TestMetricsInstrumentation.testSamplers > == > Error Message > expected:<4.0> but was:<4.508874329129532> > Stacktrace > junit.framework.AssertionFailedError: expected:<4.0> but > was:<4.508874329129532> > at junit.framework.Assert.fail(Assert.java:50) > at junit.framework.Assert.failNotEquals(Assert.java:287) > at junit.framework.Assert.assertEquals(Assert.java:102) > at junit.framework.Assert.assertEquals(Assert.java:109) > at > org.apache.oozie.util.TestMetricsInstrumentation.testSamplers(TestMetricsInstrumentation.java:167) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at junit.framework.TestCase.runTest(TestCase.java:168) > at junit.framework.TestCase.runBare(TestCase.java:134) > at junit.framework.TestResult$1.protect(TestResult.java:110) > at junit.framework.TestResult.runProtected(TestResult.java:128) > at junit.framework.TestResult.run(TestResult.java:113) > at junit.framework.TestCase.run(TestCase.java:124) > at junit.framework.TestSui
[jira] [Updated] (OOZIE-2466) Repeated failure of testForNoDuplicates and testSamplers
[ https://issues.apache.org/jira/browse/OOZIE-2466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Denes updated OOZIE-2466: Attachment: OOZIE-2466-1.patch > Repeated failure of testForNoDuplicates and testSamplers > > > Key: OOZIE-2466 > URL: https://issues.apache.org/jira/browse/OOZIE-2466 > Project: Oozie > Issue Type: Test > Components: build >Affects Versions: 4.1.0 >Reporter: Manjunath Ballur >Assignee: Ferenc Denes >Priority: Trivial > Labels: flakey, test, unit-test > Fix For: trunk > > Attachments: OOZIE-2466-1.patch, OOZIE-2466-1.patch > > > I was looking at, pre-commit builds: > https://builds.apache.org/job/oozie-trunk-precommit-build/ > Most of the times, the build is failing due to failure of following 2 test > cases: > == > org.apache.oozie.event.TestEventGeneration.testForNoDuplicates > org.apache.oozie.util.TestMetricsInstrumentation.testSamplers > Every time, these test cases fail with the same error and I feel, they need > to be fixed. > Details below: > === > org.apache.oozie.event.TestEventGeneration.testForNoDuplicates > == > Error Message > expected:<3> but was:<4> > junit.framework.AssertionFailedError: expected:<3> but was:<4> > at junit.framework.Assert.fail(Assert.java:50) > at junit.framework.Assert.failNotEquals(Assert.java:287) > at junit.framework.Assert.assertEquals(Assert.java:67) > at junit.framework.Assert.assertEquals(Assert.java:199) > at junit.framework.Assert.assertEquals(Assert.java:205) > at > org.apache.oozie.event.TestEventGeneration.testForNoDuplicates(TestEventGeneration.java:559) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at junit.framework.TestCase.runTest(TestCase.java:168) > at junit.framework.TestCase.runBare(TestCase.java:134) > at junit.framework.TestResult$1.protect(TestResult.java:110) > at junit.framework.TestResult.runProtected(TestResult.java:128) > at junit.framework.TestResult.run(TestResult.java:113) > at junit.framework.TestCase.run(TestCase.java:124) > at junit.framework.TestSuite.runTest(TestSuite.java:243) > at junit.framework.TestSuite.run(TestSuite.java:238) > at > org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83) > at org.junit.runners.Suite.runChild(Suite.java:128) > at org.junit.runners.Suite.runChild(Suite.java:24) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > > > > org.apache.oozie.util.TestMetricsInstrumentation.testSamplers > == > Error Message > expected:<4.0> but was:<4.508874329129532> > Stacktrace > junit.framework.AssertionFailedError: expected:<4.0> but > was:<4.508874329129532> > at junit.framework.Assert.fail(Assert.java:50) > at junit.framework.Assert.failNotEquals(Assert.java:287) > at junit.framework.Assert.assertEquals(Assert.java:102) > at junit.framework.Assert.assertEquals(Assert.java:109) > at > org.apache.oozie.util.TestMetricsInstrumentation.testSamplers(TestMetricsInstrumentation.java:167) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at junit.framework.TestCase.runTest(TestCase.java:168) > at junit.framework.TestCase.runBare(TestCase.java:134) > at junit.framework.TestResult$1.protect(TestResult.java:110) > at junit.framework.TestResult.runProtected(TestResult.java:128) > at junit.framework.TestResult.run(TestResult.java:113) > at junit.framework.TestCase.run(TestCase.java:124) > at junit.framework.TestSuite.runTest(TestSuite.java:243) > at junit.framework.TestSuite.run(TestSui
[jira] [Updated] (OOZIE-2466) Repeated failure of testForNoDuplicates and testSamplers
[ https://issues.apache.org/jira/browse/OOZIE-2466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Denes updated OOZIE-2466: Labels: flakey test unit-test (was: ) > Repeated failure of testForNoDuplicates and testSamplers > > > Key: OOZIE-2466 > URL: https://issues.apache.org/jira/browse/OOZIE-2466 > Project: Oozie > Issue Type: Test > Components: build >Affects Versions: 4.1.0 >Reporter: Manjunath Ballur >Assignee: Ferenc Denes >Priority: Trivial > Labels: flakey, test, unit-test > Fix For: trunk > > Attachments: OOZIE-2466-1.patch > > > I was looking at, pre-commit builds: > https://builds.apache.org/job/oozie-trunk-precommit-build/ > Most of the times, the build is failing due to failure of following 2 test > cases: > == > org.apache.oozie.event.TestEventGeneration.testForNoDuplicates > org.apache.oozie.util.TestMetricsInstrumentation.testSamplers > Every time, these test cases fail with the same error and I feel, they need > to be fixed. > Details below: > === > org.apache.oozie.event.TestEventGeneration.testForNoDuplicates > == > Error Message > expected:<3> but was:<4> > junit.framework.AssertionFailedError: expected:<3> but was:<4> > at junit.framework.Assert.fail(Assert.java:50) > at junit.framework.Assert.failNotEquals(Assert.java:287) > at junit.framework.Assert.assertEquals(Assert.java:67) > at junit.framework.Assert.assertEquals(Assert.java:199) > at junit.framework.Assert.assertEquals(Assert.java:205) > at > org.apache.oozie.event.TestEventGeneration.testForNoDuplicates(TestEventGeneration.java:559) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at junit.framework.TestCase.runTest(TestCase.java:168) > at junit.framework.TestCase.runBare(TestCase.java:134) > at junit.framework.TestResult$1.protect(TestResult.java:110) > at junit.framework.TestResult.runProtected(TestResult.java:128) > at junit.framework.TestResult.run(TestResult.java:113) > at junit.framework.TestCase.run(TestCase.java:124) > at junit.framework.TestSuite.runTest(TestSuite.java:243) > at junit.framework.TestSuite.run(TestSuite.java:238) > at > org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83) > at org.junit.runners.Suite.runChild(Suite.java:128) > at org.junit.runners.Suite.runChild(Suite.java:24) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > > > > org.apache.oozie.util.TestMetricsInstrumentation.testSamplers > == > Error Message > expected:<4.0> but was:<4.508874329129532> > Stacktrace > junit.framework.AssertionFailedError: expected:<4.0> but > was:<4.508874329129532> > at junit.framework.Assert.fail(Assert.java:50) > at junit.framework.Assert.failNotEquals(Assert.java:287) > at junit.framework.Assert.assertEquals(Assert.java:102) > at junit.framework.Assert.assertEquals(Assert.java:109) > at > org.apache.oozie.util.TestMetricsInstrumentation.testSamplers(TestMetricsInstrumentation.java:167) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at junit.framework.TestCase.runTest(TestCase.java:168) > at junit.framework.TestCase.runBare(TestCase.java:134) > at junit.framework.TestResult$1.protect(TestResult.java:110) > at junit.framework.TestResult.runProtected(TestResult.java:128) > at junit.framework.TestResult.run(TestResult.java:113) > at junit.framework.TestCase.run(TestCase.java:124) > at junit.framework.TestSuite.runTest(TestSuite.java:243) > at junit.framework.TestSuite.run(TestSuite.java:238)
[jira] [Updated] (OOZIE-2466) Repeated failure of testForNoDuplicates and testSamplers
[ https://issues.apache.org/jira/browse/OOZIE-2466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Denes updated OOZIE-2466: Attachment: OOZIE-2466-1.patch > Repeated failure of testForNoDuplicates and testSamplers > > > Key: OOZIE-2466 > URL: https://issues.apache.org/jira/browse/OOZIE-2466 > Project: Oozie > Issue Type: Test > Components: build >Reporter: Manjunath Ballur >Priority: Trivial > Attachments: OOZIE-2466-1.patch > > > I was looking at, pre-commit builds: > https://builds.apache.org/job/oozie-trunk-precommit-build/ > Most of the times, the build is failing due to failure of following 2 test > cases: > == > org.apache.oozie.event.TestEventGeneration.testForNoDuplicates > org.apache.oozie.util.TestMetricsInstrumentation.testSamplers > Every time, these test cases fail with the same error and I feel, they need > to be fixed. > Details below: > === > org.apache.oozie.event.TestEventGeneration.testForNoDuplicates > == > Error Message > expected:<3> but was:<4> > junit.framework.AssertionFailedError: expected:<3> but was:<4> > at junit.framework.Assert.fail(Assert.java:50) > at junit.framework.Assert.failNotEquals(Assert.java:287) > at junit.framework.Assert.assertEquals(Assert.java:67) > at junit.framework.Assert.assertEquals(Assert.java:199) > at junit.framework.Assert.assertEquals(Assert.java:205) > at > org.apache.oozie.event.TestEventGeneration.testForNoDuplicates(TestEventGeneration.java:559) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at junit.framework.TestCase.runTest(TestCase.java:168) > at junit.framework.TestCase.runBare(TestCase.java:134) > at junit.framework.TestResult$1.protect(TestResult.java:110) > at junit.framework.TestResult.runProtected(TestResult.java:128) > at junit.framework.TestResult.run(TestResult.java:113) > at junit.framework.TestCase.run(TestCase.java:124) > at junit.framework.TestSuite.runTest(TestSuite.java:243) > at junit.framework.TestSuite.run(TestSuite.java:238) > at > org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83) > at org.junit.runners.Suite.runChild(Suite.java:128) > at org.junit.runners.Suite.runChild(Suite.java:24) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > > > > org.apache.oozie.util.TestMetricsInstrumentation.testSamplers > == > Error Message > expected:<4.0> but was:<4.508874329129532> > Stacktrace > junit.framework.AssertionFailedError: expected:<4.0> but > was:<4.508874329129532> > at junit.framework.Assert.fail(Assert.java:50) > at junit.framework.Assert.failNotEquals(Assert.java:287) > at junit.framework.Assert.assertEquals(Assert.java:102) > at junit.framework.Assert.assertEquals(Assert.java:109) > at > org.apache.oozie.util.TestMetricsInstrumentation.testSamplers(TestMetricsInstrumentation.java:167) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at junit.framework.TestCase.runTest(TestCase.java:168) > at junit.framework.TestCase.runBare(TestCase.java:134) > at junit.framework.TestResult$1.protect(TestResult.java:110) > at junit.framework.TestResult.runProtected(TestResult.java:128) > at junit.framework.TestResult.run(TestResult.java:113) > at junit.framework.TestCase.run(TestCase.java:124) > at junit.framework.TestSuite.runTest(TestSuite.java:243) > at junit.framework.TestSuite.run(TestSuite.java:238) > at > org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83) > at org.junit.runners.Suite.runChild(Suite.java:128
[jira] [Assigned] (OOZIE-2466) Repeated failure of testForNoDuplicates and testSamplers
[ https://issues.apache.org/jira/browse/OOZIE-2466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Denes reassigned OOZIE-2466: --- Assignee: Ferenc Denes > Repeated failure of testForNoDuplicates and testSamplers > > > Key: OOZIE-2466 > URL: https://issues.apache.org/jira/browse/OOZIE-2466 > Project: Oozie > Issue Type: Test > Components: build >Reporter: Manjunath Ballur >Assignee: Ferenc Denes >Priority: Trivial > Attachments: OOZIE-2466-1.patch > > > I was looking at, pre-commit builds: > https://builds.apache.org/job/oozie-trunk-precommit-build/ > Most of the times, the build is failing due to failure of following 2 test > cases: > == > org.apache.oozie.event.TestEventGeneration.testForNoDuplicates > org.apache.oozie.util.TestMetricsInstrumentation.testSamplers > Every time, these test cases fail with the same error and I feel, they need > to be fixed. > Details below: > === > org.apache.oozie.event.TestEventGeneration.testForNoDuplicates > == > Error Message > expected:<3> but was:<4> > junit.framework.AssertionFailedError: expected:<3> but was:<4> > at junit.framework.Assert.fail(Assert.java:50) > at junit.framework.Assert.failNotEquals(Assert.java:287) > at junit.framework.Assert.assertEquals(Assert.java:67) > at junit.framework.Assert.assertEquals(Assert.java:199) > at junit.framework.Assert.assertEquals(Assert.java:205) > at > org.apache.oozie.event.TestEventGeneration.testForNoDuplicates(TestEventGeneration.java:559) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at junit.framework.TestCase.runTest(TestCase.java:168) > at junit.framework.TestCase.runBare(TestCase.java:134) > at junit.framework.TestResult$1.protect(TestResult.java:110) > at junit.framework.TestResult.runProtected(TestResult.java:128) > at junit.framework.TestResult.run(TestResult.java:113) > at junit.framework.TestCase.run(TestCase.java:124) > at junit.framework.TestSuite.runTest(TestSuite.java:243) > at junit.framework.TestSuite.run(TestSuite.java:238) > at > org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83) > at org.junit.runners.Suite.runChild(Suite.java:128) > at org.junit.runners.Suite.runChild(Suite.java:24) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > > > > org.apache.oozie.util.TestMetricsInstrumentation.testSamplers > == > Error Message > expected:<4.0> but was:<4.508874329129532> > Stacktrace > junit.framework.AssertionFailedError: expected:<4.0> but > was:<4.508874329129532> > at junit.framework.Assert.fail(Assert.java:50) > at junit.framework.Assert.failNotEquals(Assert.java:287) > at junit.framework.Assert.assertEquals(Assert.java:102) > at junit.framework.Assert.assertEquals(Assert.java:109) > at > org.apache.oozie.util.TestMetricsInstrumentation.testSamplers(TestMetricsInstrumentation.java:167) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at junit.framework.TestCase.runTest(TestCase.java:168) > at junit.framework.TestCase.runBare(TestCase.java:134) > at junit.framework.TestResult$1.protect(TestResult.java:110) > at junit.framework.TestResult.runProtected(TestResult.java:128) > at junit.framework.TestResult.run(TestResult.java:113) > at junit.framework.TestCase.run(TestCase.java:124) > at junit.framework.TestSuite.runTest(TestSuite.java:243) > at junit.framework.TestSuite.run(TestSuite.java:238) > at > org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83) > at org.junit.ru
[jira] [Commented] (OOZIE-2466) Repeated failure of testForNoDuplicates and testSamplers
[ https://issues.apache.org/jira/browse/OOZIE-2466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15177865#comment-15177865 ] Ferenc Denes commented on OOZIE-2466: - The problem here is, that we sample the 0-9 sequence several times, and expect the mean to be 4 with delta 0.5. The sampling is up to the wait above. Normally it is 20 or 21 times (which results in ~4.29). If the WAITFOR_RATIO is set to be different it is sampled ~20*WAITFOR_RATIO times. The more one samples the closer it is to 4.5 By the floating point representation it might happen, that the value goes above 4.5. An other reason might be, that the sampling window starts at or above 5, in which case the average is above 4.5. Overall it should be 4.5 with the same delta. > Repeated failure of testForNoDuplicates and testSamplers > > > Key: OOZIE-2466 > URL: https://issues.apache.org/jira/browse/OOZIE-2466 > Project: Oozie > Issue Type: Test > Components: build >Reporter: Manjunath Ballur >Priority: Trivial > > I was looking at, pre-commit builds: > https://builds.apache.org/job/oozie-trunk-precommit-build/ > Most of the times, the build is failing due to failure of following 2 test > cases: > == > org.apache.oozie.event.TestEventGeneration.testForNoDuplicates > org.apache.oozie.util.TestMetricsInstrumentation.testSamplers > Every time, these test cases fail with the same error and I feel, they need > to be fixed. > Details below: > === > org.apache.oozie.event.TestEventGeneration.testForNoDuplicates > == > Error Message > expected:<3> but was:<4> > junit.framework.AssertionFailedError: expected:<3> but was:<4> > at junit.framework.Assert.fail(Assert.java:50) > at junit.framework.Assert.failNotEquals(Assert.java:287) > at junit.framework.Assert.assertEquals(Assert.java:67) > at junit.framework.Assert.assertEquals(Assert.java:199) > at junit.framework.Assert.assertEquals(Assert.java:205) > at > org.apache.oozie.event.TestEventGeneration.testForNoDuplicates(TestEventGeneration.java:559) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at junit.framework.TestCase.runTest(TestCase.java:168) > at junit.framework.TestCase.runBare(TestCase.java:134) > at junit.framework.TestResult$1.protect(TestResult.java:110) > at junit.framework.TestResult.runProtected(TestResult.java:128) > at junit.framework.TestResult.run(TestResult.java:113) > at junit.framework.TestCase.run(TestCase.java:124) > at junit.framework.TestSuite.runTest(TestSuite.java:243) > at junit.framework.TestSuite.run(TestSuite.java:238) > at > org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83) > at org.junit.runners.Suite.runChild(Suite.java:128) > at org.junit.runners.Suite.runChild(Suite.java:24) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > > > > org.apache.oozie.util.TestMetricsInstrumentation.testSamplers > == > Error Message > expected:<4.0> but was:<4.508874329129532> > Stacktrace > junit.framework.AssertionFailedError: expected:<4.0> but > was:<4.508874329129532> > at junit.framework.Assert.fail(Assert.java:50) > at junit.framework.Assert.failNotEquals(Assert.java:287) > at junit.framework.Assert.assertEquals(Assert.java:102) > at junit.framework.Assert.assertEquals(Assert.java:109) > at > org.apache.oozie.util.TestMetricsInstrumentation.testSamplers(TestMetricsInstrumentation.java:167) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at junit.framework.TestCase.runTest(TestCase.java:168) > at junit.framework.TestCase.r
[jira] [Commented] (OOZIE-2429) TestEventGeneration test is flakey
[ https://issues.apache.org/jira/browse/OOZIE-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15177812#comment-15177812 ] Ferenc Denes commented on OOZIE-2429: - Rebuilt it, just the good old testSamplers remained, there is a jira for that one as well: OOZIE-2466. Please review the changes. > TestEventGeneration test is flakey > -- > > Key: OOZIE-2429 > URL: https://issues.apache.org/jira/browse/OOZIE-2429 > Project: Oozie > Issue Type: Bug > Components: action, tests >Affects Versions: trunk >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Minor > Fix For: trunk > > Attachments: OOZIE-2429-1.patch, OOZIE-2429-2.patch, > OOZIE-2429-3.patch, OOZIE-2429-3.patch > > > TestEventGeneration's testForNoDuplicates fails time to time depending on the > circumstances of the test. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2429) TestEventGeneration test is flakey
[ https://issues.apache.org/jira/browse/OOZIE-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Denes updated OOZIE-2429: Attachment: OOZIE-2429-3.patch > TestEventGeneration test is flakey > -- > > Key: OOZIE-2429 > URL: https://issues.apache.org/jira/browse/OOZIE-2429 > Project: Oozie > Issue Type: Bug > Components: action, tests >Affects Versions: trunk >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Minor > Fix For: trunk > > Attachments: OOZIE-2429-1.patch, OOZIE-2429-2.patch, > OOZIE-2429-3.patch, OOZIE-2429-3.patch > > > TestEventGeneration's testForNoDuplicates fails time to time depending on the > circumstances of the test. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2429) TestEventGeneration test is flakey
[ https://issues.apache.org/jira/browse/OOZIE-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15177549#comment-15177549 ] Ferenc Denes commented on OOZIE-2429: - The failed test cases are unrelated. > TestEventGeneration test is flakey > -- > > Key: OOZIE-2429 > URL: https://issues.apache.org/jira/browse/OOZIE-2429 > Project: Oozie > Issue Type: Bug > Components: action, tests >Affects Versions: trunk >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Minor > Fix For: trunk > > Attachments: OOZIE-2429-1.patch, OOZIE-2429-2.patch, > OOZIE-2429-3.patch > > > TestEventGeneration's testForNoDuplicates fails time to time depending on the > circumstances of the test. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2429) TestEventGeneration test is flakey
[ https://issues.apache.org/jira/browse/OOZIE-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15176052#comment-15176052 ] Ferenc Denes commented on OOZIE-2429: - [~puru], [~rkanter] thanks for the comments. I have attached the new patch with the required logic. > TestEventGeneration test is flakey > -- > > Key: OOZIE-2429 > URL: https://issues.apache.org/jira/browse/OOZIE-2429 > Project: Oozie > Issue Type: Bug > Components: action, tests >Affects Versions: trunk >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Minor > Fix For: trunk > > Attachments: OOZIE-2429-1.patch, OOZIE-2429-2.patch, > OOZIE-2429-3.patch > > > TestEventGeneration's testForNoDuplicates fails tome to time depending on the > circumstances of the test. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2429) TestEventGeneration test is flakey
[ https://issues.apache.org/jira/browse/OOZIE-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Denes updated OOZIE-2429: Description: TestEventGeneration's testForNoDuplicates fails time to time depending on the circumstances of the test. (was: TestEventGeneration's testForNoDuplicates fails tome to time depending on the circumstances of the test.) > TestEventGeneration test is flakey > -- > > Key: OOZIE-2429 > URL: https://issues.apache.org/jira/browse/OOZIE-2429 > Project: Oozie > Issue Type: Bug > Components: action, tests >Affects Versions: trunk >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Minor > Fix For: trunk > > Attachments: OOZIE-2429-1.patch, OOZIE-2429-2.patch, > OOZIE-2429-3.patch > > > TestEventGeneration's testForNoDuplicates fails time to time depending on the > circumstances of the test. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2429) TestEventGeneration test is flakey
[ https://issues.apache.org/jira/browse/OOZIE-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Denes updated OOZIE-2429: Attachment: OOZIE-2429-3.patch > TestEventGeneration test is flakey > -- > > Key: OOZIE-2429 > URL: https://issues.apache.org/jira/browse/OOZIE-2429 > Project: Oozie > Issue Type: Bug > Components: action, tests >Affects Versions: trunk >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Minor > Fix For: trunk > > Attachments: OOZIE-2429-1.patch, OOZIE-2429-2.patch, > OOZIE-2429-3.patch > > > TestEventGeneration's testForNoDuplicates fails tome to time depending on the > circumstances of the test. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2466) Repeated failure of testForNoDuplicates and testSamplers
[ https://issues.apache.org/jira/browse/OOZIE-2466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15167235#comment-15167235 ] Ferenc Denes commented on OOZIE-2466: - For the testForNoDuplicates there is a jira already. OOZIE-2429 it is. > Repeated failure of testForNoDuplicates and testSamplers > > > Key: OOZIE-2466 > URL: https://issues.apache.org/jira/browse/OOZIE-2466 > Project: Oozie > Issue Type: Test > Components: build >Reporter: Manjunath Ballur >Priority: Trivial > > I was looking at, pre-commit builds: > https://builds.apache.org/job/oozie-trunk-precommit-build/ > Most of the times, the build is failing due to failure of following 2 test > cases: > == > org.apache.oozie.event.TestEventGeneration.testForNoDuplicates > org.apache.oozie.util.TestMetricsInstrumentation.testSamplers > Every time, these test cases fail with the same error and I feel, they need > to be fixed. > Details below: > === > org.apache.oozie.event.TestEventGeneration.testForNoDuplicates > == > Error Message > expected:<3> but was:<4> > junit.framework.AssertionFailedError: expected:<3> but was:<4> > at junit.framework.Assert.fail(Assert.java:50) > at junit.framework.Assert.failNotEquals(Assert.java:287) > at junit.framework.Assert.assertEquals(Assert.java:67) > at junit.framework.Assert.assertEquals(Assert.java:199) > at junit.framework.Assert.assertEquals(Assert.java:205) > at > org.apache.oozie.event.TestEventGeneration.testForNoDuplicates(TestEventGeneration.java:559) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at junit.framework.TestCase.runTest(TestCase.java:168) > at junit.framework.TestCase.runBare(TestCase.java:134) > at junit.framework.TestResult$1.protect(TestResult.java:110) > at junit.framework.TestResult.runProtected(TestResult.java:128) > at junit.framework.TestResult.run(TestResult.java:113) > at junit.framework.TestCase.run(TestCase.java:124) > at junit.framework.TestSuite.runTest(TestSuite.java:243) > at junit.framework.TestSuite.run(TestSuite.java:238) > at > org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83) > at org.junit.runners.Suite.runChild(Suite.java:128) > at org.junit.runners.Suite.runChild(Suite.java:24) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > > > > org.apache.oozie.util.TestMetricsInstrumentation.testSamplers > == > Error Message > expected:<4.0> but was:<4.508874329129532> > Stacktrace > junit.framework.AssertionFailedError: expected:<4.0> but > was:<4.508874329129532> > at junit.framework.Assert.fail(Assert.java:50) > at junit.framework.Assert.failNotEquals(Assert.java:287) > at junit.framework.Assert.assertEquals(Assert.java:102) > at junit.framework.Assert.assertEquals(Assert.java:109) > at > org.apache.oozie.util.TestMetricsInstrumentation.testSamplers(TestMetricsInstrumentation.java:167) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at junit.framework.TestCase.runTest(TestCase.java:168) > at junit.framework.TestCase.runBare(TestCase.java:134) > at junit.framework.TestResult$1.protect(TestResult.java:110) > at junit.framework.TestResult.runProtected(TestResult.java:128) > at junit.framework.TestResult.run(TestResult.java:113) > at junit.framework.TestCase.run(TestCase.java:124) > at junit.framework.TestSuite.runTest(TestSuite.java:243) > at junit.framework.TestSuite.run(TestSuite.java:238) > at > org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83) >
[jira] [Created] (OOZIE-2459) TestCoordActionInputCheckXCommand.testNone is flakey
Ferenc Denes created OOZIE-2459: --- Summary: TestCoordActionInputCheckXCommand.testNone is flakey Key: OOZIE-2459 URL: https://issues.apache.org/jira/browse/OOZIE-2459 Project: Oozie Issue Type: Bug Components: action, tests Affects Versions: 4.1.0 Reporter: Ferenc Denes Priority: Minor Fix For: trunk TestCoordActionInputCheckXCommand.testNone is flakey. Also the TestCoordActionInputCheckXCommandNonUTC.testNone which is very similar. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2429) TestEventGeneration test is flakey
[ https://issues.apache.org/jira/browse/OOZIE-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15152380#comment-15152380 ] Ferenc Denes commented on OOZIE-2429: - [~puru] Thank for the review. I have spent some more time walking through the precondition verification code, and this as well. I think this solution is more correct fot he following reasons: The veryfyPrecondition throws an exception in case of a precondition issue. However this check above is to skip some actions and become a no operation with no additional external effect. Throwing an exception in this case would alter the behaviour of the verification, and probably the intent of that as well. Also this status change is up to the logic above this condition check. The code/condition mentioned here is for the administration of the changed action (like updating the modification time, SLAs, DB etc). Can you please see the full execute() function to see if this check is in the right place. Thanks a lot > TestEventGeneration test is flakey > -- > > Key: OOZIE-2429 > URL: https://issues.apache.org/jira/browse/OOZIE-2429 > Project: Oozie > Issue Type: Bug > Components: action, tests >Affects Versions: trunk >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Minor > Fix For: trunk > > Attachments: OOZIE-2429-1.patch, OOZIE-2429-2.patch > > > TestEventGeneration's testForNoDuplicates fails tome to time depending on the > circumstances of the test. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2454) Oozie Doc has a wrongly named property called oozie.coord.workflow.notification.url
[ https://issues.apache.org/jira/browse/OOZIE-2454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Denes updated OOZIE-2454: Attachment: OOZIE-2454-1.patch > Oozie Doc has a wrongly named property called > oozie.coord.workflow.notification.url > --- > > Key: OOZIE-2454 > URL: https://issues.apache.org/jira/browse/OOZIE-2454 > Project: Oozie > Issue Type: Bug > Components: docs >Affects Versions: 4.1.0 >Reporter: Ferenc Denes >Priority: Trivial > Labels: documentation, newbie > Fix For: trunk > > Attachments: OOZIE-2454-1.patch > > > The following doc for coordinator notification > https://oozie.apache.org/docs/4.1.0/CoordinatorFunctionalSpec.html#a15._Coordinator_Notifications > called oozie.coord.workflow.notification.url. It should be > oozie.coord.notification.url. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2458) Oozie "Install Oozie ShareLib" should ensure uploaded jars are world readable
[ https://issues.apache.org/jira/browse/OOZIE-2458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Denes updated OOZIE-2458: Attachment: (was: OOZIE-2458-1.patch) > Oozie "Install Oozie ShareLib" should ensure uploaded jars are world readable > - > > Key: OOZIE-2458 > URL: https://issues.apache.org/jira/browse/OOZIE-2458 > Project: Oozie > Issue Type: Bug > Components: scripts >Affects Versions: 4.1.0 >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Critical > Fix For: trunk > > Attachments: OOZIE-2458-1.patch > > > If the default umask of HDFS does not include world read then the Oozie > sharelib gets created accordingly, and then the jars can not be read at > runtime because Oozie workflows impersonate the submitting user. > The "Install Oozie ShareLib" step should ensure that the installed jars are > world readable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2458) Oozie "Install Oozie ShareLib" should ensure uploaded jars are world readable
[ https://issues.apache.org/jira/browse/OOZIE-2458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Denes updated OOZIE-2458: Attachment: OOZIE-2458-1.patch > Oozie "Install Oozie ShareLib" should ensure uploaded jars are world readable > - > > Key: OOZIE-2458 > URL: https://issues.apache.org/jira/browse/OOZIE-2458 > Project: Oozie > Issue Type: Bug > Components: scripts >Affects Versions: 4.1.0 >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Critical > Fix For: trunk > > Attachments: OOZIE-2458-1.patch, OOZIE-2458-1.patch > > > If the default umask of HDFS does not include world read then the Oozie > sharelib gets created accordingly, and then the jars can not be read at > runtime because Oozie workflows impersonate the submitting user. > The "Install Oozie ShareLib" step should ensure that the installed jars are > world readable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2458) Oozie "Install Oozie ShareLib" should ensure uploaded jars are world readable
[ https://issues.apache.org/jira/browse/OOZIE-2458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Denes updated OOZIE-2458: Attachment: OOZIE-2458-1.patch > Oozie "Install Oozie ShareLib" should ensure uploaded jars are world readable > - > > Key: OOZIE-2458 > URL: https://issues.apache.org/jira/browse/OOZIE-2458 > Project: Oozie > Issue Type: Bug > Components: scripts >Affects Versions: 4.1.0 >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Critical > Fix For: trunk > > Attachments: OOZIE-2458-1.patch > > > If the default umask of HDFS does not include world read then the Oozie > sharelib gets created accordingly, and then the jars can not be read at > runtime because Oozie workflows impersonate the submitting user. > The "Install Oozie ShareLib" step should ensure that the installed jars are > world readable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OOZIE-2458) Oozie "Install Oozie ShareLib" should ensure uploaded jars are world readable
Ferenc Denes created OOZIE-2458: --- Summary: Oozie "Install Oozie ShareLib" should ensure uploaded jars are world readable Key: OOZIE-2458 URL: https://issues.apache.org/jira/browse/OOZIE-2458 Project: Oozie Issue Type: Bug Components: scripts Affects Versions: 4.1.0 Reporter: Ferenc Denes Assignee: Ferenc Denes Priority: Critical Fix For: trunk If the default umask of HDFS does not include world read then the Oozie sharelib gets created accordingly, and then the jars can not be read at runtime because Oozie workflows impersonate the submitting user. The "Install Oozie ShareLib" step should ensure that the installed jars are world readable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OOZIE-2454) Oozie Doc has a wrongly named property called oozie.coord.workflow.notification.url
Ferenc Denes created OOZIE-2454: --- Summary: Oozie Doc has a wrongly named property called oozie.coord.workflow.notification.url Key: OOZIE-2454 URL: https://issues.apache.org/jira/browse/OOZIE-2454 Project: Oozie Issue Type: Bug Components: docs Affects Versions: 4.1.0 Reporter: Ferenc Denes Priority: Trivial Fix For: trunk The following doc for coordinator notification https://oozie.apache.org/docs/4.1.0/CoordinatorFunctionalSpec.html#a15._Coordinator_Notifications called oozie.coord.workflow.notification.url. It should be oozie.coord.notification.url. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2429) TestEventGeneration test is flakey
[ https://issues.apache.org/jira/browse/OOZIE-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15115401#comment-15115401 ] Ferenc Denes commented on OOZIE-2429: - Test cases are unrelated (the on in question disappeared). > TestEventGeneration test is flakey > -- > > Key: OOZIE-2429 > URL: https://issues.apache.org/jira/browse/OOZIE-2429 > Project: Oozie > Issue Type: Bug > Components: action, tests >Affects Versions: trunk >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Minor > Fix For: trunk > > Attachments: OOZIE-2429-1.patch, OOZIE-2429-2.patch > > > TestEventGeneration's testForNoDuplicates fails tome to time depending on the > circumstances of the test. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2432) TestPurgeXCommand fails
[ https://issues.apache.org/jira/browse/OOZIE-2432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15115394#comment-15115394 ] Ferenc Denes commented on OOZIE-2432: - Commented the requested. The stacktraces remained, so it is uniform in the test, and easier to debug. > TestPurgeXCommand fails > --- > > Key: OOZIE-2432 > URL: https://issues.apache.org/jira/browse/OOZIE-2432 > Project: Oozie > Issue Type: Bug > Components: tests >Affects Versions: trunk >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Critical > Fix For: trunk > > Attachments: OOZIE-2432-1.patch, OOZIE-2432-2.patch > > > TestPurgeXCommand's testPurgeWFWithSubWF1 and testPurgeXCommandFailed started > to fail at early 2016 without pulling in new version from the repo. > I suspect there is a hardcoded value somewhere which is makes this test valid > till end of 2015. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2432) TestPurgeXCommand fails
[ https://issues.apache.org/jira/browse/OOZIE-2432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Denes updated OOZIE-2432: Attachment: OOZIE-2432-2.patch > TestPurgeXCommand fails > --- > > Key: OOZIE-2432 > URL: https://issues.apache.org/jira/browse/OOZIE-2432 > Project: Oozie > Issue Type: Bug > Components: tests >Affects Versions: trunk >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Critical > Fix For: trunk > > Attachments: OOZIE-2432-1.patch, OOZIE-2432-2.patch > > > TestPurgeXCommand's testPurgeWFWithSubWF1 and testPurgeXCommandFailed started > to fail at early 2016 without pulling in new version from the repo. > I suspect there is a hardcoded value somewhere which is makes this test valid > till end of 2015. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2429) TestEventGeneration test is flakey
[ https://issues.apache.org/jira/browse/OOZIE-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15101769#comment-15101769 ] Ferenc Denes commented on OOZIE-2429: - Thanks for the review. 1. In the if statement (as in the patch) we update the modified time, add other commands above generating events. I think we should do all those only in case of any _real_ modification. That's why I have kept all those in there. I think it is right this way, and also safer to handle all those together. Kept it in there, if you still think it is necessary to ammend I will do it. 2. I have modified it to debug, however it was info as the other similar log messages were info as well. 3. Fixed all those. 4. Removed that. > TestEventGeneration test is flakey > -- > > Key: OOZIE-2429 > URL: https://issues.apache.org/jira/browse/OOZIE-2429 > Project: Oozie > Issue Type: Bug > Components: action, tests >Affects Versions: trunk >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Minor > Fix For: trunk > > Attachments: OOZIE-2429-1.patch, OOZIE-2429-2.patch > > > TestEventGeneration's testForNoDuplicates fails tome to time depending on the > circumstances of the test. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2429) TestEventGeneration test is flakey
[ https://issues.apache.org/jira/browse/OOZIE-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Denes updated OOZIE-2429: Attachment: OOZIE-2429-2.patch > TestEventGeneration test is flakey > -- > > Key: OOZIE-2429 > URL: https://issues.apache.org/jira/browse/OOZIE-2429 > Project: Oozie > Issue Type: Bug > Components: action, tests >Affects Versions: trunk >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Minor > Fix For: trunk > > Attachments: OOZIE-2429-1.patch, OOZIE-2429-2.patch > > > TestEventGeneration's testForNoDuplicates fails tome to time depending on the > circumstances of the test. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2435) TestCoordChangeXCommand is flakey
[ https://issues.apache.org/jira/browse/OOZIE-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15101517#comment-15101517 ] Ferenc Denes commented on OOZIE-2435: - Thanks Robert! > TestCoordChangeXCommand is flakey > - > > Key: OOZIE-2435 > URL: https://issues.apache.org/jira/browse/OOZIE-2435 > Project: Oozie > Issue Type: Bug > Components: tests >Affects Versions: trunk >Reporter: Ferenc Denes >Assignee: Ferenc Denes > Labels: test > Fix For: trunk > > Attachments: OOZIE-2435-1.patch, OOZIE-2435-2.patch > > > TestCoordChangeXCommand's subtests fail time to time in our test harness, and > also in apache jenkins. > The issue is, that the CallableQueueService modifies some actions, while we > are observing the behaviour. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2435) TestCoordChangeXCommand is flakey
[ https://issues.apache.org/jira/browse/OOZIE-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15097998#comment-15097998 ] Ferenc Denes commented on OOZIE-2435: - Thanks Robert, added the new patch. > TestCoordChangeXCommand is flakey > - > > Key: OOZIE-2435 > URL: https://issues.apache.org/jira/browse/OOZIE-2435 > Project: Oozie > Issue Type: Bug > Components: tests >Affects Versions: trunk >Reporter: Ferenc Denes >Assignee: Ferenc Denes > Labels: test > Fix For: trunk > > Attachments: OOZIE-2435-1.patch, OOZIE-2435-2.patch > > > TestCoordChangeXCommand's subtests fail time to time in our test harness, and > also in apache jenkins. > The issue is, that the CallableQueueService modifies some actions, while we > are observing the behaviour. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2435) TestCoordChangeXCommand is flakey
[ https://issues.apache.org/jira/browse/OOZIE-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Denes updated OOZIE-2435: Attachment: OOZIE-2435-2.patch > TestCoordChangeXCommand is flakey > - > > Key: OOZIE-2435 > URL: https://issues.apache.org/jira/browse/OOZIE-2435 > Project: Oozie > Issue Type: Bug > Components: tests >Affects Versions: trunk >Reporter: Ferenc Denes >Assignee: Ferenc Denes > Labels: test > Fix For: trunk > > Attachments: OOZIE-2435-1.patch, OOZIE-2435-2.patch > > > TestCoordChangeXCommand's subtests fail time to time in our test harness, and > also in apache jenkins. > The issue is, that the CallableQueueService modifies some actions, while we > are observing the behaviour. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2435) TestCoordChangeXCommand is flakey
[ https://issues.apache.org/jira/browse/OOZIE-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Denes updated OOZIE-2435: Attachment: OOZIE-2435-1.patch > TestCoordChangeXCommand is flakey > - > > Key: OOZIE-2435 > URL: https://issues.apache.org/jira/browse/OOZIE-2435 > Project: Oozie > Issue Type: Bug > Components: tests >Affects Versions: trunk >Reporter: Ferenc Denes >Assignee: Ferenc Denes > Labels: test > Fix For: trunk > > Attachments: OOZIE-2435-1.patch > > > TestCoordChangeXCommand's subtests fail time to time in our test harness, and > also in apache jenkins. > The issue is, that the CallableQueueService modifies some actions, while we > are observing the behaviour. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OOZIE-2435) TestCoordChangeXCommand is flakey
Ferenc Denes created OOZIE-2435: --- Summary: TestCoordChangeXCommand is flakey Key: OOZIE-2435 URL: https://issues.apache.org/jira/browse/OOZIE-2435 Project: Oozie Issue Type: Bug Components: tests Affects Versions: trunk Reporter: Ferenc Denes Assignee: Ferenc Denes Fix For: trunk TestCoordChangeXCommand's subtests fail time to time in our test harness, and also in apache jenkins. The issue is, that the CallableQueueService modifies some actions, while we are observing the behaviour. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2429) TestEventGeneration test is flakey
[ https://issues.apache.org/jira/browse/OOZIE-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Denes updated OOZIE-2429: Issue Type: Bug (was: Test) > TestEventGeneration test is flakey > -- > > Key: OOZIE-2429 > URL: https://issues.apache.org/jira/browse/OOZIE-2429 > Project: Oozie > Issue Type: Bug > Components: action, tests >Affects Versions: trunk >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Minor > Fix For: trunk > > Attachments: OOZIE-2429-1.patch > > > TestEventGeneration's testForNoDuplicates fails tome to time depending on the > circumstances of the test. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2432) TestPurgeXCommand fails
[ https://issues.apache.org/jira/browse/OOZIE-2432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Denes updated OOZIE-2432: Issue Type: Bug (was: Test) > TestPurgeXCommand fails > --- > > Key: OOZIE-2432 > URL: https://issues.apache.org/jira/browse/OOZIE-2432 > Project: Oozie > Issue Type: Bug > Components: tests >Affects Versions: trunk >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Critical > Fix For: trunk > > Attachments: OOZIE-2432-1.patch > > > TestPurgeXCommand's testPurgeWFWithSubWF1 and testPurgeXCommandFailed started > to fail at early 2016 without pulling in new version from the repo. > I suspect there is a hardcoded value somewhere which is makes this test valid > till end of 2015. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2432) TestPurgeXCommand fails
[ https://issues.apache.org/jira/browse/OOZIE-2432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Denes updated OOZIE-2432: Fix Version/s: trunk > TestPurgeXCommand fails > --- > > Key: OOZIE-2432 > URL: https://issues.apache.org/jira/browse/OOZIE-2432 > Project: Oozie > Issue Type: Test > Components: tests >Affects Versions: trunk >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Critical > Fix For: trunk > > Attachments: OOZIE-2432-1.patch > > > TestPurgeXCommand's testPurgeWFWithSubWF1 and testPurgeXCommandFailed started > to fail at early 2016 without pulling in new version from the repo. > I suspect there is a hardcoded value somewhere which is makes this test valid > till end of 2015. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2432) TestPurgeXCommand fails
[ https://issues.apache.org/jira/browse/OOZIE-2432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Denes updated OOZIE-2432: Attachment: OOZIE-2432-1.patch > TestPurgeXCommand fails > --- > > Key: OOZIE-2432 > URL: https://issues.apache.org/jira/browse/OOZIE-2432 > Project: Oozie > Issue Type: Test > Components: tests >Affects Versions: trunk >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Critical > Attachments: OOZIE-2432-1.patch > > > TestPurgeXCommand's testPurgeWFWithSubWF1 and testPurgeXCommandFailed started > to fail at early 2016 without pulling in new version from the repo. > I suspect there is a hardcoded value somewhere which is makes this test valid > till end of 2015. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OOZIE-2432) TestPurgeXCommand fails
Ferenc Denes created OOZIE-2432: --- Summary: TestPurgeXCommand fails Key: OOZIE-2432 URL: https://issues.apache.org/jira/browse/OOZIE-2432 Project: Oozie Issue Type: Test Components: tests Affects Versions: trunk Reporter: Ferenc Denes Priority: Critical TestPurgeXCommand's testPurgeWFWithSubWF1 and testPurgeXCommandFailed started to fail at early 2016 without pulling in new version from the repo. I suspect there is a hardcoded value somewhere which is makes this test valid till end of 2015. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2429) TestEventGeneration test is flakey
[ https://issues.apache.org/jira/browse/OOZIE-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15087279#comment-15087279 ] Ferenc Denes commented on OOZIE-2429: - With the patch I have: 1. Fixed the event generation core, such a way that if the state of the coordinator has not changed there is no event generated. 2. switched off the EventHandlerService in the test, as it has drained some events from the queue while observing the state of that. 3. Separated the testForNoDulicates test to 3, so it is more visible what failed if any. 4. Fixed the check of the order of events in the queue. > TestEventGeneration test is flakey > -- > > Key: OOZIE-2429 > URL: https://issues.apache.org/jira/browse/OOZIE-2429 > Project: Oozie > Issue Type: Test > Components: action, tests >Affects Versions: trunk >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Minor > Fix For: trunk > > Attachments: OOZIE-2429-1.patch > > > TestEventGeneration's testForNoDuplicates fails tome to time depending on the > circumstances of the test. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OOZIE-2429) TestEventGeneration test is flakey
[ https://issues.apache.org/jira/browse/OOZIE-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15085752#comment-15085752 ] Ferenc Denes commented on OOZIE-2429: - Looks like that not only the the test is wrong (where the EventHandlerService manipulates the event queue in the background), but there are really duplicate COORDINATOR_ACTION - FAILURE messages. We have to modify the event generation by checking if the coordinator action's state has really changed at the update stage. The test passes in some cases, where the background processes are slow to add the last event. W have to make sure that all the related actions are complete by the time we check the results. Also the testForNoDuplicates checks 3 things. These should be separated out to 3 tests (interestingly one of the tests inside waits for an other one to happen. This shows one of the problems with this practice). > TestEventGeneration test is flakey > -- > > Key: OOZIE-2429 > URL: https://issues.apache.org/jira/browse/OOZIE-2429 > Project: Oozie > Issue Type: Test > Components: action, tests >Affects Versions: trunk >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Minor > Fix For: trunk > > > TestEventGeneration's testForNoDuplicates fails tome to time depending on the > circumstances of the test. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2429) TestEventGeneration test is flakey
[ https://issues.apache.org/jira/browse/OOZIE-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Denes updated OOZIE-2429: Component/s: action > TestEventGeneration test is flakey > -- > > Key: OOZIE-2429 > URL: https://issues.apache.org/jira/browse/OOZIE-2429 > Project: Oozie > Issue Type: Test > Components: action, tests >Affects Versions: trunk >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Minor > Fix For: trunk > > > TestEventGeneration's testForNoDuplicates fails tome to time depending on the > circumstances of the test. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (OOZIE-2429) TestEventGeneration test is flakey
[ https://issues.apache.org/jira/browse/OOZIE-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Denes reassigned OOZIE-2429: --- Assignee: Ferenc Denes > TestEventGeneration test is flakey > -- > > Key: OOZIE-2429 > URL: https://issues.apache.org/jira/browse/OOZIE-2429 > Project: Oozie > Issue Type: Test > Components: tests >Affects Versions: trunk >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Minor > Fix For: trunk > > > TestEventGeneration's testForNoDuplicates fails tome to time depending on the > circumstances of the test. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OOZIE-2429) TestEventGeneration test is flakey
Ferenc Denes created OOZIE-2429: --- Summary: TestEventGeneration test is flakey Key: OOZIE-2429 URL: https://issues.apache.org/jira/browse/OOZIE-2429 Project: Oozie Issue Type: Test Components: tests Affects Versions: trunk Reporter: Ferenc Denes Priority: Minor Fix For: trunk TestEventGeneration's testForNoDuplicates fails tome to time depending on the circumstances of the test. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2428) TestSLAService, TestSLAEventGeneration flakey tests
[ https://issues.apache.org/jira/browse/OOZIE-2428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Denes updated OOZIE-2428: Attachment: OOZIE-2428-1.patch > TestSLAService, TestSLAEventGeneration flakey tests > --- > > Key: OOZIE-2428 > URL: https://issues.apache.org/jira/browse/OOZIE-2428 > Project: Oozie > Issue Type: Bug > Components: tests >Affects Versions: 4.2.0 >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Minor > Labels: test > Fix For: trunk > > Attachments: OOZIE-2428-1.patch > > > TestSLAService, TestSLAEventGeneration test results depend on the order of > the tests, as the variable checked for output is not reset before/after each > test. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (OOZIE-2428) TestSLAService, TestSLAEventGeneration flakey tests
[ https://issues.apache.org/jira/browse/OOZIE-2428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Denes updated OOZIE-2428: Fix Version/s: (was: 4.3.0) (was: 4.2.0) trunk > TestSLAService, TestSLAEventGeneration flakey tests > --- > > Key: OOZIE-2428 > URL: https://issues.apache.org/jira/browse/OOZIE-2428 > Project: Oozie > Issue Type: Bug > Components: tests >Affects Versions: 4.2.0 >Reporter: Ferenc Denes >Assignee: Ferenc Denes >Priority: Minor > Labels: test > Fix For: trunk > > > TestSLAService, TestSLAEventGeneration test results depend on the order of > the tests, as the variable checked for output is not reset before/after each > test. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (OOZIE-2428) TestSLAService, TestSLAEventGeneration flakey tests
Ferenc Denes created OOZIE-2428: --- Summary: TestSLAService, TestSLAEventGeneration flakey tests Key: OOZIE-2428 URL: https://issues.apache.org/jira/browse/OOZIE-2428 Project: Oozie Issue Type: Bug Components: tests Affects Versions: 4.2.0 Reporter: Ferenc Denes Assignee: Ferenc Denes Priority: Minor Fix For: 4.3.0, 4.2.0 TestSLAService, TestSLAEventGeneration test results depend on the order of the tests, as the variable checked for output is not reset before/after each test. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
