[jira] [Commented] (OOZIE-2585) Disable or remove test case TestPartitionDependencyManagerEhcache.testMemoryUsageAndSpeedOverflowToDisk and testMemoryUsageAndSpeed
[
https://issues.apache.org/jira/browse/OOZIE-2585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16396870#comment-16396870
]
Hadoop QA commented on OOZIE-2585:
--
Testing JIRA OOZIE-2585
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
.{color:red}WARNING{color}: the current HEAD has 100 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [webapp].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 2091
.Tests failed: 17
.Tests errors: 12
.The patch failed the following testcases:
testPartitionDependency(org.apache.oozie.service.TestPartitionDependencyManagerEhcache)
testCoordActionRecoveryServiceForWaitingRegisterPartition(org.apache.oozie.service.TestRecoveryService)
testCoordinatorActionSelectors(org.apache.oozie.jms.TestJMSJobEventListener)
testCoordinatorActionSelectorsNegative(org.apache.oozie.jms.TestJMSJobEventListener)
testWorkflowJobSelectors(org.apache.oozie.jms.TestJMSJobEventListener)
testOnCoordinatorJobSuccessEvent(org.apache.oozie.jms.TestJMSJobEventListener)
testOnWorkflowJobSuspendEvent(org.apache.oozie.jms.TestJMSJobEventListener)
testWorkflowJobSelectorsAnd(org.apache.oozie.jms.TestJMSJobEventListener)
testOnCoordinatorJobFailureEvent(org.apache.oozie.jms.TestJMSJobEventListener)
testConnectionDrop(org.apache.oozie.jms.TestJMSJobEventListener)
testWorkflowJobSelectorsNegative(org.apache.oozie.jms.TestJMSJobEventListener)
testWorkflowJobSelectorsOr(org.apache.oozie.jms.TestJMSJobEventListener)
testOnWorkflowJobSuccessEvent(org.apache.oozie.jms.TestJMSJobEventListener)
testOnCoordinatorActionWaitingEvent(org.apache.oozie.jms.TestJMSJobEventListener)
testOnWorkflowJobFailureEvent(org.apache.oozie.jms.TestJMSJobEventListener)
testOnCoordinatorActionStartEvent(org.apache.oozie.jms.TestJMSJobEventListener)
testOnWorkflowJobStartedEvent(org.apache.oozie.jms.TestJMSJobEventListener)
.Tests failing with errors:
testDropEventTypeMessage(org.apache.oozie.jms.TestHCatMessageHandler)
testCacheUpdateByMessage(org.apache.oozie.jms.TestHCatMessageHandler)
testOnSLADurationMetEvent(org.apache.oozie.jms.TestJMSSLAEventListener)
testOnSLAStartMetEvent(org.apache.oozie.jms.TestJMSSLAEventListener)
testSLAJobSelectorsNegative(org.apache.oozie.jms.TestJMSSLAEventListener)
testOnSLAEndMetEvent(org.apache.oozie.jms.TestJMSSLAEventListener)
testOnSLAStartMissEvent(org.apache.oozie.jms.TestJMSSLAEventListener)
testSLAJobSelectors(org.apache.oozie.jms.TestJMSSLAEventListener)
testOnSLAEndMissEvent(org.apache.oozie.jms.TestJMSSLAEventListener)
testOnSLADurationMissEvent(org.apache.oozie.jms.TestJMSSLAEventListener)
testConnectionRetry(org.apache.oozie.service.TestJMSAccessorService)
[jira] [Commented] (OOZIE-2585) Disable or remove test case TestPartitionDependencyManagerEhcache.testMemoryUsageAndSpeedOverflowToDisk and testMemoryUsageAndSpeed
[
https://issues.apache.org/jira/browse/OOZIE-2585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16396812#comment-16396812
]
Hadoop QA commented on OOZIE-2585:
--
PreCommit-OOZIE-Build started
> Disable or remove test case
> TestPartitionDependencyManagerEhcache.testMemoryUsageAndSpeedOverflowToDisk
> and testMemoryUsageAndSpeed
> ---
>
> Key: OOZIE-2585
> URL: https://issues.apache.org/jira/browse/OOZIE-2585
> Project: Oozie
> Issue Type: Sub-task
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Minor
> Attachments: OOZIE-2585-001.patch, OOZIE-2585-002.patch,
> OOZIE-2585-003.patch
>
>
> There is a problem with the test case testMemoryUsageAndSpeedOverflowToDisk.
> After 20k entries, data is spilled to the disk. However, Ehcache writes on a
> separate thread asynchronously and all put operations return immediately. The
> problem is that after storing the 60k entries, we try to read them back
> instantly, with many of them are still being flushed to the storage. As long
> as disk write in progress, these entries are unavailable.
> I didn't find a reliable way to wait for disk writes. Therefore I suggest
> disabling this test or eliminate it altogether -- it's a performance test
> that runs on all kinds of machines during a build, I don't think it makes too
> much sense to keep it.
> Also testMemoryUsageAndSpeed might be prone to test failures too - these two
> assertions might evaluate to false:
> {code}
> assertTrue((endTime - startTime) < insertTimeinMillis);
> ...
> assertTrue((System.currentTimeMillis() - endTime) < retrievalTimeinMillis);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-2585) Disable or remove test case TestPartitionDependencyManagerEhcache.testMemoryUsageAndSpeedOverflowToDisk and testMemoryUsageAndSpeed
[
https://issues.apache.org/jira/browse/OOZIE-2585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16396803#comment-16396803
]
Peter Cseh commented on OOZIE-2585:
---
+1
Thanks for the rebase!
> Disable or remove test case
> TestPartitionDependencyManagerEhcache.testMemoryUsageAndSpeedOverflowToDisk
> and testMemoryUsageAndSpeed
> ---
>
> Key: OOZIE-2585
> URL: https://issues.apache.org/jira/browse/OOZIE-2585
> Project: Oozie
> Issue Type: Sub-task
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Minor
> Attachments: OOZIE-2585-001.patch, OOZIE-2585-002.patch,
> OOZIE-2585-003.patch
>
>
> There is a problem with the test case testMemoryUsageAndSpeedOverflowToDisk.
> After 20k entries, data is spilled to the disk. However, Ehcache writes on a
> separate thread asynchronously and all put operations return immediately. The
> problem is that after storing the 60k entries, we try to read them back
> instantly, with many of them are still being flushed to the storage. As long
> as disk write in progress, these entries are unavailable.
> I didn't find a reliable way to wait for disk writes. Therefore I suggest
> disabling this test or eliminate it altogether -- it's a performance test
> that runs on all kinds of machines during a build, I don't think it makes too
> much sense to keep it.
> Also testMemoryUsageAndSpeed might be prone to test failures too - these two
> assertions might evaluate to false:
> {code}
> assertTrue((endTime - startTime) < insertTimeinMillis);
> ...
> assertTrue((System.currentTimeMillis() - endTime) < retrievalTimeinMillis);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-2585) Disable or remove test case TestPartitionDependencyManagerEhcache.testMemoryUsageAndSpeedOverflowToDisk and testMemoryUsageAndSpeed
[
https://issues.apache.org/jira/browse/OOZIE-2585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16396800#comment-16396800
]
Peter Bacsko commented on OOZIE-2585:
-
[~gezapeti] I rebased the patch. As we discussed IRL, I removed the two tests
altogether.
> Disable or remove test case
> TestPartitionDependencyManagerEhcache.testMemoryUsageAndSpeedOverflowToDisk
> and testMemoryUsageAndSpeed
> ---
>
> Key: OOZIE-2585
> URL: https://issues.apache.org/jira/browse/OOZIE-2585
> Project: Oozie
> Issue Type: Sub-task
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Minor
> Attachments: OOZIE-2585-001.patch, OOZIE-2585-002.patch,
> OOZIE-2585-003.patch
>
>
> There is a problem with the test case testMemoryUsageAndSpeedOverflowToDisk.
> After 20k entries, data is spilled to the disk. However, Ehcache writes on a
> separate thread asynchronously and all put operations return immediately. The
> problem is that after storing the 60k entries, we try to read them back
> instantly, with many of them are still being flushed to the storage. As long
> as disk write in progress, these entries are unavailable.
> I didn't find a reliable way to wait for disk writes. Therefore I suggest
> disabling this test or eliminate it altogether -- it's a performance test
> that runs on all kinds of machines during a build, I don't think it makes too
> much sense to keep it.
> Also testMemoryUsageAndSpeed might be prone to test failures too - these two
> assertions might evaluate to false:
> {code}
> assertTrue((endTime - startTime) < insertTimeinMillis);
> ...
> assertTrue((System.currentTimeMillis() - endTime) < retrievalTimeinMillis);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-2585) Disable or remove test case TestPartitionDependencyManagerEhcache.testMemoryUsageAndSpeedOverflowToDisk and testMemoryUsageAndSpeed
[
https://issues.apache.org/jira/browse/OOZIE-2585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16251523#comment-16251523
]
Hadoop QA commented on OOZIE-2585:
--
Testing JIRA OOZIE-2585
Cleaning local git workspace
{color:red}-1{color} Patch failed to apply to head of branch
> Disable or remove test case
> TestPartitionDependencyManagerEhcache.testMemoryUsageAndSpeedOverflowToDisk
> and testMemoryUsageAndSpeed
> ---
>
> Key: OOZIE-2585
> URL: https://issues.apache.org/jira/browse/OOZIE-2585
> Project: Oozie
> Issue Type: Sub-task
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Minor
> Attachments: OOZIE-2585-001.patch, OOZIE-2585-002.patch
>
>
> There is a problem with the test case testMemoryUsageAndSpeedOverflowToDisk.
> After 20k entries, data is spilled to the disk. However, Ehcache writes on a
> separate thread asynchronously and all put operations return immediately. The
> problem is that after storing the 60k entries, we try to read them back
> instantly, with many of them are still being flushed to the storage. As long
> as disk write in progress, these entries are unavailable.
> I didn't find a reliable way to wait for disk writes. Therefore I suggest
> disabling this test or eliminate it altogether -- it's a performance test
> that runs on all kinds of machines during a build, I don't think it makes too
> much sense to keep it.
> Also testMemoryUsageAndSpeed might be prone to test failures too - these two
> assertions might evaluate to false:
> {code}
> assertTrue((endTime - startTime) < insertTimeinMillis);
> ...
> assertTrue((System.currentTimeMillis() - endTime) < retrievalTimeinMillis);
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OOZIE-2585) Disable or remove test case TestPartitionDependencyManagerEhcache.testMemoryUsageAndSpeedOverflowToDisk and testMemoryUsageAndSpeed
[
https://issues.apache.org/jira/browse/OOZIE-2585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16064615#comment-16064615
]
Peter Cseh commented on OOZIE-2585:
---
[~pbacsko], can you rebase patch-2? It does not apply to master anymore.
> Disable or remove test case
> TestPartitionDependencyManagerEhcache.testMemoryUsageAndSpeedOverflowToDisk
> and testMemoryUsageAndSpeed
> ---
>
> Key: OOZIE-2585
> URL: https://issues.apache.org/jira/browse/OOZIE-2585
> Project: Oozie
> Issue Type: Bug
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Minor
> Attachments: OOZIE-2585-001.patch, OOZIE-2585-002.patch
>
>
> There is a problem with the test case testMemoryUsageAndSpeedOverflowToDisk.
> After 20k entries, data is spilled to the disk. However, Ehcache writes on a
> separate thread asynchronously and all put operations return immediately. The
> problem is that after storing the 60k entries, we try to read them back
> instantly, with many of them are still being flushed to the storage. As long
> as disk write in progress, these entries are unavailable.
> I didn't find a reliable way to wait for disk writes. Therefore I suggest
> disabling this test or eliminate it altogether -- it's a performance test
> that runs on all kinds of machines during a build, I don't think it makes too
> much sense to keep it.
> Also testMemoryUsageAndSpeed might be prone to test failures too - these two
> assertions might evaluate to false:
> {code}
> assertTrue((endTime - startTime) < insertTimeinMillis);
> ...
> assertTrue((System.currentTimeMillis() - endTime) < retrievalTimeinMillis);
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (OOZIE-2585) Disable or remove test case TestPartitionDependencyManagerEhcache.testMemoryUsageAndSpeedOverflowToDisk and testMemoryUsageAndSpeed
[
https://issues.apache.org/jira/browse/OOZIE-2585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15949320#comment-15949320
]
Peter Cseh commented on OOZIE-2585:
---
+1
Removing these tests is a good idea.
> Disable or remove test case
> TestPartitionDependencyManagerEhcache.testMemoryUsageAndSpeedOverflowToDisk
> and testMemoryUsageAndSpeed
> ---
>
> Key: OOZIE-2585
> URL: https://issues.apache.org/jira/browse/OOZIE-2585
> Project: Oozie
> Issue Type: Bug
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Minor
> Attachments: OOZIE-2585-001.patch, OOZIE-2585-002.patch
>
>
> There is a problem with the test case testMemoryUsageAndSpeedOverflowToDisk.
> After 20k entries, data is spilled to the disk. However, Ehcache writes on a
> separate thread asynchronously and all put operations return immediately. The
> problem is that after storing the 60k entries, we try to read them back
> instantly, with many of them are still being flushed to the storage. As long
> as disk write in progress, these entries are unavailable.
> I didn't find a reliable way to wait for disk writes. Therefore I suggest
> disabling this test or eliminate it altogether -- it's a performance test
> that runs on all kinds of machines during a build, I don't think it makes too
> much sense to keep it.
> Also testMemoryUsageAndSpeed might be prone to test failures too - these two
> assertions might evaluate to false:
> {code}
> assertTrue((endTime - startTime) < insertTimeinMillis);
> ...
> assertTrue((System.currentTimeMillis() - endTime) < retrievalTimeinMillis);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2585) Disable or remove test case TestPartitionDependencyManagerEhcache.testMemoryUsageAndSpeedOverflowToDisk and testMemoryUsageAndSpeed
[
https://issues.apache.org/jira/browse/OOZIE-2585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15948182#comment-15948182
]
Robert Kanter commented on OOZIE-2585:
--
I'm fine with getting rid of a flakey performance test, especially because it's
testing the performance of ehcache and not Oozie itself.
I'm not too familiar with this code, but it looks like this only applies to
{{testMemoryUsageAndSpeed}} and {{testMemoryUsageAndSpeedOverflowToDisk}}. The
others seem to be checking correct behavior. If so, I guess we should only
remove these two.
> Disable or remove test case
> TestPartitionDependencyManagerEhcache.testMemoryUsageAndSpeedOverflowToDisk
> and testMemoryUsageAndSpeed
> ---
>
> Key: OOZIE-2585
> URL: https://issues.apache.org/jira/browse/OOZIE-2585
> Project: Oozie
> Issue Type: Bug
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Minor
> Attachments: OOZIE-2585-001.patch, OOZIE-2585-002.patch
>
>
> There is a problem with the test case testMemoryUsageAndSpeedOverflowToDisk.
> After 20k entries, data is spilled to the disk. However, Ehcache writes on a
> separate thread asynchronously and all put operations return immediately. The
> problem is that after storing the 60k entries, we try to read them back
> instantly, with many of them are still being flushed to the storage. As long
> as disk write in progress, these entries are unavailable.
> I didn't find a reliable way to wait for disk writes. Therefore I suggest
> disabling this test or eliminate it altogether -- it's a performance test
> that runs on all kinds of machines during a build, I don't think it makes too
> much sense to keep it.
> Also testMemoryUsageAndSpeed might be prone to test failures too - these two
> assertions might evaluate to false:
> {code}
> assertTrue((endTime - startTime) < insertTimeinMillis);
> ...
> assertTrue((System.currentTimeMillis() - endTime) < retrievalTimeinMillis);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2585) Disable or remove test case TestPartitionDependencyManagerEhcache.testMemoryUsageAndSpeedOverflowToDisk and testMemoryUsageAndSpeed
[
https://issues.apache.org/jira/browse/OOZIE-2585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15404594#comment-15404594
]
Robert Kanter commented on OOZIE-2585:
--
ping [~rohini]
> Disable or remove test case
> TestPartitionDependencyManagerEhcache.testMemoryUsageAndSpeedOverflowToDisk
> and testMemoryUsageAndSpeed
> ---
>
> Key: OOZIE-2585
> URL: https://issues.apache.org/jira/browse/OOZIE-2585
> Project: Oozie
> Issue Type: Bug
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Minor
> Attachments: OOZIE-2585-001.patch, OOZIE-2585-002.patch
>
>
> There is a problem with the test case testMemoryUsageAndSpeedOverflowToDisk.
> After 20k entries, data is spilled to the disk. However, Ehcache writes on a
> separate thread asynchronously and all put operations return immediately. The
> problem is that after storing the 60k entries, we try to read them back
> instantly, with many of them are still being flushed to the storage. As long
> as disk write in progress, these entries are unavailable.
> I didn't find a reliable way to wait for disk writes. Therefore I suggest
> disabling this test or eliminate it altogether -- it's a performance test
> that runs on all kinds of machines during a build, I don't think it makes too
> much sense to keep it.
> Also testMemoryUsageAndSpeed might be prone to test failures too - these two
> assertions might evaluate to false:
> {code}
> assertTrue((endTime - startTime) < insertTimeinMillis);
> ...
> assertTrue((System.currentTimeMillis() - endTime) < retrievalTimeinMillis);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-2585) Disable or remove test case TestPartitionDependencyManagerEhcache.testMemoryUsageAndSpeedOverflowToDisk and testMemoryUsageAndSpeed
[
https://issues.apache.org/jira/browse/OOZIE-2585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15396912#comment-15396912
]
Hadoop QA commented on OOZIE-2585:
--
Testing JIRA OOZIE-2585
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch does adds/modifies 2 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
{color:red}-1 COMPILE{color}
.{color:red}-1{color} HEAD does not compile
.{color:red}-1{color} patch does not compile
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 1788
.Tests failed: 5
.Tests errors: 2
.The patch failed the following testcases:
. testMain(org.apache.oozie.action.hadoop.TestHiveMain)
. testPigScript(org.apache.oozie.action.hadoop.TestPigMain)
. testEmbeddedPigWithinPython(org.apache.oozie.action.hadoop.TestPigMain)
. testPig_withNullExternalID(org.apache.oozie.action.hadoop.TestPigMain)
. testPigScript(org.apache.oozie.action.hadoop.TestPigMainWithOldAPI)
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/oozie-trunk-precommit-build/3091/
> Disable or remove test case
> TestPartitionDependencyManagerEhcache.testMemoryUsageAndSpeedOverflowToDisk
> and testMemoryUsageAndSpeed
> ---
>
> Key: OOZIE-2585
> URL: https://issues.apache.org/jira/browse/OOZIE-2585
> Project: Oozie
> Issue Type: Bug
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Minor
> Attachments: OOZIE-2585-001.patch, OOZIE-2585-002.patch
>
>
> There is a problem with the test case testMemoryUsageAndSpeedOverflowToDisk.
> After 20k entries, data is spilled to the disk. However, Ehcache writes on a
> separate thread asynchronously and all put operations return immediately. The
> problem is that after storing the 60k entries, we try to read them back
> instantly, with many of them are still being flushed to the storage. As long
> as disk write in progress, these entries are unavailable.
> I didn't find a reliable way to wait for disk writes. Therefore I suggest
> disabling this test or eliminate it altogether -- it's a performance test
> that runs on all kinds of machines during a build, I don't think it makes too
> much sense to keep it.
> Also testMemoryUsageAndSpeed might be prone to test failures too - these two
> assertions might evaluate to false:
> {code}
> assertTrue((endTime - startTime) < insertTimeinMillis);
> ...
> assertTrue((System.currentTimeMillis() - endTime) < retrievalTimeinMillis);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-2585) Disable or remove test case TestPartitionDependencyManagerEhcache.testMemoryUsageAndSpeedOverflowToDisk and testMemoryUsageAndSpeed
[
https://issues.apache.org/jira/browse/OOZIE-2585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15373965#comment-15373965
]
Robert Kanter commented on OOZIE-2585:
--
[~rohini], what do you think about doing this? We've seen this test being very
flakey for quite a while.
> Disable or remove test case
> TestPartitionDependencyManagerEhcache.testMemoryUsageAndSpeedOverflowToDisk
> and testMemoryUsageAndSpeed
> ---
>
> Key: OOZIE-2585
> URL: https://issues.apache.org/jira/browse/OOZIE-2585
> Project: Oozie
> Issue Type: Bug
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Minor
> Attachments: OOZIE-2585-001.patch, OOZIE-2585-002.patch
>
>
> There is a problem with the test case testMemoryUsageAndSpeedOverflowToDisk.
> After 20k entries, data is spilled to the disk. However, Ehcache writes on a
> separate thread asynchronously and all put operations return immediately. The
> problem is that after storing the 60k entries, we try to read them back
> instantly, with many of them are still being flushed to the storage. As long
> as disk write in progress, these entries are unavailable.
> I didn't find a reliable way to wait for disk writes. Therefore I suggest
> disabling this test or eliminate it altogether -- it's a performance test
> that runs on all kinds of machines during a build, I don't think it makes too
> much sense to keep it.
> Also testMemoryUsageAndSpeed might be prone to test failures too - these two
> assertions might evaluate to false:
> {code}
> assertTrue((endTime - startTime) < insertTimeinMillis);
> ...
> assertTrue((System.currentTimeMillis() - endTime) < retrievalTimeinMillis);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-2585) Disable or remove test case TestPartitionDependencyManagerEhcache.testMemoryUsageAndSpeedOverflowToDisk and testMemoryUsageAndSpeed
[
https://issues.apache.org/jira/browse/OOZIE-2585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15355238#comment-15355238
]
Hadoop QA commented on OOZIE-2585:
--
Testing JIRA OOZIE-2585
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch does adds/modifies 2 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 1784
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:green}*+1 Overall result, good!, no -1s*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/oozie-trunk-precommit-build/3017/
> Disable or remove test case
> TestPartitionDependencyManagerEhcache.testMemoryUsageAndSpeedOverflowToDisk
> and testMemoryUsageAndSpeed
> ---
>
> Key: OOZIE-2585
> URL: https://issues.apache.org/jira/browse/OOZIE-2585
> Project: Oozie
> Issue Type: Bug
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Minor
> Attachments: OOZIE-2585-001.patch, OOZIE-2585-002.patch
>
>
> There is a problem with the test case testMemoryUsageAndSpeedOverflowToDisk.
> After 20k entries, data is spilled to the disk. However, Ehcache writes on a
> separate thread asynchronously and all put operations return immediately. The
> problem is that after storing the 60k entries, we try to read them back
> instantly, with many of them are still being flushed to the storage. As long
> as disk write in progress, these entries are unavailable.
> I didn't find a reliable way to wait for disk writes. Therefore I suggest
> disabling this test or eliminate it altogether -- it's a performance test
> that runs on all kinds of machines during a build, I don't think it makes too
> much sense to keep it.
> Also testMemoryUsageAndSpeed might be prone to test failures too - these two
> assertions might evaluate to false:
> {code}
> assertTrue((endTime - startTime) < insertTimeinMillis);
> ...
> assertTrue((System.currentTimeMillis() - endTime) < retrievalTimeinMillis);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OOZIE-2585) Disable or remove test case TestPartitionDependencyManagerEhcache.testMemoryUsageAndSpeedOverflowToDisk and testMemoryUsageAndSpeed
[
https://issues.apache.org/jira/browse/OOZIE-2585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15355022#comment-15355022
]
Hadoop QA commented on OOZIE-2585:
--
Testing JIRA OOZIE-2585
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch does adds/modifies 2 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 1787
.Tests failed: 5
.Tests errors: 2
.The patch failed the following testcases:
. testMain(org.apache.oozie.action.hadoop.TestHiveMain)
. testPigScript(org.apache.oozie.action.hadoop.TestPigMain)
. testPig_withNullExternalID(org.apache.oozie.action.hadoop.TestPigMain)
. testEmbeddedPigWithinPython(org.apache.oozie.action.hadoop.TestPigMain)
. testPigScript(org.apache.oozie.action.hadoop.TestPigMainWithOldAPI)
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/oozie-trunk-precommit-build/3016/
> Disable or remove test case
> TestPartitionDependencyManagerEhcache.testMemoryUsageAndSpeedOverflowToDisk
> and testMemoryUsageAndSpeed
> ---
>
> Key: OOZIE-2585
> URL: https://issues.apache.org/jira/browse/OOZIE-2585
> Project: Oozie
> Issue Type: Bug
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Minor
> Attachments: OOZIE-2585-001.patch
>
>
> There is a problem with the test case testMemoryUsageAndSpeedOverflowToDisk.
> After 20k entries, data is spilled to the disk. However, Ehcache writes on a
> separate thread asynchronously and all put operations return immediately. The
> problem is that after storing the 60k entries, we try to read them back
> instantly, with many of them are still being flushed to the storage. As long
> as disk write in progress, these entries are unavailable.
> I didn't find a reliable way to wait for disk writes. Therefore I suggest
> disabling this test or eliminate it altogether -- it's a performance test
> that runs on all kinds of machines during a build, I don't think it makes too
> much sense to keep it.
> Also testMemoryUsageAndSpeed might be prone to test failures too - these two
> assertions might evaluate to false:
> {code}
> assertTrue((endTime - startTime) < insertTimeinMillis);
> ...
> assertTrue((System.currentTimeMillis() - endTime) < retrievalTimeinMillis);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
