[
https://issues.apache.org/jira/browse/OOZIE-2382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Kanter updated OOZIE-2382:
---------------------------------
Description:
{{rg.apache.oozie.action.hadoop.TestPigMain.testPig_withNullExternalID}} is
flakey.
{noformat}
junit.framework.AssertionFailedError
at junit.framework.Assert.fail(Assert.java:48)
at junit.framework.Assert.assertTrue(Assert.java:20)
at junit.framework.Assert.assertFalse(Assert.java:34)
at junit.framework.Assert.assertFalse(Assert.java:41)
at
org.apache.oozie.action.hadoop.PigTestCase.testPig_withNullExternalID(PigTestCase.java:112)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at junit.framework.TestCase.runTest(TestCase.java:168)
at junit.framework.TestCase.runBare(TestCase.java:134)
at junit.framework.TestResult$1.protect(TestResult.java:110)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.framework.TestResult.run(TestResult.java:113)
at junit.framework.TestCase.run(TestCase.java:124)
at junit.framework.TestSuite.runTest(TestSuite.java:243)
at junit.framework.TestSuite.run(TestSuite.java:238)
at
org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
at org.junit.runners.Suite.runChild(Suite.java:128)
at org.junit.runners.Suite.runChild(Suite.java:24)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
{noformat}
We've seen this for months but has been really tricky to track down. The
output of the test has this:
{noformat}
Run pig script using PigRunner.run() for Pig version 0.8+
Apache Pig version 0.12.0-cdh5.5.0-SNAPSHOT (rexported)
compiled Sep 30 2015, 22:11:56
Hadoop Job IDs executed by Pig: job_20151001151153510_0001
Run pig script using PigRunner.run() for Pig version 0.8+
{noformat}
Compared with a successful run of the test, the difference is that it shows a
Hadoop Job ID. This test is supposed to make Pig fail and not run any jobs,
which can be confirmed in the test output, yet there's a Hadoop Job ID here!
It turns out that {{PigRunner}} eventually gets to {{PigStats}}, which is a
singleton. And because we're running all of the Pig tests in the same JVM,
{{PigStats}} is actually leaking through the tests. It doesn't seem to be a
problem most of the time because it must be getting overwritten at each run,
however, because {{testPig_withNullExternalID}} is purposefully failing the Pig
job, it's not, and we end up with the {{PigStats}} from the previous test or a
non-initialized {{PigStats}} depending on the order of the tests. If it's a
previous {{PigStats}}, then {{PigMain}} will write the Hadoop Job ID to a new
file for the {{testPig_withNullExternalID}} test, which will then fail because
it's not expecting this file.
Unfortunately, there isn't a clean way to reset {{PigStats}}. Pig v 0.9 has a
{{set}} method which we can pass {{null}} to in order to reset the
{{PigStats}}. However, this was changed in Pig v 0.13 to the {{start}} method.
In either case, they're both package private, so we have to usereflection to
find the existing method for whichever version of Pig we're using and make it
public. We should do this after every Pig test, even if they don't all need it
just to make sure that we're starting with a fresh Pig each time as unit tests
are supposed to start fresh anyway.
was:
{{rg.apache.oozie.action.hadoop.TestPigMain.testPig_withNullExternalID}} is
flakey.
{noformat}
junit.framework.AssertionFailedError
at junit.framework.Assert.fail(Assert.java:48)
at junit.framework.Assert.assertTrue(Assert.java:20)
at junit.framework.Assert.assertFalse(Assert.java:34)
at junit.framework.Assert.assertFalse(Assert.java:41)
at
org.apache.oozie.action.hadoop.PigTestCase.testPig_withNullExternalID(PigTestCase.java:112)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at junit.framework.TestCase.runTest(TestCase.java:168)
at junit.framework.TestCase.runBare(TestCase.java:134)
at junit.framework.TestResult$1.protect(TestResult.java:110)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.framework.TestResult.run(TestResult.java:113)
at junit.framework.TestCase.run(TestCase.java:124)
at junit.framework.TestSuite.runTest(TestSuite.java:243)
at junit.framework.TestSuite.run(TestSuite.java:238)
at
org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
at org.junit.runners.Suite.runChild(Suite.java:128)
at org.junit.runners.Suite.runChild(Suite.java:24)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
{noformat}
We've seen this for months but has been really tricky to track down. The
output of the test has this:
{noformat}
Run pig script using PigRunner.run() for Pig version 0.8+
Apache Pig version 0.12.0-cdh5.5.0-SNAPSHOT (rexported)
compiled Sep 30 2015, 22:11:56
Hadoop Job IDs executed by Pig: job_20151001151153510_0001
Run pig script using PigRunner.run() for Pig version 0.8+
{noformat}
Compared with a successful run of the test, the difference is that it shows a
Hadoop Job ID. This test is supposed to make Pig fail and not run any jobs,
which can be confirmed in the test output, yet there's a Hadoop Job ID here!
It turns out that {{PigRunner}} eventually gets to {{PigStats}}, which is a
singleton. And because we're running all of the Pig tests in the same JVM,
{{PigStats}} is actually leaking through the tests. It doesn't seem to be a
problem most of the time because it must be getting overwritten at each run,
however, because {{testPig_withNullExternalID}} is purposefully failing the Pig
job, it's not, and we end up with the {{PigStats}} from the previous test or a
non-initialized {{PigStats}} depending on the order of the tests. If it's a
previous {{PigStats}}, then {{PigMain}} will write the Hadoop Job ID to a new
file for
Unfortunately, there isn't a clean way to reset {{PigStats}}. Pig v 0.9 has a
{{set}} method which we can pass {{null}} to in order to reset the
{{PigStats}}. However, this was changed in Pig v 0.13 to the {{start}} method.
In either case, they're both package private, so we have to usereflection to
find the existing method for whichever version of Pig we're using and make it
public. We should do this after every Pig test, even if they don't all need it
just to make sure that we're starting with a fresh Pig each time as unit tests
are supposed to start fresh anyway.
> rg.apache.oozie.action.hadoop.TestPigMain.testPig_withNullExternalID is flakey
> ------------------------------------------------------------------------------
>
> Key: OOZIE-2382
> URL: https://issues.apache.org/jira/browse/OOZIE-2382
> Project: Oozie
> Issue Type: Bug
> Components: tests
> Affects Versions: trunk
> Reporter: Robert Kanter
> Assignee: Robert Kanter
> Fix For: trunk
>
>
> {{rg.apache.oozie.action.hadoop.TestPigMain.testPig_withNullExternalID}} is
> flakey.
> {noformat}
> junit.framework.AssertionFailedError
> at junit.framework.Assert.fail(Assert.java:48)
> at junit.framework.Assert.assertTrue(Assert.java:20)
> at junit.framework.Assert.assertFalse(Assert.java:34)
> at junit.framework.Assert.assertFalse(Assert.java:41)
> at
> org.apache.oozie.action.hadoop.PigTestCase.testPig_withNullExternalID(PigTestCase.java:112)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at junit.framework.TestCase.runTest(TestCase.java:168)
> at junit.framework.TestCase.runBare(TestCase.java:134)
> at junit.framework.TestResult$1.protect(TestResult.java:110)
> at junit.framework.TestResult.runProtected(TestResult.java:128)
> at junit.framework.TestResult.run(TestResult.java:113)
> at junit.framework.TestCase.run(TestCase.java:124)
> at junit.framework.TestSuite.runTest(TestSuite.java:243)
> at junit.framework.TestSuite.run(TestSuite.java:238)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
> at org.junit.runners.Suite.runChild(Suite.java:128)
> at org.junit.runners.Suite.runChild(Suite.java:24)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> {noformat}
> We've seen this for months but has been really tricky to track down. The
> output of the test has this:
> {noformat}
> Run pig script using PigRunner.run() for Pig version 0.8+
> Apache Pig version 0.12.0-cdh5.5.0-SNAPSHOT (rexported)
> compiled Sep 30 2015, 22:11:56
> Hadoop Job IDs executed by Pig: job_20151001151153510_0001
> Run pig script using PigRunner.run() for Pig version 0.8+
> {noformat}
> Compared with a successful run of the test, the difference is that it shows a
> Hadoop Job ID. This test is supposed to make Pig fail and not run any jobs,
> which can be confirmed in the test output, yet there's a Hadoop Job ID here!
> It turns out that {{PigRunner}} eventually gets to {{PigStats}}, which is a
> singleton. And because we're running all of the Pig tests in the same JVM,
> {{PigStats}} is actually leaking through the tests. It doesn't seem to be a
> problem most of the time because it must be getting overwritten at each run,
> however, because {{testPig_withNullExternalID}} is purposefully failing the
> Pig job, it's not, and we end up with the {{PigStats}} from the previous test
> or a non-initialized {{PigStats}} depending on the order of the tests. If
> it's a previous {{PigStats}}, then {{PigMain}} will write the Hadoop Job ID
> to a new file for the {{testPig_withNullExternalID}} test, which will then
> fail because it's not expecting this file.
> Unfortunately, there isn't a clean way to reset {{PigStats}}. Pig v 0.9 has
> a {{set}} method which we can pass {{null}} to in order to reset the
> {{PigStats}}. However, this was changed in Pig v 0.13 to the {{start}}
> method. In either case, they're both package private, so we have to
> usereflection to find the existing method for whichever version of Pig we're
> using and make it public. We should do this after every Pig test, even if
> they don't all need it just to make sure that we're starting with a fresh Pig
> each time as unit tests are supposed to start fresh anyway.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
