[ https://issues.apache.org/jira/browse/OOZIE-2382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
abhishek bafna updated OOZIE-2382: ---------------------------------- Fix Version/s: (was: trunk) 4.3.0 > org.apache.oozie.action.hadoop.TestPigMain.testPig_withNullExternalID is > flakey > ------------------------------------------------------------------------------- > > Key: OOZIE-2382 > URL: https://issues.apache.org/jira/browse/OOZIE-2382 > Project: Oozie > Issue Type: Bug > Components: tests > Affects Versions: trunk > Reporter: Robert Kanter > Assignee: Robert Kanter > Fix For: 4.3.0 > > Attachments: OOZIE-2382.001.patch, OOZIE-2382.addendum.001.patch > > > {{rg.apache.oozie.action.hadoop.TestPigMain.testPig_withNullExternalID}} is > flakey. > {noformat} > junit.framework.AssertionFailedError > at junit.framework.Assert.fail(Assert.java:48) > at junit.framework.Assert.assertTrue(Assert.java:20) > at junit.framework.Assert.assertFalse(Assert.java:34) > at junit.framework.Assert.assertFalse(Assert.java:41) > at > org.apache.oozie.action.hadoop.PigTestCase.testPig_withNullExternalID(PigTestCase.java:112) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at junit.framework.TestCase.runTest(TestCase.java:168) > at junit.framework.TestCase.runBare(TestCase.java:134) > at junit.framework.TestResult$1.protect(TestResult.java:110) > at junit.framework.TestResult.runProtected(TestResult.java:128) > at junit.framework.TestResult.run(TestResult.java:113) > at junit.framework.TestCase.run(TestCase.java:124) > at junit.framework.TestSuite.runTest(TestSuite.java:243) > at junit.framework.TestSuite.run(TestSuite.java:238) > at > org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83) > at org.junit.runners.Suite.runChild(Suite.java:128) > at org.junit.runners.Suite.runChild(Suite.java:24) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:744) > {noformat} > We've seen this for months but has been really tricky to track down. The > output of the test has this: > {noformat} > Run pig script using PigRunner.run() for Pig version 0.8+ > Apache Pig version 0.12.0-cdh5.5.0-SNAPSHOT (rexported) > compiled Sep 30 2015, 22:11:56 > Hadoop Job IDs executed by Pig: job_20151001151153510_0001 > Run pig script using PigRunner.run() for Pig version 0.8+ > {noformat} > Compared with a successful run of the test, the difference is that it shows a > Hadoop Job ID. This test is supposed to make Pig fail and not run any jobs, > which can be confirmed in the test output, yet there's a Hadoop Job ID here! > It turns out that {{PigRunner}} eventually gets to {{PigStats}}, which is a > singleton. And because we're running all of the Pig tests in the same JVM, > {{PigStats}} is actually leaking through the tests. It doesn't seem to be a > problem most of the time because it must be getting overwritten at each run, > however, because {{testPig_withNullExternalID}} is purposefully failing the > Pig job, it's not, and we end up with the {{PigStats}} from the previous test > or a non-initialized {{PigStats}} depending on the order of the tests. If > it's a previous {{PigStats}}, then {{PigMain}} will write the Hadoop Job ID > to a new file for the {{testPig_withNullExternalID}} test, which will then > fail because it's not expecting this file. > Unfortunately, there isn't a clean way to reset {{PigStats}}. Pig v 0.9 has > a {{set}} method which we can pass {{null}} to in order to reset the > {{PigStats}}. However, this was changed in Pig v 0.13 to the {{start}} > method. In either case, they're both package private, so we have to > usereflection to find the existing method for whichever version of Pig we're > using and make it public. We should do this after every Pig test, even if > they don't all need it just to make sure that we're starting with a fresh Pig > each time as unit tests are supposed to start fresh anyway. -- This message was sent by Atlassian JIRA (v6.3.4#6332)