Hi, Mike Thanks for sharing your experiments.
> CommitsImplTest.testListCommits > CommitsImplTest.testGetCommit_generation_notfound > CommitsImplTest.testGetSegments > DocumentsImplTest.testGetDocumentFIelds I also found CommitsImplTest and DocumentsImplTest fail frequently, especially CommitsImplTest is unhappy with lucene test framework (I pointed that in my previous post.) > I wonder if this is somehow related to running mvn from command line vs running in IntelliJ since previously I was doing the latter In my personal experience, when I was running those suspicious tests on IntelliJ IDEA, they were always green - but I am not sure that `mvn test` is the cause. Thanks, Tomoko 2018年8月21日(火) 22:53 Michael Sokolov <msoko...@gmail.com>: > I was running these luke tests a bunch and found the following tests fail > intermittently; pretty frequently. Once I @Ignore them I can get a > consistent pass: > > > CommitsImplTest.testListCommits > CommitsImplTest.testGetCommit_generation_notfound > CommitsImplTest.testGetSegments > DocumentsImplTest.testGetDocumentFIelds > > I did not attempt to figure out why the tests were failing, but to do that, > I would: > > Run repeatedly until you get a failure -- save the test "seed" from this > run that should be printed out in the failure message Then you should be > able to reliably reproduce this failure by re-running with system property > "tests.seed" set to that value. This is used to initialize the > randomization that LuceneTestCase does. > > My best guess is that the failures may have to do with randomly using some > Directory implementation or other Lucene feature that Luke doesn't properly > handle? > > Hmm I was trying this again to see if I could get an example, and strangely > these tests are no longer failing for me after several runs, when > previously they failed quite often. I wonder if this is somehow related to > running mvn from command line vs running in IntelliJ since previously I was > doing the latter > > -Mike > > On Tue, Aug 21, 2018 at 9:01 AM Tomoko Uchida < > tomoko.uchida.1...@gmail.com> > wrote: > > > Hello, > > > > Could you give me some advice or comments about usage of LuceneTestCase. > > > > Some of our unit tests extending LuceneTestCase fail by assertion error > -- > > sometimes, randomly. > > I suppose we use LuceneTestCase in inappropriate way, but cannot find out > > how to fix it. > > > > Here is some information about failed tests. > > > > * The full test code is here: > > > > > https://github.com/DmitryKey/luke/blob/master/src/test/java/org/apache/lucene/luke/models/commits/CommitsImplTest.java > > * We run tests by `mvn test` on Mac PC or Travis CI (oracle jdk8/9/10, > > openjdk 8/9/10), assertion errors occur regardless of platform or jdk > > version. > > * Stack trace of an assertion error is at the end of this mail. > > > > Any advice are appreciated. Please tell me if more information is needed. > > > > Thanks, > > Tomoko > > > > > > ------------------------------------------------------- > > T E S T S > > ------------------------------------------------------- > > Running org.apache.lucene.luke.models.commits.CommitsImplTest > > NOTE: reproduce with: ant test -Dtestcase=CommitsImplTest > > -Dtests.method=testGetSegmentAttributes -Dtests.seed=35AF58F652536895 > > -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=de > > -Dtests.timezone=Africa/Kigali -Dtests.asserts=true > > -Dtests.file.encoding=UTF-8 > > NOTE: leaving temporary files on disk at: > > > > > /private/var/folders/xr/mrs6w1m15y1f4wkgfhn_x1dm0000gp/T/lucene.luke.models.commits.CommitsImplTest_35AF58F652536895-001 > > NOTE: test params are: > > > > > codec=HighCompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=HIGH_COMPRESSION, > > chunkSize=6, maxDocsPerChunk=7, blockSize=2), > > > > > termVectorsFormat=CompressingTermVectorsFormat(compressionMode=HIGH_COMPRESSION, > > chunkSize=6, blockSize=2)), sim=RandomSimilarity(queryNorm=true): {}, > > locale=de, timezone=Africa/Kigali > > NOTE: Mac OS X 10.13.6 x86_64/Oracle Corporation 1.8.0_181 > > (64-bit)/cpus=4,threads=1,free=201929064,total=257425408 > > NOTE: All tests run in this JVM: [CommitsImplTest] > > Tests run: 13, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.44 sec > > <<< FAILURE! > > > > > testGetSegmentAttributes(org.apache.lucene.luke.models.commits.CommitsImplTest) > > Time elapsed: 0.047 sec <<< FAILURE! > > java.lang.AssertionError > > at > __randomizedtesting.SeedInfo.seed([35AF58F652536895:AE37E8467BC01918]:0) > > at org.junit.Assert.fail(Assert.java:92) > > at org.junit.Assert.assertTrue(Assert.java:43) > > at org.junit.Assert.assertTrue(Assert.java:54) > > at > > > > > org.apache.lucene.luke.models.commits.CommitsImplTest.testGetSegmentAttributes(CommitsImplTest.java:151) > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > at > > > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > > at > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > at java.lang.reflect.Method.invoke(Method.java:498) > > at > > > > > com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737) > > at > > > > > com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934) > > at > > > > > com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970) > > at > > > > > com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984) > > at > > > > > org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49) > > at > > > > > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) > > at > > > > > org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) > > at > > > > > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) > > at > > > > > org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) > > at > > > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > > at > > > > > com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) > > at > > > > > com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817) > > at > > > > > com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468) > > at > > > > > com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943) > > at > > > > > com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829) > > at > > > > > com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879) > > at > > > > > com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890) > > at > > > > > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) > > at > > > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > > at > > > > > org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41) > > at > > > > > com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) > > at > > > > > com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) > > at > > > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > > at > > > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > > at > > > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > > at > > > > > org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) > > at > > > > > org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) > > at > > > > > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) > > at > > > > > org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54) > > at > > > > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > > at > > > > > com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) > > at java.lang.Thread.run(Thread.java:748) > > > -- Tomoko Uchida
