I was able to repro locally and track this down to an invalid test assumption that incorrectly relies on doc ordering. We can relax some of the randomness in the test case to ensure the doc ordering, but I think the right thing to do is to relax the test assertions instead. The failing assertion isn’t really necessary to the test, and I like keeping the randomness.
I pushed a fix for this on main and branch_9x (captured in https://github.com/apache/lucene/pull/12920). If anyone disagrees with the approach to the fix, I'm happy to iterate. Cheers, -Greg On Mon, Dec 11, 2023 at 12:49 Greg Miller <[email protected]> wrote: > Shoot, looking into this. This is a new test I added a month ago in > GH#12640. Not sure if we hit a random bug in the test or if the bug fix I > made in that change was incomplete somehow and this uncovered it. > > Cheers, > -Greg > > On Mon, Dec 11, 2023 at 11:17 AM Apache Jenkins Server < > [email protected]> wrote: > >> Build: https://ci-builds.apache.org/job/Lucene/job/Lucene-Check-9.x/7134/ >> >> 2 tests failed. >> FAILED: >> org.apache.lucene.facet.TestDrillSideways.testCollectionTerminated >> >> Error Message: >> java.lang.AssertionError: expected:<1> but was:<2> >> >> Stack Trace: >> java.lang.AssertionError: expected:<1> but was:<2> >> at >> __randomizedtesting.SeedInfo.seed([18C2A46594D4CAB4:8AD53C0B3D6F9A5]:0) >> at [email protected]/org.junit.Assert.fail(Assert.java:89) >> at [email protected]/org.junit.Assert.failNotEquals(Assert.java:835) >> at [email protected]/org.junit.Assert.assertEquals(Assert.java:647) >> at [email protected]/org.junit.Assert.assertEquals(Assert.java:633) >> at >> org.apache.lucene.facet.TestDrillSideways.testCollectionTerminated(TestDrillSideways.java:332) >> at >> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native >> Method) >> at >> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >> at >> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.base/java.lang.reflect.Method.invoke(Method.java:566) >> at [email protected] >> /com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758) >> at [email protected] >> /com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946) >> at [email protected] >> /com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982) >> at [email protected] >> /com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996) >> at [email protected] >> /org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48) >> at [email protected] >> /org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) >> at [email protected] >> /org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45) >> at [email protected] >> /org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) >> at [email protected] >> /org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) >> at [email protected] >> /org.junit.rules.RunRules.evaluate(RunRules.java:20) >> at [email protected] >> /com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) >> at [email protected] >> /com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) >> at [email protected] >> /com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843) >> at [email protected] >> /com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490) >> at [email protected] >> /com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955) >> at [email protected] >> /com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840) >> at [email protected] >> /com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891) >> at [email protected] >> /com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902) >> at [email protected] >> /org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) >> at [email protected] >> /com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) >> at [email protected] >> /org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38) >> at [email protected] >> /com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) >> at [email protected] >> /com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) >> at [email protected] >> /com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) >> at [email protected] >> /com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) >> at [email protected] >> /org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) >> at [email protected] >> /org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) >> at [email protected] >> /org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) >> at [email protected] >> /org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) >> at [email protected] >> /org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47) >> at [email protected] >> /org.junit.rules.RunRules.evaluate(RunRules.java:20) >> at [email protected] >> /com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) >> at [email protected] >> /com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) >> at [email protected] >> /com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850) >> at java.base/java.lang.Thread.run(Thread.java:829) >> >> >> FAILED: >> org.apache.lucene.facet.TestParallelDrillSideways.testCollectionTerminated >> >> Error Message: >> java.lang.AssertionError: expected:<1> but was:<2> >> >> Stack Trace: >> java.lang.AssertionError: expected:<1> but was:<2> >> at >> __randomizedtesting.SeedInfo.seed([18C2A46594D4CAB4:8AD53C0B3D6F9A5]:0) >> at [email protected]/org.junit.Assert.fail(Assert.java:89) >> at [email protected]/org.junit.Assert.failNotEquals(Assert.java:835) >> at [email protected]/org.junit.Assert.assertEquals(Assert.java:647) >> at [email protected]/org.junit.Assert.assertEquals(Assert.java:633) >> at >> org.apache.lucene.facet.TestDrillSideways.testCollectionTerminated(TestDrillSideways.java:332) >> at >> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native >> Method) >> at >> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >> at >> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.base/java.lang.reflect.Method.invoke(Method.java:566) >> at [email protected] >> /com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758) >> at [email protected] >> /com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946) >> at [email protected] >> /com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982) >> at [email protected] >> /com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996) >> at [email protected] >> /org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48) >> at [email protected] >> /org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) >> at [email protected] >> /org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45) >> at [email protected] >> /org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) >> at [email protected] >> /org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) >> at [email protected] >> /org.junit.rules.RunRules.evaluate(RunRules.java:20) >> at [email protected] >> /com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) >> at [email protected] >> /com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) >> at [email protected] >> /com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843) >> at [email protected] >> /com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490) >> at [email protected] >> /com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955) >> at [email protected] >> /com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840) >> at [email protected] >> /com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891) >> at [email protected] >> /com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902) >> at [email protected] >> /org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) >> at [email protected] >> /com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) >> at [email protected] >> /org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38) >> at [email protected] >> /com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) >> at [email protected] >> /com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) >> at [email protected] >> /com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) >> at [email protected] >> /com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) >> at [email protected] >> /org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) >> at [email protected] >> /org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) >> at [email protected] >> /org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) >> at [email protected] >> /org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) >> at [email protected] >> /org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47) >> at [email protected] >> /org.junit.rules.RunRules.evaluate(RunRules.java:20) >> at [email protected] >> /com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) >> at [email protected] >> /com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) >> at [email protected] >> /com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850) >> at java.base/java.lang.Thread.run(Thread.java:829) >> >> >> >> >> Build Log: >> [...truncated 1485 lines...] >> ERROR: The following test(s) have failed: >> - org.apache.lucene.facet.TestDrillSideways.testCollectionTerminated >> (:lucene:facet) >> Test output: >> /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Check-9.x/lucene/facet/build/test-results/test/outputs/OUTPUT-org.apache.lucene.facet.TestDrillSideways.txt >> Reproduce with: gradlew :lucene:facet:test --tests >> "org.apache.lucene.facet.TestDrillSideways.testCollectionTerminated" >> -Ptests.jvms=4 -Ptests.jvmargs= -Ptests.seed=18C2A46594D4CAB4 >> -Ptests.multiplier=2 -Ptests.badapples=false -Ptests.gui=true >> -Ptests.file.encoding=ISO-8859-1 -Ptests.vectorsize=256 >> >> - >> org.apache.lucene.facet.TestParallelDrillSideways.testCollectionTerminated >> (:lucene:facet) >> Test output: >> /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Check-9.x/lucene/facet/build/test-results/test/outputs/OUTPUT-org.apache.lucene.facet.TestParallelDrillSideways.txt >> Reproduce with: gradlew :lucene:facet:test --tests >> "org.apache.lucene.facet.TestParallelDrillSideways.testCollectionTerminated" >> -Ptests.jvms=4 -Ptests.jvmargs= -Ptests.seed=18C2A46594D4CAB4 >> -Ptests.multiplier=2 -Ptests.badapples=false -Ptests.gui=true >> -Ptests.file.encoding=ISO-8859-1 -Ptests.vectorsize=256 >> >> >> Deprecated Gradle features were used in this build, making it >> incompatible with Gradle 9.0. >> >> You can use '--warning-mode all' to show the individual deprecation >> warnings and determine if they come from your own scripts or plugins. >> >> For more on this, please refer to >> https://docs.gradle.org/8.4/userguide/command_line_interface.html#sec:command_line_warnings >> in the Gradle documentation. >> >> BUILD SUCCESSFUL in 36m 40s >> 824 actionable tasks: 824 executed >> >> Publishing build scan... >> https://ge.apache.org/s/kuwgbciekwsoe >> >> Build step 'Invoke Gradle script' changed build result to SUCCESS >> Archiving artifacts >> hudson.FilePath$ValidateAntFileMask$1Cancel >> at >> hudson.FilePath$ValidateAntFileMask$1.isCaseSensitive(FilePath.java:3300) >> at >> org.apache.tools.ant.DirectoryScanner.lambda$isIncluded$3(DirectoryScanner.java:1374) >> at >> java.base/java.util.stream.MatchOps$1MatchSink.accept(MatchOps.java:90) >> at >> java.base/java.util.Spliterators$ArraySpliterator.tryAdvance(Spliterators.java:958) >> at >> java.base/java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:127) >> at >> java.base/java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:502) >> at >> java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:488) >> at >> java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) >> at >> java.base/java.util.stream.MatchOps$MatchOp.evaluateSequential(MatchOps.java:230) >> at >> java.base/java.util.stream.MatchOps$MatchOp.evaluateSequential(MatchOps.java:196) >> at >> java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) >> at >> java.base/java.util.stream.ReferencePipeline.anyMatch(ReferencePipeline.java:528) >> at >> org.apache.tools.ant.DirectoryScanner.isIncluded(DirectoryScanner.java:1374) >> at >> org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1241) >> at >> org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1267) >> at >> org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1267) >> at >> org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1267) >> at >> org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1267) >> at >> org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1267) >> at >> org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1267) >> at >> org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1267) >> at >> org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1267) >> at >> org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1267) >> at >> org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1267) >> at >> org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1267) >> at >> org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1194) >> at >> org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1156) >> at >> org.apache.tools.ant.DirectoryScanner.checkIncludePatterns(DirectoryScanner.java:954) >> at >> org.apache.tools.ant.DirectoryScanner.scan(DirectoryScanner.java:912) >> at >> hudson.FilePath$ValidateAntFileMask.hasMatch(FilePath.java:3313) >> Caused: hudson.FilePath$FileMaskNoMatchesFoundException: no matches found >> within 10000 >> at >> hudson.FilePath$ValidateAntFileMask.hasMatch(FilePath.java:3318) >> at hudson.FilePath$ValidateAntFileMask.invoke(FilePath.java:3196) >> at hudson.FilePath$ValidateAntFileMask.invoke(FilePath.java:3174) >> at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3578) >> Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to >> lucene-solr-2 >> at >> hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1784) >> at >> hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356) >> at hudson.remoting.Channel.call(Channel.java:1000) >> at hudson.FilePath.act(FilePath.java:1192) >> at hudson.FilePath.act(FilePath.java:1181) >> at hudson.FilePath.validateAntFileMask(FilePath.java:3171) >> at >> hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:271) >> at >> hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:80) >> at >> hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) >> at >> hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:818) >> at >> hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:767) >> at hudson.model.Build$BuildExecution.post2(Build.java:179) >> at >> hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:711) >> at hudson.model.Run.execute(Run.java:1925) >> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:44) >> at >> hudson.model.ResourceController.execute(ResourceController.java:101) >> at hudson.model.Executor.run(Executor.java:442) >> Caused: hudson.FilePath$TunneledInterruptedException >> at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3580) >> at hudson.remoting.UserRequest.perform(UserRequest.java:211) >> at hudson.remoting.UserRequest.perform(UserRequest.java:54) >> at hudson.remoting.Request$2.run(Request.java:377) >> at >> hudson.remoting.InterceptingExecutorService.lambda$wrap$0(InterceptingExecutorService.java:78) >> at >> java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) >> at >> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) >> at >> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) >> at java.base/java.lang.Thread.run(Thread.java:829) >> Caused: java.lang.InterruptedException: >> hudson.FilePath$FileMaskNoMatchesFoundException: no matches found within >> 10000 >> at hudson.FilePath.act(FilePath.java:1194) >> at hudson.FilePath.act(FilePath.java:1181) >> at hudson.FilePath.validateAntFileMask(FilePath.java:3171) >> at >> hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:271) >> at >> hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:80) >> at >> hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) >> at >> hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:818) >> at >> hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:767) >> at hudson.model.Build$BuildExecution.post2(Build.java:179) >> at >> hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:711) >> at hudson.model.Run.execute(Run.java:1925) >> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:44) >> at >> hudson.model.ResourceController.execute(ResourceController.java:101) >> at hudson.model.Executor.run(Executor.java:442) >> No artifacts found that match the file pattern >> "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error? >> Recording test results >> [Checks API] No suitable checks publisher found. >> Build step 'Publish JUnit test result report' changed build result to >> UNSTABLE >> Email was triggered for: Unstable (Test Failures) >> Sending email for trigger: Unstable (Test Failures) >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] > >
