These failures relate to the way Arrays.binarySearch works when there are repeated values, in which case the result is undefined (it can be any of the indexes with the value), but in SlowCompositeReaderWrapper we are relying on finding the lowest-indexed of the repeats. I'll work on a fix
On Sat, Sep 28, 2024 at 3:32 PM Policeman Jenkins Server <jenk...@thetaphi.de> wrote: > > Build: https://jenkins.thetaphi.de/job/Lucene-main-Linux/50364/ > Java: 64bit/hotspot/jdk-24-ea+16 -XX:-UseCompressedOops -XX:+UseParallelGC > > 2 tests failed. > FAILED: > org.apache.lucene.backward_codecs.lucene90.TestLucene90HnswVectorsFormat.testSparseVectors > > Error Message: > org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed > > Stack Trace: > org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed > at > app/org.apache.lucene.core@10.0.0-SNAPSHOT/org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:916) > at > app/org.apache.lucene.core@10.0.0-SNAPSHOT/org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:929) > at > app/org.apache.lucene.core@10.0.0-SNAPSHOT/org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1550) > at > app/org.apache.lucene.core@10.0.0-SNAPSHOT/org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1838) > at > app/org.apache.lucene.core@10.0.0-SNAPSHOT/org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1478) > at > app/org.apache.lucene.test_framework@10.0.0-SNAPSHOT/org.apache.lucene.tests.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:224) > at > app/org.apache.lucene.test_framework@10.0.0-SNAPSHOT/org.apache.lucene.tests.index.BaseKnnVectorsFormatTestCase.testSparseVectors(BaseKnnVectorsFormatTestCase.java:873) > at > java.base@24-ea/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) > at java.base@24-ea/java.lang.reflect.Method.invoke(Method.java:573) > at > app/randomizedtesting.runner@2.8.1/com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758) > at > app/randomizedtesting.runner@2.8.1/com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946) > at > app/randomizedtesting.runner@2.8.1/com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982) > at > app/randomizedtesting.runner@2.8.1/com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996) > at > app/org.apache.lucene.test_framework@10.0.0-SNAPSHOT/org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48) > at > app/org.apache.lucene.test_framework@10.0.0-SNAPSHOT/org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) > at > app/org.apache.lucene.test_framework@10.0.0-SNAPSHOT/org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45) > at > app/org.apache.lucene.test_framework@10.0.0-SNAPSHOT/org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) > at > app/org.apache.lucene.test_framework@10.0.0-SNAPSHOT/org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) > at > app/junit@4.13.1/org.junit.rules.RunRules.evaluate(RunRules.java:20) > at > app/randomizedtesting.runner@2.8.1/com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > app/randomizedtesting.runner@2.8.1/com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) > at > app/randomizedtesting.runner@2.8.1/com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843) > at > app/randomizedtesting.runner@2.8.1/com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490) > at > app/randomizedtesting.runner@2.8.1/com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955) > at > app/randomizedtesting.runner@2.8.1/com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840) > at > app/randomizedtesting.runner@2.8.1/com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891) > at > app/randomizedtesting.runner@2.8.1/com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902) > at > app/org.apache.lucene.test_framework@10.0.0-SNAPSHOT/org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) > at > app/randomizedtesting.runner@2.8.1/com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > app/org.apache.lucene.test_framework@10.0.0-SNAPSHOT/org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38) > at > app/randomizedtesting.runner@2.8.1/com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) > at > app/randomizedtesting.runner@2.8.1/com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) > at > app/randomizedtesting.runner@2.8.1/com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > app/randomizedtesting.runner@2.8.1/com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > app/org.apache.lucene.test_framework@10.0.0-SNAPSHOT/org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) > at > app/org.apache.lucene.test_framework@10.0.0-SNAPSHOT/org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) > at > app/org.apache.lucene.test_framework@10.0.0-SNAPSHOT/org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) > at > app/org.apache.lucene.test_framework@10.0.0-SNAPSHOT/org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) > at > app/org.apache.lucene.test_framework@10.0.0-SNAPSHOT/org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47) > at > app/junit@4.13.1/org.junit.rules.RunRules.evaluate(RunRules.java:20) > at > app/randomizedtesting.runner@2.8.1/com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > app/randomizedtesting.runner@2.8.1/com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) > at > app/randomizedtesting.runner@2.8.1/com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850) > at java.base@24-ea/java.lang.Thread.run(Thread.java:1576) > Caused by: java.lang.NullPointerException: Cannot invoke > "org.apache.lucene.index.FloatVectorValues.vectorValue(int)" because > "this.subs[this.lastSubIndex].sub" is null > at > org.apache.lucene.core@10.0.0-SNAPSHOT/org.apache.lucene.index.SlowCompositeCodecReaderWrapper$SlowCompositeKnnVectorsReaderWrapper$MergedFloatVectorValues.vectorValue(SlowCompositeCodecReaderWrapper.java:905) > at > org.apache.lucene.core@10.0.0-SNAPSHOT/org.apache.lucene.index.SortingCodecReader$SortingFloatVectorValues.vectorValue(SortingCodecReader.java:324) > at > org.apache.lucene.core@10.0.0-SNAPSHOT/org.apache.lucene.codecs.KnnVectorsWriter$MergedVectorValues$MergedFloat32VectorValues.vectorValue(KnnVectorsWriter.java:364) > at > org.apache.lucene.backward_codecs.lucene90.Lucene90HnswVectorsWriter.writeVectorData(Lucene90HnswVectorsWriter.java:194) > at > org.apache.lucene.backward_codecs.lucene90.Lucene90HnswVectorsWriter.writeField(Lucene90HnswVectorsWriter.java:122) > at > org.apache.lucene.core@10.0.0-SNAPSHOT/org.apache.lucene.codecs.BufferingKnnVectorsWriter.mergeOneField(BufferingKnnVectorsWriter.java:203) > at > org.apache.lucene.core@10.0.0-SNAPSHOT/org.apache.lucene.codecs.perfield.PerFieldKnnVectorsFormat$FieldsWriter.mergeOneField(PerFieldKnnVectorsFormat.java:121) > at > org.apache.lucene.core@10.0.0-SNAPSHOT/org.apache.lucene.codecs.KnnVectorsWriter.merge(KnnVectorsWriter.java:105) > at > org.apache.lucene.core@10.0.0-SNAPSHOT/org.apache.lucene.index.SegmentMerger.mergeVectorValues(SegmentMerger.java:271) > at > org.apache.lucene.core@10.0.0-SNAPSHOT/org.apache.lucene.index.SegmentMerger.mergeWithLogging(SegmentMerger.java:314) > at > org.apache.lucene.core@10.0.0-SNAPSHOT/org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:158) > at > org.apache.lucene.core@10.0.0-SNAPSHOT/org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:5283) > at > org.apache.lucene.core@10.0.0-SNAPSHOT/org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4749) > at > org.apache.lucene.core@10.0.0-SNAPSHOT/org.apache.lucene.index.IndexWriter$IndexWriterMergeSource.merge(IndexWriter.java:6567) > at > org.apache.lucene.core@10.0.0-SNAPSHOT/org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:668) > at > org.apache.lucene.core@10.0.0-SNAPSHOT/org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:729) > > > FAILED: > org.apache.lucene.backward_codecs.lucene92.TestLucene92HnswVectorsFormat.testSparseVectors > > Error Message: > org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed > > Stack Trace: > org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed > at > app/org.apache.lucene.core@10.0.0-SNAPSHOT/org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:916) > at > app/org.apache.lucene.core@10.0.0-SNAPSHOT/org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:929) > at > app/org.apache.lucene.core@10.0.0-SNAPSHOT/org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1550) > at > app/org.apache.lucene.core@10.0.0-SNAPSHOT/org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1838) > at > app/org.apache.lucene.core@10.0.0-SNAPSHOT/org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1478) > at > app/org.apache.lucene.test_framework@10.0.0-SNAPSHOT/org.apache.lucene.tests.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:224) > at > app/org.apache.lucene.test_framework@10.0.0-SNAPSHOT/org.apache.lucene.tests.index.BaseKnnVectorsFormatTestCase.testSparseVectors(BaseKnnVectorsFormatTestCase.java:873) > at > java.base@24-ea/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) > at java.base@24-ea/java.lang.reflect.Method.invoke(Method.java:573) > at > app/randomizedtesting.runner@2.8.1/com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758) > at > app/randomizedtesting.runner@2.8.1/com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946) > at > app/randomizedtesting.runner@2.8.1/com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982) > at > app/randomizedtesting.runner@2.8.1/com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996) > at > app/org.apache.lucene.test_framework@10.0.0-SNAPSHOT/org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48) > at > app/org.apache.lucene.test_framework@10.0.0-SNAPSHOT/org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) > at > app/org.apache.lucene.test_framework@10.0.0-SNAPSHOT/org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45) > at > app/org.apache.lucene.test_framework@10.0.0-SNAPSHOT/org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) > at > app/org.apache.lucene.test_framework@10.0.0-SNAPSHOT/org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) > at > app/junit@4.13.1/org.junit.rules.RunRules.evaluate(RunRules.java:20) > at > app/randomizedtesting.runner@2.8.1/com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > app/randomizedtesting.runner@2.8.1/com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) > at > app/randomizedtesting.runner@2.8.1/com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843) > at > app/randomizedtesting.runner@2.8.1/com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490) > at > app/randomizedtesting.runner@2.8.1/com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955) > at > app/randomizedtesting.runner@2.8.1/com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840) > at > app/randomizedtesting.runner@2.8.1/com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891) > at > app/randomizedtesting.runner@2.8.1/com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902) > at > app/org.apache.lucene.test_framework@10.0.0-SNAPSHOT/org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) > at > app/randomizedtesting.runner@2.8.1/com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > app/org.apache.lucene.test_framework@10.0.0-SNAPSHOT/org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38) > at > app/randomizedtesting.runner@2.8.1/com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) > at > app/randomizedtesting.runner@2.8.1/com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) > at > app/randomizedtesting.runner@2.8.1/com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > app/randomizedtesting.runner@2.8.1/com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > app/org.apache.lucene.test_framework@10.0.0-SNAPSHOT/org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) > at > app/org.apache.lucene.test_framework@10.0.0-SNAPSHOT/org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) > at > app/org.apache.lucene.test_framework@10.0.0-SNAPSHOT/org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) > at > app/org.apache.lucene.test_framework@10.0.0-SNAPSHOT/org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) > at > app/org.apache.lucene.test_framework@10.0.0-SNAPSHOT/org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47) > at > app/junit@4.13.1/org.junit.rules.RunRules.evaluate(RunRules.java:20) > at > app/randomizedtesting.runner@2.8.1/com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > app/randomizedtesting.runner@2.8.1/com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) > at > app/randomizedtesting.runner@2.8.1/com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850) > at java.base@24-ea/java.lang.Thread.run(Thread.java:1576) > Caused by: java.lang.NullPointerException: Cannot invoke > "org.apache.lucene.index.FloatVectorValues.vectorValue(int)" because > "this.subs[this.lastSubIndex].sub" is null > at > org.apache.lucene.core@10.0.0-SNAPSHOT/org.apache.lucene.index.SlowCompositeCodecReaderWrapper$SlowCompositeKnnVectorsReaderWrapper$MergedFloatVectorValues.vectorValue(SlowCompositeCodecReaderWrapper.java:905) > at > org.apache.lucene.core@10.0.0-SNAPSHOT/org.apache.lucene.index.SortingCodecReader$SortingFloatVectorValues.vectorValue(SortingCodecReader.java:324) > at > org.apache.lucene.core@10.0.0-SNAPSHOT/org.apache.lucene.codecs.KnnVectorsWriter$MergedVectorValues$MergedFloat32VectorValues.vectorValue(KnnVectorsWriter.java:364) > at > org.apache.lucene.backward_codecs.lucene92.Lucene92HnswVectorsWriter.writeVectorData(Lucene92HnswVectorsWriter.java:197) > at > org.apache.lucene.backward_codecs.lucene92.Lucene92HnswVectorsWriter.writeField(Lucene92HnswVectorsWriter.java:129) > at > org.apache.lucene.core@10.0.0-SNAPSHOT/org.apache.lucene.codecs.BufferingKnnVectorsWriter.mergeOneField(BufferingKnnVectorsWriter.java:203) > at > org.apache.lucene.core@10.0.0-SNAPSHOT/org.apache.lucene.codecs.perfield.PerFieldKnnVectorsFormat$FieldsWriter.mergeOneField(PerFieldKnnVectorsFormat.java:121) > at > org.apache.lucene.core@10.0.0-SNAPSHOT/org.apache.lucene.codecs.KnnVectorsWriter.merge(KnnVectorsWriter.java:105) > at > org.apache.lucene.core@10.0.0-SNAPSHOT/org.apache.lucene.index.SegmentMerger.mergeVectorValues(SegmentMerger.java:271) > at > org.apache.lucene.core@10.0.0-SNAPSHOT/org.apache.lucene.index.SegmentMerger.mergeWithLogging(SegmentMerger.java:314) > at > org.apache.lucene.core@10.0.0-SNAPSHOT/org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:158) > at > org.apache.lucene.core@10.0.0-SNAPSHOT/org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:5283) > at > org.apache.lucene.core@10.0.0-SNAPSHOT/org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4749) > at > org.apache.lucene.core@10.0.0-SNAPSHOT/org.apache.lucene.index.IndexWriter$IndexWriterMergeSource.merge(IndexWriter.java:6567) > at > org.apache.lucene.core@10.0.0-SNAPSHOT/org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:668) > at > org.apache.lucene.core@10.0.0-SNAPSHOT/org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:729) > > --------------------------------------------------------------------- > To unsubscribe, e-mail: builds-unsubscr...@lucene.apache.org > For additional commands, e-mail: builds-h...@lucene.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
