Re: [JENKINS] Lucene » Lucene-NightlyTests-main - Build # 1019 - Unstable!
: This reproduces constantly for me, I even rolls back to some (random) old : commit: 9acc6539959 (in last September) and it still reproduces. FWIW: The oldest known failure of TestGroupFacetCollector.testRandom in my archive is the week of 2021-05-31 : Found another similar failure: : https://ci-builds.apache.org/job/Lucene/job/Lucene-Check-main/8835/ : The error seems to be a more common one but somehow (so far) it only : happens inside this TestGroupFacetCollector test... : : On Wed, May 31, 2023 at 10:18 PM Apache Jenkins Server < : jenk...@builds.apache.org> wrote: : : > Build: : > https://ci-builds.apache.org/job/Lucene/job/Lucene-NightlyTests-main/1019/ : > : > 1 tests failed. : > FAILED: : > org.apache.lucene.search.grouping.TestGroupFacetCollector.testRandom : > : > Error Message: : > java.lang.IndexOutOfBoundsException: Range [0, 0 + 692) out of bounds for : > length 80 : > : > Stack Trace: : > java.lang.IndexOutOfBoundsException: Range [0, 0 + 692) out of bounds for : > length 80 : > at : > __randomizedtesting.SeedInfo.seed([46C657C9C98115E0:348A72C678E1A393]:0) : > at : > java.base/jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:64) : > at : > java.base/jdk.internal.util.Preconditions.outOfBoundsCheckFromIndexSize(Preconditions.java:82) : > at : > java.base/jdk.internal.util.Preconditions.checkFromIndexSize(Preconditions.java:361) : > at java.base/java.util.Objects.checkFromIndexSize(Objects.java:413) : > at java.base/java.nio.HeapByteBuffer.get(HeapByteBuffer.java:180) : > at org.apache.lucene.core@10.0.0-SNAPSHOT : > /org.apache.lucene.store.ByteBuffersDataInput.readBytes(ByteBuffersDataInput.java:159) : > at org.apache.lucene.core@10.0.0-SNAPSHOT : > /org.apache.lucene.store.ByteBuffersIndexInput.readBytes(ByteBuffersIndexInput.java:85) : > at org.apache.lucene.test_framework@10.0.0-SNAPSHOT : > /org.apache.lucene.tests.store.MockIndexInputWrapper.readBytes(MockIndexInputWrapper.java:148) : > at org.apache.lucene.core@10.0.0-SNAPSHOT : > /org.apache.lucene.codecs.lucene90.Lucene90DocValuesProducer$TermsDict.decompressBlock(Lucene90DocValuesProducer.java:1234) : > at org.apache.lucene.core@10.0.0-SNAPSHOT : > /org.apache.lucene.codecs.lucene90.Lucene90DocValuesProducer$TermsDict.next(Lucene90DocValuesProducer.java:1092) : > at : > org.apache.lucene.search.grouping.TermGroupFacetCollector$MV$SegmentResult.nextTerm(TermGroupFacetCollector.java:437) : > at : > org.apache.lucene.search.grouping.GroupFacetCollector.mergeSegmentResults(GroupFacetCollector.java:97) : > at : > org.apache.lucene.search.grouping.TestGroupFacetCollector.testRandom(TestGroupFacetCollector.java:429) : > at : > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native : > Method) : > at : > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) : > at : > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) : > at java.base/java.lang.reflect.Method.invoke(Method.java:568) : > at randomizedtesting.runner@2.8.1 : > /com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758) : > at randomizedtesting.runner@2.8.1 : > /com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946) : > at randomizedtesting.runner@2.8.1 : > /com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982) : > at randomizedtesting.runner@2.8.1 : > /com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996) : > at org.apache.lucene.test_framework@10.0.0-SNAPSHOT : > /org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48) : > at org.apache.lucene.test_framework@10.0.0-SNAPSHOT : > /org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) : > at org.apache.lucene.test_framework@10.0.0-SNAPSHOT : > /org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45) : > at org.apache.lucene.test_framework@10.0.0-SNAPSHOT : > /org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) : > at org.apache.lucene.test_framework@10.0.0-SNAPSHOT : > /org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) : > at junit@4.13.1 : > /org.junit.rules.RunRules.evaluate(RunRules.java:20) : > at randomizedtesting.runner@2.8.1 : > /com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) : > at randomizedtesting.runner@2.8.1 : > /com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) :
Re: [JENKINS] Lucene-9.x-MacOSX (64bit/hotspot/jdk-16.0.2) - Build # 2385 - Unstable!
FWIW: This same exact failure (same assertion msg) happened on main
recently as well.
I've included some log snippets shouwing randomization details of both
failures below.
Historically this test (with this name) has existed unmodified since long
before i started keeping records of failure rates, but it has had failure
"blips" ~14 times since 2021-04-12 -- prio to that, i have no record of it
*EVER* failing.
Looking at the log snippets, the few things that jump out at me as being
"common" between them...
- java >= 16
- -XX:+UseCompressedOops
- codec=SimpleText
- sim=Asserting(RandomSimilarity(queryNorm=false)
- common "All tests run in this JVM" ...
- TestOmitPositions
- TestStopFilter
- TestXYShape
- TestAddIndexes
- TestIndexCommit
- TestIndexingSequenceNumbers
- TestDocIDMerger
- TestLatLonMultiLineShapeQueries
- TestLucene90StoredFieldsFormatHighCompression
https://jenkins.thetaphi.de/view/Lucene/job/Lucene-main-Linux/42271/
Checking out Revision 72b91156f3d2049e66a987b9cc435c22d6c9e3af
(refs/remotes/origin/main)
[description-setter] Description set: Java: 64bit/hotspot/jdk-17.0.5
-XX:+UseCompressedOops -XX:+UseParallelGC
2> NOTE: reproduce with: gradlew test --tests
TestSizeBoundedForceMerge.testByteSizeLimit -Dtests.seed=AE682DEBC10C54AC
-Dtests.multiplier=3 -Dtests.locale=so-KE -Dtests.timezone=Asia/Kathmandu
-Dtests.asserts=true -Dtests.file.encoding=UTF-8
2> NOTE: test params are: codec=SimpleText,
sim=Asserting(RandomSimilarity(queryNorm=false): {}), locale=so-KE,
timezone=Asia/Kathmandu
2> NOTE: Linux 5.19.0-42-generic amd64/Eclipse Adoptium 17.0.5
(64-bit)/cpus=16,threads=1,free=191739968,total=335020032
2> NOTE: All tests run in this JVM: [TestDemo, TestAnalysisSPILoader,
TestCharFilter, TestStopFilter, TestPackedTokenAttributeImpl,
TestDeflateWithPresetDictCompressionMode, TestIndexedDISI,
TestLucene90NormsFormat, TestLucene90StoredFieldsFormatHighCompression,
TestStoredFieldsInt, TestBinaryDocument, TestFeatureSort,
TestKeywordField, TestLatLonMultiLineShapeQueries,
TestLatLonPointDistanceSort, TestLatLonShape, TestPerFieldConsistency,
TestXYMultiPointShapeQueries, TestXYShape, TestLine2D,
TestSimpleWKTShapeParsing, TestXYRectangle, Test2BPostings,
TestAddIndexes, TestApproximatePriorityQueue, TestBufferedUpdates,
TestConcurrentMergeScheduler, TestDeletionPolicy, TestDocIDMerger,
TestDocsWithFieldSet, TestDuelingCodecsAtNight, TestFieldUpdatesBuffer,
TestFilterMergePolicy, TestIndexCommit, TestIndexSorting,
TestIndexWriterExceptions, TestIndexWriterMergePolicy,
TestIndexWriterOutOfFileDescriptors, TestIndexingSequenceNumbers,
TestLogMergePolicy, TestMergeRateLimiter, TestMultiTermsEnum,
TestNoDeletionPolicy, TestOmitPositions, TestParallelReaderEmptyIndex,
TestPerSegmentDeletes, TestReaderClosed, TestSegmentInfos,
TestSizeBoundedForceMerge]
The details from the 9.x build were...
https://jenkins.thetaphi.de/view/Lucene/job/Lucene-9.x-MacOSX/2385/
Checking out Revision d96e4ffbb5ae1ef0b599c1a83fdba8cb486dfb65
(refs/remotes/origin/branch_9x)
[description-setter] Description set: Java: 64bit/hotspot/jdk-16.0.2
-XX:+UseCompressedOops -XX:+UseShenandoahGC
2> NOTE: reproduce with: gradlew test --tests
TestSizeBoundedForceMerge.testByteSizeLimit -Dtests.seed=1BBFFFEF44E9DD39
-Dtests.locale=bs-Cyrl-BA -Dtests.timezone=America/Thunder_Bay
-Dtests.asserts=true -Dtests.file.encoding=UTF-8
2> NOTE: test params are: codec=SimpleText,
sim=Asserting(RandomSimilarity(queryNorm=false): {}), locale=bs-Cyrl-BA,
timezone=America/Thunder_Bay
2> NOTE: Mac OS X 10.14.6 x86_64/Eclipse Foundation 16.0.2
(64-bit)/cpus=6,threads=1,free=156085320,total=471072768
2> NOTE: All tests run in this JVM: [TestSearch,
TestAutomatonToTokenStream, TestCharacterUtils, TestStopFilter,
TestCharTermAttributeImpl, TestCompetitiveFreqNormAccumulator,
TestHighCompressionMode, TestLucene90DocValuesFormat,
TestLucene90NormsFormatMergeInstance,
TestLucene90StoredFieldsFormatHighCompression,
TestCompressingTermVectorsFormat, TestPerFieldPostingsFormat,
TestDoubleRange, TestFieldType, TestLatLonDocValuesField,
TestLatLonMultiLineShapeQueries, TestLatLonPointDistanceFeatureQuery,
TestLatLonPolygonShapeDVQueries, TestLongHashSet,
TestSortedSetDocValuesSetQuery, TestXYMultiPolygonShapeQueries,
TestXYShape, TestGeoUtils, TestPolygon2D, TestXYLine, Test2BDocs,
Test2BPostingsBytes, TestAddIndexes, TestAllFilesHaveCodecHeader,
TestBinaryDocValuesUpdates, TestCodecHoldsOpenFiles, TestCrash,
TestDemoParallelLeafReader, TestDocIDMerger, TestDocsAndPositions,
TestDocumentsWriterStallControl, TestFieldInfos, TestFilterCodecReader,
TestFlex, TestIndexCommit, TestIndexReaderClose, TestIndexWriterConfig,
TestIndexWriterFromReader, TestIndexWriterNRTIsCurrent,
TestIndexWriterReader, TestIndexingSequenceNumbers, TestLazyProxSkipping,
TestMaxTermFrequency, TestMultiDocValues, TestNRTReaderWithThreads,
TestNoMergePolicy, TestOmitPositions,
Re: [JENKINS] Lucene-9.x-Linux (64bit/openj9/jdk-17.0.5) - Build # 10811 - Unstable!
> > We should sometimes run the OpenJ9 to make it explicit that it doesn't > work. > It doesn't - there are occasional hiccups that repeat from time to time and cannot be reproduced with hotspot. We can create an umbrella issue for these errors and just append logs there, perhaps it'll be of some use, I don't know. > Maybe it helps to improve their code. Unfortunately I don't know how to best report issues like this. Neither do I. Maybe chatgpt knows... D. >
Re: [JENKINS] Lucene-9.x-Linux (64bit/openj9/jdk-17.0.5) - Build # 10811 - Unstable!
If I have some time, I will update to newer version. We should sometimes run the OpenJ9 to make it explicit that it doesn't work. Maybe it helps to improve their code. Unfortunately I don't know how to best report issues like this. Uwe Am 01.06.2023 um 12:37 schrieb Dawid Weiss: There are frequent odd errors on J9. I think we should turn it off and not recommend it to people, something is wrong with it. On Thu, Jun 1, 2023 at 12:00 PM Michael McCandless wrote: Does not reproduce for me, but, it looks like it's IBM's J9: Java: 64bit/openj9/jdk-17.0.5 -XX:+UseCompressedOops -Xgcpolicy:balanced Maybe a J9 bug... Mike McCandless http://blog.mikemccandless.com On Thu, Jun 1, 2023 at 5:54 AM Michael McCandless wrote: Hmm, maybe it's a stdout/stderr issue -- the juicy failure details come out in stderr: https://jenkins.thetaphi.de/job/Lucene-9.x-Linux/10811/testReport/junit/org.apache.lucene.index/TestIndexFileDeleter/testExcInDecRef/ Mike McCandless http://blog.mikemccandless.com On Thu, Jun 1, 2023 at 5:53 AM Michael McCandless wrote: It used to be that these build failure emails included the "Reproduce with: ...", thanks to Steve Rowe's incredible regular expression (the largest real-world regexp I have ever seen!!!). But I don't see it here anymore. Does anyone know why/when it broke? I know I can click through to the Jenkins details and see the specifics, but, those are aggressively deleted ... Mike McCandless http://blog.mikemccandless.com On Thu, Jun 1, 2023 at 3:48 AM Policeman Jenkins Server wrote: Build: https://jenkins.thetaphi.de/job/Lucene-9.x-Linux/10811/ Java: 64bit/openj9/jdk-17.0.5 -XX:+UseCompressedOops -Xgcpolicy:balanced 1 tests failed. FAILED: org.apache.lucene.index.TestIndexFileDeleter.testExcInDecRef Error Message: org.apache.lucene.store.AlreadyClosedException: this DocumentsWriter is closed Stack Trace: org.apache.lucene.store.AlreadyClosedException: this DocumentsWriter is closed at __randomizedtesting.SeedInfo.seed([909AEB548A3F7E78:79079C66FCF69985]:0) at app//org.apache.lucene.index.DocumentsWriter.ensureOpen(DocumentsWriter.java:200) at app//org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:429) at app//org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1532) at app//org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1817) at app//org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1470) at app//org.apache.lucene.tests.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:222) at app//org.apache.lucene.index.TestIndexFileDeleter.testExcInDecRef(TestIndexFileDeleter.java:495) at java.base@17.0.5/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base@17.0.5/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base@17.0.5/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base@17.0.5/java.lang.reflect.Method.invoke(Method.java:568) at app//com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758) at app//com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946) at app//com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982) at app//com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996) at app//org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48) at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) at app//org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
Re: [JENKINS] Lucene-9.x-Linux (64bit/openj9/jdk-17.0.5) - Build # 10811 - Unstable!
There are frequent odd errors on J9. I think we should turn it off and not recommend it to people, something is wrong with it. On Thu, Jun 1, 2023 at 12:00 PM Michael McCandless < luc...@mikemccandless.com> wrote: > Does not reproduce for me, but, it looks like it's IBM's J9: Java: > 64bit/openj9/jdk-17.0.5 -XX:+UseCompressedOops -Xgcpolicy:balanced > > Maybe a J9 bug... > > Mike McCandless > > http://blog.mikemccandless.com > > > On Thu, Jun 1, 2023 at 5:54 AM Michael McCandless < > luc...@mikemccandless.com> wrote: > >> Hmm, maybe it's a stdout/stderr issue -- the juicy failure details come >> out in stderr: >> >> >> https://jenkins.thetaphi.de/job/Lucene-9.x-Linux/10811/testReport/junit/org.apache.lucene.index/TestIndexFileDeleter/testExcInDecRef/ >> >> Mike McCandless >> >> http://blog.mikemccandless.com >> >> >> On Thu, Jun 1, 2023 at 5:53 AM Michael McCandless < >> luc...@mikemccandless.com> wrote: >> >>> It used to be that these build failure emails included the "Reproduce >>> with: ...", thanks to Steve Rowe's incredible regular expression (the >>> largest real-world regexp I have ever seen!!!). >>> >>> But I don't see it here anymore. Does anyone know why/when it broke? >>> >>> I know I can click through to the Jenkins details and see the specifics, >>> but, those are aggressively deleted ... >>> >>> Mike McCandless >>> >>> http://blog.mikemccandless.com >>> >>> >>> On Thu, Jun 1, 2023 at 3:48 AM Policeman Jenkins Server < >>> jenk...@thetaphi.de> wrote: >>> Build: https://jenkins.thetaphi.de/job/Lucene-9.x-Linux/10811/ Java: 64bit/openj9/jdk-17.0.5 -XX:+UseCompressedOops -Xgcpolicy:balanced 1 tests failed. FAILED: org.apache.lucene.index.TestIndexFileDeleter.testExcInDecRef Error Message: org.apache.lucene.store.AlreadyClosedException: this DocumentsWriter is closed Stack Trace: org.apache.lucene.store.AlreadyClosedException: this DocumentsWriter is closed at __randomizedtesting.SeedInfo.seed([909AEB548A3F7E78:79079C66FCF69985]:0) at app//org.apache.lucene.index.DocumentsWriter.ensureOpen(DocumentsWriter.java:200) at app//org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:429) at app//org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1532) at app//org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1817) at app//org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1470) at app//org.apache.lucene.tests.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:222) at app//org.apache.lucene.index.TestIndexFileDeleter.testExcInDecRef(TestIndexFileDeleter.java:495) at java.base@17.0.5/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base@17.0.5 /jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base@17.0.5 /jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base@17.0.5 /java.lang.reflect.Method.invoke(Method.java:568) at app//com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758) at app//com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946) at app//com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982) at app//com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996) at app//org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48) at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) at app//org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45) at app//org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) at app//org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) at app//org.junit.rules.RunRules.evaluate(RunRules.java:20) at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) at app//com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843) at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490) at
Re: [JENKINS] Lucene-9.x-Linux (64bit/openj9/jdk-17.0.5) - Build # 10811 - Unstable!
Does not reproduce for me, but, it looks like it's IBM's J9: Java: 64bit/openj9/jdk-17.0.5 -XX:+UseCompressedOops -Xgcpolicy:balanced Maybe a J9 bug... Mike McCandless http://blog.mikemccandless.com On Thu, Jun 1, 2023 at 5:54 AM Michael McCandless wrote: > Hmm, maybe it's a stdout/stderr issue -- the juicy failure details come > out in stderr: > > > https://jenkins.thetaphi.de/job/Lucene-9.x-Linux/10811/testReport/junit/org.apache.lucene.index/TestIndexFileDeleter/testExcInDecRef/ > > Mike McCandless > > http://blog.mikemccandless.com > > > On Thu, Jun 1, 2023 at 5:53 AM Michael McCandless < > luc...@mikemccandless.com> wrote: > >> It used to be that these build failure emails included the "Reproduce >> with: ...", thanks to Steve Rowe's incredible regular expression (the >> largest real-world regexp I have ever seen!!!). >> >> But I don't see it here anymore. Does anyone know why/when it broke? >> >> I know I can click through to the Jenkins details and see the specifics, >> but, those are aggressively deleted ... >> >> Mike McCandless >> >> http://blog.mikemccandless.com >> >> >> On Thu, Jun 1, 2023 at 3:48 AM Policeman Jenkins Server < >> jenk...@thetaphi.de> wrote: >> >>> Build: https://jenkins.thetaphi.de/job/Lucene-9.x-Linux/10811/ >>> Java: 64bit/openj9/jdk-17.0.5 -XX:+UseCompressedOops -Xgcpolicy:balanced >>> >>> 1 tests failed. >>> FAILED: org.apache.lucene.index.TestIndexFileDeleter.testExcInDecRef >>> >>> Error Message: >>> org.apache.lucene.store.AlreadyClosedException: this DocumentsWriter is >>> closed >>> >>> Stack Trace: >>> org.apache.lucene.store.AlreadyClosedException: this DocumentsWriter is >>> closed >>> at >>> __randomizedtesting.SeedInfo.seed([909AEB548A3F7E78:79079C66FCF69985]:0) >>> at >>> app//org.apache.lucene.index.DocumentsWriter.ensureOpen(DocumentsWriter.java:200) >>> at >>> app//org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:429) >>> at >>> app//org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1532) >>> at >>> app//org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1817) >>> at >>> app//org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1470) >>> at >>> app//org.apache.lucene.tests.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:222) >>> at >>> app//org.apache.lucene.index.TestIndexFileDeleter.testExcInDecRef(TestIndexFileDeleter.java:495) >>> at >>> java.base@17.0.5/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native >>> Method) >>> at java.base@17.0.5 >>> /jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) >>> at java.base@17.0.5 >>> /jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>> at java.base@17.0.5 >>> /java.lang.reflect.Method.invoke(Method.java:568) >>> at >>> app//com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758) >>> at >>> app//com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946) >>> at >>> app//com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982) >>> at >>> app//com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996) >>> at >>> app//org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48) >>> at >>> app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) >>> at >>> app//org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45) >>> at >>> app//org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) >>> at >>> app//org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) >>> at app//org.junit.rules.RunRules.evaluate(RunRules.java:20) >>> at >>> app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) >>> at >>> app//com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) >>> at >>> app//com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843) >>> at >>> app//com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490) >>> at >>> app//com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955) >>> at >>> app//com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840) >>> at >>> app//com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891) >>> at >>>
Re: [JENKINS] Lucene-9.x-Linux (64bit/openj9/jdk-17.0.5) - Build # 10811 - Unstable!
Hmm, maybe it's a stdout/stderr issue -- the juicy failure details come out in stderr: https://jenkins.thetaphi.de/job/Lucene-9.x-Linux/10811/testReport/junit/org.apache.lucene.index/TestIndexFileDeleter/testExcInDecRef/ Mike McCandless http://blog.mikemccandless.com On Thu, Jun 1, 2023 at 5:53 AM Michael McCandless wrote: > It used to be that these build failure emails included the "Reproduce > with: ...", thanks to Steve Rowe's incredible regular expression (the > largest real-world regexp I have ever seen!!!). > > But I don't see it here anymore. Does anyone know why/when it broke? > > I know I can click through to the Jenkins details and see the specifics, > but, those are aggressively deleted ... > > Mike McCandless > > http://blog.mikemccandless.com > > > On Thu, Jun 1, 2023 at 3:48 AM Policeman Jenkins Server < > jenk...@thetaphi.de> wrote: > >> Build: https://jenkins.thetaphi.de/job/Lucene-9.x-Linux/10811/ >> Java: 64bit/openj9/jdk-17.0.5 -XX:+UseCompressedOops -Xgcpolicy:balanced >> >> 1 tests failed. >> FAILED: org.apache.lucene.index.TestIndexFileDeleter.testExcInDecRef >> >> Error Message: >> org.apache.lucene.store.AlreadyClosedException: this DocumentsWriter is >> closed >> >> Stack Trace: >> org.apache.lucene.store.AlreadyClosedException: this DocumentsWriter is >> closed >> at >> __randomizedtesting.SeedInfo.seed([909AEB548A3F7E78:79079C66FCF69985]:0) >> at >> app//org.apache.lucene.index.DocumentsWriter.ensureOpen(DocumentsWriter.java:200) >> at >> app//org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:429) >> at >> app//org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1532) >> at >> app//org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1817) >> at >> app//org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1470) >> at >> app//org.apache.lucene.tests.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:222) >> at >> app//org.apache.lucene.index.TestIndexFileDeleter.testExcInDecRef(TestIndexFileDeleter.java:495) >> at >> java.base@17.0.5/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native >> Method) >> at java.base@17.0.5 >> /jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) >> at java.base@17.0.5 >> /jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.base@17.0.5 >> /java.lang.reflect.Method.invoke(Method.java:568) >> at >> app//com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758) >> at >> app//com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946) >> at >> app//com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982) >> at >> app//com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996) >> at >> app//org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48) >> at >> app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) >> at >> app//org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45) >> at >> app//org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) >> at >> app//org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) >> at app//org.junit.rules.RunRules.evaluate(RunRules.java:20) >> at >> app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) >> at >> app//com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) >> at >> app//com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843) >> at >> app//com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490) >> at >> app//com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955) >> at >> app//com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840) >> at >> app//com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891) >> at >> app//com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902) >> at >> app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) >> at >> app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) >> at >> app//org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38) >> at >>
Re: [JENKINS] Lucene-9.x-Linux (64bit/openj9/jdk-17.0.5) - Build # 10811 - Unstable!
It used to be that these build failure emails included the "Reproduce with: ...", thanks to Steve Rowe's incredible regular expression (the largest real-world regexp I have ever seen!!!). But I don't see it here anymore. Does anyone know why/when it broke? I know I can click through to the Jenkins details and see the specifics, but, those are aggressively deleted ... Mike McCandless http://blog.mikemccandless.com On Thu, Jun 1, 2023 at 3:48 AM Policeman Jenkins Server wrote: > Build: https://jenkins.thetaphi.de/job/Lucene-9.x-Linux/10811/ > Java: 64bit/openj9/jdk-17.0.5 -XX:+UseCompressedOops -Xgcpolicy:balanced > > 1 tests failed. > FAILED: org.apache.lucene.index.TestIndexFileDeleter.testExcInDecRef > > Error Message: > org.apache.lucene.store.AlreadyClosedException: this DocumentsWriter is > closed > > Stack Trace: > org.apache.lucene.store.AlreadyClosedException: this DocumentsWriter is > closed > at > __randomizedtesting.SeedInfo.seed([909AEB548A3F7E78:79079C66FCF69985]:0) > at > app//org.apache.lucene.index.DocumentsWriter.ensureOpen(DocumentsWriter.java:200) > at > app//org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:429) > at > app//org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1532) > at > app//org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1817) > at > app//org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1470) > at > app//org.apache.lucene.tests.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:222) > at > app//org.apache.lucene.index.TestIndexFileDeleter.testExcInDecRef(TestIndexFileDeleter.java:495) > at > java.base@17.0.5/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native > Method) > at java.base@17.0.5 > /jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) > at java.base@17.0.5 > /jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.base@17.0.5 > /java.lang.reflect.Method.invoke(Method.java:568) > at > app//com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758) > at > app//com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946) > at > app//com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982) > at > app//com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996) > at > app//org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48) > at > app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) > at > app//org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45) > at > app//org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) > at > app//org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) > at app//org.junit.rules.RunRules.evaluate(RunRules.java:20) > at > app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > app//com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) > at > app//com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843) > at > app//com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490) > at > app//com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955) > at > app//com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840) > at > app//com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891) > at > app//com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902) > at > app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) > at > app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > app//org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38) > at > app//com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) > at > app//com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) > at > app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at >
Re: [JENKINS] Lucene » Lucene-NightlyTests-main - Build # 1019 - Unstable!
This reproduces constantly for me, I even rolls back to some (random) old commit: 9acc6539959 (in last September) and it still reproduces. Found another similar failure: https://ci-builds.apache.org/job/Lucene/job/Lucene-Check-main/8835/ The error seems to be a more common one but somehow (so far) it only happens inside this TestGroupFacetCollector test... On Wed, May 31, 2023 at 10:18 PM Apache Jenkins Server < jenk...@builds.apache.org> wrote: > Build: > https://ci-builds.apache.org/job/Lucene/job/Lucene-NightlyTests-main/1019/ > > 1 tests failed. > FAILED: > org.apache.lucene.search.grouping.TestGroupFacetCollector.testRandom > > Error Message: > java.lang.IndexOutOfBoundsException: Range [0, 0 + 692) out of bounds for > length 80 > > Stack Trace: > java.lang.IndexOutOfBoundsException: Range [0, 0 + 692) out of bounds for > length 80 > at > __randomizedtesting.SeedInfo.seed([46C657C9C98115E0:348A72C678E1A393]:0) > at > java.base/jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:64) > at > java.base/jdk.internal.util.Preconditions.outOfBoundsCheckFromIndexSize(Preconditions.java:82) > at > java.base/jdk.internal.util.Preconditions.checkFromIndexSize(Preconditions.java:361) > at java.base/java.util.Objects.checkFromIndexSize(Objects.java:413) > at java.base/java.nio.HeapByteBuffer.get(HeapByteBuffer.java:180) > at org.apache.lucene.core@10.0.0-SNAPSHOT > /org.apache.lucene.store.ByteBuffersDataInput.readBytes(ByteBuffersDataInput.java:159) > at org.apache.lucene.core@10.0.0-SNAPSHOT > /org.apache.lucene.store.ByteBuffersIndexInput.readBytes(ByteBuffersIndexInput.java:85) > at org.apache.lucene.test_framework@10.0.0-SNAPSHOT > /org.apache.lucene.tests.store.MockIndexInputWrapper.readBytes(MockIndexInputWrapper.java:148) > at org.apache.lucene.core@10.0.0-SNAPSHOT > /org.apache.lucene.codecs.lucene90.Lucene90DocValuesProducer$TermsDict.decompressBlock(Lucene90DocValuesProducer.java:1234) > at org.apache.lucene.core@10.0.0-SNAPSHOT > /org.apache.lucene.codecs.lucene90.Lucene90DocValuesProducer$TermsDict.next(Lucene90DocValuesProducer.java:1092) > at > org.apache.lucene.search.grouping.TermGroupFacetCollector$MV$SegmentResult.nextTerm(TermGroupFacetCollector.java:437) > at > org.apache.lucene.search.grouping.GroupFacetCollector.mergeSegmentResults(GroupFacetCollector.java:97) > at > org.apache.lucene.search.grouping.TestGroupFacetCollector.testRandom(TestGroupFacetCollector.java:429) > at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native > Method) > at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) > at > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.base/java.lang.reflect.Method.invoke(Method.java:568) > at randomizedtesting.runner@2.8.1 > /com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758) > at randomizedtesting.runner@2.8.1 > /com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946) > at randomizedtesting.runner@2.8.1 > /com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982) > at randomizedtesting.runner@2.8.1 > /com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996) > at org.apache.lucene.test_framework@10.0.0-SNAPSHOT > /org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48) > at org.apache.lucene.test_framework@10.0.0-SNAPSHOT > /org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) > at org.apache.lucene.test_framework@10.0.0-SNAPSHOT > /org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45) > at org.apache.lucene.test_framework@10.0.0-SNAPSHOT > /org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) > at org.apache.lucene.test_framework@10.0.0-SNAPSHOT > /org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) > at junit@4.13.1 > /org.junit.rules.RunRules.evaluate(RunRules.java:20) > at randomizedtesting.runner@2.8.1 > /com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at randomizedtesting.runner@2.8.1 > /com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) > at randomizedtesting.runner@2.8.1 > /com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843) > at randomizedtesting.runner@2.8.1 > /com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490) > at
