I tried reducing 500 full flushes to 250 and 5000 iters to 2500, and
then the test would fairly reliably finish in 1 minute or so, although
I did once see
* Exception: java.lang.IllegalArgumentException thrown
from the UncaughtExceptionHandler in thread "Lucene Merge Thread #102"
I added an UncaughtExceptionHandler with printStackTrace to threads
created by ConcurrentMergeScheduler but was never able to reproduce
that one
with the larger limits, this test generally seems to fail with OOM, or
else takes so long that I kill it, so I think we should reduce the
limits; then perhaps we might see some more interesting failure modes
On Sat, Nov 27, 2021 at 1:30 PM Michael Sokolov <[email protected]> wrote:
>
> Hmm, that test is taking a very long time on my laptop (running
> Ubuntu) with JDK11, so doesn't seem to be a Windows or JDK17 issue
>
> ./gradlew :lucene:core:test --tests
> "org.apache.lucene.index.TestIndexWriterMergePolicy.testStressUpdateSameDocumentWithMergeOnGetReader"
> -Dtests.nightly=true -Dtests.seed=B67ECC7381FE35B
>
> after 12m44s I killed it
>
>
> On Sat, Nov 27, 2021 at 11:59 AM Michael Sokolov <[email protected]> wrote:
> >
> > I see that we periodically get various test failures from this test.
> > Has anybody been tracking this more carefully than me, and if so, do
> > you remember if it's always on Windows where these too-many-open-files
> > (and sometimes out of disk space) errors show up? I wonder if we
> > should reduce the cardinality of these test settings:
> >
> > AtomicInteger iters = new AtomicInteger(100 +
> > random().nextInt(TEST_NIGHTLY ? 5000 : 1000));
> > AtomicInteger numFullFlushes =
> > new AtomicInteger(10 + random().nextInt(TEST_NIGHTLY ? 500 :
> > 100));
> >
> > On Sat, Nov 27, 2021 at 5:54 AM Policeman Jenkins Server
> > <[email protected]> wrote:
> > >
> > > Build: https://jenkins.thetaphi.de/job/Lucene-jdk17panama-Windows/701/
> > > Java: 64bit/jdk-17 -XX:+UseCompressedOops -XX:+UseG1GC
> > >
> > > 2 tests failed.
> > > FAILED: org.apache.lucene.index.TestIndexWriterMergePolicy.classMethod
> > >
> > > Error Message:
> > > java.lang.AssertionError: The test or suite printed 10871 bytes to stdout
> > > and stderr, even though the limit was set to 8192 bytes. Increase the
> > > limit with @Limit, ignore it completely with @SuppressSysoutChecks or run
> > > with -Dtests.verbose=true
> > >
> > > Stack Trace:
> > > java.lang.AssertionError: The test or suite printed 10871 bytes to stdout
> > > and stderr, even though the limit was set to 8192 bytes. Increase the
> > > limit with @Limit, ignore it completely with @SuppressSysoutChecks or run
> > > with -Dtests.verbose=true
> > > at __randomizedtesting.SeedInfo.seed([B67ECC7381FE35B]:0)
> > > at
> > > org.apache.lucene.util.TestRuleLimitSysouts.afterIfSuccessful(TestRuleLimitSysouts.java:283)
> > > at
> > > com.carrotsearch.randomizedtesting.rules.TestRuleAdapter$1.afterIfSuccessful(TestRuleAdapter.java:36)
> > > at
> > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:37)
> > > at
> > > org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> > > at
> > > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> > > at
> > > org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> > > at
> > > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> > > at
> > > org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
> > > at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> > > at
> > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > > at
> > > com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
> > > at
> > > com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:826)
> > > at java.base/java.lang.Thread.run(Thread.java:833)
> > >
> > >
> > > FAILED:
> > > org.apache.lucene.index.TestIndexWriterMergePolicy.testStressUpdateSameDocumentWithMergeOnGetReader
> > >
> > > Error Message:
> > > com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an
> > > uncaught exception in thread: Thread[id=869, name=Thread-739,
> > > state=RUNNABLE, group=TGRP-TestIndexWriterMergePolicy]
> > >
> > > Stack Trace:
> > > com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an
> > > uncaught exception in thread: Thread[id=869, name=Thread-739,
> > > state=RUNNABLE, group=TGRP-TestIndexWriterMergePolicy]
> > > Caused by: java.lang.AssertionError: java.nio.file.FileSystemException:
> > > C:\Users\jenkins\workspace\Lucene-jdk17panama-Windows\lucene\core\build\tmp\tests-tmp\lucene.index.TestIndexWriterMergePolicy_B67ECC7381FE35B-001\index-MMapDirectory-015\_na_Lucene90FieldsIndex-doc_ids_1ak.tmp:
> > > Too many open files
> > > at __randomizedtesting.SeedInfo.seed([B67ECC7381FE35B]:0)
> > > at
> > > org.apache.lucene.index.TestIndexWriterMergePolicy.lambda$stressUpdateSameDocumentWithMergeOnX$4(TestIndexWriterMergePolicy.java:752)
> > > at java.base/java.lang.Thread.run(Thread.java:833)
> > > Caused by: java.nio.file.FileSystemException:
> > > C:\Users\jenkins\workspace\Lucene-jdk17panama-Windows\lucene\core\build\tmp\tests-tmp\lucene.index.TestIndexWriterMergePolicy_B67ECC7381FE35B-001\index-MMapDirectory-015\_na_Lucene90FieldsIndex-doc_ids_1ak.tmp:
> > > Too many open files
> > > at
> > > org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:45)
> > > at
> > > org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82)
> > > at
> > > org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:163)
> > > at
> > > org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:194)
> > > at java.base/java.nio.file.Files.newOutputStream(Files.java:228)
> > > at
> > > org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:392)
> > > at
> > > org.apache.lucene.store.FSDirectory.createTempOutput(FSDirectory.java:234)
> > > at
> > > org.apache.lucene.store.MockDirectoryWrapper.createTempOutput(MockDirectoryWrapper.java:745)
> > > at
> > > org.apache.lucene.store.FilterDirectory.createTempOutput(FilterDirectory.java:81)
> > > at
> > > org.apache.lucene.store.TrackingDirectoryWrapper.createTempOutput(TrackingDirectoryWrapper.java:49)
> > > at
> > > org.apache.lucene.codecs.lucene90.compressing.FieldsIndexWriter.<init>(FieldsIndexWriter.java:83)
> > > at
> > > org.apache.lucene.codecs.lucene90.compressing.Lucene90CompressingStoredFieldsWriter.<init>(Lucene90CompressingStoredFieldsWriter.java:148)
> > > at
> > > org.apache.lucene.codecs.lucene90.compressing.Lucene90CompressingStoredFieldsFormat.fieldsWriter(Lucene90CompressingStoredFieldsFormat.java:140)
> > > at
> > > org.apache.lucene.codecs.lucene90.Lucene90StoredFieldsFormat.fieldsWriter(Lucene90StoredFieldsFormat.java:154)
> > > at
> > > org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:48)
> > > at
> > > org.apache.lucene.index.StoredFieldsConsumer.initStoredFieldsWriter(StoredFieldsConsumer.java:49)
> > > at
> > > org.apache.lucene.index.StoredFieldsConsumer.startDocument(StoredFieldsConsumer.java:56)
> > > at
> > > org.apache.lucene.index.IndexingChain.startStoredFields(IndexingChain.java:556)
> > > at
> > > org.apache.lucene.index.IndexingChain.processDocument(IndexingChain.java:587)
> > > at
> > > org.apache.lucene.index.DocumentsWriterPerThread.updateDocuments(DocumentsWriterPerThread.java:239)
> > > at
> > > org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:432)
> > > at
> > > org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1530)
> > > at
> > > org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1815)
> > > at
> > > org.apache.lucene.index.RandomIndexWriter.updateDocument(RandomIndexWriter.java:305)
> > > at
> > > org.apache.lucene.index.TestIndexWriterMergePolicy.lambda$stressUpdateSameDocumentWithMergeOnX$4(TestIndexWriterMergePolicy.java:746)
> > > ... 1 more
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: [email protected]
> > > For additional commands, e-mail: [email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
