It's this issue, Mike: https://issues.apache.org/jira/browse/LUCENE-10088
On Sat, Nov 27, 2021 at 8:11 PM Michael Sokolov <[email protected]> wrote: > I tried reducing 500 full flushes to 250 and 5000 iters to 2500, and > then the test would fairly reliably finish in 1 minute or so, although > I did once see > > * Exception: java.lang.IllegalArgumentException thrown > from the UncaughtExceptionHandler in thread "Lucene Merge Thread #102" > > I added an UncaughtExceptionHandler with printStackTrace to threads > created by ConcurrentMergeScheduler but was never able to reproduce > that one > > with the larger limits, this test generally seems to fail with OOM, or > else takes so long that I kill it, so I think we should reduce the > limits; then perhaps we might see some more interesting failure modes > > On Sat, Nov 27, 2021 at 1:30 PM Michael Sokolov <[email protected]> > wrote: > > > > Hmm, that test is taking a very long time on my laptop (running > > Ubuntu) with JDK11, so doesn't seem to be a Windows or JDK17 issue > > > > ./gradlew :lucene:core:test --tests > > > "org.apache.lucene.index.TestIndexWriterMergePolicy.testStressUpdateSameDocumentWithMergeOnGetReader" > > -Dtests.nightly=true -Dtests.seed=B67ECC7381FE35B > > > > after 12m44s I killed it > > > > > > On Sat, Nov 27, 2021 at 11:59 AM Michael Sokolov <[email protected]> > wrote: > > > > > > I see that we periodically get various test failures from this test. > > > Has anybody been tracking this more carefully than me, and if so, do > > > you remember if it's always on Windows where these too-many-open-files > > > (and sometimes out of disk space) errors show up? I wonder if we > > > should reduce the cardinality of these test settings: > > > > > > AtomicInteger iters = new AtomicInteger(100 + > > > random().nextInt(TEST_NIGHTLY ? 5000 : 1000)); > > > AtomicInteger numFullFlushes = > > > new AtomicInteger(10 + random().nextInt(TEST_NIGHTLY ? 500 > : 100)); > > > > > > On Sat, Nov 27, 2021 at 5:54 AM Policeman Jenkins Server > > > <[email protected]> wrote: > > > > > > > > Build: > https://jenkins.thetaphi.de/job/Lucene-jdk17panama-Windows/701/ > > > > Java: 64bit/jdk-17 -XX:+UseCompressedOops -XX:+UseG1GC > > > > > > > > 2 tests failed. > > > > FAILED: > org.apache.lucene.index.TestIndexWriterMergePolicy.classMethod > > > > > > > > Error Message: > > > > java.lang.AssertionError: The test or suite printed 10871 bytes to > stdout and stderr, even though the limit was set to 8192 bytes. Increase > the limit with @Limit, ignore it completely with @SuppressSysoutChecks or > run with -Dtests.verbose=true > > > > > > > > Stack Trace: > > > > java.lang.AssertionError: The test or suite printed 10871 bytes to > stdout and stderr, even though the limit was set to 8192 bytes. Increase > the limit with @Limit, ignore it completely with @SuppressSysoutChecks or > run with -Dtests.verbose=true > > > > at __randomizedtesting.SeedInfo.seed([B67ECC7381FE35B]:0) > > > > at > org.apache.lucene.util.TestRuleLimitSysouts.afterIfSuccessful(TestRuleLimitSysouts.java:283) > > > > at > com.carrotsearch.randomizedtesting.rules.TestRuleAdapter$1.afterIfSuccessful(TestRuleAdapter.java:36) > > > > at > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:37) > > > > at > org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) > > > > at > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) > > > > at > org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) > > > > at > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) > > > > at > org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47) > > > > at org.junit.rules.RunRules.evaluate(RunRules.java:20) > > > > at > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > > > > at > com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370) > > > > at > com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:826) > > > > at java.base/java.lang.Thread.run(Thread.java:833) > > > > > > > > > > > > FAILED: > org.apache.lucene.index.TestIndexWriterMergePolicy.testStressUpdateSameDocumentWithMergeOnGetReader > > > > > > > > Error Message: > > > > com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured > an uncaught exception in thread: Thread[id=869, name=Thread-739, > state=RUNNABLE, group=TGRP-TestIndexWriterMergePolicy] > > > > > > > > Stack Trace: > > > > com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured > an uncaught exception in thread: Thread[id=869, name=Thread-739, > state=RUNNABLE, group=TGRP-TestIndexWriterMergePolicy] > > > > Caused by: java.lang.AssertionError: > java.nio.file.FileSystemException: > C:\Users\jenkins\workspace\Lucene-jdk17panama-Windows\lucene\core\build\tmp\tests-tmp\lucene.index.TestIndexWriterMergePolicy_B67ECC7381FE35B-001\index-MMapDirectory-015\_na_Lucene90FieldsIndex-doc_ids_1ak.tmp: > Too many open files > > > > at __randomizedtesting.SeedInfo.seed([B67ECC7381FE35B]:0) > > > > at > org.apache.lucene.index.TestIndexWriterMergePolicy.lambda$stressUpdateSameDocumentWithMergeOnX$4(TestIndexWriterMergePolicy.java:752) > > > > at java.base/java.lang.Thread.run(Thread.java:833) > > > > Caused by: java.nio.file.FileSystemException: > C:\Users\jenkins\workspace\Lucene-jdk17panama-Windows\lucene\core\build\tmp\tests-tmp\lucene.index.TestIndexWriterMergePolicy_B67ECC7381FE35B-001\index-MMapDirectory-015\_na_Lucene90FieldsIndex-doc_ids_1ak.tmp: > Too many open files > > > > at > org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:45) > > > > at > org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:82) > > > > at > org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:163) > > > > at > org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:194) > > > > at > java.base/java.nio.file.Files.newOutputStream(Files.java:228) > > > > at > org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:392) > > > > at > org.apache.lucene.store.FSDirectory.createTempOutput(FSDirectory.java:234) > > > > at > org.apache.lucene.store.MockDirectoryWrapper.createTempOutput(MockDirectoryWrapper.java:745) > > > > at > org.apache.lucene.store.FilterDirectory.createTempOutput(FilterDirectory.java:81) > > > > at > org.apache.lucene.store.TrackingDirectoryWrapper.createTempOutput(TrackingDirectoryWrapper.java:49) > > > > at > org.apache.lucene.codecs.lucene90.compressing.FieldsIndexWriter.<init>(FieldsIndexWriter.java:83) > > > > at > org.apache.lucene.codecs.lucene90.compressing.Lucene90CompressingStoredFieldsWriter.<init>(Lucene90CompressingStoredFieldsWriter.java:148) > > > > at > org.apache.lucene.codecs.lucene90.compressing.Lucene90CompressingStoredFieldsFormat.fieldsWriter(Lucene90CompressingStoredFieldsFormat.java:140) > > > > at > org.apache.lucene.codecs.lucene90.Lucene90StoredFieldsFormat.fieldsWriter(Lucene90StoredFieldsFormat.java:154) > > > > at > org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:48) > > > > at > org.apache.lucene.index.StoredFieldsConsumer.initStoredFieldsWriter(StoredFieldsConsumer.java:49) > > > > at > org.apache.lucene.index.StoredFieldsConsumer.startDocument(StoredFieldsConsumer.java:56) > > > > at > org.apache.lucene.index.IndexingChain.startStoredFields(IndexingChain.java:556) > > > > at > org.apache.lucene.index.IndexingChain.processDocument(IndexingChain.java:587) > > > > at > org.apache.lucene.index.DocumentsWriterPerThread.updateDocuments(DocumentsWriterPerThread.java:239) > > > > at > org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:432) > > > > at > org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1530) > > > > at > org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1815) > > > > at > org.apache.lucene.index.RandomIndexWriter.updateDocument(RandomIndexWriter.java:305) > > > > at > org.apache.lucene.index.TestIndexWriterMergePolicy.lambda$stressUpdateSameDocumentWithMergeOnX$4(TestIndexWriterMergePolicy.java:746) > > > > ... 1 more > > > > > > > > --------------------------------------------------------------------- > > > > To unsubscribe, e-mail: [email protected] > > > > For additional commands, e-mail: [email protected] > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
