This is possible when e.g. you close but don't wait for merges, and its actually able to delete unreferenced files. then the next IW session will use the same file names, which MDW catches.
we already fixed the possible bug case where, if the files cant be deleted for some reason, the next IW will "skip past those generations" so it wont cause issues. But I don't know how to fix the test.. We could allow this case when we successfully deleted a file (only successful, to ensure above bug case "works"). But really this should only be allowed across IW sessions, and most tests don't do that, so we lose a lot of safety checks. On Sun, Sep 7, 2014 at 3:52 AM, Apache Jenkins Server <[email protected]> wrote: > Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-4.x/613/ > > 1 tests failed. > REGRESSION: org.apache.lucene.index.TestIndexWriterMerging.testNoWaitClose > > Error Message: > file "_au.fdx" was already written to > > Stack Trace: > java.io.IOException: file "_au.fdx" was already written to > at > __randomizedtesting.SeedInfo.seed([D9426C8ADCEA197A:FEB3CD5B0FFCC98E]:0) > at > org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:528) > at > org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:44) > at > org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:113) > at > org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:120) > at > org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:47) > at > org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:84) > at > org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:254) > at > org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:298) > at > org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:242) > at > org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:454) > at > org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1542) > at > org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1257) > at > org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1238) > at > org.apache.lucene.index.TestIndexWriterMerging.testNoWaitClose(TestIndexWriterMerging.java:388) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877) > at > org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) > at > org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51) > at > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) > at > com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) > at > org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) > at > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) > at > org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) > at > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) > at > com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798) > at > com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458) > at > com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783) > at > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) > at > org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) > at > com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) > at > com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) > at > com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) > at > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43) > at > org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) > at > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) > at > org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) > at > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) > at java.lang.Thread.run(Thread.java:745) > > > > > Build Log: > [...truncated 991 lines...] > [junit4] Suite: org.apache.lucene.index.TestIndexWriterMerging > [junit4] 2> NOTE: download the large Jenkins line-docs file by running > 'ant get-jenkins-line-docs' in the lucene directory. > [junit4] 2> NOTE: reproduce with: ant test > -Dtestcase=TestIndexWriterMerging -Dtests.method=testNoWaitClose > -Dtests.seed=D9426C8ADCEA197A -Dtests.multiplier=2 -Dtests.nightly=true > -Dtests.slow=true > -Dtests.linedocsfile=/home/jenkins/lucene-data/enwiki.random.lines.txt > -Dtests.locale=ar_IQ -Dtests.timezone=Europe/Dublin > -Dtests.file.encoding=ISO-8859-1 > [junit4] ERROR 2.54s J3 | TestIndexWriterMerging.testNoWaitClose <<< > [junit4] > Throwable #1: java.io.IOException: file "_au.fdx" was > already written to > [junit4] > at > __randomizedtesting.SeedInfo.seed([D9426C8ADCEA197A:FEB3CD5B0FFCC98E]:0) > [junit4] > at > org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:528) > [junit4] > at > org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:44) > [junit4] > at > org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:113) > [junit4] > at > org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:120) > [junit4] > at > org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:47) > [junit4] > at > org.apache.lucene.index.DefaultIndexingChain.initStoredFieldsWriter(DefaultIndexingChain.java:84) > [junit4] > at > org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:254) > [junit4] > at > org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:298) > [junit4] > at > org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:242) > [junit4] > at > org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:454) > [junit4] > at > org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1542) > [junit4] > at > org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1257) > [junit4] > at > org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1238) > [junit4] > at > org.apache.lucene.index.TestIndexWriterMerging.testNoWaitClose(TestIndexWriterMerging.java:388) > [junit4] > at java.lang.Thread.run(Thread.java:745) > [junit4] 2> NOTE: leaving temporary files on disk at: > /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-4.x/lucene/build/core/test/J3/./temp/lucene.index.TestIndexWriterMerging-D9426C8ADCEA197A-001 > [junit4] 2> NOTE: test params are: codec=Asserting(Lucene410), > sim=DefaultSimilarity, locale=ar_IQ, timezone=Europe/Dublin > [junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation > 1.7.0_65 (64-bit)/cpus=16,threads=1,free=287705952,total=468713472 > [junit4] 2> NOTE: All tests run in this JVM: [TestCharsRefBuilder, > TestLockFactory, TestExceedMaxTermLength, TestDoubleBarrelLRUCache, > TestMergeSchedulerExternal, TestOpenBitSet, TestTermdocPerf, > TestPrefixFilter, TestTermVectorsFormat, NestedSetupChain, > NestedTeardownChain, TestTimeLimitingCollector, TestBackwardsCompatibility3x, > TestTopFieldCollector, TestFieldValueFilter, TestForceMergeForever, > TestReaderClosed, TestSimpleExplanations, TestIndexWriterNRTIsCurrent, > TestQueryBuilder, TestMultiMMap, TestLock, TestSortDocValues, > TestMultiLevelSkipList, TestSpanExplanationsOfNonMatches, TestFieldType, > TestDisjunctionMaxQuery, TestAtomicUpdate, TestVirtualMethod, > TestBooleanMinShouldMatch, TestLucene41StoredFieldsFormat, TestFSTs, > TestUnicodeUtil, TestStoredFieldsFormat, TestRegexpQuery, > TestPayloadNearQuery, TestSearch, TestFilterIterator, TestFieldReuse, > TestDeterminizeLexicon, TestLogMergePolicy, TestSortRescorer, > TestMutableValues, TestRamUsageEstimatorOnWildAnimals, TestStressNRT, > TestPayloadSpans, TestWAH8DocIdSet, TestPositionIncrement, > TestCachingCollector, TestLucene410DocValuesFormat, TestNot, TestAddIndexes, > TestPrefixQuery, TestIndexWriterReader, TestIndexWriterMerging] > [junit4] Completed on J3 in 8.98s, 6 tests, 1 error <<< FAILURES! > > [...truncated 985 lines...] > BUILD FAILED > /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-4.x/build.xml:498: > The following error occurred while executing this line: > /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-4.x/build.xml:471: > The following error occurred while executing this line: > /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-4.x/build.xml:60: > The following error occurred while executing this line: > /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-4.x/extra-targets.xml:39: > The following error occurred while executing this line: > /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-4.x/lucene/build.xml:49: > The following error occurred while executing this line: > /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-4.x/lucene/common-build.xml:1362: > The following error occurred while executing this line: > /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-4.x/lucene/common-build.xml:965: > There were test failures: 418 suites, 3731 tests, 1 error, 120 ignored (109 > assumptions) > > Total time: 66 minutes 5 seconds > Build step 'Invoke Ant' marked build as failure > Archiving artifacts > Sending artifact delta relative to Lucene-Solr-NightlyTests-4.x #610 > Archived 5 artifacts > Archive block size is 32768 > Received 0 blocks and 4845127 bytes > Compression is 0.0% > Took 2.9 sec > Recording test results > Email was triggered for: Failure > Sending email for trigger: Failure > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
