[ https://issues.apache.org/jira/browse/SOLR-6920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14267406#comment-14267406 ]
Varun Thacker commented on SOLR-6920: ------------------------------------- When using a try-finally block code in SOLR-6440 the threads don't hang anymore. The test still fails because of errors like this - {code} 391305 T11 C196 P59014 oasu.SolrIndexWriter.close ERROR Error closing IndexWriter java.lang.AssertionError: file _2_2.liv does not exist; files=[segments_2, _1.cfs, _3.si, _2.cfe, _1.si, _1.cfe, _3.cfe, _0.cfs, _0.cfe, _2.si, _0.si, _3.cfs, _2.cfs] at org.apache.lucene.index.IndexWriter.filesExist(IndexWriter.java:4232) at org.apache.lucene.index.IndexWriter.startCommit(IndexWriter.java:4303) at org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2785) at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2888) at org.apache.lucene.index.IndexWriter.shutdown(IndexWriter.java:965) at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1008) at org.apache.solr.update.SolrIndexWriter.close(SolrIndexWriter.java:129) at org.apache.solr.update.DirectUpdateHandler2.closeWriter(DirectUpdateHandler2.java:804) at org.apache.solr.update.DefaultSolrCoreState.closeIndexWriter(DefaultSolrCoreState.java:68) at org.apache.solr.update.DefaultSolrCoreState.close(DefaultSolrCoreState.java:359) at org.apache.solr.update.SolrCoreState.decrefSolrCoreState(SolrCoreState.java:72) at org.apache.solr.core.SolrCore.close(SolrCore.java:1110) at org.apache.solr.core.SolrCores.close(SolrCores.java:117) at org.apache.solr.core.CoreContainer.shutdown(CoreContainer.java:366) at org.apache.solr.servlet.SolrDispatchFilter.destroy(SolrDispatchFilter.java:194) at org.apache.solr.cloud.ChaosMonkey.stopJettySolrRunner(ChaosMonkey.java:197) at org.apache.solr.cloud.ChaosMonkey.stop(ChaosMonkey.java:550) at org.apache.solr.cloud.AbstractFullDistribZkTestBase.destroyServers(AbstractFullDistribZkTestBase.java:1568) at org.apache.solr.BaseDistributedSearchTestCase.tearDown(BaseDistributedSearchTestCase.java:283) at org.apache.solr.cloud.AbstractDistribZkTestBase.tearDown(AbstractDistribZkTestBase.java:231) at org.apache.solr.cloud.AbstractFullDistribZkTestBase.tearDown(AbstractFullDistribZkTestBase.java:1546) at org.apache.solr.cloud.BasicDistributedZkTest.tearDown(BasicDistributedZkTest.java:1166) at org.apache.solr.cloud.ShardSplitTest.tearDown(ShardSplitTest.java:91) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:885) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at java.lang.Thread.run(Thread.java:745) {code} No .liv files get included with the logic change in ReplicationHandler.getFileList with the patch. Looking into that now. > During replication use checksums to verify if files are the same > ---------------------------------------------------------------- > > Key: SOLR-6920 > URL: https://issues.apache.org/jira/browse/SOLR-6920 > Project: Solr > Issue Type: Bug > Components: replication (java) > Reporter: Varun Thacker > Attachments: SOLR-6920.patch > > > Currently we check if an index file on the master and slave is the same by > checking if it's name and file length match. > With LUCENE-2446 we now have a checksums for each index file in the segment. > We should leverage this to verify if two files are the same. > Places like SnapPuller.isIndexStale and SnapPuller.downloadIndexFiles should > check against the checksum also. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org