Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-trunk/677/
5 tests failed. REGRESSION: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.testDistribSearch Error Message: Test abandoned because suite timeout was reached. Stack Trace: java.lang.Exception: Test abandoned because suite timeout was reached. at __randomizedtesting.SeedInfo.seed([FEA2EACA01B8699E]:0) FAILED: junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeySafeLeaderTest Error Message: Suite timeout exceeded (>= 7200000 msec). Stack Trace: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec). at __randomizedtesting.SeedInfo.seed([FEA2EACA01B8699E]:0) REGRESSION: org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testDistribSearch Error Message: There are still nodes recoverying - waited for 330 seconds Stack Trace: java.lang.AssertionError: There are still nodes recoverying - waited for 330 seconds at __randomizedtesting.SeedInfo.seed([FEA2EACA01B8699E:7F4464D276E709A2]:0) at org.junit.Assert.fail(Assert.java:93) at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:178) at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:137) at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:132) at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:834) at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.createCollection(CollectionsAPIDistributedZkTest.java:1332) at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.addReplicaTest(CollectionsAPIDistributedZkTest.java:1259) at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(CollectionsAPIDistributedZkTest.java:210) at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at java.lang.Thread.run(Thread.java:745) REGRESSION: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.testDistribSearch Error Message: Error executing query Stack Trace: org.apache.solr.client.solrj.SolrServerException: Error executing query at __randomizedtesting.SeedInfo.seed([FEA2EACA01B8699E:7F4464D276E709A2]:0) at org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:100) at org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:301) at org.apache.solr.cloud.CloudInspectUtil.compareResults(CloudInspectUtil.java:223) at org.apache.solr.cloud.CloudInspectUtil.compareResults(CloudInspectUtil.java:165) at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.testIndexingBatchPerRequestWithHttpSolrServer(FullSolrCloudDistribCmdsTest.java:414) at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.doTest(FullSolrCloudDistribCmdsTest.java:144) at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.http.ParseException: Invalid content type: at org.apache.http.entity.ContentType.parse(ContentType.java:233) at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:513) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:215) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:211) at org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:91) ... 46 more FAILED: org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload Error Message: expected:<[{indexVersion=1415292543590,generation=2,filelist=[_14q.fdt, _14q.fdx, _14q.fnm, _14q.nvd, _14q.nvm, _14q.si, _14q_Asserting_0.doc, _14q_Asserting_0.tim, _14q_Asserting_0.tip, _14r.cfe, _14r.cfs, _14r.si, _14s.cfe, _14s.cfs, _14s.si, _14t.cfe, _14t.cfs, _14t.si, _14u.cfe, _14u.cfs, _14u.si, _14v.cfe, _14v.cfs, _14v.si, _14w.cfe, _14w.cfs, _14w.si, _14x.cfe, _14x.cfs, _14x.si, _14y.cfe, _14y.cfs, _14y.si, _14z.cfe, _14z.cfs, _14z.si, _150.cfe, _150.cfs, _150.si, _151.cfe, _151.cfs, _151.si, _152.cfe, _152.cfs, _152.si, _153.cfe, _153.cfs, _153.si, _154.cfe, _154.cfs, _154.si, _155.cfe, _155.cfs, _155.si, _156.cfe, _156.cfs, _156.si, _157.cfe, _157.cfs, _157.si, _158.cfe, _158.cfs, _158.si, _159.cfe, _159.cfs, _159.si, _15a.cfe, _15a.cfs, _15a.si, _15b.cfe, _15b.cfs, _15b.si, _15c.cfe, _15c.cfs, _15c.si, _15d.cfe, _15d.cfs, _15d.si, _g0.fdt, _g0.fdx, _g0.fnm, _g0.nvd, _g0.nvm, _g0.si, _g0_Asserting_0.doc, _g0_Asserting_0.tim, _g0_Asserting_0.tip, _vd.fdt, _vd.fdx, _vd.fnm, _vd.nvd, _vd.nvm, _vd.si, _vd_Asserting_0.doc, _vd_Asserting_0.tim, _vd_Asserting_0.tip, segments_2]}]> but was:<[{indexVersion=1415292543590,generation=3,filelist=[_15e.fdt, _15e.fdx, _15e.fnm, _15e.nvd, _15e.nvm, _15e.si, _15e_Asserting_0.doc, _15e_Asserting_0.tim, _15e_Asserting_0.tip, _g0.fdt, _g0.fdx, _g0.fnm, _g0.nvd, _g0.nvm, _g0.si, _g0_Asserting_0.doc, _g0_Asserting_0.tim, _g0_Asserting_0.tip, _vd.fdt, _vd.fdx, _vd.fnm, _vd.nvd, _vd.nvm, _vd.si, _vd_Asserting_0.doc, _vd_Asserting_0.tim, _vd_Asserting_0.tip, segments_3]}, {indexVersion=1415292543590,generation=2,filelist=[_14q.fdt, _14q.fdx, _14q.fnm, _14q.nvd, _14q.nvm, _14q.si, _14q_Asserting_0.doc, _14q_Asserting_0.tim, _14q_Asserting_0.tip, _14r.cfe, _14r.cfs, _14r.si, _14s.cfe, _14s.cfs, _14s.si, _14t.cfe, _14t.cfs, _14t.si, _14u.cfe, _14u.cfs, _14u.si, _14v.cfe, _14v.cfs, _14v.si, _14w.cfe, _14w.cfs, _14w.si, _14x.cfe, _14x.cfs, _14x.si, _14y.cfe, _14y.cfs, _14y.si, _14z.cfe, _14z.cfs, _14z.si, _150.cfe, _150.cfs, _150.si, _151.cfe, _151.cfs, _151.si, _152.cfe, _152.cfs, _152.si, _153.cfe, _153.cfs, _153.si, _154.cfe, _154.cfs, _154.si, _155.cfe, _155.cfs, _155.si, _156.cfe, _156.cfs, _156.si, _157.cfe, _157.cfs, _157.si, _158.cfe, _158.cfs, _158.si, _159.cfe, _159.cfs, _159.si, _15a.cfe, _15a.cfs, _15a.si, _15b.cfe, _15b.cfs, _15b.si, _15c.cfe, _15c.cfs, _15c.si, _15d.cfe, _15d.cfs, _15d.si, _g0.fdt, _g0.fdx, _g0.fnm, _g0.nvd, _g0.nvm, _g0.si, _g0_Asserting_0.doc, _g0_Asserting_0.tim, _g0_Asserting_0.tip, _vd.fdt, _vd.fdx, _vd.fnm, _vd.nvd, _vd.nvm, _vd.si, _vd_Asserting_0.doc, _vd_Asserting_0.tim, _vd_Asserting_0.tip, segments_2]}]> Stack Trace: java.lang.AssertionError: expected:<[{indexVersion=1415292543590,generation=2,filelist=[_14q.fdt, _14q.fdx, _14q.fnm, _14q.nvd, _14q.nvm, _14q.si, _14q_Asserting_0.doc, _14q_Asserting_0.tim, _14q_Asserting_0.tip, _14r.cfe, _14r.cfs, _14r.si, _14s.cfe, _14s.cfs, _14s.si, _14t.cfe, _14t.cfs, _14t.si, _14u.cfe, _14u.cfs, _14u.si, _14v.cfe, _14v.cfs, _14v.si, _14w.cfe, _14w.cfs, _14w.si, _14x.cfe, _14x.cfs, _14x.si, _14y.cfe, _14y.cfs, _14y.si, _14z.cfe, _14z.cfs, _14z.si, _150.cfe, _150.cfs, _150.si, _151.cfe, _151.cfs, _151.si, _152.cfe, _152.cfs, _152.si, _153.cfe, _153.cfs, _153.si, _154.cfe, _154.cfs, _154.si, _155.cfe, _155.cfs, _155.si, _156.cfe, _156.cfs, _156.si, _157.cfe, _157.cfs, _157.si, _158.cfe, _158.cfs, _158.si, _159.cfe, _159.cfs, _159.si, _15a.cfe, _15a.cfs, _15a.si, _15b.cfe, _15b.cfs, _15b.si, _15c.cfe, _15c.cfs, _15c.si, _15d.cfe, _15d.cfs, _15d.si, _g0.fdt, _g0.fdx, _g0.fnm, _g0.nvd, _g0.nvm, _g0.si, _g0_Asserting_0.doc, _g0_Asserting_0.tim, _g0_Asserting_0.tip, _vd.fdt, _vd.fdx, _vd.fnm, _vd.nvd, _vd.nvm, _vd.si, _vd_Asserting_0.doc, _vd_Asserting_0.tim, _vd_Asserting_0.tip, segments_2]}]> but was:<[{indexVersion=1415292543590,generation=3,filelist=[_15e.fdt, _15e.fdx, _15e.fnm, _15e.nvd, _15e.nvm, _15e.si, _15e_Asserting_0.doc, _15e_Asserting_0.tim, _15e_Asserting_0.tip, _g0.fdt, _g0.fdx, _g0.fnm, _g0.nvd, _g0.nvm, _g0.si, _g0_Asserting_0.doc, _g0_Asserting_0.tim, _g0_Asserting_0.tip, _vd.fdt, _vd.fdx, _vd.fnm, _vd.nvd, _vd.nvm, _vd.si, _vd_Asserting_0.doc, _vd_Asserting_0.tim, _vd_Asserting_0.tip, segments_3]}, {indexVersion=1415292543590,generation=2,filelist=[_14q.fdt, _14q.fdx, _14q.fnm, _14q.nvd, _14q.nvm, _14q.si, _14q_Asserting_0.doc, _14q_Asserting_0.tim, _14q_Asserting_0.tip, _14r.cfe, _14r.cfs, _14r.si, _14s.cfe, _14s.cfs, _14s.si, _14t.cfe, _14t.cfs, _14t.si, _14u.cfe, _14u.cfs, _14u.si, _14v.cfe, _14v.cfs, _14v.si, _14w.cfe, _14w.cfs, _14w.si, _14x.cfe, _14x.cfs, _14x.si, _14y.cfe, _14y.cfs, _14y.si, _14z.cfe, _14z.cfs, _14z.si, _150.cfe, _150.cfs, _150.si, _151.cfe, _151.cfs, _151.si, _152.cfe, _152.cfs, _152.si, _153.cfe, _153.cfs, _153.si, _154.cfe, _154.cfs, _154.si, _155.cfe, _155.cfs, _155.si, _156.cfe, _156.cfs, _156.si, _157.cfe, _157.cfs, _157.si, _158.cfe, _158.cfs, _158.si, _159.cfe, _159.cfs, _159.si, _15a.cfe, _15a.cfs, _15a.si, _15b.cfe, _15b.cfs, _15b.si, _15c.cfe, _15c.cfs, _15c.si, _15d.cfe, _15d.cfs, _15d.si, _g0.fdt, _g0.fdx, _g0.fnm, _g0.nvd, _g0.nvm, _g0.si, _g0_Asserting_0.doc, _g0_Asserting_0.tim, _g0_Asserting_0.tip, _vd.fdt, _vd.fdx, _vd.fnm, _vd.nvd, _vd.nvm, _vd.si, _vd_Asserting_0.doc, _vd_Asserting_0.tim, _vd_Asserting_0.tip, segments_2]}]> at __randomizedtesting.SeedInfo.seed([FEA2EACA01B8699E:DB75F1FA71F0679D]:0) at org.junit.Assert.fail(Assert.java:93) at org.junit.Assert.failNotEquals(Assert.java:647) at org.junit.Assert.assertEquals(Assert.java:128) at org.junit.Assert.assertEquals(Assert.java:147) at org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload(TestReplicationHandler.java:1175) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at java.lang.Thread.run(Thread.java:745) Build Log: [...truncated 11497 lines...] [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest [junit4] 2> Creating dataDir: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-FEA2EACA01B8699E-001/init-core-data-001 [junit4] 2> 114306 T303 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (false) [junit4] 2> 114306 T303 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: / [junit4] 2> 114314 T303 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch [junit4] 2> 114348 T303 oasc.ZkTestServer.run STARTING ZK TEST SERVER [junit4] 1> client port:0.0.0.0/0.0.0.0:0 [junit4] 2> 114359 T304 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server [junit4] 2> 114660 T303 oasc.ZkTestServer.run start zk server on port:22662 [junit4] 2> 114673 T303 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 114979 T303 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 115161 T310 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4942c49a name:ZooKeeperConnection Watcher:127.0.0.1:22662 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 115161 T303 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 115162 T303 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 115174 T303 oascc.SolrZkClient.makePath makePath: /solr [junit4] 2> 115301 T303 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 115302 T303 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 115304 T312 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@b861b68 name:ZooKeeperConnection Watcher:127.0.0.1:22662/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 115305 T303 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 115305 T303 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 115313 T303 oascc.SolrZkClient.makePath makePath: /collections/collection1 [junit4] 2> 115318 T303 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards [junit4] 2> 115322 T303 oascc.SolrZkClient.makePath makePath: /collections/control_collection [junit4] 2> 115325 T303 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards [junit4] 2> 115330 T303 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml [junit4] 2> 115331 T303 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml [junit4] 2> 115337 T303 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml [junit4] 2> 115337 T303 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml [junit4] 2> 115342 T303 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml [junit4] 2> 115343 T303 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml [junit4] 2> 115348 T303 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt [junit4] 2> 115348 T303 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt [junit4] 2> 115352 T303 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt [junit4] 2> 115353 T303 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt [junit4] 2> 115357 T303 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml [junit4] 2> 115358 T303 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml [junit4] 2> 115362 T303 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml [junit4] 2> 115363 T303 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml [junit4] 2> 115367 T303 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json [junit4] 2> 115368 T303 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json [junit4] 2> 115372 T303 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt [junit4] 2> 115373 T303 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt [junit4] 2> 115377 T303 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt [junit4] 2> 115377 T303 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt [junit4] 2> 115381 T303 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt [junit4] 2> 115382 T303 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt [junit4] 2> 115406 T303 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 115407 T303 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 115409 T314 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5ae568e3 name:ZooKeeperConnection Watcher:127.0.0.1:22662/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 115410 T303 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 115410 T303 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> Using old style solr.xml [junit4] 2> 115414 T303 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 115415 T303 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 115416 T316 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@624e4cd1 name:ZooKeeperConnection Watcher:127.0.0.1:22662 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 115417 T303 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 115417 T303 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 115417 T303 oascc.SolrZkClient.makePath makePath: /solr [junit4] 2> 115436 T303 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 115437 T303 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 115439 T318 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5d34f657 name:ZooKeeperConnection Watcher:127.0.0.1:22662/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 115439 T303 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 115440 T303 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 115442 T303 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml [junit4] 2> 115442 T303 oascc.SolrZkClient.makePath makePath: /configs/conf2/solrconfig.xml [junit4] 2> 115448 T303 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml [junit4] 2> 115449 T303 oascc.SolrZkClient.makePath makePath: /configs/conf2/schema.xml [junit4] 2> 115453 T303 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml [junit4] 2> 115454 T303 oascc.SolrZkClient.makePath makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml [junit4] 2> 115458 T303 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt [junit4] 2> 115458 T303 oascc.SolrZkClient.makePath makePath: /configs/conf2/stopwords.txt [junit4] 2> 115462 T303 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt [junit4] 2> 115463 T303 oascc.SolrZkClient.makePath makePath: /configs/conf2/protwords.txt [junit4] 2> 115467 T303 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml [junit4] 2> 115467 T303 oascc.SolrZkClient.makePath makePath: /configs/conf2/currency.xml [junit4] 2> 115471 T303 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json [junit4] 2> 115472 T303 oascc.SolrZkClient.makePath makePath: /configs/conf2/open-exchange-rates.json [junit4] 2> 115476 T303 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt [junit4] 2> 115477 T303 oascc.SolrZkClient.makePath makePath: /configs/conf2/mapping-ISOLatin1Accent.txt [junit4] 2> 115480 T303 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt [junit4] 2> 115481 T303 oascc.SolrZkClient.makePath makePath: /configs/conf2/old_synonyms.txt [junit4] 2> 115485 T303 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt [junit4] 2> 115486 T303 oascc.SolrZkClient.makePath makePath: /configs/conf2/synonyms.txt [junit4] 2> 115490 T303 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml [junit4] 2> 115491 T303 oascc.SolrZkClient.makePath makePath: /configs/conf2/elevate.xml [junit4] 2> 116925 T303 oejs.Server.doStart jetty-8.1.10.v20130312 [junit4] 2> 116993 T303 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] [junit4] 2> 116999 T303 oejs.AbstractConnector.doStart Started SslSocketConnector@127.0.0.1:22668 [junit4] 2> 117003 T303 oass.SolrDispatchFilter.init SolrDispatchFilter.init() [junit4] 2> 117004 T303 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 117004 T303 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-FEA2EACA01B8699E-001/tempDir-002 [junit4] 2> 117005 T303 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-FEA2EACA01B8699E-001/tempDir-002/' [junit4] 2> 117040 T303 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-FEA2EACA01B8699E-001/tempDir-002/solr.xml [junit4] 2> 117094 T303 oasc.CoreContainer.<init> New CoreContainer 79307965 [junit4] 2> 117094 T303 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-FEA2EACA01B8699E-001/tempDir-002/] [junit4] 2> 117095 T303 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000 [junit4] 2> 117096 T303 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null [junit4] 2> 117096 T303 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000 [junit4] 2> 117096 T303 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20 [junit4] 2> 117097 T303 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000 [junit4] 2> 117097 T303 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0 [junit4] 2> 117097 T303 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647 [junit4] 2> 117097 T303 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5 [junit4] 2> 117098 T303 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1 [junit4] 2> 117098 T303 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false [junit4] 2> 117101 T303 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=3400000&connTimeout=45000&retry=false [junit4] 2> 117103 T303 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 117104 T303 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 117104 T303 oasc.CoreContainer.load Host Name: 127.0.0.1 [junit4] 2> 117104 T303 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:22662/solr [junit4] 2> 117137 T303 oasc.ZkController.checkChrootPath zkHost includes chroot [junit4] 2> 117138 T303 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 117138 T303 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 117140 T329 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2075ade1 name:ZooKeeperConnection Watcher:127.0.0.1:22662 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 117141 T303 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 117141 T303 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 117154 T303 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 117156 T331 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@322ba6d7 name:ZooKeeperConnection Watcher:127.0.0.1:22662/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 117156 T303 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 117176 T303 oascc.SolrZkClient.makePath makePath: /overseer/queue [junit4] 2> 117191 T303 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work [junit4] 2> 117201 T303 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running [junit4] 2> 117206 T303 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed [junit4] 2> 117211 T303 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure [junit4] 2> 117240 T303 oascc.SolrZkClient.makePath makePath: /live_nodes [junit4] 2> 117242 T303 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:22668_ [junit4] 2> 117251 T303 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:22668_ [junit4] 2> 117255 T303 oascc.SolrZkClient.makePath makePath: /overseer_elect [junit4] 2> 117258 T303 oascc.SolrZkClient.makePath makePath: /overseer_elect/election [junit4] 2> 117261 T303 oasc.Overseer.close Overseer (id=null) closing [junit4] 2> 117285 T303 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:22668_ [junit4] 2> 117286 T303 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader [junit4] 2> 117289 T303 oasc.Overseer.start Overseer (id=92752555564007430-127.0.0.1:22668_-n_0000000000) starting [junit4] 2> 117308 T303 oascc.SolrZkClient.makePath makePath: /overseer/queue-work [junit4] 2> 117345 T303 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000 [junit4] 2> 117367 T333 oasc.OverseerCollectionProcessor.run Process current queue of collection creations [junit4] 2> 117368 T303 oascc.SolrZkClient.makePath makePath: /clusterstate.json [junit4] 2> 117371 T303 oascc.SolrZkClient.makePath makePath: /aliases.json [junit4] 2> 117373 T303 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 117404 T332 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue [junit4] 2> 117414 T303 oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3 [junit4] 2> 117415 T303 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done [junit4] 2> 117461 T303 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 117462 T303 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 117464 T337 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6ec94dc8 name:ZooKeeperConnection Watcher:127.0.0.1:22662/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 117465 T303 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 117465 T303 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 117467 T303 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 117474 T303 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false [junit4] 2> 117475 T303 oasc.AbstractFullDistribZkTestBase.createJettys Creating collection1 with stateFormat=2 [junit4] 2> 117475 T303 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 117476 T303 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 117477 T339 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@20afaa0a name:ZooKeeperConnection Watcher:127.0.0.1:22662/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 117477 T303 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 117478 T303 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 117492 T340 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 117509 T332 oasc.Overseer$ClusterStateUpdater.buildCollection building a new collection: collection1 [junit4] 2> 117510 T332 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2] [junit4] 2> 117520 T332 oasc.Overseer$ClusterStateUpdater.createCollection state version collection1 2 [junit4] 2> 117530 T332 oasc.Overseer$ClusterStateUpdater.updateZkStates going to create_collection /collections/collection1/state.json [junit4] 2> 117533 T340 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1) [junit4] 2> 117533 T341 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1) [junit4] 2> 118592 T303 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1 [junit4] 2> 118593 T303 oejs.Server.doStart jetty-8.1.10.v20130312 [junit4] 2> 118599 T303 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] [junit4] 2> 118602 T303 oejs.AbstractConnector.doStart Started SslSocketConnector@127.0.0.1:22673 [junit4] 2> 118605 T303 oass.SolrDispatchFilter.init SolrDispatchFilter.init() [junit4] 2> 118606 T303 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 118606 T303 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-FEA2EACA01B8699E-001/tempDir-003 [junit4] 2> 118606 T303 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-FEA2EACA01B8699E-001/tempDir-003/' [junit4] 2> 118642 T303 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-FEA2EACA01B8699E-001/tempDir-003/solr.xml [junit4] 2> 118697 T303 oasc.CoreContainer.<init> New CoreContainer 1226200913 [junit4] 2> 118698 T303 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-FEA2EACA01B8699E-001/tempDir-003/] [junit4] 2> 118699 T303 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000 [junit4] 2> 118699 T303 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null [junit4] 2> 118699 T303 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000 [junit4] 2> 118699 T303 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20 [junit4] 2> 118700 T303 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000 [junit4] 2> 118700 T303 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0 [junit4] 2> 118700 T303 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647 [junit4] 2> 118701 T303 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5 [junit4] 2> 118701 T303 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1 [junit4] 2> 118701 T303 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false [junit4] 2> 118704 T303 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=3400000&connTimeout=45000&retry=false [junit4] 2> 118706 T303 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 118707 T303 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 118707 T303 oasc.CoreContainer.load Host Name: 127.0.0.1 [junit4] 2> 118707 T303 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:22662/solr [junit4] 2> 118708 T303 oasc.ZkController.checkChrootPath zkHost includes chroot [junit4] 2> 118708 T303 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 118709 T303 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 118711 T352 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@77347fda name:ZooKeeperConnection Watcher:127.0.0.1:22662 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 118711 T303 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 118712 T303 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 118714 T303 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 118716 T354 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@41e8505a name:ZooKeeperConnection Watcher:127.0.0.1:22662/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 118716 T303 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 118727 T303 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 119737 T303 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:22673_ [junit4] 2> 119739 T303 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:22673_ [junit4] 2> 119746 T303 oasc.Overseer.close Overseer (id=null) closing [junit4] 2> 119762 T303 oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3 [junit4] 2> 119762 T303 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done [junit4] 2> 121080 T303 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2 [junit4] 2> 121086 T303 oejs.Server.doStart jetty-8.1.10.v20130312 [junit4] 2> 121093 T303 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] [junit4] 2> 121095 T303 oejs.AbstractConnector.doStart Started SslSocketConnector@127.0.0.1:22676 [junit4] 2> 121098 T303 oass.SolrDispatchFilter.init SolrDispatchFilter.init() [junit4] 2> 121098 T303 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 121099 T303 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-FEA2EACA01B8699E-001/tempDir-004 [junit4] 2> 121099 T303 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-FEA2EACA01B8699E-001/tempDir-004/' [junit4] 2> 121130 T303 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-FEA2EACA01B8699E-001/tempDir-004/solr.xml [junit4] 2> 121218 T303 oasc.CoreContainer.<init> New CoreContainer 1777797839 [junit4] 2> 121219 T303 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-FEA2EACA01B8699E-001/tempDir-004/] [junit4] 2> 121220 T303 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000 [junit4] 2> 121220 T303 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null [junit4] 2> 121221 T303 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000 [junit4] 2> 121221 T303 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20 [junit4] 2> 121221 T303 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000 [junit4] 2> 121221 T303 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0 [junit4] 2> 121222 T303 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647 [junit4] 2> 121222 T303 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5 [junit4] 2> 121222 T303 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1 [junit4] 2> 121223 T303 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false [junit4] 2> 121225 T303 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=3400000&connTimeout=45000&retry=false [junit4] 2> 121228 T303 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 121228 T303 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 121228 T303 oasc.CoreContainer.load Host Name: 127.0.0.1 [junit4] 2> 121229 T303 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:22662/solr [junit4] 2> 121229 T303 oasc.ZkController.checkChrootPath zkHost includes chroot [junit4] 2> 121229 T303 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 121230 T303 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 121233 T367 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2e40dc59 name:ZooKeeperConnection Watcher:127.0.0.1:22662 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 121233 T303 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 121234 T303 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 121237 T303 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 121239 T369 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@54505bcd name:ZooKeeperConnection Watcher:127.0.0.1:22662/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 121239 T303 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 121251 T303 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 122261 T303 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:22676_ [junit4] 2> 122263 T303 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:22676_ [junit4] 2> 122269 T303 oasc.Overseer.close Overseer (id=null) closing [junit4] 2> 122276 T303 oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3 [junit4] 2> 122276 T303 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done [junit4] 2> 123924 T303 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3 [junit4] 2> 123925 T303 oejs.Server.doStart jetty-8.1.10.v20130312 [junit4] 2> 123934 T303 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] [junit4] 2> 123938 T303 oejs.AbstractConnector.doStart Started SslSocketConnector@127.0.0.1:22679 [junit4] 2> 123942 T303 oass.SolrDispatchFilter.init SolrDispatchFilter.init() [junit4] 2> 123943 T303 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 123943 T303 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-FEA2EACA01B8699E-001/tempDir-005 [junit4] 2> 123943 T303 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-FEA2EACA01B8699E-001/tempDir-005/' [junit4] 2> 123990 T303 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-FEA2EACA01B8699E-001/tempDir-005/solr.xml [junit4] 2> 124074 T303 oasc.CoreContainer.<init> New CoreContainer 770659873 [junit4] 2> 124074 T303 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-FEA2EACA01B8699E-001/tempDir-005/] [junit4] 2> 124076 T303 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000 [junit4] 2> 124076 T303 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null [junit4] 2> 124077 T303 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000 [junit4] 2> 124077 T303 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20 [junit4] 2> 124077 T303 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000 [junit4] 2> 124078 T303 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0 [junit4] 2> 124078 T303 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647 [junit4] 2> 124079 T303 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5 [junit4] 2> 124079 T303 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1 [junit4] 2> 124080 T303 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false [junit4] 2> 124083 T303 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=3400000&connTimeout=45000&retry=false [junit4] 2> 124087 T303 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 124087 T303 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 124088 T303 oasc.CoreContainer.load Host Name: 127.0.0.1 [junit4] 2> 124088 T303 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:22662/solr [junit4] 2> 124088 T303 oasc.ZkController.checkChrootPath zkHost includes chroot [junit4] 2> 124089 T303 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 124090 T303 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 124092 T382 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@42e550fd name:ZooKeeperConnection Watcher:127.0.0.1:22662 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 124093 T303 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 124094 T303 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 124097 T303 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 124098 T384 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@c6a5b72 name:ZooKeeperConnection Watcher:127.0.0.1:22662/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 124099 T303 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 124110 T303 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 125120 T303 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:22679_ [junit4] 2> 125122 T303 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:22679_ [junit4] 2> 125128 T303 oasc.Overseer.close Overseer (id=null) closing [junit4] 2> 125136 T303 oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3 [junit4] 2> 125136 T303 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done [junit4] 2> 126794 T303 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 4 [junit4] 2> 126796 T303 oejs.Server.doStart jetty-8.1.10.v20130312 [junit4] 2> 126803 T303 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] [junit4] 2> 126807 T303 oejs.AbstractConnector.doStart Started SslSocketConnector@127.0.0.1:22682 [junit4] 2> 126811 T303 oass.SolrDispatchFilter.init SolrDispatchFilter.init() [junit4] 2> 126811 T303 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 126812 T303 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-FEA2EACA01B8699E-001/tempDir-006 [junit4] 2> 126812 T303 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-FEA2EACA01B8699E-001/tempDir-006/' [junit4] 2> 126864 T303 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-FEA2EACA01B8699E-001/tempDir-006/solr.xml [junit4] 2> 126947 T303 oasc.CoreContainer.<init> New CoreContainer 1469336945 [junit4] 2> 126948 T303 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-FEA2EACA01B8699E-001/tempDir-006/] [junit4] 2> 126949 T303 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000 [junit4] 2> 126949 T303 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null [junit4] 2> 126950 T303 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000 [junit4] 2> 126950 T303 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20 [junit4] 2> 126951 T303 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000 [junit4] 2> 126951 T303 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0 [junit4] 2> 126951 T303 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647 [junit4] 2> 126952 T303 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5 [junit4] 2> 126952 T303 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1 [junit4] 2> 126953 T303 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false [junit4] 2> 126957 T303 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=3400000&connTimeout=45000&retry=false [junit4] 2> 126960 T303 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 126961 T303 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 126961 T303 oasc.CoreContainer.load Host Name: 127.0.0.1 [junit4] 2> 126962 T303 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:22662/solr [junit4] 2> 126962 T303 oasc.ZkController.checkChrootPath zkHost includes chroot [junit4] 2> 126962 T303 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 126964 T303 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 126966 T397 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1f4e7db name:ZooKeeperConnection Watcher:127.0.0.1:22662 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 126967 T303 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 126967 T303 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 126970 T303 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 126972 T399 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@79025994 name:ZooKeeperConnection Watcher:127.0.0.1:22662/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 126972 T303 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 126982 T303 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 127991 T303 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:22682_ [junit4] 2> 127993 T303 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:22682_ [junit4] 2> 128000 T303 oasc.Overseer.close Overseer (id=null) closing [junit4] 2> 128007 T303 oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3 [junit4] 2> 128008 T303 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done [junit4] 2> 128323 T349 oasha.CollectionsHandler.handleCreateAction Creating Collection : name=solrj_collection&wt=javabin&collection.configName=conf1&replicationFactor=2&action=CREATE&numShards=2&version=2&router.field=myOwnField [junit4] 2> 128326 T340 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/collection-queue-work state SyncConnected [junit4] 2> 128328 T333 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{ [junit4] 2> "operation":"create", [junit4] 2> "fromApi":"true", [junit4] 2> "name":"solrj_collection", [junit4] 2> "replicationFactor":"2", [junit4] 2> "collection.configName":"conf1", [junit4] 2> "numShards":"2", [junit4] 2> "router.field":"myOwnField"} [junit4] 2> 128335 T403 oasc.OverseerCollectionProcessor.processMessage WARN OverseerCollectionProcessor.processMessage : create , { [junit4] 2> "operation":"create", [junit4] 2> "fromApi":"true", [junit4] 2> "name":"solrj_collection", [junit4] 2> "replicationFactor":"2", [junit4] 2> "collection.configName":"conf1", [junit4] 2> "numShards":"2", [junit4] 2> "router.field":"myOwnField"} [junit4] 2> 128343 T403 oasc.OverseerCollectionProcessor.createConfNode creating collections conf node /collections/solrj_collection [junit4] 2> 128344 T403 oascc.SolrZkClient.makePath makePath: /collections/solrj_collection [junit4] 2> 128351 T340 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 128358 T332 oasc.Overseer$ClusterStateUpdater.buildCollection building a new collection: solrj_collection [junit4] 2> 128358 T332 oasc.Overseer$ClusterStateUpdater.createCollection Create collection solrj_collection with shards [shard1, shard2] [junit4] 2> 128358 T332 oasc.Overseer$ClusterStateUpdater.createCollection state version solrj_collection 1 [junit4] 2> 128364 T370 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4] 2> 128364 T340 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4] 2> 128364 T355 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4] 2> 128364 T400 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4] 2> 128364 T341 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4] 2> 128364 T385 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4] 2> 128451 T403 oasc.OverseerCollectionProcessor.createCollection Creating SolrCores for new collection solrj_collection, shardNames [shard1, shard2] , replicationFactor : 2 [junit4] 2> 128452 T403 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard1_replica1 as part of slice shard1 of collection solrj_collection on 127.0.0.1:22682_ [junit4] 2> 128454 T403 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard1_replica2 as part of slice shard1 of collection solrj_collection on 127.0.0.1:22668_ [junit4] 2> 128457 T403 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard2_replica1 as part of slice shard2 of collection solrj_collection on 127.0.0.1:22679_ [junit4] 2> 128460 T403 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard2_replica2 as part of slice shard2 of collection solrj_collection on 127.0.0.1:22676_ [junit4] 2> 128602 T323 oasha.CoreAdminHandler.handleCreateAction core create command name=solrj_collection_shard1_replica2&wt=javabin&shard=shard1&collection.configName=conf1&action=CREATE&numShards=2&version=2&qt=/admin/cores&collection=solrj_collection [junit4] 2> 128606 T374 oasha.CoreAdminHandler.handleCreateAction core create command name=solrj_collection_shard2_replica1&wt=javabin&shard=shard2&collection.configName=conf1&action=CREATE&numShards=2&version=2&qt=/admin/cores&collection=solrj_collection [junit4] 2> 128610 T389 oasha.CoreAdminHandler.handleCreateAction core create command name=solrj_collection_shard1_replica1&wt=javabin&shard=shard1&collection.configName=conf1&action=CREATE&numShards=2&version=2&qt=/admin/cores&collection=solrj_collection [junit4] 2> 128612 T361 oasha.CoreAdminHandler.handleCreateAction core create command name=solrj_collection_shard2_replica2&wt=javabin&shard=shard2&collection.configName=conf1&action=CREATE&numShards=2&version=2&qt=/admin/cores&collection=solrj_collection [junit4] 2> 128615 T374 oasc.ZkController.publish publishing core=solrj_collection_shard2_replica1 state=down collection=solrj_collection [junit4] 2> 128616 T323 oasc.ZkController.publish publishing core=solrj_collection_shard1_replica2 state=down collection=solrj_collection [junit4] 2> 128616 T389 oasc.ZkController.publish publishing core=solrj_collection_shard1_replica1 state=down collection=solrj_collection [junit4] 2> 128617 T361 oasc.ZkController.publish publishing core=solrj_collection_shard2_replica2 state=down collection=solrj_collection [junit4] 2> 128625 T340 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 128625 T323 oasc.ZkController.waitForCoreNodeName look for our core node name [junit4] 2> 128626 T361 oasc.ZkController.waitForCoreNodeName look for our core node name [junit4] 2> 128625 T389 oasc.ZkController.waitForCoreNodeName look for our core node name [junit4] 2> 128626 T374 oasc.ZkController.waitForCoreNodeName look for our core node name [junit4] 2> 128638 T332 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={ [junit4] 2> "roles":null, [junit4] 2> "operation":"state", [junit4] 2> "base_url":"https://127.0.0.1:22682", [junit4] 2> "shard":"shard1", [junit4] 2> "node_name":"127.0.0.1:22682_", [junit4] 2> "numShards":"2", [junit4] 2> "core":"solrj_collection_shard1_replica1", [junit4] 2> "collection":"solrj_collection", [junit4] 2> "state":"down"} [junit4] 2> 128651 T332 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={ [junit4] 2> "roles":null, [junit4] 2> "operation":"state", [junit4] 2> "base_url":"https://127.0.0.1:22668", [junit4] 2> "shard":"shard1", [junit4] 2> "node_name":"127.0.0.1:22668_", [junit4] 2> "numShards":"2", [junit4] 2> "core":"solrj_collection_shard1_replica2", [junit4] 2> "collection":"solrj_collection", [junit4] 2> "state":"down"} [junit4] 2> 128656 T340 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 128658 T332 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={ [junit4] 2> "roles":null, [junit4] 2> "operation":"state", [junit4] 2> "base_url":"https://127.0.0.1:22679", [junit4] 2> "shard":"shard2", [junit4] 2> "node_name":"127.0.0.1:22679_", [junit4] 2> "numShards":"2", [junit4] 2> "core":"solrj_collection_shard2_replica1", [junit4] 2> "collection":"solrj_collection", [junit4] 2> "state":"down"} [junit4] 2> 128665 T332 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={ [junit4] 2> "roles":null, [junit4] 2> "operation":"state", [junit4] 2> "base_url":"https://127.0.0.1:22676", [junit4] 2> "shard":"shard2", [junit4] 2> "node_name":"127.0.0.1:22676_", [junit4] 2> "numShards":"2", [junit4] 2> "core":"solrj_collection_shard2_replica2", [junit4] 2> "collection":"solrj_collection", [junit4] 2> "state":"down"} [junit4] 2> 128669 T340 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 128671 T385 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4] 2> 128672 T400 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4] 2> 128672 T370 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4] 2> 128671 T340 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4] 2> 128671 T355 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4] 2> 128671 T341 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4] 2> 129627 T361 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard2_replica2 [junit4] 2> 129627 T323 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard1_replica2 [junit4] 2> 129627 T389 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard1_replica1 [junit4] 2> 129627 T374 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard2_replica1 [junit4] 2> 129627 T361 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection [junit4] 2> 129629 T374 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection [junit4] 2> 129628 T389 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection [junit4] 2> 129628 T323 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection [junit4] 2> 129630 T374 oasc.ZkController.createCollectionZkNode Collection zkNode exists [junit4] 2> 129631 T323 oasc.ZkController.createCollectionZkNode Collection zkNode exists [junit4] 2> 129630 T361 oasc.ZkController.createCollectionZkNode Collection zkNode exists [junit4] 2> 129631 T323 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection [junit4] 2> 129631 T374 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection [junit4] 2> 129630 T389 oasc.ZkController.createCollectionZkNode Collection zkNode exists [junit4] 2> 129632 T361 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection [junit4] 2> 129633 T323 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper [junit4] 2> 129633 T389 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection [junit4] 2> 129634 T374 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper [junit4] 2> 129634 T361 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper [junit4] 2> 129635 T389 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper [junit4] 2> 129640 T374 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-FEA2EACA01B8699E-001/tempDir-005/solrj_collection_shard2_replica1/' [junit4] 2> 129640 T389 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-FEA2EACA01B8699E-001/tempDir-006/solrj_collection_shard1_replica1/' [junit4] 2> 129640 T323 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-FEA2EACA01B8699E-001/tempDir-002/solrj_collection_shard1_replica2/' [junit4] 2> 129640 T361 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-FEA2EACA01B8699E-001/tempDir-004/solrj_collection_shard2_replica2/' [junit4] 2> 129852 T374 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0 [junit4] 2> 129853 T323 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0 [junit4] 2> 129854 T361 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0 [junit4] 2> 129858 T389 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0 [junit4] 2> 129915 T361 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml [junit4] 2> 129917 T374 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml [junit4] 2> 129917 T361 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml [junit4] 2> 129918 T323 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml [junit4] 2> 129919 T374 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml [junit4] 2> 129920 T323 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml [junit4] 2> 129921 T389 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml [junit4] 2> 129922 T389 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml [junit4] 2> 129931 T361 oass.IndexSchema.readSchema [solrj_collection_shard2_replica2] Schema name=test [junit4] 2> 129948 T374 oass.IndexSchema.readSchema [solrj_collection_shard2_replica1] Schema name=test [junit4] 2> 129950 T323 oass.IndexSchema.readSchema [solrj_collection_shard1_replica2] Schema name=test [junit4] 2> 129954 T389 oass.IndexSchema.readSchema [solrj_collection_shard1_replica1] Schema name=test [junit4] 2> 130439 T374 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440. [junit4] 2> 130443 T323 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440. [junit4] 2> 130443 T389 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440. [junit4] 2> 130444 T361 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440. [junit4] 2> 130453 T374 oass.IndexSchema.readSchema default search field in schema is text [junit4] 2> 130453 T361 oass.IndexSchema.readSchema default search field in schema is text [junit4] 2> 130454 T389 oass.IndexSchema.readSchema default search field in schema is text [junit4] 2> 130455 T323 oass.IndexSchema.readSchema default search field in schema is text [junit4] 2> 130456 T374 oass.IndexSchema.readSchema unique key field: id [junit4] 2> 130457 T361 oass.IndexSchema.readSchema unique key field: id [junit4] 2> 130457 T389 oass.IndexSchema.readSchema unique key field: id [junit4] 2> 130458 T323 oass.IndexSchema.readSchema unique key field: id [junit4] 2> 130481 T389 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 130482 T323 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 130482 T374 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 130483 T361 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 130488 T323 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 130488 T374 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 130489 T389 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 130490 T361 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 130494 T323 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4] 2> 130495 T374 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4] 2> 130495 T323 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE [junit4] 2> 130496 T361 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4] 2> 130496 T389 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4] 2> 130497 T374 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE [junit4] 2> 130498 T361 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE [junit4] 2> 130499 T389 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE [junit4] 2> 130497 T323 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING [junit4] 2> 130499 T389 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING [junit4] 2> 130499 T361 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING [junit4] 2> 130498 T374 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING [junit4] 2> 130501 T361 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4] 2> 130500 T389 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4] 2> 130500 T323 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4] 2> 130501 T374 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4] 2> 130502 T361 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE [junit4] 2> 130503 T361 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING [junit4] 2> 130503 T389 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE [junit4] 2> 130504 T361 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard2_replica2' using configuration from collection solrj_collection [junit4] 2> 130504 T374 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE [junit4] 2> 130503 T323 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE [junit4] 2> 130505 T374 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING [junit4] 2> 130504 T361 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory [junit4] 2> 130504 T389 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING [junit4] 2> 130506 T361 oasc.SolrCore.<init> [solrj_collection_shard2_replica2] Opening new SolrCore at ../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-FEA2EACA01B8699E-001/tempDir-004/solrj_collection_shard2_replica2/, dataDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-FEA2EACA01B8699E-001/tempDir-004/solrj_collection_shard2_replica2/data/ [junit4] 2> 130506 T374 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard2_replica1' using configuration from collection solrj_collection [junit4] 2> 130505 T323 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING [junit4] 2> 130508 T374 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory [junit4] 2> 130508 T361 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1ac9a702 [junit4] 2> 130507 T389 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard1_replica1' using configuration from collection solrj_collection [junit4] 2> 130509 T374 oasc.SolrCore.<init> [solrj_collection_shard2_replica1] Opening new SolrCore at /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-FEA2EACA01B8699E-001/tempDir-005/solrj_collection_shard2_replica1/, dataDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-FEA2EACA01B8699E-001/tempDir-005/solrj_collection_shard2_replica1/data/ [junit4] 2> 130508 T323 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard1_replica2' using configuration from collection solrj_collection [junit4] 2> 130510 T374 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1ac9a702 [junit4] 2> 130510 T389 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory [junit4] 2> 130511 T323 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory [junit4] 2> 130512 T389 oasc.SolrCore.<init> [solrj_collection_shard1_replica1] Opening new SolrCore at /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-FEA2EACA01B8699E-001/tempDir-006/solrj_collection_shard1_replica1/, dataDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-FEA2EACA01B8699E-001/tempDir-006/solrj_collection_shard1_replica1/data/ [junit4] 2> 130512 T323 oasc.SolrCore.<init> [solrj_collection_shard1_replica2] Opening new SolrCore at /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-FEA2EACA01B8699E-001/tempDir-002/solrj_collection_shard1_replica2/, dataDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J3/temp/solr.cloud.CollectionsAPIDistributedZkTest-FEA2EACA01B8699E-001/tempDir-002/solrj_collection_shard1_replica2/data/ [junit4] 2> 130513 T389 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1ac9a702 [junit4] 2> 130513 T323 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: [...truncated too long message...] junit4] 2> at java.lang.Thread.run(Thread.java:745) [junit4] 2> 65) Thread[id=2186, name=searcherExecutor-1123-thread-1, state=WAITING, group=TGRP-ChaosMonkeySafeLeaderTest] [junit4] 2> at sun.misc.Unsafe.park(Native Method) [junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) [junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) [junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) [junit4] 2> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) [junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [junit4] 2> at java.lang.Thread.run(Thread.java:745) [junit4] 2> 66) Thread[id=2172, name=qtp1257495698-2172 Acceptor0 SocketConnector@127.0.0.1:31806, state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderTest] [junit4] 2> at java.net.PlainSocketImpl.socketAccept(Native Method) [junit4] 2> at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398) [junit4] 2> at java.net.ServerSocket.implAccept(ServerSocket.java:530) [junit4] 2> at java.net.ServerSocket.accept(ServerSocket.java:498) [junit4] 2> at org.eclipse.jetty.server.bio.SocketConnector.accept(SocketConnector.java:117) [junit4] 2> at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938) [junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) [junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) [junit4] 2> at java.lang.Thread.run(Thread.java:745) [junit4] 2> 67) Thread[id=1873, name=coreZkRegister-1018-thread-1, state=WAITING, group=TGRP-ChaosMonkeySafeLeaderTest] [junit4] 2> at sun.misc.Unsafe.park(Native Method) [junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) [junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) [junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) [junit4] 2> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) [junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [junit4] 2> at java.lang.Thread.run(Thread.java:745) [junit4] 2> 68) Thread[id=1934, name=qtp1892819812-1934 Acceptor0 SocketConnector@127.0.0.1:51454, state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderTest] [junit4] 2> at java.net.PlainSocketImpl.socketAccept(Native Method) [junit4] 2> at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398) [junit4] 2> at java.net.ServerSocket.implAccept(ServerSocket.java:530) [junit4] 2> at java.net.ServerSocket.accept(ServerSocket.java:498) [junit4] 2> at org.eclipse.jetty.server.bio.SocketConnector.accept(SocketConnector.java:117) [junit4] 2> at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938) [junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) [junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) [junit4] 2> at java.lang.Thread.run(Thread.java:745) [junit4] 2> 69) Thread[id=1875, name=qtp1669094124-1875 Acceptor0 SocketConnector@127.0.0.1:27878, state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderTest] [junit4] 2> at java.net.PlainSocketImpl.socketAccept(Native Method) [junit4] 2> at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398) [junit4] 2> at java.net.ServerSocket.implAccept(ServerSocket.java:530) [junit4] 2> at java.net.ServerSocket.accept(ServerSocket.java:498) [junit4] 2> at org.eclipse.jetty.server.bio.SocketConnector.accept(SocketConnector.java:117) [junit4] 2> at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938) [junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) [junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) [junit4] 2> at java.lang.Thread.run(Thread.java:745) [junit4] 2> 70) Thread[id=1913, name=qtp1383421744-1913 Acceptor0 SocketConnector@127.0.0.1:30451, state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderTest] [junit4] 2> at java.net.PlainSocketImpl.socketAccept(Native Method) [junit4] 2> at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398) [junit4] 2> at java.net.ServerSocket.implAccept(ServerSocket.java:530) [junit4] 2> at java.net.ServerSocket.accept(ServerSocket.java:498) [junit4] 2> at org.eclipse.jetty.server.bio.SocketConnector.accept(SocketConnector.java:117) [junit4] 2> at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938) [junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) [junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) [junit4] 2> at java.lang.Thread.run(Thread.java:745) [junit4] 2> NOTE: test params are: codec=Asserting(Lucene50), sim=RandomSimilarityProvider(queryNorm=false,coord=no): {}, locale=lv_LV, timezone=US/Michigan [junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_65 (64-bit)/cpus=16,threads=72,free=123046616,total=225968128 [junit4] 2> NOTE: All tests run in this JVM: [TestExceedMaxTermLength, TestSchemaResource, TestSolr4Spatial2, PathHierarchyTokenizerFactoryTest, TestAtomicUpdateErrorCases, TestBinaryField, TestLMDirichletSimilarityFactory, DateFieldTest, RequiredFieldsTest, AutoCommitTest, DebugComponentTest, TestStandardQParsers, DeleteReplicaTest, TestDefaultSearchFieldResource, TestSweetSpotSimilarityFactory, SolrRequestParserTest, ShowFileRequestHandlerTest, SystemInfoHandlerTest, TestQuerySenderNoQuery, DirectUpdateHandlerTest, TestSchemaSimilarityResource, ZkControllerTest, FastVectorHighlighterTest, TestCodecSupport, CachingDirectoryFactoryTest, HdfsBasicDistributedZkTest, SolrCoreCheckLockOnStartupTest, TestCollationField, TestSolrConfigHandler, SuggestComponentTest, TestDistribDocBasedVersion, TriLevelCompositeIdRoutingTest, InfoHandlerTest, TestRecovery, TestManagedStopFilterFactory, TestCollectionAPI, TestFieldTypeCollectionResource, TestDocumentBuilder, ClusterStateTest, CurrencyFieldOpenExchangeTest, BlockCacheTest, TestFreeTextSuggestions, ConnectionManagerTest, SolrCoreTest, NoCacheHeaderTest, PreAnalyzedFieldTest, TestInitParams, DocValuesMissingTest, TestLFUCache, ChaosMonkeySafeLeaderTest] [junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory. [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=ChaosMonkeySafeLeaderTest -Dtests.seed=FEA2EACA01B8699E -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=lv_LV -Dtests.timezone=US/Michigan -Dtests.asserts=true -Dtests.file.encoding=US-ASCII [junit4] ERROR 0.00s J1 | ChaosMonkeySafeLeaderTest (suite) <<< [junit4] > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec). [junit4] > at __randomizedtesting.SeedInfo.seed([FEA2EACA01B8699E]:0) [junit4] Completed on J1 in 7224.57s, 1 test, 2 errors <<< FAILURES! [...truncated 25 lines...] BUILD FAILED /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:532: The following error occurred while executing this line: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:473: The following error occurred while executing this line: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:61: The following error occurred while executing this line: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/extra-targets.xml:39: The following error occurred while executing this line: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build.xml:189: The following error occurred while executing this line: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/common-build.xml:502: The following error occurred while executing this line: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:1358: The following error occurred while executing this line: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:965: There were test failures: 447 suites, 1820 tests, 1 suite-level error, 2 errors, 2 failures, 75 ignored (9 assumptions) Total time: 273 minutes 8 seconds Build step 'Invoke Ant' marked build as failure Archiving artifacts Sending artifact delta relative to Lucene-Solr-NightlyTests-trunk #675 Archived 5 artifacts Archive block size is 32768 Received 0 blocks and 841793481 bytes Compression is 0.0% Took 2 min 33 sec Recording test results Email was triggered for: Failure Sending email for trigger: Failure
--------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org