Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Linux/839/ Java: 64bit/jdk-9-ea+121 -XX:+UseCompressedOops -XX:+UseParallelGC
17 tests failed. FAILED: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.test Error Message: Error from server at http://127.0.0.1:45528/collection1: Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 500 </title> </head> <body> <h2>HTTP ERROR: 500</h2> <p>Problem accessing /collection1/update. Reason: <pre> {metadata={error-class=org.apache.solr.common.SolrException,root-error-class=org.apache.solr.common.SolrException},msg=SolrCore 'collection1' is not available due to init failure: Timed out getting coreNodeName for collection1,trace=org.apache.solr.common.SolrException: SolrCore 'collection1' is not available due to init failure: Timed out getting coreNodeName for collection1 at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:1080) at org.apache.solr.servlet.HttpSolrCall.init(HttpSolrCall.java:256) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:418) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:399) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) at org.eclipse.jetty.server.Server.handle(Server.java:518) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) Caused by: org.apache.solr.common.SolrException: Timed out getting coreNodeName for collection1 at org.apache.solr.cloud.ZkController.waitForCoreNodeName(ZkController.java:1387) at org.apache.solr.cloud.ZkController.doGetShardIdAndNodeNameProcess(ZkController.java:1365) at org.apache.solr.cloud.ZkController.preRegister(ZkController.java:1456) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:809) at org.apache.solr.core.CoreContainer.lambda$load$0(CoreContainer.java:468) at java.util.concurrent.FutureTask.run(java.base@9-ea/FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229) at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@9-ea/ThreadPoolExecutor.java:1158) at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@9-ea/ThreadPoolExecutor.java:632) ... 1 more ,code=500}</pre></p> <hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.8.v20160314</a><hr/> </body> </html> Stack Trace: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:45528/collection1: Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 500 </title> </head> <body> <h2>HTTP ERROR: 500</h2> <p>Problem accessing /collection1/update. Reason: <pre> {metadata={error-class=org.apache.solr.common.SolrException,root-error-class=org.apache.solr.common.SolrException},msg=SolrCore 'collection1' is not available due to init failure: Timed out getting coreNodeName for collection1,trace=org.apache.solr.common.SolrException: SolrCore 'collection1' is not available due to init failure: Timed out getting coreNodeName for collection1 at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:1080) at org.apache.solr.servlet.HttpSolrCall.init(HttpSolrCall.java:256) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:418) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:399) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) at org.eclipse.jetty.server.Server.handle(Server.java:518) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) Caused by: org.apache.solr.common.SolrException: Timed out getting coreNodeName for collection1 at org.apache.solr.cloud.ZkController.waitForCoreNodeName(ZkController.java:1387) at org.apache.solr.cloud.ZkController.doGetShardIdAndNodeNameProcess(ZkController.java:1365) at org.apache.solr.cloud.ZkController.preRegister(ZkController.java:1456) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:809) at org.apache.solr.core.CoreContainer.lambda$load$0(CoreContainer.java:468) at java.util.concurrent.FutureTask.run(java.base@9-ea/FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229) at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@9-ea/ThreadPoolExecutor.java:1158) at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@9-ea/ThreadPoolExecutor.java:632) ... 1 more ,code=500}</pre></p> <hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.8.v20160314</a><hr/> </body> </html> at __randomizedtesting.SeedInfo.seed([B721922F7358ADDD:3F75ADF5DDA4C025]:0) at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:558) at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259) at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248) at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149) at org.apache.solr.client.solrj.SolrClient.deleteByQuery(SolrClient.java:895) at org.apache.solr.client.solrj.SolrClient.deleteByQuery(SolrClient.java:858) at org.apache.solr.client.solrj.SolrClient.deleteByQuery(SolrClient.java:873) at org.apache.solr.cloud.AbstractFullDistribZkTestBase.del(AbstractFullDistribZkTestBase.java:835) at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.tryDelete(ChaosMonkeySafeLeaderTest.java:190) at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.test(ChaosMonkeySafeLeaderTest.java:92) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@9-ea/Native Method) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@9-ea/NativeMethodAccessorImpl.java:62) at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@9-ea/DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(java.base@9-ea/Method.java:531) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) FAILED: org.apache.solr.cloud.SimpleCollectionCreateDeleteTest.test Error Message: No registered leader was found after waiting for 10000ms , collection: collection1 slice: shard1 Stack Trace: org.apache.solr.common.SolrException: No registered leader was found after waiting for 10000ms , collection: collection1 slice: shard1 at __randomizedtesting.SeedInfo.seed([B721922F7358ADDD:3F75ADF5DDA4C025]:0) at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:718) at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createJettys(AbstractFullDistribZkTestBase.java:427) at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:332) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:990) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) FAILED: junit.framework.TestSuite.org.apache.solr.cloud.LeaderElectionIntegrationTest Error Message: 17 threads leaked from SUITE scope at org.apache.solr.cloud.LeaderElectionIntegrationTest: 1) Thread[id=1081, name=TEST-LeaderElectionIntegrationTest.testSimpleSliceLeaderElection-seed#[B721922F7358ADDD]-SendThread(127.0.0.1:42177), state=TIMED_WAITING, group=TGRP-LeaderElectionIntegrationTest] at java.lang.Thread.sleep(java.base@9-ea/Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003) 2) Thread[id=1060, name=Thread-389, state=WAITING, group=TGRP-LeaderElectionIntegrationTest] at java.lang.Object.wait(java.base@9-ea/Native Method) at java.lang.Object.wait(java.base@9-ea/Object.java:516) at org.apache.solr.core.CloserThread.run(CoreContainer.java:1253) 3) Thread[id=1054, name=TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[B721922F7358ADDD]-SendThread(127.0.0.1:34628), state=TIMED_WAITING, group=TGRP-LeaderElectionIntegrationTest] at java.lang.Thread.sleep(java.base@9-ea/Native Method) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994) 4) Thread[id=1082, name=TEST-LeaderElectionIntegrationTest.testSimpleSliceLeaderElection-seed#[B721922F7358ADDD]-EventThread, state=WAITING, group=TGRP-LeaderElectionIntegrationTest] at jdk.internal.misc.Unsafe.park(java.base@9-ea/Native Method) at java.util.concurrent.locks.LockSupport.park(java.base@9-ea/LockSupport.java:190) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@9-ea/AbstractQueuedSynchronizer.java:2064) at java.util.concurrent.LinkedBlockingQueue.take(java.base@9-ea/LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494) 5) Thread[id=1086, name=OverseerHdfsCoreFailoverThread-96026543810281475-127.0.0.1:7000_solr-n_0000000000, state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.] at java.lang.Thread.sleep(java.base@9-ea/Native Method) at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) 6) Thread[id=1059, name=OverseerHdfsCoreFailoverThread-96026522827489283-127.0.0.1:7000_solr-n_0000000000, state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.] at java.lang.Thread.sleep(java.base@9-ea/Native Method) at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) 7) Thread[id=1091, name=zkCallback-183-thread-4, state=TIMED_WAITING, group=TGRP-LeaderElectionIntegrationTest] at jdk.internal.misc.Unsafe.park(java.base@9-ea/Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(java.base@9-ea/LockSupport.java:230) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(java.base@9-ea/SynchronousQueue.java:461) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(java.base@9-ea/SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(java.base@9-ea/SynchronousQueue.java:937) at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@9-ea/ThreadPoolExecutor.java:1082) at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@9-ea/ThreadPoolExecutor.java:1143) at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@9-ea/ThreadPoolExecutor.java:632) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) 8) Thread[id=1057, name=OverseerStateUpdate-96026522827489283-127.0.0.1:7000_solr-n_0000000000, state=TIMED_WAITING, group=Overseer state updater.] at java.lang.Thread.sleep(java.base@9-ea/Native Method) at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:102) at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:70) at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:345) at org.apache.solr.cloud.Overseer$ClusterStateUpdater.amILeader(Overseer.java:418) at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:145) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) 9) Thread[id=1085, name=OverseerCollectionConfigSetProcessor-96026543810281475-127.0.0.1:7000_solr-n_0000000000, state=TIMED_WAITING, group=Overseer collection creation process.] at java.lang.Thread.sleep(java.base@9-ea/Native Method) at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:102) at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:70) at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:345) at org.apache.solr.cloud.OverseerTaskProcessor.amILeader(OverseerTaskProcessor.java:328) at org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:167) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) 10) Thread[id=1058, name=OverseerCollectionConfigSetProcessor-96026522827489283-127.0.0.1:7000_solr-n_0000000000, state=TIMED_WAITING, group=Overseer collection creation process.] at java.lang.Thread.sleep(java.base@9-ea/Native Method) at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:102) at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:70) at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:345) at org.apache.solr.cloud.OverseerTaskProcessor.amILeader(OverseerTaskProcessor.java:328) at org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:167) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) 11) Thread[id=1087, name=Thread-391, state=WAITING, group=TGRP-LeaderElectionIntegrationTest] at java.lang.Object.wait(java.base@9-ea/Native Method) at java.lang.Object.wait(java.base@9-ea/Object.java:516) at org.apache.solr.core.CloserThread.run(CoreContainer.java:1253) 12) Thread[id=1093, name=zkCallback-183-thread-6, state=TIMED_WAITING, group=TGRP-LeaderElectionIntegrationTest] at jdk.internal.misc.Unsafe.park(java.base@9-ea/Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(java.base@9-ea/LockSupport.java:230) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(java.base@9-ea/SynchronousQueue.java:461) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(java.base@9-ea/SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(java.base@9-ea/SynchronousQueue.java:937) at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@9-ea/ThreadPoolExecutor.java:1082) at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@9-ea/ThreadPoolExecutor.java:1143) at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@9-ea/ThreadPoolExecutor.java:632) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) 13) Thread[id=1055, name=TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[B721922F7358ADDD]-EventThread, state=WAITING, group=TGRP-LeaderElectionIntegrationTest] at jdk.internal.misc.Unsafe.park(java.base@9-ea/Native Method) at java.util.concurrent.locks.LockSupport.park(java.base@9-ea/LockSupport.java:190) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@9-ea/AbstractQueuedSynchronizer.java:2064) at java.util.concurrent.LinkedBlockingQueue.take(java.base@9-ea/LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494) 14) Thread[id=1084, name=OverseerStateUpdate-96026543810281475-127.0.0.1:7000_solr-n_0000000000, state=TIMED_WAITING, group=Overseer state updater.] at java.lang.Thread.sleep(java.base@9-ea/Native Method) at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:102) at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:70) at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:328) at org.apache.solr.cloud.DistributedQueue.fetchZkChildren(DistributedQueue.java:309) at org.apache.solr.cloud.DistributedQueue.firstChild(DistributedQueue.java:288) at org.apache.solr.cloud.DistributedQueue.firstElement(DistributedQueue.java:357) at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:164) at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:142) at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:196) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) 15) Thread[id=1089, name=zkCallback-183-thread-2, state=TIMED_WAITING, group=TGRP-LeaderElectionIntegrationTest] at jdk.internal.misc.Unsafe.park(java.base@9-ea/Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(java.base@9-ea/LockSupport.java:230) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(java.base@9-ea/SynchronousQueue.java:461) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(java.base@9-ea/SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(java.base@9-ea/SynchronousQueue.java:937) at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@9-ea/ThreadPoolExecutor.java:1082) at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@9-ea/ThreadPoolExecutor.java:1143) at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@9-ea/ThreadPoolExecutor.java:632) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) 16) Thread[id=1090, name=zkCallback-183-thread-3, state=TIMED_WAITING, group=TGRP-LeaderElectionIntegrationTest] at jdk.internal.misc.Unsafe.park(java.base@9-ea/Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(java.base@9-ea/LockSupport.java:230) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(java.base@9-ea/SynchronousQueue.java:461) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(java.base@9-ea/SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(java.base@9-ea/SynchronousQueue.java:937) at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@9-ea/ThreadPoolExecutor.java:1082) at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@9-ea/ThreadPoolExecutor.java:1143) at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@9-ea/ThreadPoolExecutor.java:632) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) 17) Thread[id=1092, name=zkCallback-183-thread-5, state=TIMED_WAITING, group=TGRP-LeaderElectionIntegrationTest] at jdk.internal.misc.Unsafe.park(java.base@9-ea/Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(java.base@9-ea/LockSupport.java:230) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(java.base@9-ea/SynchronousQueue.java:461) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(java.base@9-ea/SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(java.base@9-ea/SynchronousQueue.java:937) at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@9-ea/ThreadPoolExecutor.java:1082) at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@9-ea/ThreadPoolExecutor.java:1143) at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@9-ea/ThreadPoolExecutor.java:632) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) Stack Trace: com.carrotsearch.randomizedtesting.ThreadLeakError: 17 threads leaked from SUITE scope at org.apache.solr.cloud.LeaderElectionIntegrationTest: 1) Thread[id=1081, name=TEST-LeaderElectionIntegrationTest.testSimpleSliceLeaderElection-seed#[B721922F7358ADDD]-SendThread(127.0.0.1:42177), state=TIMED_WAITING, group=TGRP-LeaderElectionIntegrationTest] at java.lang.Thread.sleep(java.base@9-ea/Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003) 2) Thread[id=1060, name=Thread-389, state=WAITING, group=TGRP-LeaderElectionIntegrationTest] at java.lang.Object.wait(java.base@9-ea/Native Method) at java.lang.Object.wait(java.base@9-ea/Object.java:516) at org.apache.solr.core.CloserThread.run(CoreContainer.java:1253) 3) Thread[id=1054, name=TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[B721922F7358ADDD]-SendThread(127.0.0.1:34628), state=TIMED_WAITING, group=TGRP-LeaderElectionIntegrationTest] at java.lang.Thread.sleep(java.base@9-ea/Native Method) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994) 4) Thread[id=1082, name=TEST-LeaderElectionIntegrationTest.testSimpleSliceLeaderElection-seed#[B721922F7358ADDD]-EventThread, state=WAITING, group=TGRP-LeaderElectionIntegrationTest] at jdk.internal.misc.Unsafe.park(java.base@9-ea/Native Method) at java.util.concurrent.locks.LockSupport.park(java.base@9-ea/LockSupport.java:190) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@9-ea/AbstractQueuedSynchronizer.java:2064) at java.util.concurrent.LinkedBlockingQueue.take(java.base@9-ea/LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494) 5) Thread[id=1086, name=OverseerHdfsCoreFailoverThread-96026543810281475-127.0.0.1:7000_solr-n_0000000000, state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.] at java.lang.Thread.sleep(java.base@9-ea/Native Method) at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) 6) Thread[id=1059, name=OverseerHdfsCoreFailoverThread-96026522827489283-127.0.0.1:7000_solr-n_0000000000, state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.] at java.lang.Thread.sleep(java.base@9-ea/Native Method) at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) 7) Thread[id=1091, name=zkCallback-183-thread-4, state=TIMED_WAITING, group=TGRP-LeaderElectionIntegrationTest] at jdk.internal.misc.Unsafe.park(java.base@9-ea/Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(java.base@9-ea/LockSupport.java:230) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(java.base@9-ea/SynchronousQueue.java:461) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(java.base@9-ea/SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(java.base@9-ea/SynchronousQueue.java:937) at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@9-ea/ThreadPoolExecutor.java:1082) at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@9-ea/ThreadPoolExecutor.java:1143) at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@9-ea/ThreadPoolExecutor.java:632) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) 8) Thread[id=1057, name=OverseerStateUpdate-96026522827489283-127.0.0.1:7000_solr-n_0000000000, state=TIMED_WAITING, group=Overseer state updater.] at java.lang.Thread.sleep(java.base@9-ea/Native Method) at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:102) at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:70) at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:345) at org.apache.solr.cloud.Overseer$ClusterStateUpdater.amILeader(Overseer.java:418) at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:145) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) 9) Thread[id=1085, name=OverseerCollectionConfigSetProcessor-96026543810281475-127.0.0.1:7000_solr-n_0000000000, state=TIMED_WAITING, group=Overseer collection creation process.] at java.lang.Thread.sleep(java.base@9-ea/Native Method) at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:102) at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:70) at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:345) at org.apache.solr.cloud.OverseerTaskProcessor.amILeader(OverseerTaskProcessor.java:328) at org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:167) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) 10) Thread[id=1058, name=OverseerCollectionConfigSetProcessor-96026522827489283-127.0.0.1:7000_solr-n_0000000000, state=TIMED_WAITING, group=Overseer collection creation process.] at java.lang.Thread.sleep(java.base@9-ea/Native Method) at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:102) at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:70) at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:345) at org.apache.solr.cloud.OverseerTaskProcessor.amILeader(OverseerTaskProcessor.java:328) at org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:167) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) 11) Thread[id=1087, name=Thread-391, state=WAITING, group=TGRP-LeaderElectionIntegrationTest] at java.lang.Object.wait(java.base@9-ea/Native Method) at java.lang.Object.wait(java.base@9-ea/Object.java:516) at org.apache.solr.core.CloserThread.run(CoreContainer.java:1253) 12) Thread[id=1093, name=zkCallback-183-thread-6, state=TIMED_WAITING, group=TGRP-LeaderElectionIntegrationTest] at jdk.internal.misc.Unsafe.park(java.base@9-ea/Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(java.base@9-ea/LockSupport.java:230) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(java.base@9-ea/SynchronousQueue.java:461) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(java.base@9-ea/SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(java.base@9-ea/SynchronousQueue.java:937) at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@9-ea/ThreadPoolExecutor.java:1082) at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@9-ea/ThreadPoolExecutor.java:1143) at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@9-ea/ThreadPoolExecutor.java:632) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) 13) Thread[id=1055, name=TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[B721922F7358ADDD]-EventThread, state=WAITING, group=TGRP-LeaderElectionIntegrationTest] at jdk.internal.misc.Unsafe.park(java.base@9-ea/Native Method) at java.util.concurrent.locks.LockSupport.park(java.base@9-ea/LockSupport.java:190) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@9-ea/AbstractQueuedSynchronizer.java:2064) at java.util.concurrent.LinkedBlockingQueue.take(java.base@9-ea/LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494) 14) Thread[id=1084, name=OverseerStateUpdate-96026543810281475-127.0.0.1:7000_solr-n_0000000000, state=TIMED_WAITING, group=Overseer state updater.] at java.lang.Thread.sleep(java.base@9-ea/Native Method) at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:102) at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:70) at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:328) at org.apache.solr.cloud.DistributedQueue.fetchZkChildren(DistributedQueue.java:309) at org.apache.solr.cloud.DistributedQueue.firstChild(DistributedQueue.java:288) at org.apache.solr.cloud.DistributedQueue.firstElement(DistributedQueue.java:357) at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:164) at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:142) at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:196) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) 15) Thread[id=1089, name=zkCallback-183-thread-2, state=TIMED_WAITING, group=TGRP-LeaderElectionIntegrationTest] at jdk.internal.misc.Unsafe.park(java.base@9-ea/Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(java.base@9-ea/LockSupport.java:230) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(java.base@9-ea/SynchronousQueue.java:461) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(java.base@9-ea/SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(java.base@9-ea/SynchronousQueue.java:937) at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@9-ea/ThreadPoolExecutor.java:1082) at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@9-ea/ThreadPoolExecutor.java:1143) at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@9-ea/ThreadPoolExecutor.java:632) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) 16) Thread[id=1090, name=zkCallback-183-thread-3, state=TIMED_WAITING, group=TGRP-LeaderElectionIntegrationTest] at jdk.internal.misc.Unsafe.park(java.base@9-ea/Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(java.base@9-ea/LockSupport.java:230) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(java.base@9-ea/SynchronousQueue.java:461) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(java.base@9-ea/SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(java.base@9-ea/SynchronousQueue.java:937) at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@9-ea/ThreadPoolExecutor.java:1082) at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@9-ea/ThreadPoolExecutor.java:1143) at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@9-ea/ThreadPoolExecutor.java:632) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) 17) Thread[id=1092, name=zkCallback-183-thread-5, state=TIMED_WAITING, group=TGRP-LeaderElectionIntegrationTest] at jdk.internal.misc.Unsafe.park(java.base@9-ea/Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(java.base@9-ea/LockSupport.java:230) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(java.base@9-ea/SynchronousQueue.java:461) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(java.base@9-ea/SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(java.base@9-ea/SynchronousQueue.java:937) at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@9-ea/ThreadPoolExecutor.java:1082) at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@9-ea/ThreadPoolExecutor.java:1143) at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@9-ea/ThreadPoolExecutor.java:632) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) at __randomizedtesting.SeedInfo.seed([B721922F7358ADDD]:0) FAILED: junit.framework.TestSuite.org.apache.solr.cloud.LeaderElectionIntegrationTest Error Message: There are still zombie threads that couldn't be terminated: 1) Thread[id=1081, name=TEST-LeaderElectionIntegrationTest.testSimpleSliceLeaderElection-seed#[B721922F7358ADDD]-SendThread(127.0.0.1:42177), state=TIMED_WAITING, group=TGRP-LeaderElectionIntegrationTest] at java.lang.Thread.sleep(java.base@9-ea/Native Method) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994) 2) Thread[id=1087, name=Thread-391, state=WAITING, group=TGRP-LeaderElectionIntegrationTest] at java.lang.Object.wait(java.base@9-ea/Native Method) at java.lang.Object.wait(java.base@9-ea/Object.java:516) at org.apache.solr.core.CloserThread.run(CoreContainer.java:1253) 3) Thread[id=1093, name=zkCallback-183-thread-6, state=TIMED_WAITING, group=TGRP-LeaderElectionIntegrationTest] at jdk.internal.misc.Unsafe.park(java.base@9-ea/Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(java.base@9-ea/LockSupport.java:230) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(java.base@9-ea/SynchronousQueue.java:461) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(java.base@9-ea/SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(java.base@9-ea/SynchronousQueue.java:937) at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@9-ea/ThreadPoolExecutor.java:1082) at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@9-ea/ThreadPoolExecutor.java:1143) at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@9-ea/ThreadPoolExecutor.java:632) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) 4) Thread[id=1060, name=Thread-389, state=WAITING, group=TGRP-LeaderElectionIntegrationTest] at java.lang.Object.wait(java.base@9-ea/Native Method) at java.lang.Object.wait(java.base@9-ea/Object.java:516) at org.apache.solr.core.CloserThread.run(CoreContainer.java:1253) 5) Thread[id=1054, name=TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[B721922F7358ADDD]-SendThread(127.0.0.1:34628), state=TIMED_WAITING, group=TGRP-LeaderElectionIntegrationTest] at java.lang.Thread.sleep(java.base@9-ea/Native Method) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994) 6) Thread[id=1086, name=OverseerHdfsCoreFailoverThread-96026543810281475-127.0.0.1:7000_solr-n_0000000000, state=RUNNABLE, group=Overseer Hdfs SolrCore Failover Thread.] at java.lang.Thread.sleep(java.base@9-ea/Native Method) at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) 7) Thread[id=1059, name=OverseerHdfsCoreFailoverThread-96026522827489283-127.0.0.1:7000_solr-n_0000000000, state=RUNNABLE, group=Overseer Hdfs SolrCore Failover Thread.] at java.lang.Thread.sleep(java.base@9-ea/Native Method) at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) 8) Thread[id=1091, name=zkCallback-183-thread-4, state=TIMED_WAITING, group=TGRP-LeaderElectionIntegrationTest] at jdk.internal.misc.Unsafe.park(java.base@9-ea/Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(java.base@9-ea/LockSupport.java:230) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(java.base@9-ea/SynchronousQueue.java:461) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(java.base@9-ea/SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(java.base@9-ea/SynchronousQueue.java:937) at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@9-ea/ThreadPoolExecutor.java:1082) at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@9-ea/ThreadPoolExecutor.java:1143) at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@9-ea/ThreadPoolExecutor.java:632) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) 9) Thread[id=1089, name=zkCallback-183-thread-2, state=TIMED_WAITING, group=TGRP-LeaderElectionIntegrationTest] at jdk.internal.misc.Unsafe.park(java.base@9-ea/Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(java.base@9-ea/LockSupport.java:230) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(java.base@9-ea/SynchronousQueue.java:461) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(java.base@9-ea/SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(java.base@9-ea/SynchronousQueue.java:937) at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@9-ea/ThreadPoolExecutor.java:1082) at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@9-ea/ThreadPoolExecutor.java:1143) at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@9-ea/ThreadPoolExecutor.java:632) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) 10) Thread[id=1090, name=zkCallback-183-thread-3, state=TIMED_WAITING, group=TGRP-LeaderElectionIntegrationTest] at jdk.internal.misc.Unsafe.park(java.base@9-ea/Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(java.base@9-ea/LockSupport.java:230) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(java.base@9-ea/SynchronousQueue.java:461) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(java.base@9-ea/SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(java.base@9-ea/SynchronousQueue.java:937) at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@9-ea/ThreadPoolExecutor.java:1082) at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@9-ea/ThreadPoolExecutor.java:1143) at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@9-ea/ThreadPoolExecutor.java:632) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) 11) Thread[id=1092, name=zkCallback-183-thread-5, state=TIMED_WAITING, group=TGRP-LeaderElectionIntegrationTest] at jdk.internal.misc.Unsafe.park(java.base@9-ea/Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(java.base@9-ea/LockSupport.java:230) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(java.base@9-ea/SynchronousQueue.java:461) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(java.base@9-ea/SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(java.base@9-ea/SynchronousQueue.java:937) at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@9-ea/ThreadPoolExecutor.java:1082) at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@9-ea/ThreadPoolExecutor.java:1143) at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@9-ea/ThreadPoolExecutor.java:632) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) Stack Trace: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated: 1) Thread[id=1081, name=TEST-LeaderElectionIntegrationTest.testSimpleSliceLeaderElection-seed#[B721922F7358ADDD]-SendThread(127.0.0.1:42177), state=TIMED_WAITING, group=TGRP-LeaderElectionIntegrationTest] at java.lang.Thread.sleep(java.base@9-ea/Native Method) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994) 2) Thread[id=1087, name=Thread-391, state=WAITING, group=TGRP-LeaderElectionIntegrationTest] at java.lang.Object.wait(java.base@9-ea/Native Method) at java.lang.Object.wait(java.base@9-ea/Object.java:516) at org.apache.solr.core.CloserThread.run(CoreContainer.java:1253) 3) Thread[id=1093, name=zkCallback-183-thread-6, state=TIMED_WAITING, group=TGRP-LeaderElectionIntegrationTest] at jdk.internal.misc.Unsafe.park(java.base@9-ea/Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(java.base@9-ea/LockSupport.java:230) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(java.base@9-ea/SynchronousQueue.java:461) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(java.base@9-ea/SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(java.base@9-ea/SynchronousQueue.java:937) at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@9-ea/ThreadPoolExecutor.java:1082) at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@9-ea/ThreadPoolExecutor.java:1143) at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@9-ea/ThreadPoolExecutor.java:632) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) 4) Thread[id=1060, name=Thread-389, state=WAITING, group=TGRP-LeaderElectionIntegrationTest] at java.lang.Object.wait(java.base@9-ea/Native Method) at java.lang.Object.wait(java.base@9-ea/Object.java:516) at org.apache.solr.core.CloserThread.run(CoreContainer.java:1253) 5) Thread[id=1054, name=TEST-LeaderElectionIntegrationTest.testLeaderElectionAfterClientTimeout-seed#[B721922F7358ADDD]-SendThread(127.0.0.1:34628), state=TIMED_WAITING, group=TGRP-LeaderElectionIntegrationTest] at java.lang.Thread.sleep(java.base@9-ea/Native Method) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994) 6) Thread[id=1086, name=OverseerHdfsCoreFailoverThread-96026543810281475-127.0.0.1:7000_solr-n_0000000000, state=RUNNABLE, group=Overseer Hdfs SolrCore Failover Thread.] at java.lang.Thread.sleep(java.base@9-ea/Native Method) at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) 7) Thread[id=1059, name=OverseerHdfsCoreFailoverThread-96026522827489283-127.0.0.1:7000_solr-n_0000000000, state=RUNNABLE, group=Overseer Hdfs SolrCore Failover Thread.] at java.lang.Thread.sleep(java.base@9-ea/Native Method) at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) 8) Thread[id=1091, name=zkCallback-183-thread-4, state=TIMED_WAITING, group=TGRP-LeaderElectionIntegrationTest] at jdk.internal.misc.Unsafe.park(java.base@9-ea/Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(java.base@9-ea/LockSupport.java:230) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(java.base@9-ea/SynchronousQueue.java:461) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(java.base@9-ea/SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(java.base@9-ea/SynchronousQueue.java:937) at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@9-ea/ThreadPoolExecutor.java:1082) at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@9-ea/ThreadPoolExecutor.java:1143) at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@9-ea/ThreadPoolExecutor.java:632) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) 9) Thread[id=1089, name=zkCallback-183-thread-2, state=TIMED_WAITING, group=TGRP-LeaderElectionIntegrationTest] at jdk.internal.misc.Unsafe.park(java.base@9-ea/Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(java.base@9-ea/LockSupport.java:230) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(java.base@9-ea/SynchronousQueue.java:461) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(java.base@9-ea/SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(java.base@9-ea/SynchronousQueue.java:937) at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@9-ea/ThreadPoolExecutor.java:1082) at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@9-ea/ThreadPoolExecutor.java:1143) at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@9-ea/ThreadPoolExecutor.java:632) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) 10) Thread[id=1090, name=zkCallback-183-thread-3, state=TIMED_WAITING, group=TGRP-LeaderElectionIntegrationTest] at jdk.internal.misc.Unsafe.park(java.base@9-ea/Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(java.base@9-ea/LockSupport.java:230) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(java.base@9-ea/SynchronousQueue.java:461) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(java.base@9-ea/SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(java.base@9-ea/SynchronousQueue.java:937) at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@9-ea/ThreadPoolExecutor.java:1082) at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@9-ea/ThreadPoolExecutor.java:1143) at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@9-ea/ThreadPoolExecutor.java:632) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) 11) Thread[id=1092, name=zkCallback-183-thread-5, state=TIMED_WAITING, group=TGRP-LeaderElectionIntegrationTest] at jdk.internal.misc.Unsafe.park(java.base@9-ea/Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(java.base@9-ea/LockSupport.java:230) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(java.base@9-ea/SynchronousQueue.java:461) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(java.base@9-ea/SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(java.base@9-ea/SynchronousQueue.java:937) at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@9-ea/ThreadPoolExecutor.java:1082) at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@9-ea/ThreadPoolExecutor.java:1143) at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@9-ea/ThreadPoolExecutor.java:632) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) at __randomizedtesting.SeedInfo.seed([B721922F7358ADDD]:0) FAILED: org.apache.solr.cloud.AliasIntegrationTest.test Error Message: Error from server at https://127.0.0.1:42951/lee/v/collection1: Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 500 </title> </head> <body> <h2>HTTP ERROR: 500</h2> <p>Problem accessing /lee/v/collection1/update. Reason: <pre> {metadata={error-class=org.apache.solr.common.SolrException,root-error-class=org.apache.solr.common.SolrException},msg=SolrCore 'collection1' is not available due to init failure: Timed out getting coreNodeName for collection1,trace=org.apache.solr.common.SolrException: SolrCore 'collection1' is not available due to init failure: Timed out getting coreNodeName for collection1 at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:1080) at org.apache.solr.servlet.HttpSolrCall.init(HttpSolrCall.java:256) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:418) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:399) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) at org.eclipse.jetty.server.Server.handle(Server.java:518) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:186) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) Caused by: org.apache.solr.common.SolrException: Timed out getting coreNodeName for collection1 at org.apache.solr.cloud.ZkController.waitForCoreNodeName(ZkController.java:1387) at org.apache.solr.cloud.ZkController.doGetShardIdAndNodeNameProcess(ZkController.java:1365) at org.apache.solr.cloud.ZkController.preRegister(ZkController.java:1456) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:809) at org.apache.solr.core.CoreContainer.lambda$load$0(CoreContainer.java:468) at java.util.concurrent.FutureTask.run(java.base@9-ea/FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229) at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@9-ea/ThreadPoolExecutor.java:1158) at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@9-ea/ThreadPoolExecutor.java:632) ... 1 more ,code=500}</pre></p> <hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.8.v20160314</a><hr/> </body> </html> Stack Trace: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:42951/lee/v/collection1: Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 500 </title> </head> <body> <h2>HTTP ERROR: 500</h2> <p>Problem accessing /lee/v/collection1/update. Reason: <pre> {metadata={error-class=org.apache.solr.common.SolrException,root-error-class=org.apache.solr.common.SolrException},msg=SolrCore 'collection1' is not available due to init failure: Timed out getting coreNodeName for collection1,trace=org.apache.solr.common.SolrException: SolrCore 'collection1' is not available due to init failure: Timed out getting coreNodeName for collection1 at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:1080) at org.apache.solr.servlet.HttpSolrCall.init(HttpSolrCall.java:256) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:418) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:399) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) at org.eclipse.jetty.server.Server.handle(Server.java:518) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:186) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) Caused by: org.apache.solr.common.SolrException: Timed out getting coreNodeName for collection1 at org.apache.solr.cloud.ZkController.waitForCoreNodeName(ZkController.java:1387) at org.apache.solr.cloud.ZkController.doGetShardIdAndNodeNameProcess(ZkController.java:1365) at org.apache.solr.cloud.ZkController.preRegister(ZkController.java:1456) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:809) at org.apache.solr.core.CoreContainer.lambda$load$0(CoreContainer.java:468) at java.util.concurrent.FutureTask.run(java.base@9-ea/FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229) at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@9-ea/ThreadPoolExecutor.java:1158) at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@9-ea/ThreadPoolExecutor.java:632) ... 1 more ,code=500}</pre></p> <hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.8.v20160314</a><hr/> </body> </html> at __randomizedtesting.SeedInfo.seed([B721922F7358ADDD:3F75ADF5DDA4C025]:0) at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:558) at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259) at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248) at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149) at org.apache.solr.client.solrj.SolrClient.deleteByQuery(SolrClient.java:895) at org.apache.solr.client.solrj.SolrClient.deleteByQuery(SolrClient.java:858) at org.apache.solr.client.solrj.SolrClient.deleteByQuery(SolrClient.java:873) at org.apache.solr.cloud.AbstractFullDistribZkTestBase.del(AbstractFullDistribZkTestBase.java:835) at org.apache.solr.cloud.AliasIntegrationTest.test(AliasIntegrationTest.java:69) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@9-ea/Native Method) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@9-ea/NativeMethodAccessorImpl.java:62) at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@9-ea/DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(java.base@9-ea/Method.java:531) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) FAILED: org.apache.solr.cloud.AsyncMigrateRouteKeyTest.test Error Message: Error from server at http://127.0.0.1:33488/at_jgc/collection1: Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 500 </title> </head> <body> <h2>HTTP ERROR: 500</h2> <p>Problem accessing /at_jgc/collection1/update. Reason: <pre> {metadata={error-class=org.apache.solr.common.SolrException,root-error-class=org.apache.solr.common.SolrException},msg=SolrCore 'collection1' is not available due to init failure: Timed out getting coreNodeName for collection1,trace=org.apache.solr.common.SolrException: SolrCore 'collection1' is not available due to init failure: Timed out getting coreNodeName for collection1 at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:1080) at org.apache.solr.servlet.HttpSolrCall.init(HttpSolrCall.java:256) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:418) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:399) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) at org.eclipse.jetty.server.Server.handle(Server.java:518) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) Caused by: org.apache.solr.common.SolrException: Timed out getting coreNodeName for collection1 at org.apache.solr.cloud.ZkController.waitForCoreNodeName(ZkController.java:1387) at org.apache.solr.cloud.ZkController.doGetShardIdAndNodeNameProcess(ZkController.java:1365) at org.apache.solr.cloud.ZkController.preRegister(ZkController.java:1456) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:809) at org.apache.solr.core.CoreContainer.lambda$load$0(CoreContainer.java:468) at java.util.concurrent.FutureTask.run(java.base@9-ea/FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229) at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@9-ea/ThreadPoolExecutor.java:1158) at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@9-ea/ThreadPoolExecutor.java:632) ... 1 more ,code=500}</pre></p> <hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.8.v20160314</a><hr/> </body> </html> Stack Trace: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:33488/at_jgc/collection1: Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 500 </title> </head> <body> <h2>HTTP ERROR: 500</h2> <p>Problem accessing /at_jgc/collection1/update. Reason: <pre> {metadata={error-class=org.apache.solr.common.SolrException,root-error-class=org.apache.solr.common.SolrException},msg=SolrCore 'collection1' is not available due to init failure: Timed out getting coreNodeName for collection1,trace=org.apache.solr.common.SolrException: SolrCore 'collection1' is not available due to init failure: Timed out getting coreNodeName for collection1 at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:1080) at org.apache.solr.servlet.HttpSolrCall.init(HttpSolrCall.java:256) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:418) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:399) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) at org.eclipse.jetty.server.Server.handle(Server.java:518) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) Caused by: org.apache.solr.common.SolrException: Timed out getting coreNodeName for collection1 at org.apache.solr.cloud.ZkController.waitForCoreNodeName(ZkController.java:1387) at org.apache.solr.cloud.ZkController.doGetShardIdAndNodeNameProcess(ZkController.java:1365) at org.apache.solr.cloud.ZkController.preRegister(ZkController.java:1456) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:809) at org.apache.solr.core.CoreContainer.lambda$load$0(CoreContainer.java:468) at java.util.concurrent.FutureTask.run(java.base@9-ea/FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229) at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@9-ea/ThreadPoolExecutor.java:1158) at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@9-ea/ThreadPoolExecutor.java:632) ... 1 more ,code=500}</pre></p> <hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.8.v20160314</a><hr/> </body> </html> at __randomizedtesting.SeedInfo.seed([B721922F7358ADDD:3F75ADF5DDA4C025]:0) at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:558) at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259) at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248) at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149) at org.apache.solr.client.solrj.SolrClient.deleteByQuery(SolrClient.java:895) at org.apache.solr.client.solrj.SolrClient.deleteByQuery(SolrClient.java:858) at org.apache.solr.client.solrj.SolrClient.deleteByQuery(SolrClient.java:873) at org.apache.solr.cloud.AbstractFullDistribZkTestBase.del(AbstractFullDistribZkTestBase.java:835) at org.apache.solr.cloud.MigrateRouteKeyTest.multipleShardMigrateTest(MigrateRouteKeyTest.java:135) at org.apache.solr.cloud.AsyncMigrateRouteKeyTest.test(AsyncMigrateRouteKeyTest.java:44) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@9-ea/Native Method) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@9-ea/NativeMethodAccessorImpl.java:62) at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@9-ea/DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(java.base@9-ea/Method.java:531) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) FAILED: org.apache.solr.cloud.BasicDistributedZk2Test.test Error Message: Error from server at http://127.0.0.1:34084: Error CREATEing SolrCore 'onenodecollectioncore': Unable to create core [onenodecollectioncore] Caused by: Timed out getting coreNodeName for onenodecollectioncore Stack Trace: java.lang.AssertionError: Error from server at http://127.0.0.1:34084: Error CREATEing SolrCore 'onenodecollectioncore': Unable to create core [onenodecollectioncore] Caused by: Timed out getting coreNodeName for onenodecollectioncore at __randomizedtesting.SeedInfo.seed([B721922F7358ADDD:3F75ADF5DDA4C025]:0) at org.junit.Assert.fail(Assert.java:93) at org.apache.solr.cloud.BasicDistributedZk2Test.testNodeWithoutCollectionForwarding(BasicDistributedZk2Test.java:162) at org.apache.solr.cloud.BasicDistributedZk2Test.test(BasicDistributedZk2Test.java:71) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@9-ea/Native Method) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@9-ea/NativeMethodAccessorImpl.java:62) at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@9-ea/DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(java.base@9-ea/Method.java:531) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367) at java.lang.Thread.run(java.base@9-ea/Thread.java:843) FAILED: org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.test Error Message: Error from server at http://127.0.0.1:34798/collection1: Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 500 </title> </head> <body> <h2>HTTP ERROR: 500</h2> <p>Problem accessing /collection1/update. Reason: <pre> {metadata={error-class=org.apache.solr.common.SolrException,root-error-class=org.apache.solr.common.SolrException},msg=SolrCore 'collection1' is not available due to init failure: Timed out getting coreNodeName for collection1,trace=org.apache.solr.common.SolrException: SolrCore 'collection1' is not available due to init failure: Timed out getting coreNodeName for collection1 at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:1080) at org.apache.solr.servlet.HttpSolrCall.init(HttpSolrCall.java:256) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:418) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676) at org.apache.solr.client.solrj.embedded.JettySolr [...truncated too long message...] rDispatchFilter.java:208) [junit4] > at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676) [junit4] > at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109) [junit4] > at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676) [junit4] > at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581) [junit4] > at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224) [junit4] > at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160) [junit4] > at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511) [junit4] > at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) [junit4] > at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092) [junit4] > at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) [junit4] > at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:399) [junit4] > at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) [junit4] > at org.eclipse.jetty.server.Server.handle(Server.java:518) [junit4] > at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308) [junit4] > at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244) [junit4] > at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) [junit4] > at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) [junit4] > at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) [junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246) [junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156) [junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654) [junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572) [junit4] > at java.lang.Thread.run(java.base@9-ea/Thread.java:843) [junit4] > Caused by: org.apache.solr.common.SolrException: Timed out getting coreNodeName for collection1 [junit4] > at org.apache.solr.cloud.ZkController.waitForCoreNodeName(ZkController.java:1387) [junit4] > at org.apache.solr.cloud.ZkController.doGetShardIdAndNodeNameProcess(ZkController.java:1365) [junit4] > at org.apache.solr.cloud.ZkController.preRegister(ZkController.java:1456) [junit4] > at org.apache.solr.core.CoreContainer.create(CoreContainer.java:809) [junit4] > at org.apache.solr.core.CoreContainer.lambda$load$0(CoreContainer.java:468) [junit4] > at java.util.concurrent.FutureTask.run(java.base@9-ea/FutureTask.java:266) [junit4] > at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229) [junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@9-ea/ThreadPoolExecutor.java:1158) [junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@9-ea/ThreadPoolExecutor.java:632) [junit4] > ... 1 more [junit4] > ,code=500}</pre></p> [junit4] > <hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.8.v20160314</a><hr/> [junit4] > </body> [junit4] > </html> [junit4] > at __randomizedtesting.SeedInfo.seed([B721922F7358ADDD:3F75ADF5DDA4C025]:0) [junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:558) [junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259) [junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248) [junit4] > at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149) [junit4] > at org.apache.solr.client.solrj.SolrClient.deleteByQuery(SolrClient.java:895) [junit4] > at org.apache.solr.client.solrj.SolrClient.deleteByQuery(SolrClient.java:858) [junit4] > at org.apache.solr.client.solrj.SolrClient.deleteByQuery(SolrClient.java:873) [junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.del(AbstractFullDistribZkTestBase.java:835) [junit4] > at org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.test(ChaosMonkeyNothingIsSafeTest.java:128) [junit4] > at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@9-ea/Native Method) [junit4] > at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@9-ea/NativeMethodAccessorImpl.java:62) [junit4] > at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@9-ea/DelegatingMethodAccessorImpl.java:43) [junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992) [junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967) [junit4] > at java.lang.Thread.run(java.base@9-ea/Thread.java:843) [junit4] 2> 10414172 INFO (SUITE-ChaosMonkeyNothingIsSafeTest-seed#[B721922F7358ADDD]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore [junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_B721922F7358ADDD-001 [junit4] 2> Jun 06, 2016 11:54:29 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks [junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s). [junit4] 2> NOTE: test params are: codec=Asserting(Lucene60): {}, docValues:{}, maxPointsInLeafNode=242, maxMBSortInHeap=5.517363650119625, sim=RandomSimilarity(queryNorm=false,coord=crazy): {}, locale=lu, timezone=Antarctica/Casey [junit4] 2> NOTE: Linux 4.4.0-21-generic amd64/Oracle Corporation 9-ea (64-bit)/cpus=12,threads=1,free=160036344,total=455606272 [junit4] 2> NOTE: All tests run in this JVM: [TestQuerySenderListener, TestStressUserVersions, ChaosMonkeySafeLeaderTest, DistanceFunctionTest, TestScoreJoinQPNoScore, TestConfigSets, TestLegacyFieldCache, FileUtilsTest, TestQueryUtils, TestPivotHelperCode, TestSolr4Spatial2, TestUniqueKeyFieldResource, CollectionsAPIDistributedZkTest, TestChildDocTransformer, TestMiniSolrCloudClusterBase, SpellPossibilityIteratorTest, DirectSolrSpellCheckerTest, DeleteReplicaTest, HdfsWriteToMultipleCollectionsTest, ExitableDirectoryReaderTest, TestFieldTypeCollectionResource, DistributedQueueTest, TestTrie, OpenCloseCoreStressTest, SolrCmdDistributorTest, HighlighterMaxOffsetTest, TestSha256AuthenticationProvider, SharedFSAutoReplicaFailoverUtilsTest, TestLRUStatsCache, TestGeoJSONResponseWriter, HdfsRecoveryZkTest, UniqFieldsUpdateProcessorFactoryTest, HdfsBasicDistributedZkTest, TestAnalyzeInfixSuggestions, TestDistribIDF, AsyncMigrateRouteKeyTest, TestPartialUpdateDeduplication, TestSolrCoreProperties, MultiTermTest, TestFieldCacheReopen, TestDistributedGrouping, OutOfBoxZkACLAndCredentialsProvidersTest, TestCoreContainer, ShardRoutingCustomTest, CSVRequestHandlerTest, TestStressVersions, TestTestInjection, TestUseDocValuesAsStored2, SpellCheckComponentTest, AliasIntegrationTest, TestSubQueryTransformerDistrib, TestFiltering, TolerantUpdateProcessorTest, ZkStateReaderTest, DistributedIntervalFacetingTest, RestartWhileUpdatingTest, SolrXmlInZkTest, NotRequiredUniqueKeyTest, TestFreeTextSuggestions, TestTolerantUpdateProcessorCloud, SuggesterFSTTest, TestCollectionAPI, DistributedExpandComponentTest, SecurityConfHandlerTest, HttpPartitionTest, HLLSerializationTest, TestBlobHandler, TestConfigOverlay, CollectionReloadTest, SolrInfoMBeanTest, TestSurroundQueryParser, DistributedMLTComponentTest, TestManagedSchemaAPI, TestDynamicFieldResource, BlockDirectoryTest, DistributedQueryComponentCustomSortTest, TestFieldSortValues, RecoveryAfterSoftCommitTest, TestDownShardTolerantSearch, ShardRoutingTest, TestSolrDeletionPolicy2, TestCursorMarkWithoutUniqueKey, BadCopyFieldTest, DOMUtilTest, BasicFunctionalityTest, TestRestManager, TestCloudDeleteByQuery, TestReplicaProperties, TestSolrConfigHandlerCloud, TestInfoStreamLogging, HdfsCollectionsAPIDistributedZkTest, TestQuerySenderNoQuery, ChaosMonkeyNothingIsSafeTest] [junit4] Completed [607/613 (14!)] on J1 in 612.35s, 1 test, 1 error <<< FAILURES! [...truncated 38 lines...] BUILD FAILED /home/jenkins/workspace/Lucene-Solr-6.x-Linux/build.xml:740: The following error occurred while executing this line: /home/jenkins/workspace/Lucene-Solr-6.x-Linux/build.xml:684: The following error occurred while executing this line: /home/jenkins/workspace/Lucene-Solr-6.x-Linux/build.xml:59: The following error occurred while executing this line: /home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build.xml:233: The following error occurred while executing this line: /home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/common-build.xml:531: The following error occurred while executing this line: /home/jenkins/workspace/Lucene-Solr-6.x-Linux/lucene/common-build.xml:1427: The following error occurred while executing this line: /home/jenkins/workspace/Lucene-Solr-6.x-Linux/lucene/common-build.xml:984: There were test failures: 613 suites (10 ignored), 2567 tests, 2 suite-level errors, 11 errors, 4 failures, 1777 ignored (63 assumptions) [seed: B721922F7358ADDD] Total time: 197 minutes 14 seconds Build step 'Invoke Ant' marked build as failure Archiving artifacts [WARNINGS] Skipping publisher since build result is FAILURE Recording test results Email was triggered for: Failure - Any Sending email for trigger: Failure - Any
--------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org