[JENKINS] Lucene-Solr-5.x-Linux (64bit/jdk1.8.0_20) - Build # 11678 - Failure!

Policeman Jenkins Server Sun, 04 Jan 2015 14:00:42 -0800

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Linux/11678/
Java: 64bit/jdk1.8.0_20 -XX:+UseCompressedOops -XX:+UseSerialGC (asserts: false)


5 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.HttpPartitionTest

Error Message:
ERROR: SolrIndexSearcher opens=4 closes=1

Stack Trace:
java.lang.AssertionError: ERROR: SolrIndexSearcher opens=4 closes=1
        at __randomizedtesting.SeedInfo.seed([D9864374E8E5CEF0]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at 
org.apache.solr.SolrTestCaseJ4.endTrackingSearchers(SolrTestCaseJ4.java:447)
        at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:187)
        at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:483)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:790)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
        at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.HttpPartitionTest

Error Message:
46 threads leaked from SUITE scope at org.apache.solr.cloud.HttpPartitionTest:  
   1) Thread[id=7846, name=qtp859836886-7846 Acceptor1 
SelectChannelConnector@127.0.0.1:53819, state=BLOCKED, 
group=TGRP-HttpPartitionTest]         at 
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:224)     
    at 
org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
         at 
org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) 
        at java.lang.Thread.run(Thread.java:745)    2) Thread[id=7859, 
name=Thread-3293, state=WAITING, group=TGRP-HttpPartitionTest]         at 
java.lang.Object.wait(Native Method)         at 
java.lang.Object.wait(Object.java:502)         at 
org.apache.solr.core.CloserThread.run(CoreContainer.java:929)    3) 
Thread[id=7823, name=qtp1575822333-7823 Selector1, state=RUNNABLE, 
group=TGRP-HttpPartitionTest]         at 
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at 
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at 
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at 
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at 
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at 
org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:569)
         at 
org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)        
 at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) 
        at java.lang.Thread.run(Thread.java:745)    4) Thread[id=7825, 
name=qtp1575822333-7825 Acceptor1 SelectChannelConnector@127.0.0.1:55660, 
state=BLOCKED, group=TGRP-HttpPartitionTest]         at 
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:224)     
    at 
org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
         at 
org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) 
        at java.lang.Thread.run(Thread.java:745)    5) Thread[id=7813, 
name=TEST-HttpPartitionTest.testDistribSearch-seed#[D9864374E8E5CEF0]-EventThread,
 state=WAITING, group=TGRP-HttpPartitionTest]         at 
sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)     
    at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)    
6) Thread[id=7802, name=qtp1406611874-7802 Selector1, state=RUNNABLE, 
group=TGRP-HttpPartitionTest]         at 
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at 
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at 
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at 
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at 
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at 
org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:569)
         at 
org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)        
 at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) 
        at java.lang.Thread.run(Thread.java:745)    7) Thread[id=7845, 
name=qtp859836886-7845 Selector1, state=RUNNABLE, group=TGRP-HttpPartitionTest] 
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at 
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at 
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at 
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at 
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at 
org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:569)
         at 
org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)        
 at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) 
        at java.lang.Thread.run(Thread.java:745)    8) Thread[id=7824, 
name=qtp1575822333-7824 Acceptor0 SelectChannelConnector@127.0.0.1:55660, 
state=RUNNABLE, group=TGRP-HttpPartitionTest]         at 
sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at 
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:241)     
    at 
org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
         at 
org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) 
        at java.lang.Thread.run(Thread.java:745)    9) Thread[id=7805, 
name=qtp1406611874-7805 Acceptor1 SelectChannelConnector@127.0.0.1:40279, 
state=BLOCKED, group=TGRP-HttpPartitionTest]         at 
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:224)     
    at 
org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
         at 
org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) 
        at java.lang.Thread.run(Thread.java:745)   10) Thread[id=7834, 
name=TEST-HttpPartitionTest.testDistribSearch-seed#[D9864374E8E5CEF0]-EventThread,
 state=WAITING, group=TGRP-HttpPartitionTest]         at 
sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)     
    at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)   
11) Thread[id=7850, name=qtp859836886-7850, state=TIMED_WAITING, 
group=TGRP-HttpPartitionTest]         at sun.misc.Unsafe.park(Native Method)    
     at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)  
       at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
         at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)     
    at 
org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572) 
        at java.lang.Thread.run(Thread.java:745)   12) Thread[id=7847, 
name=qtp859836886-7847, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]      
   at sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
         at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)     
    at 
org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572) 
        at java.lang.Thread.run(Thread.java:745)   13) Thread[id=7854, 
name=TEST-HttpPartitionTest.testDistribSearch-seed#[D9864374E8E5CEF0]-SendThread(127.0.0.1:42676),
 state=TIMED_WAITING, group=TGRP-HttpPartitionTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
         at 
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)    
     at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)   
14) Thread[id=7821, name=HashSessionScavenger-194, state=TIMED_WAITING, 
group=TGRP-HttpPartitionTest]         at java.lang.Object.wait(Native Method)   
      at java.util.TimerThread.mainLoop(Timer.java:552)         at 
java.util.TimerThread.run(Timer.java:505)   15) Thread[id=7808, 
name=qtp1406611874-7808, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]     
    at sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
         at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)     
    at 
org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572) 
        at java.lang.Thread.run(Thread.java:745)   16) Thread[id=7837, 
name=searcherExecutor-4165-thread-1, state=WAITING, 
group=TGRP-HttpPartitionTest]         at sun.misc.Unsafe.park(Native Method)    
     at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)       
  at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)     
    at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
        at java.lang.Thread.run(Thread.java:745)   17) Thread[id=7871, 
name=OverseerHdfsCoreFailoverThread-93087894478782471-127.0.0.1:43088_oa_s-n_0000000001,
 state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:108)   
      at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:76)
         at 
org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:311)         
at 
org.apache.solr.common.cloud.ZkStateReader.getClusterProps(ZkStateReader.java:727)
         at 
org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.doWork(OverseerAutoReplicaFailoverThread.java:149)
         at 
org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:128)
         at java.lang.Thread.run(Thread.java:745)   18) Thread[id=7855, 
name=TEST-HttpPartitionTest.testDistribSearch-seed#[D9864374E8E5CEF0]-EventThread,
 state=WAITING, group=TGRP-HttpPartitionTest]         at 
sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)     
    at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)   
19) Thread[id=7840, name=coreZkRegister-4160-thread-1, state=WAITING, 
group=TGRP-HttpPartitionTest]         at sun.misc.Unsafe.park(Native Method)    
     at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)       
  at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)     
    at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
        at java.lang.Thread.run(Thread.java:745)   20) Thread[id=7803, 
name=qtp1406611874-7803, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]     
    at sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
         at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)     
    at 
org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572) 
        at java.lang.Thread.run(Thread.java:745)   21) Thread[id=7870, 
name=OverseerCollectionProcessor-93087894478782471-127.0.0.1:43088_oa_s-n_0000000001,
 state=TIMED_WAITING, group=Overseer collection creation process.]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:108)   
      at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:76)
         at 
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:345)        
 at 
org.apache.solr.cloud.OverseerCollectionProcessor.amILeader(OverseerCollectionProcessor.java:543)
         at 
org.apache.solr.cloud.OverseerCollectionProcessor.run(OverseerCollectionProcessor.java:301)
         at java.lang.Thread.run(Thread.java:745)   22) Thread[id=7804, 
name=qtp1406611874-7804, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]     
    at sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
         at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)     
    at 
org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572) 
        at java.lang.Thread.run(Thread.java:745)   23) Thread[id=7806, 
name=qtp1406611874-7806 Acceptor0 SelectChannelConnector@127.0.0.1:40279, 
state=RUNNABLE, group=TGRP-HttpPartitionTest]         at 
sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at 
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:241)     
    at 
org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
         at 
org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) 
        at java.lang.Thread.run(Thread.java:745)   24) Thread[id=7829, 
name=qtp1575822333-7829, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]     
    at sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
         at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)     
    at 
org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572) 
        at java.lang.Thread.run(Thread.java:745)   25) Thread[id=7822, 
name=qtp1575822333-7822 Selector0, state=RUNNABLE, 
group=TGRP-HttpPartitionTest]         at 
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at 
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at 
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at 
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at 
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at 
org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:569)
         at 
org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)        
 at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) 
        at java.lang.Thread.run(Thread.java:745)   26) Thread[id=7861, 
name=coreZkRegister-4169-thread-1, state=WAITING, group=TGRP-HttpPartitionTest] 
        at sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)     
    at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
        at java.lang.Thread.run(Thread.java:745)   27) Thread[id=7858, 
name=searcherExecutor-4174-thread-1, state=WAITING, 
group=TGRP-HttpPartitionTest]         at sun.misc.Unsafe.park(Native Method)    
     at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)       
  at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)     
    at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
        at java.lang.Thread.run(Thread.java:745)   28) Thread[id=7844, 
name=qtp859836886-7844 Selector0, state=RUNNABLE, group=TGRP-HttpPartitionTest] 
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at 
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at 
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at 
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at 
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at 
org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:569)
         at 
org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)        
 at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) 
        at java.lang.Thread.run(Thread.java:745)   29) Thread[id=7816, 
name=searcherExecutor-4156-thread-1, state=WAITING, 
group=TGRP-HttpPartitionTest]         at sun.misc.Unsafe.park(Native Method)    
     at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)       
  at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)     
    at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
        at java.lang.Thread.run(Thread.java:745)   30) Thread[id=7807, 
name=qtp1406611874-7807, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]     
    at sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
         at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)     
    at 
org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572) 
        at java.lang.Thread.run(Thread.java:745)   31) Thread[id=7828, 
name=qtp1575822333-7828, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]     
    at sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
         at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)     
    at 
org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572) 
        at java.lang.Thread.run(Thread.java:745)   32) Thread[id=7812, 
name=TEST-HttpPartitionTest.testDistribSearch-seed#[D9864374E8E5CEF0]-SendThread(127.0.0.1:42676),
 state=TIMED_WAITING, group=TGRP-HttpPartitionTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994)   33) 
Thread[id=7869, 
name=OverseerStateUpdate-93087894478782471-127.0.0.1:43088_oa_s-n_0000000001, 
state=TIMED_WAITING, group=Overseer state updater.]         at 
java.lang.Object.wait(Native Method)         at 
org.apache.solr.cloud.DistributedQueue$LatchWatcher.await(DistributedQueue.java:273)
         at 
org.apache.solr.cloud.DistributedQueue.getChildren(DistributedQueue.java:317)   
      at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:591) 
        at 
org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:569)         
at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:215)    
     at java.lang.Thread.run(Thread.java:745)   34) Thread[id=7827, 
name=qtp1575822333-7827, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]     
    at sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
         at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)     
    at 
org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572) 
        at java.lang.Thread.run(Thread.java:745)   35) Thread[id=7842, 
name=HashSessionScavenger-195, state=TIMED_WAITING, 
group=TGRP-HttpPartitionTest]         at java.lang.Object.wait(Native Method)   
      at java.util.TimerThread.mainLoop(Timer.java:552)         at 
java.util.TimerThread.run(Timer.java:505)   36) Thread[id=7817, 
name=Thread-3271, state=WAITING, group=TGRP-HttpPartitionTest]         at 
java.lang.Object.wait(Native Method)         at 
java.lang.Object.wait(Object.java:502)         at 
org.apache.solr.core.CloserThread.run(CoreContainer.java:929)   37) 
Thread[id=7838, name=Thread-3282, state=WAITING, group=TGRP-HttpPartitionTest]  
       at java.lang.Object.wait(Native Method)         at 
java.lang.Object.wait(Object.java:502)         at 
org.apache.solr.core.CloserThread.run(CoreContainer.java:929)   38) 
Thread[id=7819, name=coreZkRegister-4151-thread-1, state=WAITING, 
group=TGRP-HttpPartitionTest]         at sun.misc.Unsafe.park(Native Method)    
     at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)       
  at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)     
    at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
        at java.lang.Thread.run(Thread.java:745)   39) Thread[id=7849, 
name=qtp859836886-7849, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]      
   at sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
         at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)     
    at 
org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572) 
        at java.lang.Thread.run(Thread.java:745)   40) Thread[id=7826, 
name=qtp1575822333-7826, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]     
    at sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
         at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)     
    at 
org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572) 
        at java.lang.Thread.run(Thread.java:745)   41) Thread[id=7862, 
name=RecoveryThread-collection1, state=TIMED_WAITING, 
group=TGRP-HttpPartitionTest]         at java.lang.Thread.sleep(Native Method)  
       at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:108)   
      at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:76)
         at 
org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:380)         
at org.apache.solr.cloud.DistributedQueue.createData(DistributedQueue.java:377) 
        at 
org.apache.solr.cloud.DistributedQueue.offer(DistributedQueue.java:361)         
at org.apache.solr.cloud.ZkController.publish(ZkController.java:1167)         
at org.apache.solr.cloud.ZkController.publish(ZkController.java:1085)         
at org.apache.solr.cloud.ZkController.publish(ZkController.java:1081)         
at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:355) 
        at 
org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:235)   42) 
Thread[id=7848, name=qtp859836886-7848, state=TIMED_WAITING, 
group=TGRP-HttpPartitionTest]         at sun.misc.Unsafe.park(Native Method)    
     at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)  
       at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
         at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)     
    at 
org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572) 
        at java.lang.Thread.run(Thread.java:745)   43) Thread[id=7801, 
name=qtp1406611874-7801 Selector0, state=RUNNABLE, 
group=TGRP-HttpPartitionTest]         at 
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at 
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at 
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at 
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at 
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at 
org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:569)
         at 
org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)        
 at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) 
        at java.lang.Thread.run(Thread.java:745)   44) Thread[id=7833, 
name=TEST-HttpPartitionTest.testDistribSearch-seed#[D9864374E8E5CEF0]-SendThread(127.0.0.1:42676),
 state=TIMED_WAITING, group=TGRP-HttpPartitionTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
         at 
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)    
     at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)   
45) Thread[id=7800, name=HashSessionScavenger-193, state=TIMED_WAITING, 
group=TGRP-HttpPartitionTest]         at java.lang.Object.wait(Native Method)   
      at java.util.TimerThread.mainLoop(Timer.java:552)         at 
java.util.TimerThread.run(Timer.java:505)   46) Thread[id=7843, 
name=qtp859836886-7843 Acceptor0 SelectChannelConnector@127.0.0.1:53819, 
state=RUNNABLE, group=TGRP-HttpPartitionTest]         at 
sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at 
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:241)     
    at 
org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
         at 
org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) 
        at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 46 threads leaked from 
SUITE scope at org.apache.solr.cloud.HttpPartitionTest: 
   1) Thread[id=7846, name=qtp859836886-7846 Acceptor1 
SelectChannelConnector@127.0.0.1:53819, state=BLOCKED, 
group=TGRP-HttpPartitionTest]
        at 
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:224)
        at 
org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at 
org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:745)
   2) Thread[id=7859, name=Thread-3293, state=WAITING, 
group=TGRP-HttpPartitionTest]
        at java.lang.Object.wait(Native Method)
        at java.lang.Object.wait(Object.java:502)
        at org.apache.solr.core.CloserThread.run(CoreContainer.java:929)
   3) Thread[id=7823, name=qtp1575822333-7823 Selector1, state=RUNNABLE, 
group=TGRP-HttpPartitionTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at 
org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:569)
        at 
org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:745)
   4) Thread[id=7825, name=qtp1575822333-7825 Acceptor1 
SelectChannelConnector@127.0.0.1:55660, state=BLOCKED, 
group=TGRP-HttpPartitionTest]
        at 
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:224)
        at 
org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at 
org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:745)
   5) Thread[id=7813, 
name=TEST-HttpPartitionTest.testDistribSearch-seed#[D9864374E8E5CEF0]-EventThread,
 state=WAITING, group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
   6) Thread[id=7802, name=qtp1406611874-7802 Selector1, state=RUNNABLE, 
group=TGRP-HttpPartitionTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at 
org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:569)
        at 
org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:745)
   7) Thread[id=7845, name=qtp859836886-7845 Selector1, state=RUNNABLE, 
group=TGRP-HttpPartitionTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at 
org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:569)
        at 
org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:745)
   8) Thread[id=7824, name=qtp1575822333-7824 Acceptor0 
SelectChannelConnector@127.0.0.1:55660, state=RUNNABLE, 
group=TGRP-HttpPartitionTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at 
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:241)
        at 
org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at 
org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:745)
   9) Thread[id=7805, name=qtp1406611874-7805 Acceptor1 
SelectChannelConnector@127.0.0.1:40279, state=BLOCKED, 
group=TGRP-HttpPartitionTest]
        at 
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:224)
        at 
org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at 
org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:745)
  10) Thread[id=7834, 
name=TEST-HttpPartitionTest.testDistribSearch-seed#[D9864374E8E5CEF0]-EventThread,
 state=WAITING, group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
  11) Thread[id=7850, name=qtp859836886-7850, state=TIMED_WAITING, 
group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
        at java.lang.Thread.run(Thread.java:745)
  12) Thread[id=7847, name=qtp859836886-7847, state=TIMED_WAITING, 
group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
        at java.lang.Thread.run(Thread.java:745)
  13) Thread[id=7854, 
name=TEST-HttpPartitionTest.testDistribSearch-seed#[D9864374E8E5CEF0]-SendThread(127.0.0.1:42676),
 state=TIMED_WAITING, group=TGRP-HttpPartitionTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
        at 
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
  14) Thread[id=7821, name=HashSessionScavenger-194, state=TIMED_WAITING, 
group=TGRP-HttpPartitionTest]
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        at java.util.TimerThread.run(Timer.java:505)
  15) Thread[id=7808, name=qtp1406611874-7808, state=TIMED_WAITING, 
group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
        at java.lang.Thread.run(Thread.java:745)
  16) Thread[id=7837, name=searcherExecutor-4165-thread-1, state=WAITING, 
group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
  17) Thread[id=7871, 
name=OverseerHdfsCoreFailoverThread-93087894478782471-127.0.0.1:43088_oa_s-n_0000000001,
 state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:108)
        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:76)
        at 
org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:311)
        at 
org.apache.solr.common.cloud.ZkStateReader.getClusterProps(ZkStateReader.java:727)
        at 
org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.doWork(OverseerAutoReplicaFailoverThread.java:149)
        at 
org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:128)
        at java.lang.Thread.run(Thread.java:745)
  18) Thread[id=7855, 
name=TEST-HttpPartitionTest.testDistribSearch-seed#[D9864374E8E5CEF0]-EventThread,
 state=WAITING, group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
  19) Thread[id=7840, name=coreZkRegister-4160-thread-1, state=WAITING, 
group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
  20) Thread[id=7803, name=qtp1406611874-7803, state=TIMED_WAITING, 
group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
        at java.lang.Thread.run(Thread.java:745)
  21) Thread[id=7870, 
name=OverseerCollectionProcessor-93087894478782471-127.0.0.1:43088_oa_s-n_0000000001,
 state=TIMED_WAITING, group=Overseer collection creation process.]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:108)
        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:76)
        at 
org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:345)
        at 
org.apache.solr.cloud.OverseerCollectionProcessor.amILeader(OverseerCollectionProcessor.java:543)
        at 
org.apache.solr.cloud.OverseerCollectionProcessor.run(OverseerCollectionProcessor.java:301)
        at java.lang.Thread.run(Thread.java:745)
  22) Thread[id=7804, name=qtp1406611874-7804, state=TIMED_WAITING, 
group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
        at java.lang.Thread.run(Thread.java:745)
  23) Thread[id=7806, name=qtp1406611874-7806 Acceptor0 
SelectChannelConnector@127.0.0.1:40279, state=RUNNABLE, 
group=TGRP-HttpPartitionTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at 
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:241)
        at 
org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at 
org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:745)
  24) Thread[id=7829, name=qtp1575822333-7829, state=TIMED_WAITING, 
group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
        at java.lang.Thread.run(Thread.java:745)
  25) Thread[id=7822, name=qtp1575822333-7822 Selector0, state=RUNNABLE, 
group=TGRP-HttpPartitionTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at 
org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:569)
        at 
org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:745)
  26) Thread[id=7861, name=coreZkRegister-4169-thread-1, state=WAITING, 
group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
  27) Thread[id=7858, name=searcherExecutor-4174-thread-1, state=WAITING, 
group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
  28) Thread[id=7844, name=qtp859836886-7844 Selector0, state=RUNNABLE, 
group=TGRP-HttpPartitionTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at 
org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:569)
        at 
org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:745)
  29) Thread[id=7816, name=searcherExecutor-4156-thread-1, state=WAITING, 
group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
  30) Thread[id=7807, name=qtp1406611874-7807, state=TIMED_WAITING, 
group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
        at java.lang.Thread.run(Thread.java:745)
  31) Thread[id=7828, name=qtp1575822333-7828, state=TIMED_WAITING, 
group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
        at java.lang.Thread.run(Thread.java:745)
  32) Thread[id=7812, 
name=TEST-HttpPartitionTest.testDistribSearch-seed#[D9864374E8E5CEF0]-SendThread(127.0.0.1:42676),
 state=TIMED_WAITING, group=TGRP-HttpPartitionTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994)
  33) Thread[id=7869, 
name=OverseerStateUpdate-93087894478782471-127.0.0.1:43088_oa_s-n_0000000001, 
state=TIMED_WAITING, group=Overseer state updater.]
        at java.lang.Object.wait(Native Method)
        at 
org.apache.solr.cloud.DistributedQueue$LatchWatcher.await(DistributedQueue.java:273)
        at 
org.apache.solr.cloud.DistributedQueue.getChildren(DistributedQueue.java:317)
        at 
org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:591)
        at 
org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:569)
        at 
org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:215)
        at java.lang.Thread.run(Thread.java:745)
  34) Thread[id=7827, name=qtp1575822333-7827, state=TIMED_WAITING, 
group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
        at java.lang.Thread.run(Thread.java:745)
  35) Thread[id=7842, name=HashSessionScavenger-195, state=TIMED_WAITING, 
group=TGRP-HttpPartitionTest]
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        at java.util.TimerThread.run(Timer.java:505)
  36) Thread[id=7817, name=Thread-3271, state=WAITING, 
group=TGRP-HttpPartitionTest]
        at java.lang.Object.wait(Native Method)
        at java.lang.Object.wait(Object.java:502)
        at org.apache.solr.core.CloserThread.run(CoreContainer.java:929)
  37) Thread[id=7838, name=Thread-3282, state=WAITING, 
group=TGRP-HttpPartitionTest]
        at java.lang.Object.wait(Native Method)
        at java.lang.Object.wait(Object.java:502)
        at org.apache.solr.core.CloserThread.run(CoreContainer.java:929)
  38) Thread[id=7819, name=coreZkRegister-4151-thread-1, state=WAITING, 
group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
  39) Thread[id=7849, name=qtp859836886-7849, state=TIMED_WAITING, 
group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
        at java.lang.Thread.run(Thread.java:745)
  40) Thread[id=7826, name=qtp1575822333-7826, state=TIMED_WAITING, 
group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
        at java.lang.Thread.run(Thread.java:745)
  41) Thread[id=7862, name=RecoveryThread-collection1, state=TIMED_WAITING, 
group=TGRP-HttpPartitionTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:108)
        at 
org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:76)
        at 
org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:380)
        at 
org.apache.solr.cloud.DistributedQueue.createData(DistributedQueue.java:377)
        at 
org.apache.solr.cloud.DistributedQueue.offer(DistributedQueue.java:361)
        at org.apache.solr.cloud.ZkController.publish(ZkController.java:1167)
        at org.apache.solr.cloud.ZkController.publish(ZkController.java:1085)
        at org.apache.solr.cloud.ZkController.publish(ZkController.java:1081)
        at 
org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:355)
        at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:235)
  42) Thread[id=7848, name=qtp859836886-7848, state=TIMED_WAITING, 
group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at 
org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
        at java.lang.Thread.run(Thread.java:745)
  43) Thread[id=7801, name=qtp1406611874-7801 Selector0, state=RUNNABLE, 
group=TGRP-HttpPartitionTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at 
org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:569)
        at 
org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:745)
  44) Thread[id=7833, 
name=TEST-HttpPartitionTest.testDistribSearch-seed#[D9864374E8E5CEF0]-SendThread(127.0.0.1:42676),
 state=TIMED_WAITING, group=TGRP-HttpPartitionTest]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
        at 
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
  45) Thread[id=7800, name=HashSessionScavenger-193, state=TIMED_WAITING, 
group=TGRP-HttpPartitionTest]
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        at java.util.TimerThread.run(Timer.java:505)
  46) Thread[id=7843, name=qtp859836886-7843 Acceptor0 
SelectChannelConnector@127.0.0.1:53819, state=RUNNABLE, 
group=TGRP-HttpPartitionTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at 
sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:241)
        at 
org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:109)
        at 
org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:745)
        at __randomizedtesting.SeedInfo.seed([D9864374E8E5CEF0]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.HttpPartitionTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) 
Thread[id=7846, name=qtp859836886-7846 Acceptor1 
SelectChannelConnector@127.0.0.1:53819, state=RUNNABLE, 
group=TGRP-HttpPartitionTest]         at 
org.eclipse.jetty.server.nio.SelectChannelConnector.getConnection(SelectChannelConnector.java:160)
         at 
org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:934)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) 
        at java.lang.Thread.run(Thread.java:745)    2) Thread[id=7859, 
name=Thread-3293, state=WAITING, group=TGRP-HttpPartitionTest]         at 
java.lang.Object.wait(Native Method)         at 
java.lang.Object.wait(Object.java:502)         at 
org.apache.solr.core.CloserThread.run(CoreContainer.java:929)    3) 
Thread[id=7823, name=qtp1575822333-7823 Selector1, state=RUNNABLE, 
group=TGRP-HttpPartitionTest]         at 
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at 
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at 
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at 
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at 
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at 
org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:569)
         at 
org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)        
 at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) 
        at java.lang.Thread.run(Thread.java:745)    4) Thread[id=7825, 
name=qtp1575822333-7825 Acceptor1 SelectChannelConnector@127.0.0.1:55660, 
state=RUNNABLE, group=TGRP-HttpPartitionTest]         at 
org.eclipse.jetty.server.nio.SelectChannelConnector.getConnection(SelectChannelConnector.java:160)
         at 
org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:934)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) 
        at java.lang.Thread.run(Thread.java:745)    5) Thread[id=7802, 
name=qtp1406611874-7802 Selector1, state=RUNNABLE, 
group=TGRP-HttpPartitionTest]         at 
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at 
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at 
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at 
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at 
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at 
org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:569)
         at 
org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)        
 at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) 
        at java.lang.Thread.run(Thread.java:745)    6) Thread[id=7845, 
name=qtp859836886-7845 Selector1, state=RUNNABLE, group=TGRP-HttpPartitionTest] 
        at sun.nio.ch.IOUtil.drain(Native Method)         at 
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:90)         at 
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at 
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at 
org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:569)
         at 
org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)        
 at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) 
        at java.lang.Thread.run(Thread.java:745)    7) Thread[id=7824, 
name=qtp1575822333-7824 Acceptor0 SelectChannelConnector@127.0.0.1:55660, 
state=RUNNABLE, group=TGRP-HttpPartitionTest]         at 
org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:956)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) 
        at java.lang.Thread.run(Thread.java:745)    8) Thread[id=7805, 
name=qtp1406611874-7805 Acceptor1 SelectChannelConnector@127.0.0.1:40279, 
state=BLOCKED, group=TGRP-HttpPartitionTest]         at 
org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:102)
         at 
org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) 
        at java.lang.Thread.run(Thread.java:745)    9) Thread[id=7854, 
name=TEST-HttpPartitionTest.testDistribSearch-seed#[D9864374E8E5CEF0]-SendThread(127.0.0.1:42676),
 state=TIMED_WAITING, group=TGRP-HttpPartitionTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994)   10) 
Thread[id=7821, name=HashSessionScavenger-194, state=TIMED_WAITING, 
group=TGRP-HttpPartitionTest]         at java.lang.Object.wait(Native Method)   
      at java.util.TimerThread.mainLoop(Timer.java:552)         at 
java.util.TimerThread.run(Timer.java:505)   11) Thread[id=7837, 
name=searcherExecutor-4165-thread-1, state=WAITING, 
group=TGRP-HttpPartitionTest]         at sun.misc.Unsafe.park(Native Method)    
     at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)       
  at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)     
    at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
        at java.lang.Thread.run(Thread.java:745)   12) Thread[id=7871, 
name=OverseerHdfsCoreFailoverThread-93087894478782471-127.0.0.1:43088_oa_s-n_0000000001,
 state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:136)
         at java.lang.Thread.run(Thread.java:745)   13) Thread[id=7840, 
name=coreZkRegister-4160-thread-1, state=WAITING, group=TGRP-HttpPartitionTest] 
        at sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)     
    at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
        at java.lang.Thread.run(Thread.java:745)   14) Thread[id=7806, 
name=qtp1406611874-7806 Acceptor0 SelectChannelConnector@127.0.0.1:40279, 
state=RUNNABLE, group=TGRP-HttpPartitionTest]         at 
org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:956)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) 
        at java.lang.Thread.run(Thread.java:745)   15) Thread[id=7822, 
name=qtp1575822333-7822 Selector0, state=RUNNABLE, 
group=TGRP-HttpPartitionTest]         at 
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at 
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at 
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at 
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at 
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at 
org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:569)
         at 
org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)        
 at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) 
        at java.lang.Thread.run(Thread.java:745)   16) Thread[id=7861, 
name=coreZkRegister-4169-thread-1, state=WAITING, group=TGRP-HttpPartitionTest] 
        at sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)     
    at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
        at java.lang.Thread.run(Thread.java:745)   17) Thread[id=7858, 
name=searcherExecutor-4174-thread-1, state=WAITING, 
group=TGRP-HttpPartitionTest]         at sun.misc.Unsafe.park(Native Method)    
     at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)       
  at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)     
    at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
        at java.lang.Thread.run(Thread.java:745)   18) Thread[id=7844, 
name=qtp859836886-7844 Selector0, state=RUNNABLE, group=TGRP-HttpPartitionTest] 
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at 
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at 
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at 
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at 
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at 
org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:569)
         at 
org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)        
 at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) 
        at java.lang.Thread.run(Thread.java:745)   19) Thread[id=7816, 
name=searcherExecutor-4156-thread-1, state=WAITING, 
group=TGRP-HttpPartitionTest]         at sun.misc.Unsafe.park(Native Method)    
     at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)       
  at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)     
    at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
        at java.lang.Thread.run(Thread.java:745)   20) Thread[id=7812, 
name=TEST-HttpPartitionTest.testDistribSearch-seed#[D9864374E8E5CEF0]-SendThread(127.0.0.1:42676),
 state=TIMED_WAITING, group=TGRP-HttpPartitionTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
         at 
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)    
     at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)   
21) Thread[id=7842, name=HashSessionScavenger-195, state=TIMED_WAITING, 
group=TGRP-HttpPartitionTest]         at java.lang.Object.wait(Native Method)   
      at java.util.TimerThread.mainLoop(Timer.java:552)         at 
java.util.TimerThread.run(Timer.java:505)   22) Thread[id=7817, 
name=Thread-3271, state=WAITING, group=TGRP-HttpPartitionTest]         at 
java.lang.Object.wait(Native Method)         at 
java.lang.Object.wait(Object.java:502)         at 
org.apache.solr.core.CloserThread.run(CoreContainer.java:929)   23) 
Thread[id=7838, name=Thread-3282, state=WAITING, group=TGRP-HttpPartitionTest]  
       at java.lang.Object.wait(Native Method)         at 
java.lang.Object.wait(Object.java:502)         at 
org.apache.solr.core.CloserThread.run(CoreContainer.java:929)   24) 
Thread[id=7819, name=coreZkRegister-4151-thread-1, state=WAITING, 
group=TGRP-HttpPartitionTest]         at sun.misc.Unsafe.park(Native Method)    
     at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)       
  at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)     
    at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
        at java.lang.Thread.run(Thread.java:745)   25) Thread[id=7801, 
name=qtp1406611874-7801 Selector0, state=RUNNABLE, 
group=TGRP-HttpPartitionTest]         at 
sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at 
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at 
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)         at 
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at 
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at 
org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:569)
         at 
org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)        
 at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) 
        at java.lang.Thread.run(Thread.java:745)   26) Thread[id=7833, 
name=TEST-HttpPartitionTest.testDistribSearch-seed#[D9864374E8E5CEF0]-SendThread(127.0.0.1:42676),
 state=TIMED_WAITING, group=TGRP-HttpPartitionTest]         at 
java.lang.Thread.sleep(Native Method)         at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994)   27) 
Thread[id=7800, name=HashSessionScavenger-193, state=TIMED_WAITING, 
group=TGRP-HttpPartitionTest]         at java.lang.Object.wait(Native Method)   
      at java.util.TimerThread.mainLoop(Timer.java:552)         at 
java.util.TimerThread.run(Timer.java:505)   28) Thread[id=7843, 
name=qtp859836886-7843 Acceptor0 SelectChannelConnector@127.0.0.1:53819, 
state=RUNNABLE, group=TGRP-HttpPartitionTest]         at 
org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:102)
         at 
org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) 
        at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie 
threads that couldn't be terminated:
   1) Thread[id=7846, name=qtp859836886-7846 Acceptor1 
SelectChannelConnector@127.0.0.1:53819, state=RUNNABLE, 
group=TGRP-HttpPartitionTest]
        at 
org.eclipse.jetty.server.nio.SelectChannelConnector.getConnection(SelectChannelConnector.java:160)
        at 
org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:934)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:745)
   2) Thread[id=7859, name=Thread-3293, state=WAITING, 
group=TGRP-HttpPartitionTest]
        at java.lang.Object.wait(Native Method)
        at java.lang.Object.wait(Object.java:502)
        at org.apache.solr.core.CloserThread.run(CoreContainer.java:929)
   3) Thread[id=7823, name=qtp1575822333-7823 Selector1, state=RUNNABLE, 
group=TGRP-HttpPartitionTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at 
org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:569)
        at 
org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:745)
   4) Thread[id=7825, name=qtp1575822333-7825 Acceptor1 
SelectChannelConnector@127.0.0.1:55660, state=RUNNABLE, 
group=TGRP-HttpPartitionTest]
        at 
org.eclipse.jetty.server.nio.SelectChannelConnector.getConnection(SelectChannelConnector.java:160)
        at 
org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:934)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:745)
   5) Thread[id=7802, name=qtp1406611874-7802 Selector1, state=RUNNABLE, 
group=TGRP-HttpPartitionTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at 
org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:569)
        at 
org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:745)
   6) Thread[id=7845, name=qtp859836886-7845 Selector1, state=RUNNABLE, 
group=TGRP-HttpPartitionTest]
        at sun.nio.ch.IOUtil.drain(Native Method)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:90)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at 
org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:569)
        at 
org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:745)
   7) Thread[id=7824, name=qtp1575822333-7824 Acceptor0 
SelectChannelConnector@127.0.0.1:55660, state=RUNNABLE, 
group=TGRP-HttpPartitionTest]
        at 
org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:956)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:745)
   8) Thread[id=7805, name=qtp1406611874-7805 Acceptor1 
SelectChannelConnector@127.0.0.1:40279, state=BLOCKED, 
group=TGRP-HttpPartitionTest]
        at 
org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:102)
        at 
org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:745)
   9) Thread[id=7854, 
name=TEST-HttpPartitionTest.testDistribSearch-seed#[D9864374E8E5CEF0]-SendThread(127.0.0.1:42676),
 state=TIMED_WAITING, group=TGRP-HttpPartitionTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994)
  10) Thread[id=7821, name=HashSessionScavenger-194, state=TIMED_WAITING, 
group=TGRP-HttpPartitionTest]
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:552)
        at java.util.TimerThread.run(Timer.java:505)
  11) Thread[id=7837, name=searcherExecutor-4165-thread-1, state=WAITING, 
group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
  12) Thread[id=7871, 
name=OverseerHdfsCoreFailoverThread-93087894478782471-127.0.0.1:43088_oa_s-n_0000000001,
 state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.]
        at java.lang.Thread.sleep(Native Method)
        at 
org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:136)
        at java.lang.Thread.run(Thread.java:745)
  13) Thread[id=7840, name=coreZkRegister-4160-thread-1, state=WAITING, 
group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
  14) Thread[id=7806, name=qtp1406611874-7806 Acceptor0 
SelectChannelConnector@127.0.0.1:40279, state=RUNNABLE, 
group=TGRP-HttpPartitionTest]
        at 
org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:956)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
        at java.lang.Thread.run(Thread.java:745)
  15) Thread[id=7822, name=qtp1575822333-7822 Selector0, state=RUNNABLE, 
group=TGRP-HttpPartitionTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at 
org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:569)
        at 
org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)
        at org.eclipse.jetty.util.thread.Queued

[...truncated too long message...]

Support.java:175)
   [junit4]    >         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]    >         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   17) Thread[id=7858, name=searcherExecutor-4174-thread-1, 
state=WAITING, group=TGRP-HttpPartitionTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]    >         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]    >         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   18) Thread[id=7844, name=qtp859836886-7844 Selector0, 
state=RUNNABLE, group=TGRP-HttpPartitionTest]
   [junit4]    >         at sun.nio.ch.EPollArrayWrapper.epollWait(Native 
Method)
   [junit4]    >         at 
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
   [junit4]    >         at 
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
   [junit4]    >         at 
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
   [junit4]    >         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
   [junit4]    >         at 
org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:569)
   [junit4]    >         at 
org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)
   [junit4]    >         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
   [junit4]    >         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   19) Thread[id=7816, name=searcherExecutor-4156-thread-1, 
state=WAITING, group=TGRP-HttpPartitionTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]    >         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]    >         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   20) Thread[id=7812, 
name=TEST-HttpPartitionTest.testDistribSearch-seed#[D9864374E8E5CEF0]-SendThread(127.0.0.1:42676),
 state=TIMED_WAITING, group=TGRP-HttpPartitionTest]
   [junit4]    >         at java.lang.Thread.sleep(Native Method)
   [junit4]    >         at 
org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
   [junit4]    >         at 
org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
   [junit4]    >         at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
   [junit4]    >   21) Thread[id=7842, name=HashSessionScavenger-195, 
state=TIMED_WAITING, group=TGRP-HttpPartitionTest]
   [junit4]    >         at java.lang.Object.wait(Native Method)
   [junit4]    >         at java.util.TimerThread.mainLoop(Timer.java:552)
   [junit4]    >         at java.util.TimerThread.run(Timer.java:505)
   [junit4]    >   22) Thread[id=7817, name=Thread-3271, state=WAITING, 
group=TGRP-HttpPartitionTest]
   [junit4]    >         at java.lang.Object.wait(Native Method)
   [junit4]    >         at java.lang.Object.wait(Object.java:502)
   [junit4]    >         at 
org.apache.solr.core.CloserThread.run(CoreContainer.java:929)
   [junit4]    >   23) Thread[id=7838, name=Thread-3282, state=WAITING, 
group=TGRP-HttpPartitionTest]
   [junit4]    >         at java.lang.Object.wait(Native Method)
   [junit4]    >         at java.lang.Object.wait(Object.java:502)
   [junit4]    >         at 
org.apache.solr.core.CloserThread.run(CoreContainer.java:929)
   [junit4]    >   24) Thread[id=7819, name=coreZkRegister-4151-thread-1, 
state=WAITING, group=TGRP-HttpPartitionTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]    >         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]    >         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   25) Thread[id=7801, name=qtp1406611874-7801 Selector0, 
state=RUNNABLE, group=TGRP-HttpPartitionTest]
   [junit4]    >         at sun.nio.ch.EPollArrayWrapper.epollWait(Native 
Method)
   [junit4]    >         at 
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
   [junit4]    >         at 
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
   [junit4]    >         at 
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
   [junit4]    >         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
   [junit4]    >         at 
org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:569)
   [junit4]    >         at 
org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)
   [junit4]    >         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
   [junit4]    >         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   26) Thread[id=7833, 
name=TEST-HttpPartitionTest.testDistribSearch-seed#[D9864374E8E5CEF0]-SendThread(127.0.0.1:42676),
 state=TIMED_WAITING, group=TGRP-HttpPartitionTest]
   [junit4]    >         at java.lang.Thread.sleep(Native Method)
   [junit4]    >         at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994)
   [junit4]    >   27) Thread[id=7800, name=HashSessionScavenger-193, 
state=TIMED_WAITING, group=TGRP-HttpPartitionTest]
   [junit4]    >         at java.lang.Object.wait(Native Method)
   [junit4]    >         at java.util.TimerThread.mainLoop(Timer.java:552)
   [junit4]    >         at java.util.TimerThread.run(Timer.java:505)
   [junit4]    >   28) Thread[id=7843, name=qtp859836886-7843 Acceptor0 
SelectChannelConnector@127.0.0.1:53819, state=RUNNABLE, 
group=TGRP-HttpPartitionTest]
   [junit4]    >         at 
org.eclipse.jetty.server.nio.SelectChannelConnector.accept(SelectChannelConnector.java:102)
   [junit4]    >         at 
org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
   [junit4]    >         at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
   [junit4]    >         at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([D9864374E8E5CEF0]:0)
   [junit4] Completed on J1 in 156.75s, 1 test, 1 failure, 3 errors <<< 
FAILURES!

[...truncated 256 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/build.xml:529: The following 
error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/build.xml:477: The following 
error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/build.xml:61: The following 
error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/extra-targets.xml:39: The 
following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build.xml:190: The 
following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/common-build.xml:508: The 
following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/common-build.xml:1363: 
The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/common-build.xml:970: 
There were test failures: 460 suites, 1869 tests, 3 suite-level errors, 1 
error, 1 failure, 246 ignored (22 assumptions)

Total time: 56 minutes 30 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 64bit/jdk1.8.0_20 
-XX:+UseCompressedOops -XX:+UseSerialGC (asserts: false)
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-5.x-Linux (64bit/jdk1.8.0_20) - Build # 11678 - Failure!

Reply via email to