Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-trunk/844/

3 tests failed.
FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Error from server at http://127.0.0.1:55033/nv_vr/r/awholynewcollection_3: 
Expected mime type application/octet-stream but got text/html. <html> <head> 
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> 
<title>Error 500 </title> </head> <body> <h2>HTTP ERROR: 500</h2> <p>Problem 
accessing /nv_vr/r/awholynewcollection_3/select. Reason: <pre>    
{trace=java.lang.NullPointerException  at 
org.apache.solr.servlet.HttpSolrCall.getCoreByCollection(HttpSolrCall.java:786) 
 at org.apache.solr.servlet.HttpSolrCall.init(HttpSolrCall.java:270)  at 
org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:415)  at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)
  at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)
  at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
  at 
org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)
  at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
  at 
org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)  
at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:364)  at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
  at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585) 
 at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)
  at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
  at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)  
at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
  at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
  at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)  
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)  
at org.eclipse.jetty.server.Server.handle(Server.java:499)  at 
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)  at 
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)  at 
org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)  at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
  at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) 
 at java.lang.Thread.run(Thread.java:745) ,code=500}</pre></p> <hr 
/><i><small>Powered by Jetty://</small></i> </body> </html> 

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at http://127.0.0.1:55033/nv_vr/r/awholynewcollection_3: Expected 
mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /nv_vr/r/awholynewcollection_3/select. Reason:
<pre>    {trace=java.lang.NullPointerException
        at 
org.apache.solr.servlet.HttpSolrCall.getCoreByCollection(HttpSolrCall.java:786)
        at org.apache.solr.servlet.HttpSolrCall.init(HttpSolrCall.java:270)
        at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:415)
        at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)
        at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)
        at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
        at 
org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)
        at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
        at 
org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)
        at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:364)
        at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
        at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
        at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)
        at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
        at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
        at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
        at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
        at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
        at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
        at org.eclipse.jetty.server.Server.handle(Server.java:499)
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
        at 
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
        at 
org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
        at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
        at java.lang.Thread.run(Thread.java:745)
,code=500}</pre></p>
<hr /><i><small>Powered by Jetty://</small></i>
</body>
</html>

        at 
__randomizedtesting.SeedInfo.seed([7205036CC31451AA:FA513CB66DE83C52]:0)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:543)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
        at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)
        at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:943)
        at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:958)
        at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForNon403or404or503(AbstractFullDistribZkTestBase.java:1754)
        at 
org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:665)
        at 
org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:167)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at java.lang.Thread.run(Thread.java:745)


FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest

Error Message:
5 threads leaked from SUITE scope at 
org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest:     1) 
Thread[id=17425, name=zkCallback-909-thread-3, state=TIMED_WAITING, 
group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at 
sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         
at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
         at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
         at 
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
        at java.lang.Thread.run(Thread.java:745)    2) Thread[id=17424, 
name=zkCallback-909-thread-2, state=TIMED_WAITING, 
group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at 
sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         
at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
         at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
         at 
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
        at java.lang.Thread.run(Thread.java:745)    3) Thread[id=16989, 
name=TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]-SendThread(127.0.0.1:46837),
 state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         
at java.lang.Thread.sleep(Native Method)         at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994)    4) 
Thread[id=16991, name=zkCallback-909-thread-1, state=TIMED_WAITING, 
group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at 
sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         
at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
         at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
         at 
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
        at java.lang.Thread.run(Thread.java:745)    5) Thread[id=16990, 
name=TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]-EventThread,
 state=WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at 
sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)     
    at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 5 threads leaked from SUITE 
scope at org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest: 
   1) Thread[id=17425, name=zkCallback-909-thread-3, state=TIMED_WAITING, 
group=TGRP-HdfsCollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   2) Thread[id=17424, name=zkCallback-909-thread-2, state=TIMED_WAITING, 
group=TGRP-HdfsCollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   3) Thread[id=16989, 
name=TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]-SendThread(127.0.0.1:46837),
 state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994)
   4) Thread[id=16991, name=zkCallback-909-thread-1, state=TIMED_WAITING, 
group=TGRP-HdfsCollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   5) Thread[id=16990, 
name=TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]-EventThread,
 state=WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
        at __randomizedtesting.SeedInfo.seed([7205036CC31451AA]:0)


FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) 
Thread[id=17425, name=zkCallback-909-thread-3, state=TIMED_WAITING, 
group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at 
sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         
at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
         at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
         at 
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
        at java.lang.Thread.run(Thread.java:745)    2) Thread[id=17424, 
name=zkCallback-909-thread-2, state=TIMED_WAITING, 
group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at 
sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         
at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
         at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
         at 
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
        at java.lang.Thread.run(Thread.java:745)    3) Thread[id=16989, 
name=TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]-SendThread(127.0.0.1:46837),
 state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         
at java.lang.Thread.sleep(Native Method)         at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994)    4) 
Thread[id=16991, name=zkCallback-909-thread-1, state=TIMED_WAITING, 
group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at 
sun.misc.Unsafe.park(Native Method)         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         
at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
         at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
         at 
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
        at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie 
threads that couldn't be terminated:
   1) Thread[id=17425, name=zkCallback-909-thread-3, state=TIMED_WAITING, 
group=TGRP-HdfsCollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   2) Thread[id=17424, name=zkCallback-909-thread-2, state=TIMED_WAITING, 
group=TGRP-HdfsCollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   3) Thread[id=16989, 
name=TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]-SendThread(127.0.0.1:46837),
 state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994)
   4) Thread[id=16991, name=zkCallback-909-thread-1, state=TIMED_WAITING, 
group=TGRP-HdfsCollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
        at __randomizedtesting.SeedInfo.seed([7205036CC31451AA]:0)




Build Log:
[...truncated 10517 lines...]
   [junit4] Suite: 
org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_7205036CC31451AA-001/init-core-data-001
   [junit4]   2> 1200970 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[7205036CC31451AA]-worker) [    
] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 1200970 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[7205036CC31451AA]-worker) [    
] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: 
/nv_vr/r
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 1201017 WARN  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[7205036CC31451AA]-worker) [    
] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried 
hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 1201024 WARN  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[7205036CC31451AA]-worker) [    
] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 1201026 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[7205036CC31451AA]-worker) [    
] o.m.log jetty-6.1.26
   [junit4]   2> 1201053 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[7205036CC31451AA]-worker) [    
] o.m.log Extract 
jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs
 to ./temp/Jetty_localhost_53615_hdfs____.sfeu2/webapp
   [junit4]   2> 1201157 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[7205036CC31451AA]-worker) [    
] o.m.log NO JSP Support for /, did not find 
org.apache.jasper.servlet.JspServlet
   [junit4]   2> 1201331 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[7205036CC31451AA]-worker) [    
] o.m.log Started 
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:53615
   [junit4]   2> 1201420 WARN  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[7205036CC31451AA]-worker) [    
] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 1201421 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[7205036CC31451AA]-worker) [    
] o.m.log jetty-6.1.26
   [junit4]   2> 1201434 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[7205036CC31451AA]-worker) [    
] o.m.log Extract 
jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode
 to ./temp/Jetty_localhost_41571_datanode____.51x23r/webapp
   [junit4]   2> 1201540 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[7205036CC31451AA]-worker) [    
] o.m.log NO JSP Support for /, did not find 
org.apache.jasper.servlet.JspServlet
   [junit4]   2> 1201710 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[7205036CC31451AA]-worker) [    
] o.m.log Started 
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:41571
   [junit4]   2> 1201748 WARN  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[7205036CC31451AA]-worker) [    
] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 1201749 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[7205036CC31451AA]-worker) [    
] o.m.log jetty-6.1.26
   [junit4]   2> 1201762 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[7205036CC31451AA]-worker) [    
] o.m.log Extract 
jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode
 to ./temp/Jetty_localhost_37905_datanode____.aa1j3z/webapp
   [junit4]   2> 1201816 INFO  (IPC Server handler 3 on 51218) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-315151a4-ceb9-4f0d-ac3f-1a24b5077ba5 node DatanodeRegistration(127.0.0.1, 
datanodeUuid=f39b4f0c-9ddd-44fe-b3c2-f97feb8988af, infoPort=41571, 
ipcPort=48241, storageInfo=lv=-56;cid=testClusterID;nsid=349147342;c=0), 
blocks: 0, hasStaleStorages: true, processing time: 1 msecs
   [junit4]   2> 1201816 INFO  (IPC Server handler 3 on 51218) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-f5d7f3d3-ffdc-4e8f-8e63-4e0a3d1f6b6c node DatanodeRegistration(127.0.0.1, 
datanodeUuid=f39b4f0c-9ddd-44fe-b3c2-f97feb8988af, infoPort=41571, 
ipcPort=48241, storageInfo=lv=-56;cid=testClusterID;nsid=349147342;c=0), 
blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 1201888 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[7205036CC31451AA]-worker) [    
] o.m.log NO JSP Support for /, did not find 
org.apache.jasper.servlet.JspServlet
   [junit4]   2> 1202061 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[7205036CC31451AA]-worker) [    
] o.m.log Started 
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:37905
   [junit4]   2> 1202146 INFO  (IPC Server handler 7 on 51218) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-7de94fe4-e400-4148-a12c-80bb6da79b6f node DatanodeRegistration(127.0.0.1, 
datanodeUuid=4137508e-46c3-4960-ab01-92b4de0cedb3, infoPort=37905, 
ipcPort=55881, storageInfo=lv=-56;cid=testClusterID;nsid=349147342;c=0), 
blocks: 0, hasStaleStorages: true, processing time: 1 msecs
   [junit4]   2> 1202146 INFO  (IPC Server handler 7 on 51218) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-5a068994-9493-4dac-bd15-99e267052c30 node DatanodeRegistration(127.0.0.1, 
datanodeUuid=4137508e-46c3-4960-ab01-92b4de0cedb3, infoPort=37905, 
ipcPort=55881, storageInfo=lv=-56;cid=testClusterID;nsid=349147342;c=0), 
blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 1202259 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1202259 INFO  (Thread-12460) [    ] o.a.s.c.ZkTestServer 
client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1202259 INFO  (Thread-12460) [    ] o.a.s.c.ZkTestServer 
Starting server
   [junit4]   2> 1202359 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.ZkTestServer start zk server on port:46837
   [junit4]   2> 1202359 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1202360 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1202363 INFO  (zkCallback-875-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@7307f581 
name:ZooKeeperConnection Watcher:127.0.0.1:46837 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1202363 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1202363 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1202363 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1202365 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1202366 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1202368 INFO  (zkCallback-876-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@67c8255d 
name:ZooKeeperConnection Watcher:127.0.0.1:46837/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1202368 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1202368 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1202368 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1202370 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1202372 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1202373 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1202375 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml
 to /configs/conf1/solrconfig.xml
   [junit4]   2> 1202376 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1202378 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/schema.xml
 to /configs/conf1/schema.xml
   [junit4]   2> 1202378 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1202380 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml
 to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1202381 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.SolrZkClient makePath: 
/configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1202382 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/stopwords.txt
 to /configs/conf1/stopwords.txt
   [junit4]   2> 1202382 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1202384 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/protwords.txt
 to /configs/conf1/protwords.txt
   [junit4]   2> 1202384 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1202385 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/currency.xml
 to /configs/conf1/currency.xml
   [junit4]   2> 1202385 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1202387 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml
 to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1202387 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1202389 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json
 to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1202389 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1202390 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt
 to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1202390 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1202392 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt
 to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1202392 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1202393 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/synonyms.txt
 to /configs/conf1/synonyms.txt
   [junit4]   2> 1202394 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1202396 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1202396 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1202399 INFO  (zkCallback-877-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@6eee350e 
name:ZooKeeperConnection Watcher:127.0.0.1:46837 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1202400 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1202400 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1202400 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1202401 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1202402 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1202407 INFO  (zkCallback-878-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@3c272e37 
name:ZooKeeperConnection Watcher:127.0.0.1:46837/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1202407 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1202407 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1202408 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml
 to /configs/conf2/solrconfig.xml
   [junit4]   2> 1202408 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 1202410 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/schema.xml
 to /configs/conf2/schema.xml
   [junit4]   2> 1202410 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf2/schema.xml
   [junit4]   2> 1202412 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml
 to /configs/conf2/enumsConfig.xml
   [junit4]   2> 1202412 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf2/enumsConfig.xml
   [junit4]   2> 1202413 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml
 to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1202413 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.SolrZkClient makePath: 
/configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1202415 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/stopwords.txt
 to /configs/conf2/stopwords.txt
   [junit4]   2> 1202415 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 1202417 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/protwords.txt
 to /configs/conf2/protwords.txt
   [junit4]   2> 1202417 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf2/protwords.txt
   [junit4]   2> 1202418 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/currency.xml
 to /configs/conf2/currency.xml
   [junit4]   2> 1202418 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf2/currency.xml
   [junit4]   2> 1202420 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json
 to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 1202420 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 1202421 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt
 to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1202422 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1202423 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt
 to /configs/conf2/old_synonyms.txt
   [junit4]   2> 1202423 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 1202425 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/synonyms.txt
 to /configs/conf2/synonyms.txt
   [junit4]   2> 1202425 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 1202426 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/elevate.xml
 to /configs/conf2/elevate.xml
   [junit4]   2> 1202426 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf2/elevate.xml
   [junit4]   2> 1202761 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.SolrTestCaseJ4 Writing core.properties file to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_7205036CC31451AA-001/control-001/cores/collection1
   [junit4]   2> 1202762 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1202764 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@6d32653e{/nv_vr/r,null,AVAILABLE}
   [junit4]   2> 1202765 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.e.j.s.ServerConnector Started 
ServerConnector@bdb1c2f{HTTP/1.1}{127.0.0.1:55033}
   [junit4]   2> 1202765 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.e.j.s.Server Started @1205605ms
   [junit4]   2> 1202765 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: 
{solr.data.dir=hdfs://localhost:51218/hdfs__localhost_51218__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-trunk_solr_build_solr-core_test_J2_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_7205036CC31451AA-001_tempDir-002_control_data,
 hostContext=/nv_vr/r, hostPort=55033, 
coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_7205036CC31451AA-001/control-001/cores}
   [junit4]   2> 1202765 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): 
sun.misc.Launcher$AppClassLoader@4e0e2f2a
   [junit4]   2> 1202765 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: 
'/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_7205036CC31451AA-001/control-001/'
   [junit4]   2> 1202765 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1202765 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find 
system property or JNDI)
   [junit4]   2> 1202781 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1202781 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1202783 INFO  (zkCallback-879-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@2fa1374c 
name:ZooKeeperConnection Watcher:127.0.0.1:46837/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1202783 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1202784 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1202785 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in 
ZooKeeper)
   [junit4]   2> 1202785 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.SolrXmlConfig Loading container configuration from 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_7205036CC31451AA-001/control-001/solr.xml
   [junit4]   2> 1202789 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.CoresLocator Config-defined core root directory: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_7205036CC31451AA-001/control-001/cores
   [junit4]   2> 1202789 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.CoreContainer New CoreContainer 262882375
   [junit4]   2> 1202789 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.CoreContainer Loading cores into CoreContainer 
[instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_7205036CC31451AA-001/control-001/]
   [junit4]   2> 1202790 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.CoreContainer loading shared library: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_7205036CC31451AA-001/control-001/lib
   [junit4]   2> 1202790 WARN  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to 
classloader: lib (resolved as: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_7205036CC31451AA-001/control-001/lib).
   [junit4]   2> 1202797 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme 
: ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 
10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 
5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1202798 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: 
socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1202799 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1202799 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.l.LogWatcher Registering Log Listener [Log4j 
(org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1202799 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46837/solr
   [junit4]   2> 1202799 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1202799 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1202800 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1202801 INFO  (zkCallback-881-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@55a5d326 
name:ZooKeeperConnection Watcher:127.0.0.1:46837 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1202802 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1202802 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1202804 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) 
[n:127.0.0.1:55033_nv_vr%2Fr    ] o.a.s.c.c.ConnectionManager Waiting for 
client to connect to ZooKeeper
   [junit4]   2> 1202805 INFO  
(zkCallback-882-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr    ] o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@4ec566bd 
name:ZooKeeperConnection Watcher:127.0.0.1:46837/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1202806 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) 
[n:127.0.0.1:55033_nv_vr%2Fr    ] o.a.s.c.c.ConnectionManager Client is 
connected to ZooKeeper
   [junit4]   2> 1202807 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) 
[n:127.0.0.1:55033_nv_vr%2Fr    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/queue
   [junit4]   2> 1202808 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) 
[n:127.0.0.1:55033_nv_vr%2Fr    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/collection-queue-work
   [junit4]   2> 1202811 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) 
[n:127.0.0.1:55033_nv_vr%2Fr    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/collection-map-running
   [junit4]   2> 1202812 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) 
[n:127.0.0.1:55033_nv_vr%2Fr    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/collection-map-completed
   [junit4]   2> 1202814 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) 
[n:127.0.0.1:55033_nv_vr%2Fr    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/collection-map-failure
   [junit4]   2> 1202816 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) 
[n:127.0.0.1:55033_nv_vr%2Fr    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1202817 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) 
[n:127.0.0.1:55033_nv_vr%2Fr    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1202818 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) 
[n:127.0.0.1:55033_nv_vr%2Fr    ] o.a.s.c.c.SolrZkClient makePath: 
/clusterstate.json
   [junit4]   2> 1202819 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) 
[n:127.0.0.1:55033_nv_vr%2Fr    ] o.a.s.c.c.SolrZkClient makePath: 
/security.json
   [junit4]   2> 1202820 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) 
[n:127.0.0.1:55033_nv_vr%2Fr    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:55033_nv_vr%2Fr
   [junit4]   2> 1202821 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) 
[n:127.0.0.1:55033_nv_vr%2Fr    ] o.a.s.c.c.SolrZkClient makePath: 
/live_nodes/127.0.0.1:55033_nv_vr%2Fr
   [junit4]   2> 1202822 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) 
[n:127.0.0.1:55033_nv_vr%2Fr    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer_elect
   [junit4]   2> 1202823 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) 
[n:127.0.0.1:55033_nv_vr%2Fr    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer_elect/election
   [junit4]   2> 1202824 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) 
[n:127.0.0.1:55033_nv_vr%2Fr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1202824 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) 
[n:127.0.0.1:55033_nv_vr%2Fr    ] o.a.s.c.LeaderElector Joined leadership 
election with path: 
/overseer_elect/election/94819512677629958-127.0.0.1:55033_nv_vr%2Fr-n_0000000000
   [junit4]   2> 1202826 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) 
[n:127.0.0.1:55033_nv_vr%2Fr    ] o.a.s.c.OverseerElectionContext I am going to 
be the leader 127.0.0.1:55033_nv_vr%2Fr
   [junit4]   2> 1202826 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) 
[n:127.0.0.1:55033_nv_vr%2Fr    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer_elect/leader
   [junit4]   2> 1202827 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) 
[n:127.0.0.1:55033_nv_vr%2Fr    ] o.a.s.c.Overseer Overseer 
(id=94819512677629958-127.0.0.1:55033_nv_vr%2Fr-n_0000000000) starting
   [junit4]   2> 1202829 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) 
[n:127.0.0.1:55033_nv_vr%2Fr    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/queue-work
   [junit4]   2> 1202834 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) 
[n:127.0.0.1:55033_nv_vr%2Fr    ] o.a.s.c.OverseerAutoReplicaFailoverThread 
Starting OverseerAutoReplicaFailoverThread 
autoReplicaFailoverWorkLoopDelay=10000 
autoReplicaFailoverWaitAfterExpiration=10000 
autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1202834 INFO  
(OverseerCollectionConfigSetProcessor-94819512677629958-127.0.0.1:55033_nv_vr%2Fr-n_0000000000)
 [n:127.0.0.1:55033_nv_vr%2Fr    ] o.a.s.c.OverseerTaskProcessor Process 
current queue of overseer operations
   [junit4]   2> 1202834 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) 
[n:127.0.0.1:55033_nv_vr%2Fr    ] o.a.s.c.c.ZkStateReader Updating cluster 
state from ZooKeeper... 
   [junit4]   2> 1202835 INFO  
(OverseerStateUpdate-94819512677629958-127.0.0.1:55033_nv_vr%2Fr-n_0000000000) 
[n:127.0.0.1:55033_nv_vr%2Fr    ] o.a.s.c.Overseer Starting to work on the main 
queue
   [junit4]   2> 1202849 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) 
[n:127.0.0.1:55033_nv_vr%2Fr    ] o.a.s.c.CoreContainer Security conf doesn't 
exist. Skipping setup for authorization module.
   [junit4]   2> 1202849 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) 
[n:127.0.0.1:55033_nv_vr%2Fr    ] o.a.s.c.CoreContainer No authentication 
plugin used.
   [junit4]   2> 1202849 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) 
[n:127.0.0.1:55033_nv_vr%2Fr    ] o.a.s.c.CoresLocator Looking for core 
definitions underneath 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_7205036CC31451AA-001/control-001/cores
   [junit4]   2> 1202850 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) 
[n:127.0.0.1:55033_nv_vr%2Fr    ] o.a.s.c.SolrCore Created CoreDescriptor: 
{name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, 
loadOnStartup=true, configSetProperties=configsetprops.json, 
instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_7205036CC31451AA-001/control-001/cores/collection1,
 collection=control_collection, 
absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_7205036CC31451AA-001/control-001/cores/collection1/,
 coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 1202850 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) 
[n:127.0.0.1:55033_nv_vr%2Fr    ] o.a.s.c.CoresLocator Found core collection1 
in 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_7205036CC31451AA-001/control-001/cores/collection1/
   [junit4]   2> 1202850 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) 
[n:127.0.0.1:55033_nv_vr%2Fr    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1202852 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection   x:collection1] 
o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1202852 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection   x:collection1] 
o.a.s.c.ZkController numShards not found on descriptor - reading it from system 
property
   [junit4]   2> 1202852 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) 
[n:127.0.0.1:55033_nv_vr%2Fr    ] o.a.s.s.SolrDispatchFilter 
user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2
   [junit4]   2> 1202852 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]) 
[n:127.0.0.1:55033_nv_vr%2Fr    ] o.a.s.s.SolrDispatchFilter 
SolrDispatchFilter.init() done
   [junit4]   2> 1202853 INFO  
(OverseerStateUpdate-94819512677629958-127.0.0.1:55033_nv_vr%2Fr-n_0000000000) 
[n:127.0.0.1:55033_nv_vr%2Fr    ] o.a.s.c.Overseer processMessage: queueSize: 
1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:55033/nv_vr/r";,
   [junit4]   2>   "node_name":"127.0.0.1:55033_nv_vr%2Fr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 1202853 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection   x:collection1] 
o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1202854 INFO  
(OverseerStateUpdate-94819512677629958-127.0.0.1:55033_nv_vr%2Fr-n_0000000000) 
[n:127.0.0.1:55033_nv_vr%2Fr    ] o.a.s.c.o.ReplicaMutator Update state 
numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:55033/nv_vr/r";,
   [junit4]   2>   "node_name":"127.0.0.1:55033_nv_vr%2Fr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1202854 INFO  
(OverseerStateUpdate-94819512677629958-127.0.0.1:55033_nv_vr%2Fr-n_0000000000) 
[n:127.0.0.1:55033_nv_vr%2Fr    ] o.a.s.c.o.ClusterStateMutator building a new 
cName: control_collection
   [junit4]   2> 1202854 INFO  
(OverseerStateUpdate-94819512677629958-127.0.0.1:55033_nv_vr%2Fr-n_0000000000) 
[n:127.0.0.1:55033_nv_vr%2Fr    ] o.a.s.c.o.ReplicaMutator Assigning new node 
to shard shard=shard1
   [junit4]   2> 1202856 INFO  
(zkCallback-882-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr    ] o.a.s.c.c.ZkStateReader A cluster state 
change: WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1203854 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection   x:collection1] 
o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1203854 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection   x:collection1] 
o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1203855 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection   x:collection1] 
o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1203855 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection   x:collection1] 
o.a.s.c.c.ZkStateReader Load collection config 
from:/collections/control_collection
   [junit4]   2> 1203855 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection   x:collection1] 
o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 
specified config exists in ZooKeeper
   [junit4]   2> 1203856 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection   x:collection1] 
o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: 
'/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_7205036CC31451AA-001/control-001/cores/collection1/'
   [junit4]   2> 1203856 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection   x:collection1] 
o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1203856 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection   x:collection1] 
o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find 
system property or JNDI)
   [junit4]   2> 1203872 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection   x:collection1] 
o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1203878 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection   x:collection1] 
o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1203899 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection   x:collection1] 
o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 1203912 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection   x:collection1] 
o.a.s.c.Config Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1203913 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection   x:collection1] 
o.a.s.s.IndexSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1203920 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection   x:collection1] 
o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1204021 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection   x:collection1] 
o.a.s.s.OpenExchangeRatesOrgProvider Initialized with 
rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1204028 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection   x:collection1] 
o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1204029 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection   x:collection1] 
o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1204038 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection   x:collection1] 
o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1204040 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection   x:collection1] 
o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1204042 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection   x:collection1] 
o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from 
open-exchange-rates.json
   [junit4]   2> 1204044 WARN  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection   x:collection1] 
o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1204044 WARN  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection   x:collection1] 
o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1204044 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection   x:collection1] 
o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from 
open-exchange-rates.json
   [junit4]   2> 1204045 WARN  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection   x:collection1] 
o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 1204045 WARN  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection   x:collection1] 
o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 1204046 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection   x:collection1] 
o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default 
properties: Can't find resource 'configsetprops.json' in classpath or 
'/configs/conf1', 
cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2
   [junit4]   2> 1204046 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection   x:collection1] 
o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from 
collection control_collection
   [junit4]   2> 1204046 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 1204046 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.HdfsDirectoryFactory 
solr.hdfs.home=hdfs://localhost:51218/solr_hdfs_home
   [junit4]   2> 1204047 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication 
disabled
   [junit4]   2> 1204047 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at 
[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_7205036CC31451AA-001/control-001/cores/collection1/],
 dataDir=[null]
   [junit4]   2> 1204047 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr 
mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@631b4b36
   [junit4]   2> 1204047 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path 
hdfs://localhost:51218/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 1204066 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for 
hdfs://localhost:51218/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 1204067 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrCore New index directory detected: old=null 
new=hdfs://localhost:51218/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 1204077 WARN  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 
'hdfs:/localhost:51218/solr_hdfs_home/control_collection/core_node1/data/index' 
doesn't exist. Creating new index...
   [junit4]   2> 1204077 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path 
hdfs://localhost:51218/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 1204088 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for 
hdfs://localhost:51218/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 1204088 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: 
maxMergeAtOnce=37, maxMergeAtOnceExplicit=36, maxMergedSegmentMB=87.4072265625, 
floorSegmentMB=2.1171875, forceMergeDeletesPctAllowed=4.299913233179417, 
segmentsPerTier=16.0, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.3022367295207219
   [junit4]   2> 1204114 INFO  (IPC Server handler 9 on 51218) [    ] 
BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:38531 is 
added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, 
primaryNodeIndex=-1, 
replicas=[ReplicaUnderConstruction[[DISK]DS-7de94fe4-e400-4148-a12c-80bb6da79b6f:NORMAL:127.0.0.1:35751|RBW],
 
ReplicaUnderConstruction[[DISK]DS-315151a4-ceb9-4f0d-ac3f-1a24b5077ba5:NORMAL:127.0.0.1:38531|RBW]]}
 size 0
   [junit4]   2> 1204116 INFO  (IPC Server handler 0 on 51218) [    ] 
BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:35751 is 
added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, 
primaryNodeIndex=-1, 
replicas=[ReplicaUnderConstruction[[DISK]DS-315151a4-ceb9-4f0d-ac3f-1a24b5077ba5:NORMAL:127.0.0.1:38531|RBW],
 
ReplicaUnderConstruction[[DISK]DS-5a068994-9493-4dac-bd15-99e267052c30:NORMAL:127.0.0.1:35751|FINALIZED]]}
 size 0
   [junit4]   2> 1204123 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2>        commit{dir=NRTCachingDirectory(HdfsDirectory@b81c8180 
lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6b3560ac; 
maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 1204123 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1204125 INFO  
(OldIndexDirectoryCleanupThreadForCore-collection1) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup 
for core collection1 in 
hdfs://localhost:51218/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 1204128 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating 
updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1204128 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating 
updateRequestProcessorChain "dedupe"
   [junit4]   2> 1204128 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting 
DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1204129 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating 
updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1204129 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting 
DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1204129 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating 
updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1204129 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating 
updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1204129 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting 
DistributedUpdateProcessorFactory into updateRequestProcessorChain 
"distrib-dup-test-chain-implicit"
   [junit4]   2> 1204129 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as 
default, creating implicit default
   [junit4]   2> 1204129 WARN  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler 
{type = requestHandler,name = /dump,class = DumpRequestHandler,args = 
{defaults={a=A,b=B}}}
   [junit4]   2> 1204131 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1204132 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1204133 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1204134 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1204142 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.RequestHandlers Registered paths: 
/admin/mbeans,standard,/dump,/update/csv,/update/json/docs,/admin/luke,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/schema,/admin/plugins,/admin/logging,/update/json,/admin/threads,/admin/ping,/update,/admin/file
   [junit4]   2> 1204142 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrCore Using default statsCache cache: 
org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1204143 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: 
org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 1204143 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.UpdateLog Initializing HdfsUpdateLog: dataDir= 
defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 
tlogDfsReplication=2
   [junit4]   2> 1204155 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 1204155 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 1204156 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: 
minMergeSize=0, mergeFactor=10, maxMergeSize=1171467545, 
maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, 
maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.1]
   [junit4]   2> 1204173 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2>        commit{dir=NRTCachingDirectory(HdfsDirectory@b81c8180 
lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6b3560ac; 
maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 1204173 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1204173 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@715e3165[collection1] 
main
   [junit4]   2> 1204173 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.c.ZkStateReader Load collection config 
from:/collections/control_collection
   [junit4]   2> 1204174 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.c.ZkStateReader path=/collections/control_collection 
configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1204174 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based 
storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1204174 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO 
with znodeBase: /configs/conf1
   [junit4]   2> 1204174 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1204174 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1204175 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode 
/configs/conf1/_rest_managed.json
   [junit4]   2> 1204175 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path 
_rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1204175 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1204175 INFO  
(coreLoadExecutor-2552-thread-1-processing-n:127.0.0.1:55033_nv_vr%2Fr) 
[n:127.0.0.1:55033_nv_vr%2Fr c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 120417

[...truncated too long message...]

y.
   [junit4]   2> NOTE: reproduce with: ant test  
-Dtestcase=HdfsCollectionsAPIDistributedZkTest -Dtests.seed=7205036CC31451AA 
-Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true 
-Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt 
-Dtests.locale=bg_BG -Dtests.timezone=Europe/Kaliningrad -Dtests.asserts=true 
-Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J2 | HdfsCollectionsAPIDistributedZkTest (suite) <<<
   [junit4]    > Throwable #1: 
com.carrotsearch.randomizedtesting.ThreadLeakError: 5 threads leaked from SUITE 
scope at org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest: 
   [junit4]    >    1) Thread[id=17425, name=zkCallback-909-thread-3, 
state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]    >         at 
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    2) Thread[id=17424, name=zkCallback-909-thread-2, 
state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]    >         at 
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    3) Thread[id=16989, 
name=TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]-SendThread(127.0.0.1:46837),
 state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
   [junit4]    >         at java.lang.Thread.sleep(Native Method)
   [junit4]    >         at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994)
   [junit4]    >    4) Thread[id=16991, name=zkCallback-909-thread-1, 
state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]    >         at 
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    5) Thread[id=16990, 
name=TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]-EventThread,
 state=WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]    >         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]    >         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at 
org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([7205036CC31451AA]:0)Throwable #2: 
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie 
threads that couldn't be terminated:
   [junit4]    >    1) Thread[id=17425, name=zkCallback-909-thread-3, 
state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]    >         at 
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    2) Thread[id=17424, name=zkCallback-909-thread-2, 
state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]    >         at 
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    3) Thread[id=16989, 
name=TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[7205036CC31451AA]-SendThread(127.0.0.1:46837),
 state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
   [junit4]    >         at java.lang.Thread.sleep(Native Method)
   [junit4]    >         at 
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994)
   [junit4]    >    4) Thread[id=16991, name=zkCallback-909-thread-1, 
state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at 
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at 
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]    >         at 
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([7205036CC31451AA]:0)
   [junit4] Completed [239/555] on J2 in 86.36s, 1 test, 3 errors <<< FAILURES!

[...truncated 977 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:782:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:719:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:59:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build.xml:233:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/common-build.xml:516:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:1432:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:987:
 There were test failures: 555 suites (8 ignored), 2185 tests, 2 suite-level 
errors, 1 error, 967 ignored (16 assumptions) [seed: 7205036CC31451AA]

Total time: 177 minutes 46 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Reply via email to