Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.5/2/

8 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.core.TestLazyCores

Error Message:
ObjectTracker found 4 object(s) that were not released!!! 
[MockDirectoryWrapper, RawDirectoryWrapper, MDCAwareThreadPoolExecutor, 
SolrCore]

Stack Trace:
java.lang.AssertionError: ObjectTracker found 4 object(s) that were not 
released!!! [MockDirectoryWrapper, RawDirectoryWrapper, 
MDCAwareThreadPoolExecutor, SolrCore]
        at __randomizedtesting.SeedInfo.seed([63E09FED207FBA1]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertNull(Assert.java:551)
        at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:228)
        at sun.reflect.GeneratedMethodAccessor68.invoke(Unknown Source)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.core.TestLazyCores

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.core.TestLazyCores:     1) 
Thread[id=28792, name=searcherExecutor-5358-thread-1, state=WAITING, 
group=TGRP-TestLazyCores]         at sun.misc.Unsafe.park(Native Method)        
 at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)     
    at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
        at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE 
scope at org.apache.solr.core.TestLazyCores: 
   1) Thread[id=28792, name=searcherExecutor-5358-thread-1, state=WAITING, 
group=TGRP-TestLazyCores]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
        at __randomizedtesting.SeedInfo.seed([63E09FED207FBA1]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.core.TestLazyCores

Error Message:
There are still zombie threads that couldn't be terminated:    1) 
Thread[id=28792, name=searcherExecutor-5358-thread-1, state=WAITING, 
group=TGRP-TestLazyCores]         at sun.misc.Unsafe.park(Native Method)        
 at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)     
    at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)   
      at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
        at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie 
threads that couldn't be terminated:
   1) Thread[id=28792, name=searcherExecutor-5358-thread-1, state=WAITING, 
group=TGRP-TestLazyCores]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
        at __randomizedtesting.SeedInfo.seed([63E09FED207FBA1]:0)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Timeout occured while waiting response from server at: 
http://127.0.0.1:55648/_s/yv

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting 
response from server at: http://127.0.0.1:55648/_s/yv
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:586)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
        at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
        at 
org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:400)
        at 
org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:458)
        at 
org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:179)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:996)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:971)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketTimeoutException: Read timed out
        at java.net.SocketInputStream.socketRead0(Native Method)
        at java.net.SocketInputStream.read(SocketInputStream.java:152)
        at java.net.SocketInputStream.read(SocketInputStream.java:122)
        at 
org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
        at 
org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
        at 
org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
        at 
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
        at 
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
        at 
org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
        at 
org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
        at 
org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
        at 
org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
        at 
org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
        at 
org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
        at 
org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
        at 
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
        at 
org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
        at 
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
        at 
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
        at 
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:482)
        ... 47 more


FAILED:  org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test

Error Message:


Stack Trace:
java.lang.AssertionError
        at 
__randomizedtesting.SeedInfo.seed([63E09FED207FBA1:8E6A36247CFB9659]:0)
        at org.junit.Assert.fail(Assert.java:92)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertTrue(Assert.java:54)
        at 
org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.testBasics(SharedFSAutoReplicaFailoverTest.java:226)
        at 
org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test(SharedFSAutoReplicaFailoverTest.java:107)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:996)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:971)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:


Stack Trace:
java.lang.AssertionError
        at 
__randomizedtesting.SeedInfo.seed([63E09FED207FBA1:8E6A36247CFB9659]:0)
        at org.junit.Assert.fail(Assert.java:92)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertNull(Assert.java:551)
        at org.junit.Assert.assertNull(Assert.java:562)
        at 
org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testNoConfigSetExist(CollectionsAPIDistributedZkTest.java:615)
        at 
org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:176)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:996)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:971)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at java.lang.Thread.run(Thread.java:745)


FAILED:  
org.apache.solr.cloud.overseer.ZkStateReaderTest.testStateFormatUpdateWithTimeDelay

Error Message:
Could not find collection : c1

Stack Trace:
org.apache.solr.common.SolrException: Could not find collection : c1
        at 
__randomizedtesting.SeedInfo.seed([63E09FED207FBA1:79A0BE7BBB65D62B]:0)
        at 
org.apache.solr.common.cloud.ClusterState.getCollection(ClusterState.java:170)
        at 
org.apache.solr.cloud.overseer.ZkStateReaderTest.testStateFormatUpdate(ZkStateReaderTest.java:129)
        at 
org.apache.solr.cloud.overseer.ZkStateReaderTest.testStateFormatUpdateWithTimeDelay(ZkStateReaderTest.java:52)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at java.lang.Thread.run(Thread.java:745)


FAILED:  
org.apache.solr.cloud.overseer.ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh

Error Message:
Could not find collection : c1

Stack Trace:
org.apache.solr.common.SolrException: Could not find collection : c1
        at 
__randomizedtesting.SeedInfo.seed([63E09FED207FBA1:1984780902673D64]:0)
        at 
org.apache.solr.common.cloud.ClusterState.getCollection(ClusterState.java:170)
        at 
org.apache.solr.cloud.overseer.ZkStateReaderTest.testStateFormatUpdate(ZkStateReaderTest.java:136)
        at 
org.apache.solr.cloud.overseer.ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh(ZkStateReaderTest.java:42)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11192 lines...]
   [junit4] Suite: 
org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_63E09FED207FBA1-001/init-core-data-001
   [junit4]   2> 413631 WARN  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[63E09FED207FBA1]-worker) [    
] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=6 numCloses=6
   [junit4]   2> 413631 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[63E09FED207FBA1]-worker) [    
] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false)
   [junit4]   2> 413631 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[63E09FED207FBA1]-worker) [    
] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: 
/ojwf/
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 413666 WARN  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[63E09FED207FBA1]-worker) [    
] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried 
hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 413674 WARN  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[63E09FED207FBA1]-worker) [    
] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 413679 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[63E09FED207FBA1]-worker) [    
] o.m.log jetty-6.1.26
   [junit4]   2> 413700 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[63E09FED207FBA1]-worker) [    
] o.m.log Extract 
jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs
 to ./temp/Jetty_localhost_53272_hdfs____.otzoyv/webapp
   [junit4]   2> 413792 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[63E09FED207FBA1]-worker) [    
] o.m.log NO JSP Support for /, did not find 
org.apache.jasper.servlet.JspServlet
   [junit4]   2> 414162 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[63E09FED207FBA1]-worker) [    
] o.m.log Started 
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:53272
   [junit4]   2> 414270 WARN  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[63E09FED207FBA1]-worker) [    
] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 414273 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[63E09FED207FBA1]-worker) [    
] o.m.log jetty-6.1.26
   [junit4]   2> 414281 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[63E09FED207FBA1]-worker) [    
] o.m.log Extract 
jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode
 to ./temp/Jetty_localhost_43764_datanode____lq7z99/webapp
   [junit4]   2> 414561 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[63E09FED207FBA1]-worker) [    
] o.m.log NO JSP Support for /, did not find 
org.apache.jasper.servlet.JspServlet
   [junit4]   2> 414992 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[63E09FED207FBA1]-worker) [    
] o.m.log Started 
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43764
   [junit4]   2> 415156 WARN  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[63E09FED207FBA1]-worker) [    
] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 415158 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[63E09FED207FBA1]-worker) [    
] o.m.log jetty-6.1.26
   [junit4]   2> 415168 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[63E09FED207FBA1]-worker) [    
] o.m.log Extract 
jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode
 to ./temp/Jetty_localhost_55537_datanode____.c45dk0/webapp
   [junit4]   2> 415364 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[63E09FED207FBA1]-worker) [    
] o.m.log NO JSP Support for /, did not find 
org.apache.jasper.servlet.JspServlet
   [junit4]   2> 415594 INFO  (IPC Server handler 3 on 52446) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-ae316d87-005d-46f3-88b0-8e202fb1372c node DatanodeRegistration(127.0.0.1, 
datanodeUuid=afa803da-88e7-4b65-890f-27a3aa7e2203, infoPort=43764, 
ipcPort=35638, storageInfo=lv=-56;cid=testClusterID;nsid=767148586;c=0), 
blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 415594 INFO  (IPC Server handler 3 on 52446) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-37163f07-9c83-46d5-813f-57ec8431bd64 node DatanodeRegistration(127.0.0.1, 
datanodeUuid=afa803da-88e7-4b65-890f-27a3aa7e2203, infoPort=43764, 
ipcPort=35638, storageInfo=lv=-56;cid=testClusterID;nsid=767148586;c=0), 
blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 415876 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[63E09FED207FBA1]-worker) [    
] o.m.log Started 
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:55537
   [junit4]   2> 416103 INFO  (IPC Server handler 7 on 52446) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-3007f692-1538-4e10-a2dd-28dabd561e1f node DatanodeRegistration(127.0.0.1, 
datanodeUuid=96195079-579d-4844-8da9-b604454ea605, infoPort=55537, 
ipcPort=60506, storageInfo=lv=-56;cid=testClusterID;nsid=767148586;c=0), 
blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 416103 INFO  (IPC Server handler 7 on 52446) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-5867a145-c133-4d0b-a7b4-639d75c3c9d9 node DatanodeRegistration(127.0.0.1, 
datanodeUuid=96195079-579d-4844-8da9-b604454ea605, infoPort=55537, 
ipcPort=60506, storageInfo=lv=-56;cid=testClusterID;nsid=767148586;c=0), 
blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 416229 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 416241 INFO  (Thread-1098) [    ] o.a.s.c.ZkTestServer client 
port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 416242 INFO  (Thread-1098) [    ] o.a.s.c.ZkTestServer 
Starting server
   [junit4]   2> 416340 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.ZkTestServer start zk server on port:36934
   [junit4]   2> 416340 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 416350 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 416374 INFO  (zkCallback-298-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@47c4fb3c 
name:ZooKeeperConnection Watcher:127.0.0.1:36934 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 416376 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 416376 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 416376 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 416542 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 416557 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 416569 INFO  (zkCallback-299-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@192df244 
name:ZooKeeperConnection Watcher:127.0.0.1:36934/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 416570 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 416570 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 416570 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 416625 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 416672 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 416673 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 416674 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml
 to /configs/conf1/solrconfig.xml
   [junit4]   2> 416674 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 416676 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/core/src/test-files/solr/collection1/conf/schema.xml
 to /configs/conf1/schema.xml
   [junit4]   2> 416676 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 416677 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml
 to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 416677 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.c.SolrZkClient makePath: 
/configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 416678 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/core/src/test-files/solr/collection1/conf/stopwords.txt
 to /configs/conf1/stopwords.txt
   [junit4]   2> 416678 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 416679 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/core/src/test-files/solr/collection1/conf/protwords.txt
 to /configs/conf1/protwords.txt
   [junit4]   2> 416679 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 416680 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/core/src/test-files/solr/collection1/conf/currency.xml
 to /configs/conf1/currency.xml
   [junit4]   2> 416680 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 416681 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml
 to /configs/conf1/enumsConfig.xml
   [junit4]   2> 416681 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 416682 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json
 to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 416682 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 416683 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt
 to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 416683 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 416684 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt
 to /configs/conf1/old_synonyms.txt
   [junit4]   2> 416684 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 416685 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/core/src/test-files/solr/collection1/conf/synonyms.txt
 to /configs/conf1/synonyms.txt
   [junit4]   2> 416685 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 416686 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 416687 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 416688 INFO  (zkCallback-300-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@4e5ee59f 
name:ZooKeeperConnection Watcher:127.0.0.1:36934/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 416688 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 416688 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 416752 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.SolrTestCaseJ4 Writing core.properties file to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_63E09FED207FBA1-001/control-001/cores/collection1
   [junit4]   2> 416757 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 416826 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@5768d2ea{/ojwf,null,AVAILABLE}
   [junit4]   2> 416832 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.e.j.s.ServerConnector Started 
ServerConnector@22ef2d7b{SSL-http/1.1}{127.0.0.1:60639}
   [junit4]   2> 416832 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.e.j.s.Server Started @422054ms
   [junit4]   2> 416832 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/ojwf, 
hostPort=60639, 
solr.data.dir=hdfs://localhost:52446/hdfs__localhost_52446__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-5.5_solr_build_solr-core_test_J2_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_63E09FED207FBA1-001_tempDir-002_control_data,
 
coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_63E09FED207FBA1-001/control-001/cores}
   [junit4]   2> 416832 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): 
sun.misc.Launcher$AppClassLoader@7b3cb2c6
   [junit4]   2> 416832 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: 
'/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_63E09FED207FBA1-001/control-001'
   [junit4]   2> 416832 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 416832 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find 
system property or JNDI)
   [junit4]   2> 416833 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 416849 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 416858 INFO  (zkCallback-301-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@421b3e10 
name:ZooKeeperConnection Watcher:127.0.0.1:36934/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 416861 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 416862 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 416876 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in 
ZooKeeper)
   [junit4]   2> 416876 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.SolrXmlConfig Loading container configuration from 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_63E09FED207FBA1-001/control-001/solr.xml
   [junit4]   2> 416886 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.CorePropertiesLocator Config-defined core root directory: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_63E09FED207FBA1-001/control-001/cores
   [junit4]   2> 416886 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.CoreContainer New CoreContainer 665686237
   [junit4]   2> 416886 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.CoreContainer Loading cores into CoreContainer 
[instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_63E09FED207FBA1-001/control-001]
   [junit4]   2> 416887 WARN  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.CoreContainer Couldn't add files from 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_63E09FED207FBA1-001/control-001/lib
 to classpath: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_63E09FED207FBA1-001/control-001/lib
   [junit4]   2> 416887 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme 
: ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 
10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 
5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 416889 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: 
socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 416890 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 416890 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.l.LogWatcher Registering Log Listener [Log4j 
(org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 416890 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:36934/solr
   [junit4]   2> 416890 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 416890 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 416891 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 416981 INFO  (zkCallback-304-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@487f81d9 
name:ZooKeeperConnection Watcher:127.0.0.1:36934 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 416982 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 416982 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 416983 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) 
[n:127.0.0.1:60639_ojwf    ] o.a.s.c.ZkController Added new OnReconnect 
listener org.apache.solr.cloud.ZkController$6@78aecba1
   [junit4]   2> 416984 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) 
[n:127.0.0.1:60639_ojwf    ] o.a.s.c.c.ConnectionManager Waiting for client to 
connect to ZooKeeper
   [junit4]   2> 417002 INFO  
(zkCallback-305-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf    ] o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@6c6e9bab 
name:ZooKeeperConnection Watcher:127.0.0.1:36934/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 417002 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) 
[n:127.0.0.1:60639_ojwf    ] o.a.s.c.c.ConnectionManager Client is connected to 
ZooKeeper
   [junit4]   2> 417003 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) 
[n:127.0.0.1:60639_ojwf    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 417004 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) 
[n:127.0.0.1:60639_ojwf    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/collection-queue-work
   [junit4]   2> 417005 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) 
[n:127.0.0.1:60639_ojwf    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/collection-map-running
   [junit4]   2> 417006 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) 
[n:127.0.0.1:60639_ojwf    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/collection-map-completed
   [junit4]   2> 417007 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) 
[n:127.0.0.1:60639_ojwf    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/collection-map-failure
   [junit4]   2> 417008 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) 
[n:127.0.0.1:60639_ojwf    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 417009 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) 
[n:127.0.0.1:60639_ojwf    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 417009 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) 
[n:127.0.0.1:60639_ojwf    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 417010 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) 
[n:127.0.0.1:60639_ojwf    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 417010 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) 
[n:127.0.0.1:60639_ojwf    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:60639_ojwf
   [junit4]   2> 417011 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) 
[n:127.0.0.1:60639_ojwf    ] o.a.s.c.c.SolrZkClient makePath: 
/live_nodes/127.0.0.1:60639_ojwf
   [junit4]   2> 417022 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) 
[n:127.0.0.1:60639_ojwf    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 417027 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) 
[n:127.0.0.1:60639_ojwf    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer_elect/election
   [junit4]   2> 417045 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) 
[n:127.0.0.1:60639_ojwf    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 417050 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) 
[n:127.0.0.1:60639_ojwf    ] o.a.s.c.LeaderElector Joined leadership election 
with path: 
/overseer_elect/election/95780176274718725-127.0.0.1:60639_ojwf-n_0000000000
   [junit4]   2> 417053 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) 
[n:127.0.0.1:60639_ojwf    ] o.a.s.c.OverseerElectionContext I am going to be 
the leader 127.0.0.1:60639_ojwf
   [junit4]   2> 417054 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) 
[n:127.0.0.1:60639_ojwf    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer_elect/leader
   [junit4]   2> 417074 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) 
[n:127.0.0.1:60639_ojwf    ] o.a.s.c.Overseer Overseer 
(id=95780176274718725-127.0.0.1:60639_ojwf-n_0000000000) starting
   [junit4]   2> 417098 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) 
[n:127.0.0.1:60639_ojwf    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/queue-work
   [junit4]   2> 417194 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) 
[n:127.0.0.1:60639_ojwf    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting 
OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 
autoReplicaFailoverWaitAfterExpiration=10000 
autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 417199 INFO  
(OverseerCollectionConfigSetProcessor-95780176274718725-127.0.0.1:60639_ojwf-n_0000000000)
 [n:127.0.0.1:60639_ojwf    ] o.a.s.c.OverseerTaskProcessor Process current 
queue of overseer operations
   [junit4]   2> 417199 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) 
[n:127.0.0.1:60639_ojwf    ] o.a.s.c.c.ZkStateReader Updating cluster state 
from ZooKeeper... 
   [junit4]   2> 417201 INFO  
(OverseerStateUpdate-95780176274718725-127.0.0.1:60639_ojwf-n_0000000000) 
[n:127.0.0.1:60639_ojwf    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 417253 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) 
[n:127.0.0.1:60639_ojwf    ] o.a.s.c.CoreContainer Security conf doesn't exist. 
Skipping setup for authorization module.
   [junit4]   2> 417253 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) 
[n:127.0.0.1:60639_ojwf    ] o.a.s.c.CoreContainer No authentication plugin 
used.
   [junit4]   2> 417270 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) 
[n:127.0.0.1:60639_ojwf    ] o.a.s.c.CorePropertiesLocator Looking for core 
definitions underneath 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_63E09FED207FBA1-001/control-001/cores
   [junit4]   2> 417271 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) 
[n:127.0.0.1:60639_ojwf    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: 
{coreNodeName=, collection=control_collection, loadOnStartup=true, 
config=solrconfig.xml, shard=, name=collection1, 
configSetProperties=configsetprops.json, dataDir=data/, transient=false, 
schema=schema.xml}
   [junit4]   2> 417271 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) 
[n:127.0.0.1:60639_ojwf    ] o.a.s.c.CorePropertiesLocator Found core 
collection1 in 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_63E09FED207FBA1-001/control-001/cores/collection1
   [junit4]   2> 417271 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) 
[n:127.0.0.1:60639_ojwf    ] o.a.s.c.CorePropertiesLocator Found 1 core 
definitions
   [junit4]   2> 417278 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) 
[n:127.0.0.1:60639_ojwf    ] o.a.s.s.SolrDispatchFilter 
user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J2
   [junit4]   2> 417278 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[63E09FED207FBA1]) 
[n:127.0.0.1:60639_ojwf    ] o.a.s.s.SolrDispatchFilter 
SolrDispatchFilter.init() done
   [junit4]   2> 417278 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection   x:collection1] 
o.a.s.c.ZkController publishing state=down
   [junit4]   2> 417278 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection   x:collection1] 
o.a.s.c.ZkController numShards not found on descriptor - reading it from system 
property
   [junit4]   2> 417297 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection   x:collection1] 
o.a.s.c.ZkController Collection control_collection not visible yet, but 
flagging it so a watch is registered when it becomes visible
   [junit4]   2> 417298 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection   x:collection1] 
o.a.s.c.c.ZkStateReader addZkWatch [control_collection]
   [junit4]   2> 417309 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection   x:collection1] 
o.a.s.c.c.ZkStateReader Deleting data for [control_collection]
   [junit4]   2> 417310 INFO  
(OverseerStateUpdate-95780176274718725-127.0.0.1:60639_ojwf-n_0000000000) 
[n:127.0.0.1:60639_ojwf    ] o.a.s.c.Overseer processMessage: queueSize: 1, 
message = {
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:60639_ojwf",
   [junit4]   2>   "base_url":"https://127.0.0.1:60639/ojwf";,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 417314 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection   x:collection1] 
o.a.s.c.ZkController look for our core node name
   [junit4]   2> 417314 INFO  
(OverseerStateUpdate-95780176274718725-127.0.0.1:60639_ojwf-n_0000000000) 
[n:127.0.0.1:60639_ojwf    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 
message={
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "node_name":"127.0.0.1:60639_ojwf",
   [junit4]   2>   "base_url":"https://127.0.0.1:60639/ojwf";,
   [junit4]   2>   "state":"down",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 417314 INFO  
(OverseerStateUpdate-95780176274718725-127.0.0.1:60639_ojwf-n_0000000000) 
[n:127.0.0.1:60639_ojwf    ] o.a.s.c.o.ClusterStateMutator building a new 
cName: control_collection
   [junit4]   2> 417315 INFO  
(OverseerStateUpdate-95780176274718725-127.0.0.1:60639_ojwf-n_0000000000) 
[n:127.0.0.1:60639_ojwf    ] o.a.s.c.o.ReplicaMutator Assigning new node to 
shard shard=shard1
   [junit4]   2> 417316 INFO  
(zkCallback-305-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf    ] o.a.s.c.c.ZkStateReader A cluster state change: 
[WatchedEvent state:SyncConnected type:NodeDataChanged 
path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 418314 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection   x:collection1] 
o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 418314 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection   x:collection1] 
o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 418314 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection   x:collection1] 
o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 418314 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection   x:collection1] 
o.a.s.c.c.ZkStateReader Load collection config from: 
[/collections/control_collection]
   [junit4]   2> 418315 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection   x:collection1] 
o.a.s.c.c.ZkStateReader path=[/collections/control_collection] 
[configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 418315 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection   x:collection1] 
o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: 
'/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_63E09FED207FBA1-001/control-001/cores/collection1'
   [junit4]   2> 418315 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection   x:collection1] 
o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 418315 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection   x:collection1] 
o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find 
system property or JNDI)
   [junit4]   2> 418319 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection   x:collection1] o.a.s.c.Config 
loaded config solrconfig.xml with version 0 
   [junit4]   2> 418345 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection   x:collection1] 
o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 418360 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection   x:collection1] 
o.a.s.c.SolrConfig Using Lucene MatchVersion: 5.5.0
   [junit4]   2> 418393 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection   x:collection1] 
o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 418586 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection   x:collection1] 
o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 418769 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection   x:collection1] 
o.a.s.s.OpenExchangeRatesOrgProvider Initialized with 
rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 418774 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection   x:collection1] 
o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 418776 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection   x:collection1] 
o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 418782 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection   x:collection1] 
o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 418784 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection   x:collection1] 
o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 418786 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection   x:collection1] 
o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from 
open-exchange-rates.json
   [junit4]   2> 418787 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection   x:collection1] 
o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from 
open-exchange-rates.json
   [junit4]   2> 418788 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection   x:collection1] 
o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default 
properties: Can't find resource 'configsetprops.json' in classpath or 
'/configs/conf1', 
cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J2
   [junit4]   2> 418788 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection   x:collection1] 
o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from 
collection control_collection
   [junit4]   2> 418788 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 418788 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.HdfsDirectoryFactory 
solr.hdfs.home=hdfs://localhost:52446/solr_hdfs_home
   [junit4]   2> 418788 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication 
disabled
   [junit4]   2> 418788 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at 
[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_63E09FED207FBA1-001/control-001/cores/collection1],
 
dataDir=[hdfs://localhost:52446/solr_hdfs_home/control_collection/core_node1/data/]
   [junit4]   2> 418788 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr 
mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@14e6c243
   [junit4]   2> 418789 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path 
hdfs://localhost:52446/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 418812 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for 
hdfs://localhost:52446/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 418824 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrCore New index directory detected: old=null 
new=hdfs://localhost:52446/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 418839 WARN  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 
'hdfs:/localhost:52446/solr_hdfs_home/control_collection/core_node1/data/index' 
doesn't exist. Creating new index...
   [junit4]   2> 418839 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path 
hdfs://localhost:52446/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 418852 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for 
hdfs://localhost:52446/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 418852 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: 
minMergeSize=1677721, mergeFactor=17, maxMergeSize=2147483648, 
maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, 
maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.0]
   [junit4]   2> 418994 INFO  (IPC Server handler 5 on 52446) [    ] 
BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:58521 is 
added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, 
primaryNodeIndex=-1, 
replicas=[ReplicaUnderConstruction[[DISK]DS-ae316d87-005d-46f3-88b0-8e202fb1372c:NORMAL:127.0.0.1:58521|RBW],
 
ReplicaUnderConstruction[[DISK]DS-5867a145-c133-4d0b-a7b4-639d75c3c9d9:NORMAL:127.0.0.1:36627|RBW]]}
 size 0
   [junit4]   2> 418997 INFO  (IPC Server handler 8 on 52446) [    ] 
BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:36627 is 
added to blk_1073741825_1001 size 71
   [junit4]   2> 419014 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: 
num=1
   [junit4]   2>        
commit{dir=NRTCachingDirectory(HdfsDirectory@hdfs://localhost:52446/solr_hdfs_home/control_collection/core_node1/data/index
 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4c5721f0; 
maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 419014 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 419037 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating 
updateRequestProcessorChain "nodistrib"
   [junit4]   2> 419037 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating 
updateRequestProcessorChain "dedupe"
   [junit4]   2> 419037 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting 
DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 419037 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating 
updateRequestProcessorChain "stored_sig"
   [junit4]   2> 419037 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting 
DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 419037 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating 
updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 419037 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating 
updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 419038 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting 
DistributedUpdateProcessorFactory into updateRequestProcessorChain 
"distrib-dup-test-chain-implicit"
   [junit4]   2> 419038 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as 
default, creating implicit default
   [junit4]   2> 419038 WARN  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler 
{type = requestHandler,name = /dump,class = DumpRequestHandler,args = 
{defaults={a=A,b=B}}}
   [junit4]   2> 419040 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 419041 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 419049 INFO  
(OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:60639_ojwf 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore 
Looking for old index directories to cleanup for core collection1 in 
hdfs://localhost:52446/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 419062 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 419063 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 419066 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.RequestHandlers Registered paths: 
/admin/mbeans,/update,/update/json/docs,/admin/ping,/admin/segments,/admin/luke,/admin/file,standard,/admin/properties,/admin/system,/admin/plugins,/schema,/replication,/admin/logging,/config,/dump,/get,/admin/threads,/update/csv,/update/json
   [junit4]   2> 419067 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrCore Using default statsCache cache: 
org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 419075 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: 
org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 419075 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: dataDir= 
defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 
tlogDfsReplication=2
   [junit4]   2> 419127 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 419127 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 419128 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: 
maxMergeAtOnce=47, maxMergeAtOnceExplicit=42, maxMergedSegmentMB=9.568359375, 
floorSegmentMB=1.9638671875, forceMergeDeletesPctAllowed=19.739951894956217, 
segmentsPerTier=10.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 419193 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: 
num=1
   [junit4]   2>        
commit{dir=NRTCachingDirectory(HdfsDirectory@hdfs://localhost:52446/solr_hdfs_home/control_collection/core_node1/data/index
 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4c5721f0; 
maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 419193 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 419193 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.s.SolrIndexSearcher Opening Searcher@53bf87d1[collection1] 
main
   [junit4]   2> 419193 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: 
[/collections/control_collection]
   [junit4]   2> 419194 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] 
[configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 419194 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based 
storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 419195 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO 
with znodeBase: /configs/conf1
   [junit4]   2> 419195 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 419195 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 419195 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode 
/configs/conf1/_rest_managed.json
   [junit4]   2> 419195 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path 
_rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 419195 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 419195 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 419199 INFO  
(searcherExecutor-890-thread-1-processing-s:shard1 c:control_collection 
r:core_node1 n:127.0.0.1:60639_ojwf x:collection1) [n:127.0.0.1:60639_ojwf 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore 
[collection1] Registered new searcher Searcher@53bf87d1[collection1] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 419216 WARN  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.s.h.HdfsLocalityReporter Could not retrieve locality 
information for hdfs://localhost:57230/solr3 due to exception: 
java.net.ConnectException: Call From lucene1-us-west/10.41.0.5 to 
localhost:57230 failed on connection exception: java.net.ConnectException: 
Connection refused; For more details see:  
http://wiki.apache.org/hadoop/ConnectionRefused
   [junit4]   2> 419216 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed 
version buckets
   [junit4]   2> 419216 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 
256 version buckets from index
   [junit4]   2> 419216 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed 
version bucket highest value from index
   [junit4]   2> 419216 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent 
updates, using new clock 1532482823505575936
   [junit4]   2> 419216 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.u.UpdateLog Took 0.0ms to seed version buckets with 
highest version 1532482823505575936
   [junit4]   2> 419216 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 419222 INFO  
(coreLoadExecutor-889-thread-1-processing-n:127.0.0.1:60639_ojwf) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 419223 INFO  (coreZkRegister-883-thread-1-processing-s:shard1 
c:control_collection r:core_node1 n:127.0.0.1:60639_ojwf x:collection1) 
[n:127.0.0.1:60639_ojwf c:control_collection s:shard1 r:core_node1 
x:collection1] o.a.s.c.ZkController Regis

[...truncated too long message...]

ead leaked from SUITE scope at org.apache.solr.core.TestLazyCores: 
   [junit4]   2>    1) Thread[id=28792, name=searcherExecutor-5358-thread-1, 
state=WAITING, group=TGRP-TestLazyCores]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]   2>         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]   2>         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2>         at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]   2>         at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]   2>         at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Apr 24, 2016 10:12:13 AM 
com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: Starting to interrupt leaked threads:
   [junit4]   2>    1) Thread[id=28792, name=searcherExecutor-5358-thread-1, 
state=WAITING, group=TGRP-TestLazyCores]
   [junit4]   2> Apr 24, 2016 10:12:16 AM 
com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> SEVERE: There are still zombie threads that couldn't be 
terminated:
   [junit4]   2>    1) Thread[id=28792, name=searcherExecutor-5358-thread-1, 
state=WAITING, group=TGRP-TestLazyCores]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]   2>         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]   2>         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2>         at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]   2>         at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]   2>         at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene54): 
{v_t=PostingsFormat(name=Direct), v_s=BlockTreeOrds(blocksize=128), 
v_f=PostingsFormat(name=Memory doPackFST= true), 
id=PostingsFormat(name=Direct)}, docValues:{}, 
sim=RandomSimilarity(queryNorm=true,coord=crazy): {}, locale=da, 
timezone=America/Grenada
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 
1.7.0_80 (64-bit)/cpus=4,threads=2,free=346599432,total=532676608
   [junit4]   2> NOTE: All tests run in this JVM: 
[TestExclusionRuleCollectionAccess, ClusterStateUpdateTest, TestCryptoKeys, 
TestReloadAndDeleteDocs, ShardRoutingTest, TestBulkSchemaConcurrent, 
TestSurroundQueryParser, SharedFSAutoReplicaFailoverUtilsTest, TestUpdate, 
TestSolrConfigHandlerConcurrent, RecoveryAfterSoftCommitTest, 
HdfsBasicDistributedZk2Test, DistributedQueryComponentOptimizationTest, 
AtomicUpdatesTest, TestManagedSchemaFieldTypeResource, 
HdfsChaosMonkeySafeLeaderTest, TestFiltering, DistanceFunctionTest, 
HdfsRecoverLeaseTest, TimeZoneUtilsTest, TestClassicSimilarityFactory, 
ConvertedLegacyTest, TestCloudInspectUtil, TestRawTransformer, 
TestRequestStatusCollectionAPI, TestSolr4Spatial2, TestLRUStatsCache, 
SolrIndexSplitterTest, TestMiniSolrCloudClusterSSL, 
TestCursorMarkWithoutUniqueKey, TestSolrDeletionPolicy2, 
TestMaxScoreQueryParser, TestWordDelimiterFilterFactory, 
TestSerializedLuceneMatchVersion, ZkControllerTest, CustomCollectionTest, 
TestUseDocValuesAsStored2, TestStressVersions, CSVRequestHandlerTest, 
TestObjectReleaseTracker, ZkStateReaderTest, TestConfigOverlay, 
SoftAutoCommitTest, TestDistributedStatsComponentCardinality, 
ConfigSetsAPITest, DocumentBuilderTest, DistributedMLTComponentTest, 
DistributedFacetPivotLargeTest, TestLMJelinekMercerSimilarityFactory, 
SolrInfoMBeanTest, SimplePostToolTest, TestHashPartitioner, 
DocValuesMissingTest, TestCollapseQParserPlugin, 
OpenExchangeRatesOrgProviderTest, AliasIntegrationTest, 
LeaderInitiatedRecoveryOnShardRestartTest, TestAnalyzeInfixSuggestions, 
TestDistributedGrouping, TestSearcherReuse, TestCodecSupport, 
TestCoreContainer, TestReRankQParserPlugin, TestSchemaResource, 
HttpPartitionTest, SpatialFilterTest, TestChildDocTransformer, 
TestReqParamsAPI, MinimalSchemaTest, NumericFieldsTest, 
IgnoreCommitOptimizeUpdateProcessorFactoryTest, RulesTest, 
ChaosMonkeySafeLeaderTest, SortSpecParsingTest, TestBulkSchemaAPI, 
DocValuesTest, HdfsSyncSliceTest, ShardRoutingCustomTest, 
SolrCmdDistributorTest, TestInfoStreamLogging, ChaosMonkeyNothingIsSafeTest, 
DistributedFacetPivotSmallTest, TestPartialUpdateDeduplication, 
CoreMergeIndexesAdminHandlerTest, UniqFieldsUpdateProcessorFactoryTest, 
SpellCheckCollatorTest, TestDistribIDF, TestFastLRUCache, 
SpellCheckComponentTest, MultiTermTest, ZkSolrClientTest, TestQueryTypes, 
FullSolrCloudDistribCmdsTest, TestJoin, DistributedTermsComponentTest, 
SolrCoreTest, PeerSyncTest, TestSort, BasicFunctionalityTest, TestLazyCores]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 
'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestLazyCores 
-Dtests.seed=63E09FED207FBA1 -Dtests.multiplier=2 -Dtests.nightly=true 
-Dtests.slow=true 
-Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt 
-Dtests.locale=da -Dtests.timezone=America/Grenada -Dtests.asserts=true 
-Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J1 | TestLazyCores (suite) <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: ObjectTracker found 4 
object(s) that were not released!!! [MockDirectoryWrapper, RawDirectoryWrapper, 
MDCAwareThreadPoolExecutor, SolrCore]
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([63E09FED207FBA1]:0)
   [junit4]    >        at 
org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:228)
   [junit4]    >        at java.lang.Thread.run(Thread.java:745)Throwable #2: 
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE 
scope at org.apache.solr.core.TestLazyCores: 
   [junit4]    >    1) Thread[id=28792, name=searcherExecutor-5358-thread-1, 
state=WAITING, group=TGRP-TestLazyCores]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([63E09FED207FBA1]:0)Throwable #3: 
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie 
threads that couldn't be terminated:
   [junit4]    >    1) Thread[id=28792, name=searcherExecutor-5358-thread-1, 
state=WAITING, group=TGRP-TestLazyCores]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([63E09FED207FBA1]:0)
   [junit4] Completed [360/571 (5!)] on J1 in 154.63s, 8 tests, 1 failure, 2 
errors <<< FAILURES!

[...truncated 653 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/build.xml:757: 
The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/build.xml:694: 
The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/build.xml:59: 
The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/build.xml:233:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/solr/common-build.xml:534:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/lucene/common-build.xml:1477:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.5/lucene/common-build.xml:1033:
 There were test failures: 571 suites (7 ignored), 2289 tests, 3 suite-level 
errors, 3 errors, 2 failures, 593 ignored (10 assumptions) [seed: 
63E09FED207FBA1]

Total time: 256 minutes 29 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Reply via email to