Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-trunk/821/

4 tests failed.
FAILED:  
org.apache.solr.cloud.SolrCloudExampleTest.testLoadDocsIntoGettingStartedCollection

Error Message:
Delete action failed!

Stack Trace:
java.lang.AssertionError: Delete action failed!
        at 
__randomizedtesting.SeedInfo.seed([B49F89C8FEBF92DB:A7FCBBA7CFD02B7D]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at 
org.apache.solr.cloud.SolrCloudExampleTest.doTestDeleteAction(SolrCloudExampleTest.java:169)
        at 
org.apache.solr.cloud.SolrCloudExampleTest.testLoadDocsIntoGettingStartedCollection(SolrCloudExampleTest.java:145)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:864)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:900)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:914)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:873)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:775)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:809)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:820)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=3196, name=collection3, 
state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught 
exception in thread: Thread[id=3196, name=collection3, state=RUNNABLE, 
group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: 
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at http://127.0.0.1:58356: Could not find collection : 
awholynewstresscollection_collection3_0
        at __randomizedtesting.SeedInfo.seed([B49F89C8FEBF92DB]:0)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
        at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
        at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
        at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
        at 
org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=44001, name=collection0, 
state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught 
exception in thread: Thread[id=44001, name=collection0, state=RUNNABLE, 
group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: java.lang.RuntimeException: 
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available 
to handle this request:[http://127.0.0.1:32967/tv_mx, 
http://127.0.0.1:52574/tv_mx, http://127.0.0.1:35816/tv_mx, 
http://127.0.0.1:42093/tv_mx, http://127.0.0.1:58837/tv_mx]
        at __randomizedtesting.SeedInfo.seed([B49F89C8FEBF92DB]:0)
        at 
org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:898)
Caused by: org.apache.solr.client.solrj.SolrServerException: No live 
SolrServers available to handle this request:[http://127.0.0.1:32967/tv_mx, 
http://127.0.0.1:52574/tv_mx, http://127.0.0.1:35816/tv_mx, 
http://127.0.0.1:42093/tv_mx, http://127.0.0.1:58837/tv_mx]
        at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
        at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
        at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1574)
        at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1595)
        at 
org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:888)
Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured 
while waiting response from server at: http://127.0.0.1:52574/tv_mx
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:587)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
        at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
        at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
        ... 7 more
Caused by: java.net.SocketTimeoutException: Read timed out
        at java.net.SocketInputStream.socketRead0(Native Method)
        at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
        at java.net.SocketInputStream.read(SocketInputStream.java:170)
        at java.net.SocketInputStream.read(SocketInputStream.java:141)
        at 
org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
        at 
org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
        at 
org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
        at 
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
        at 
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
        at 
org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
        at 
org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
        at 
org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
        at 
org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
        at 
org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
        at 
org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
        at 
org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
        at 
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
        at 
org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
        at 
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
        at 
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
        at 
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:480)
        ... 11 more


FAILED:  
org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR

Error Message:
Captured an uncaught exception in thread: Thread[id=51242, 
name=coreZkRegister-5187-thread-1, state=RUNNABLE, 
group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught 
exception in thread: Thread[id=51242, name=coreZkRegister-5187-thread-1, 
state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
Caused by: java.lang.AssertionError
        at __randomizedtesting.SeedInfo.seed([B49F89C8FEBF92DB]:0)
        at 
org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2126)
        at 
org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:433)
        at 
org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
        at 
org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
        at 
org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
        at 
org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
        at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
        at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
        at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
        at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 9926 lines...]
   [junit4] Suite: 
org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B49F89C8FEBF92DB-001/init-core-data-001
   [junit4]   2> 284955 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B49F89C8FEBF92DB]-worker) [    
] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 284955 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B49F89C8FEBF92DB]-worker) [    
] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 285002 WARN  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B49F89C8FEBF92DB]-worker) [    
] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried 
hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 285013 WARN  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B49F89C8FEBF92DB]-worker) [    
] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 285016 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B49F89C8FEBF92DB]-worker) [    
] o.m.log jetty-6.1.26
   [junit4]   2> 285027 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B49F89C8FEBF92DB]-worker) [    
] o.m.log Extract 
jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs
 to ./temp/Jetty_localhost_51370_hdfs____yo2ca/webapp
   [junit4]   2> 285144 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B49F89C8FEBF92DB]-worker) [    
] o.m.log NO JSP Support for /, did not find 
org.apache.jasper.servlet.JspServlet
   [junit4]   2> 285456 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B49F89C8FEBF92DB]-worker) [    
] o.m.log Started 
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:51370
   [junit4]   2> 285594 WARN  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B49F89C8FEBF92DB]-worker) [    
] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 285595 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B49F89C8FEBF92DB]-worker) [    
] o.m.log jetty-6.1.26
   [junit4]   2> 285611 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B49F89C8FEBF92DB]-worker) [    
] o.m.log Extract 
jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode
 to ./temp/Jetty_localhost_60252_datanode____.ervdxg/webapp
   [junit4]   2> 285728 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B49F89C8FEBF92DB]-worker) [    
] o.m.log NO JSP Support for /, did not find 
org.apache.jasper.servlet.JspServlet
   [junit4]   2> 285994 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B49F89C8FEBF92DB]-worker) [    
] o.m.log Started 
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:60252
   [junit4]   2> 286050 WARN  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B49F89C8FEBF92DB]-worker) [    
] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 286051 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B49F89C8FEBF92DB]-worker) [    
] o.m.log jetty-6.1.26
   [junit4]   2> 286085 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B49F89C8FEBF92DB]-worker) [    
] o.m.log Extract 
jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode
 to ./temp/Jetty_localhost_35232_datanode____yndiae/webapp
   [junit4]   2> 286214 INFO  (IPC Server handler 3 on 42096) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-aef57e15-51c3-46a6-bf2c-96ba5749335a node DatanodeRegistration(127.0.0.1, 
datanodeUuid=241c8848-7ba1-469b-aa60-5d7eeba9b606, infoPort=60252, 
ipcPort=46117, storageInfo=lv=-56;cid=testClusterID;nsid=1340850404;c=0), 
blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 286214 INFO  (IPC Server handler 3 on 42096) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-23c65777-3ea2-41b2-8ec4-fa2ba211c787 node DatanodeRegistration(127.0.0.1, 
datanodeUuid=241c8848-7ba1-469b-aa60-5d7eeba9b606, infoPort=60252, 
ipcPort=46117, storageInfo=lv=-56;cid=testClusterID;nsid=1340850404;c=0), 
blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 286227 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B49F89C8FEBF92DB]-worker) [    
] o.m.log NO JSP Support for /, did not find 
org.apache.jasper.servlet.JspServlet
   [junit4]   2> 286410 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B49F89C8FEBF92DB]-worker) [    
] o.m.log Started 
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:35232
   [junit4]   2> 286489 INFO  (IPC Server handler 8 on 42096) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-97a4113a-3b2c-4bc2-92f2-31d96372015d node DatanodeRegistration(127.0.0.1, 
datanodeUuid=751a18cc-629c-4787-9c2a-fa9152fb21cf, infoPort=35232, 
ipcPort=42302, storageInfo=lv=-56;cid=testClusterID;nsid=1340850404;c=0), 
blocks: 0, hasStaleStorages: true, processing time: 0 msecs
   [junit4]   2> 286490 INFO  (IPC Server handler 8 on 42096) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-253de805-41bf-4423-869c-02a1c1d16a38 node DatanodeRegistration(127.0.0.1, 
datanodeUuid=751a18cc-629c-4787-9c2a-fa9152fb21cf, infoPort=35232, 
ipcPort=42302, storageInfo=lv=-56;cid=testClusterID;nsid=1340850404;c=0), 
blocks: 0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 286566 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 286566 INFO  (Thread-1424) [    ] o.a.s.c.ZkTestServer client 
port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 286566 INFO  (Thread-1424) [    ] o.a.s.c.ZkTestServer 
Starting server
   [junit4]   2> 286666 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.ZkTestServer start zk server on port:40220
   [junit4]   2> 286667 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 286667 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 286669 INFO  (zkCallback-225-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@694e9548 
name:ZooKeeperConnection Watcher:127.0.0.1:40220 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 286669 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 286670 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 286670 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 286673 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 286673 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 286675 INFO  (zkCallback-226-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@23717684 
name:ZooKeeperConnection Watcher:127.0.0.1:40220/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 286675 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 286675 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 286675 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 286677 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 286678 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 286679 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 286683 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml
 to /configs/conf1/solrconfig.xml
   [junit4]   2> 286683 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 286686 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/schema.xml
 to /configs/conf1/schema.xml
   [junit4]   2> 286686 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 286687 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml
 to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 286688 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.c.SolrZkClient makePath: 
/configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 286689 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/stopwords.txt
 to /configs/conf1/stopwords.txt
   [junit4]   2> 286689 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 286691 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/protwords.txt
 to /configs/conf1/protwords.txt
   [junit4]   2> 286691 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 286692 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/currency.xml
 to /configs/conf1/currency.xml
   [junit4]   2> 286693 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 286694 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml
 to /configs/conf1/enumsConfig.xml
   [junit4]   2> 286694 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 286696 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json
 to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 286696 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 286697 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt
 to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 286697 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 286699 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt
 to /configs/conf1/old_synonyms.txt
   [junit4]   2> 286699 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 286701 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/synonyms.txt
 to /configs/conf1/synonyms.txt
   [junit4]   2> 286701 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 287012 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.SolrTestCaseJ4 Writing core.properties file to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B49F89C8FEBF92DB-001/control-001/cores/collection1
   [junit4]   2> 287013 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 287015 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@64e1042a{/,null,AVAILABLE}
   [junit4]   2> 287016 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.e.j.s.ServerConnector Started 
ServerConnector@20bf053{HTTP/1.1}{127.0.0.1:58356}
   [junit4]   2> 287016 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.e.j.s.Server Started @290203ms
   [junit4]   2> 287016 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: 
{solr.data.dir=hdfs://localhost:42096/hdfs__localhost_42096__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-trunk_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B49F89C8FEBF92DB-001_tempDir-002_control_data,
 hostContext=/, hostPort=58356, 
coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B49F89C8FEBF92DB-001/control-001/cores}
   [junit4]   2> 287016 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): 
sun.misc.Launcher$AppClassLoader@4e0e2f2a
   [junit4]   2> 287016 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: 
'/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B49F89C8FEBF92DB-001/control-001/'
   [junit4]   2> 287017 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 287017 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find 
system property or JNDI)
   [junit4]   2> 287031 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 287032 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 287033 INFO  (zkCallback-227-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@7ee49501 
name:ZooKeeperConnection Watcher:127.0.0.1:40220/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 287033 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 287033 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 287035 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in 
ZooKeeper)
   [junit4]   2> 287035 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.SolrXmlConfig Loading container configuration from 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B49F89C8FEBF92DB-001/control-001/solr.xml
   [junit4]   2> 287040 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.CoresLocator Config-defined core root directory: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B49F89C8FEBF92DB-001/control-001/cores
   [junit4]   2> 287041 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.CoreContainer New CoreContainer 397060055
   [junit4]   2> 287041 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.CoreContainer Loading cores into CoreContainer 
[instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B49F89C8FEBF92DB-001/control-001/]
   [junit4]   2> 287041 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.CoreContainer loading shared library: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B49F89C8FEBF92DB-001/control-001/lib
   [junit4]   2> 287041 WARN  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to 
classloader: lib (resolved as: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B49F89C8FEBF92DB-001/control-001/lib).
   [junit4]   2> 287048 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme 
: ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 
10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 
5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 287049 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: 
socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 287049 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 287049 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.l.LogWatcher Registering Log Listener [Log4j 
(org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 287050 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:40220/solr
   [junit4]   2> 287050 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 287050 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 287050 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 287051 INFO  (zkCallback-229-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@50def9c4 
name:ZooKeeperConnection Watcher:127.0.0.1:40220 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 287052 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 287052 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 287058 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) 
[n:127.0.0.1:58356_    ] o.a.s.c.c.ConnectionManager Waiting for client to 
connect to ZooKeeper
   [junit4]   2> 287058 INFO  
(zkCallback-230-thread-1-processing-n:127.0.0.1:58356_) [n:127.0.0.1:58356_    
] o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@5c945bb6 
name:ZooKeeperConnection Watcher:127.0.0.1:40220/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 287058 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) 
[n:127.0.0.1:58356_    ] o.a.s.c.c.ConnectionManager Client is connected to 
ZooKeeper
   [junit4]   2> 287059 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) 
[n:127.0.0.1:58356_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 287061 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) 
[n:127.0.0.1:58356_    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/collection-queue-work
   [junit4]   2> 287063 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) 
[n:127.0.0.1:58356_    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/collection-map-running
   [junit4]   2> 287065 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) 
[n:127.0.0.1:58356_    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/collection-map-completed
   [junit4]   2> 287067 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) 
[n:127.0.0.1:58356_    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/collection-map-failure
   [junit4]   2> 287069 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) 
[n:127.0.0.1:58356_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 287070 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) 
[n:127.0.0.1:58356_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 287071 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) 
[n:127.0.0.1:58356_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 287072 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) 
[n:127.0.0.1:58356_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 287073 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) 
[n:127.0.0.1:58356_    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:58356_
   [junit4]   2> 287074 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) 
[n:127.0.0.1:58356_    ] o.a.s.c.c.SolrZkClient makePath: 
/live_nodes/127.0.0.1:58356_
   [junit4]   2> 287079 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) 
[n:127.0.0.1:58356_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 287080 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) 
[n:127.0.0.1:58356_    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer_elect/election
   [junit4]   2> 287081 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) 
[n:127.0.0.1:58356_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 287081 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) 
[n:127.0.0.1:58356_    ] o.a.s.c.LeaderElector Joined leadership election with 
path: /overseer_elect/election/94687538570199044-127.0.0.1:58356_-n_0000000000
   [junit4]   2> 287082 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) 
[n:127.0.0.1:58356_    ] o.a.s.c.OverseerElectionContext I am going to be the 
leader 127.0.0.1:58356_
   [junit4]   2> 287082 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) 
[n:127.0.0.1:58356_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 287085 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) 
[n:127.0.0.1:58356_    ] o.a.s.c.Overseer Overseer 
(id=94687538570199044-127.0.0.1:58356_-n_0000000000) starting
   [junit4]   2> 287087 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) 
[n:127.0.0.1:58356_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 287093 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) 
[n:127.0.0.1:58356_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting 
OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 
autoReplicaFailoverWaitAfterExpiration=10000 
autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 287094 INFO  
(OverseerCollectionConfigSetProcessor-94687538570199044-127.0.0.1:58356_-n_0000000000)
 [n:127.0.0.1:58356_    ] o.a.s.c.OverseerTaskProcessor Process current queue 
of overseer operations
   [junit4]   2> 287094 INFO  
(OverseerStateUpdate-94687538570199044-127.0.0.1:58356_-n_0000000000) 
[n:127.0.0.1:58356_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 287094 INFO  
(OverseerStateUpdate-94687538570199044-127.0.0.1:58356_-n_0000000000) 
[n:127.0.0.1:58356_    ] o.a.s.c.c.ZkStateReader Updating cluster state from 
ZooKeeper... 
   [junit4]   2> 287098 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) 
[n:127.0.0.1:58356_    ] o.a.s.c.c.ZkStateReader Updating cluster state from 
ZooKeeper... 
   [junit4]   2> 287113 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) 
[n:127.0.0.1:58356_    ] o.a.s.c.CoreContainer Security conf doesn't exist. 
Skipping setup for authorization module.
   [junit4]   2> 287113 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) 
[n:127.0.0.1:58356_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 287114 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) 
[n:127.0.0.1:58356_    ] o.a.s.c.CoresLocator Looking for core definitions 
underneath 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B49F89C8FEBF92DB-001/control-001/cores
   [junit4]   2> 287114 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) 
[n:127.0.0.1:58356_    ] o.a.s.c.SolrCore Created CoreDescriptor: 
{name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, 
loadOnStartup=true, configSetProperties=configsetprops.json, 
instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B49F89C8FEBF92DB-001/control-001/cores/collection1,
 collection=control_collection, 
absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B49F89C8FEBF92DB-001/control-001/cores/collection1/,
 coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 287114 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) 
[n:127.0.0.1:58356_    ] o.a.s.c.CoresLocator Found core collection1 in 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B49F89C8FEBF92DB-001/control-001/cores/collection1/
   [junit4]   2> 287114 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) 
[n:127.0.0.1:58356_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 287116 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection   x:collection1] o.a.s.c.ZkController 
publishing state=down
   [junit4]   2> 287116 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection   x:collection1] o.a.s.c.ZkController 
numShards not found on descriptor - reading it from system property
   [junit4]   2> 287116 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) 
[n:127.0.0.1:58356_    ] o.a.s.s.SolrDispatchFilter 
user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1
   [junit4]   2> 287116 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) 
[n:127.0.0.1:58356_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() 
done
   [junit4]   2> 287117 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection   x:collection1] o.a.s.c.ZkController 
look for our core node name
   [junit4]   2> 287118 INFO  
(OverseerStateUpdate-94687538570199044-127.0.0.1:58356_-n_0000000000) 
[n:127.0.0.1:58356_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message 
= {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:58356";,
   [junit4]   2>   "node_name":"127.0.0.1:58356_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 287118 INFO  
(OverseerStateUpdate-94687538570199044-127.0.0.1:58356_-n_0000000000) 
[n:127.0.0.1:58356_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 
message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:58356";,
   [junit4]   2>   "node_name":"127.0.0.1:58356_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 287119 INFO  
(OverseerStateUpdate-94687538570199044-127.0.0.1:58356_-n_0000000000) 
[n:127.0.0.1:58356_    ] o.a.s.c.o.ClusterStateMutator building a new cName: 
control_collection
   [junit4]   2> 287119 INFO  
(OverseerStateUpdate-94687538570199044-127.0.0.1:58356_-n_0000000000) 
[n:127.0.0.1:58356_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard 
shard=shard1
   [junit4]   2> 287121 INFO  
(zkCallback-230-thread-1-processing-n:127.0.0.1:58356_) [n:127.0.0.1:58356_    
] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred 
- updating... (live nodes size: 1)
   [junit4]   2> 287121 INFO  
(zkCallback-230-thread-2-processing-n:127.0.0.1:58356_) [n:127.0.0.1:58356_    
] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred 
- updating... (live nodes size: 1)
   [junit4]   2> 288118 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection   x:collection1] o.a.s.c.ZkController 
waiting to find shard id in clusterstate for collection1
   [junit4]   2> 288118 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection   x:collection1] o.a.s.c.ZkController 
Check for collection zkNode:control_collection
   [junit4]   2> 288118 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection   x:collection1] o.a.s.c.ZkController 
Collection zkNode exists
   [junit4]   2> 288118 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection   x:collection1] 
o.a.s.c.c.ZkStateReader Load collection config 
from:/collections/control_collection
   [junit4]   2> 288119 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection   x:collection1] 
o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 
specified config exists in ZooKeeper
   [junit4]   2> 288119 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection   x:collection1] 
o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: 
'/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B49F89C8FEBF92DB-001/control-001/cores/collection1/'
   [junit4]   2> 288119 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection   x:collection1] 
o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 288119 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection   x:collection1] 
o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find 
system property or JNDI)
   [junit4]   2> 288133 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection   x:collection1] o.a.s.c.Config loaded 
config solrconfig.xml with version 0 
   [junit4]   2> 288138 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection   x:collection1] o.a.s.c.SolrConfig 
current version of requestparams : -1
   [junit4]   2> 288142 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection   x:collection1] o.a.s.c.SolrConfig 
Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 288152 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection   x:collection1] o.a.s.c.Config Loaded 
SolrConfig: solrconfig.xml
   [junit4]   2> 288153 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection   x:collection1] o.a.s.s.IndexSchema 
Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 288158 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection   x:collection1] o.a.s.s.IndexSchema 
[collection1] Schema name=test
   [junit4]   2> 288259 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection   x:collection1] 
o.a.s.s.OpenExchangeRatesOrgProvider Initialized with 
rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 288264 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection   x:collection1] o.a.s.s.IndexSchema 
default search field in schema is text
   [junit4]   2> 288265 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection   x:collection1] o.a.s.s.IndexSchema 
unique key field: id
   [junit4]   2> 288277 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection   x:collection1] 
o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 288280 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection   x:collection1] 
o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 288281 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection   x:collection1] 
o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from 
open-exchange-rates.json
   [junit4]   2> 288282 WARN  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection   x:collection1] 
o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 288282 WARN  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection   x:collection1] 
o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 288282 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection   x:collection1] 
o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from 
open-exchange-rates.json
   [junit4]   2> 288283 WARN  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection   x:collection1] 
o.a.s.s.OpenExchangeRatesOrgProvider Unknown key IMPORTANT NOTE
   [junit4]   2> 288283 WARN  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection   x:collection1] 
o.a.s.s.OpenExchangeRatesOrgProvider Expected key, got STRING
   [junit4]   2> 288284 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection   x:collection1] 
o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default 
properties: Can't find resource 'configsetprops.json' in classpath or 
'/configs/conf1', 
cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1
   [junit4]   2> 288284 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection   x:collection1] o.a.s.c.CoreContainer 
Creating SolrCore 'collection1' using configuration from collection 
control_collection
   [junit4]   2> 288284 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 288294 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.HdfsDirectoryFactory 
solr.hdfs.home=hdfs://localhost:42096/solr_hdfs_home
   [junit4]   2> 288294 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 288295 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at 
[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B49F89C8FEBF92DB-001/control-001/cores/collection1/],
 dataDir=[null]
   [junit4]   2> 288295 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX 
Server: com.sun.jmx.mbeanserver.JmxMBeanServer@802de6d
   [junit4]   2> 288295 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.HdfsDirectoryFactory creating directory factory for path 
hdfs://localhost:42096/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 288317 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.CachingDirectoryFactory return new directory for 
hdfs://localhost:42096/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 288322 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore New index directory detected: old=null 
new=hdfs://localhost:42096/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 288338 WARN  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore [collection1] Solr index directory 
'hdfs:/localhost:42096/solr_hdfs_home/control_collection/core_node1/data/index' 
doesn't exist. Creating new index...
   [junit4]   2> 288338 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.HdfsDirectoryFactory creating directory factory for path 
hdfs://localhost:42096/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 288353 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.CachingDirectoryFactory return new directory for 
hdfs://localhost:42096/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 288354 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.MockRandomMergePolicy: 
org.apache.lucene.index.MockRandomMergePolicy@7de1da74
   [junit4]   2> 288629 INFO  (IPC Server handler 0 on 42096) [    ] 
BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:50737 is 
added to blk_1073741825_1001{blockUCState=COMMITTED, primaryNodeIndex=-1, 
replicas=[ReplicaUnderConstruction[[DISK]DS-aef57e15-51c3-46a6-bf2c-96ba5749335a:NORMAL:127.0.0.1:50737|RBW],
 
ReplicaUnderConstruction[[DISK]DS-97a4113a-3b2c-4bc2-92f2-31d96372015d:NORMAL:127.0.0.1:35135|RBW]]}
 size 71
   [junit4]   2> 288629 INFO  (IPC Server handler 1 on 42096) [    ] 
BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:35135 is 
added to blk_1073741825_1001 size 71
   [junit4]   2> 289027 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2>        commit{dir=NRTCachingDirectory(HdfsDirectory@b70d3926 
lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@23fce20f; 
maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 289028 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 289038 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain 
"nodistrib"
   [junit4]   2> 289039 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain 
"dedupe"
   [junit4]   2> 289039 INFO  
(OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:58356_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore 
Looking for old index directories to cleanup for core collection1 in 
hdfs://localhost:42096/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 289039 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.UpdateRequestProcessorChain inserting 
DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 289039 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain 
"stored_sig"
   [junit4]   2> 289040 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.UpdateRequestProcessorChain inserting 
DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 289040 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain 
"distrib-dup-test-chain-explicit"
   [junit4]   2> 289040 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain 
"distrib-dup-test-chain-implicit"
   [junit4]   2> 289040 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.UpdateRequestProcessorChain inserting 
DistributedUpdateProcessorFactory into updateRequestProcessorChain 
"distrib-dup-test-chain-implicit"
   [junit4]   2> 289040 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating 
implicit default
   [junit4]   2> 289042 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 289043 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 289044 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 289045 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 289050 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.RequestHandlers Registered paths: 
/admin/mbeans,standard,/update/csv,/update/json/docs,/admin/luke,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/schema,/admin/plugins,/admin/logging,/update/json,/admin/threads,/admin/ping,/update,/admin/file
   [junit4]   2> 289051 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore Using default statsCache cache: 
org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 289051 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.UpdateHandler Using UpdateLog implementation: 
org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 289052 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.UpdateLog Initializing HdfsUpdateLog: dataDir= defaultSyncLevel=FLUSH 
numRecordsToKeep=100 maxNumLogsToKeep=10 tlogDfsReplication=2
   [junit4]   2> 289080 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 289080 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 289082 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: 
minMergeSize=0, mergeFactor=10, maxMergeSize=1979649289, 
maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, 
maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.1]
   [junit4]   2> 289153 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2>        commit{dir=NRTCachingDirectory(HdfsDirectory@b70d3926 
lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@23fce20f; 
maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 289153 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 289153 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.s.SolrIndexSearcher Opening Searcher@146beb94[collection1] main
   [junit4]   2> 289153 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.c.ZkStateReader Load collection config 
from:/collections/control_collection
   [junit4]   2> 289154 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 
specified config exists in ZooKeeper
   [junit4]   2> 289154 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the 
RestManager with znodeBase: /configs/conf1
   [junit4]   2> 289155 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: 
/configs/conf1
   [junit4]   2> 289155 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 289155 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 289156 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.r.ManagedResourceStorage No data found for znode 
/configs/conf1/_rest_managed.json
   [junit4]   2> 289157 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 289157 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 289159 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 289161 INFO  
(searcherExecutor-449-thread-1-processing-n:127.0.0.1:58356_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58356_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore 
[collection1] Registered new searcher Searcher@146beb94[collection1] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 289161 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 289161 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.VersionInfo Refreshing highest value of _version_ for 256 version 
buckets from index
   [junit4]   2> 289161 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket 
highest value from index
   [junit4]   2> 289161 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.UpdateLog Could not find max version in index or recent updates, using 
new clock 1515000619842142208
   [junit4]   2> 289161 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.UpdateLog Took 0.0ms to seed version buckets with highest version 
1515000619842142208
   [junit4]   2> 289161 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 289162 INFO  
(coreLoadExecutor-448-thread-1-processing-n:127.0.0.1:58356_) 
[n:127.0.0.1:58356_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 289162 INFO  
(coreZkRegister-442-thread-1-processing-n:127.0.0.1:58356_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58356_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController 
Register replica - core:collection1 address:http://127.0.0.1:58356 
collection:control_collection shard:shard1
   [junit4]   2> 289163 INFO  
(coreZkRegister-442-thread-1-processing-n:127.0.0.1:58356_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58356_ 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.c.SolrZkClient makePath: 
/collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 289166 INFO  
(coreZkRegister-442-thread-1-processing-n:127.0.0.1:58356_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58356_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector 
Joined leadership election with path: 
/collections/control_collection/leader_elect/shard1/election/94687538570199044-core_node1-n_0000000000
   [junit4]   2> 289167 INFO  
(coreZkRegister-442-thread-1-processing-n:127.0.0.1:58356_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58356_ 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 289168 INFO  
(coreZkRegister-442-thread-1-processing-n:127.0.0.1:58356_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58356_ 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 289168 INFO  
(coreZkRegister-442-thread-1-processing-n:127.0.0.1:58356_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58356_ 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 289168 INFO  
(coreZkRegister-442-thread-1-processing-n:127.0.0.1:58356_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58356_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy 
Sync replicas to http://127.0.0.1:58356/collection1/
   [junit4]   2> 289168 INFO  
(coreZkRegister-442-thread-1-processing-n:127.0.0.1:58356_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58356_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy 
Sync Success - now sync replicas to me
   [junit4]   2> 289168 INFO  
(coreZkRegister-442-thread-1-processing-n:127.0.0.1:58356_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58356_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy 
http://127.0.0.1:58356/collection1/ has no replicas
   [junit4]   2> 289168 INFO  
(coreZkRegister-442-thread-1-processing-n:127.0.0.1:58356_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58356_ 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.ShardLeaderElectionContext I am the new leader: 
http://127.0.0.1:58356/collection1/ shard1
   [junit4]   2> 289169 INFO  
(OverseerStateUpdate-94687538570199044-127.0.0.1:58356_-n_0000000000) 
[n:127.0.0.1:58356_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message 
= {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 289169 INFO  
(coreZkRegister-442-thread-1-processing-n:127.0.0.1:58356_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58356_ 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 289170 INFO  
(zkCallback-230-thread-2-processing-n:127.0.0.1:58356_) [n:127.0.0.1:58356_    
] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred 
- updating... (live nodes size: 1)
   [junit4]   2> 289170 INFO  
(zkCallback-230-thread-1-processing-n:127.0.0.1:58356_) [n:127.0.0.1:58356_    
] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred 
- updating... (live nodes size: 1)
   [junit4]   2> 289172 INFO  
(coreZkRegister-442-thread-1-processing-n:127.0.0.1:58356_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58356_ 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 289175 INFO  
(OverseerStateUpdate-94687538570199044-127.0.0.1:58356_-n_0000000000) 
[n:127.0.0.1:58356_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message 
= {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:58356";,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 289278 INFO  
(zkCallback-230-thread-1-processing-n:127.0.0.1:58356_) [n:127.0.0.1:58356_    
] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred 
- updating... (live nodes size: 1)
   [junit4]   2> 289278 INFO  
(zkCallback-230-thread-2-processing-n:127.0.0.1:58356_) [n:127.0.0.1:58356_    
] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred 
- updating... (live nodes size: 1)
   [junit4]   2> 289326 INFO  
(coreZkRegister-442-thread-1-processing-n:127.0.0.1:58356_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58356_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController 
We are http://127.0.0.1:58356/collection1/ and leader is 
http://127.0.0.1:58356/collection1/
   [junit4]   2> 289326 INFO  
(coreZkRegister-442-thread-1-processing-n:127.0.0.1:58356_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58356_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController 
No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:58356
   [junit4]   2> 289326 INFO  
(coreZkRegister-442-thread-1-processing-n:127.0.0.1:58356_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58356_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController 
I am the leader, no recovery necessary
   [junit4]   2> 289326 INFO  
(coreZkRegister-442-thread-1-processing-n:127.0.0.1:58356_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58356_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController 
publishing state=active
   [junit4]   2> 289326 INFO  
(coreZkRegister-442-thread-1-processing-n:127.0.0.1:58356_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:58356_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController 
numShards not found on descriptor - reading it from system property
   [junit4]   2> 289334 INFO  
(OverseerStateUpdate-94687538570199044-127.0.0.1:58356_-n_0000000000) 
[n:127.0.0.1:58356_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message 
= {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:58356";,
   [junit4]   2>   "node_name":"127.0.0.1:58356_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 3
   [junit4]   2> 289336 INFO  
(OverseerStateUpdate-94687538570199044-127.0.0.1:58356_-n_0000000000) 
[n:127.0.0.1:58356_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 
message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:58356";,
   [junit4]   2>   "node_name":"127.0.0.1:58356_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 289438 INFO  
(zkCallback-230-thread-1-processing-n:127.0.0.1:58356_) [n:127.0.0.1:58356_    
] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred 
- updating... (live nodes size: 1)
   [junit4]   2> 289438 INFO  
(zkCallback-230-thread-2-processing-n:127.0.0.1:58356_) [n:127.0.0.1:58356_    
] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred 
- updating... (live nodes size: 1)
   [junit4]   2> 289619 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 289620 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 289621 INFO  (zkCallback-232-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@2e8cbd15 
name:ZooKeeperConnection Watcher:127.0.0.1:40220/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 289621 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 289621 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 289621 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 289624 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection 
loss:false
   [junit4]   2> 289624 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 289624 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 289627 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 289628 INFO  (zkCallback-233-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@7ffc8167 
name:ZooKeeperConnection Watcher:127.0.0.1:40220/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 289628 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 289628 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 289631 INFO  
(OverseerStateUpdate-94687538570199044-127.0.0.1:58356_-n_0000000000) 
[n:127.0.0.1:58356_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message 
= {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 4
   [junit4]   2> 289631 INFO  
(OverseerStateUpdate-94687538570199044-127.0.0.1:58356_-n_0000000000) 
[n:127.0.0.1:58356_    ] o.a.s.c.o.ClusterStateMutator building a new cName: 
collection1
   [junit4]   2> 289740 INFO  
(OverseerStateUpdate-94687538570199044-127.0.0.1:58356_-n_0000000000) 
[n:127.0.0.1:58356_    ] o.a.s.c.o.ZkStateWriter going to create_collection 
/collections/collection1/state.json
   [junit4]   2> 289950 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.SolrTestCaseJ4 Writing core.properties file to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B49F89C8FEBF92DB-001/shard-1-001/cores/collection1
   [junit4]   2> 289951 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B49F89C8FEBF92DB-001/shard-1-001
   [junit4]   2> 289952 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 289954 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@1ba85db6{/,null,AVAILABLE}
   [junit4]   2> 289955 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.e.j.s.ServerConnector Started 
ServerConnector@46348666{HTTP/1.1}{127.0.0.1:54613}
   [junit4]   2> 289955 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.e.j.s.Server Started @293142ms
   [junit4]   2> 289955 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: 
{solr.data.dir=hdfs://localhost:42096/hdfs__localhost_42096__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-trunk_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B49F89C8FEBF92DB-001_tempDir-002_jetty1,
 solrconfig=solrconfig.xml, hostContext=/, hostPort=54613, 
coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B49F89C8FEBF92DB-001/shard-1-001/cores}
   [junit4]   2> 289955 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): 
sun.misc.Launcher$AppClassLoader@4e0e2f2a
   [junit4]   2> 289955 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: 
'/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B49F89C8FEBF92DB-001/shard-1-001/'
   [junit4]   2> 289956 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 289956 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find 
system property or JNDI)
   [junit4]   2> 289978 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 289979 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 289982 INFO  (zkCallback-234-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@2fab4f40 
name:ZooKeeperConnection Watcher:127.0.0.1:40220/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 289982 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 289982 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 289988 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in 
ZooKeeper)
   [junit4]   2> 289988 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.SolrXmlConfig Loading container configuration from 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B49F89C8FEBF92DB-001/shard-1-001/solr.xml
   [junit4]   2> 289993 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.CoresLocator Config-defined core root directory: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B49F89C8FEBF92DB-001/shard-1-001/cores
   [junit4]   2> 289994 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.CoreContainer New CoreContainer 396859537
   [junit4]   2> 289994 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.CoreContainer Loading cores into CoreContainer 
[instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B49F89C8FEBF92DB-001/shard-1-001/]
   [junit4]   2> 289994 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.CoreContainer loading shared library: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B49F89C8FEBF92DB-001/shard-1-001/lib
   [junit4]   2> 289994 WARN  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to 
classloader: lib (resolved as: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B49F89C8FEBF92DB-001/shard-1-001/lib).
   [junit4]   2> 290001 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme 
: ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 
10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 
5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 290003 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: 
socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 290003 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 290003 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.l.LogWatcher Registering Log Listener [Log4j 
(org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 290004 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:40220/solr
   [junit4]   2> 290004 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 290004 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 290004 INFO  
(TEST-HdfsCollectionsAPIDistributedZkTest.test-seed#[B49F89C8FEBF92DB]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 290006 INFO  (zkCallback-236-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@2196679f name:ZooKeeperConnecti

[...truncated too long message...]

>       at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 3851210 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[B49F89C8FEBF92DB])
 [    ] o.e.j.s.ServerConnector Stopped 
ServerConnector@79b6d2bb{HTTP/1.1}{127.0.0.1:40698}
   [junit4]   2> 3851210 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[B49F89C8FEBF92DB])
 [    ] o.e.j.s.h.ContextHandler Stopped 
o.e.j.s.ServletContextHandler@3e99ad48{/,null,UNAVAILABLE}
   [junit4]   2> 3851212 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[B49F89C8FEBF92DB])
 [n:127.0.0.1:40698_ c:collection1 s:shard1 r:core_node2 x:collection1] 
o.a.s.c.ZkTestServer connecting to 127.0.0.1:49063 49063
   [junit4]   2> 3851396 INFO  (Thread-43888) [    ] o.a.s.c.ZkTestServer 
connecting to 127.0.0.1:49063 49063
   [junit4]   2> 3851397 WARN  (Thread-43888) [    ] o.a.s.c.ZkTestServer Watch 
limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2>        7       /solr/aliases.json
   [junit4]   2>        6       /solr/security.json
   [junit4]   2>        5       /solr/configs/conf1
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2>        35      /solr/clusterstate.json
   [junit4]   2>        3       
/solr/overseer_elect/election/94687748503830532-127.0.0.1:54250_-n_0000000000
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2>        11      /solr/live_nodes
   [junit4]   2>        8       /solr/collections
   [junit4]   2>        3       /solr/overseer/queue
   [junit4]   2>        3       /solr/overseer/collection-queue-work
   [junit4]   2> 
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 
'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  
-Dtestcase=LeaderInitiatedRecoveryOnShardRestartTest 
-Dtests.method=testRestartWithAllInLIR -Dtests.seed=B49F89C8FEBF92DB 
-Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true 
-Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt 
-Dtests.locale=it_IT -Dtests.timezone=Singapore -Dtests.asserts=true 
-Dtests.file.encoding=UTF-8
   [junit4] ERROR    362s J2 | 
LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: There are still nodes 
recoverying - waited for 330 seconds
   [junit4]    >        at 
org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:172)
   [junit4]    >        at 
org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:133)
   [junit4]    >        at 
org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:128)
   [junit4]    >        at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:830)
   [junit4]    >        at 
org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR(LeaderInitiatedRecoveryOnShardRestartTest.java:81)
   [junit4]    >        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
   [junit4]    >        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
   [junit4]    >        at java.lang.Thread.run(Thread.java:745)Throwable #2: 
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught 
exception in thread: Thread[id=51242, name=coreZkRegister-5187-thread-1, 
state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
   [junit4]    > Caused by: java.lang.AssertionError
   [junit4]    >        at 
__randomizedtesting.SeedInfo.seed([B49F89C8FEBF92DB]:0)
   [junit4]    >        at 
org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2126)
   [junit4]    >        at 
org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:433)
   [junit4]    >        at 
org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
   [junit4]    >        at 
org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
   [junit4]    >        at 
org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
   [junit4]    >        at 
org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
   [junit4]    >        at 
org.apache.solr.cloud.ZkController.register(ZkController.java:926)
   [junit4]    >        at 
org.apache.solr.cloud.ZkController.register(ZkController.java:881)
   [junit4]    >        at 
org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
   [junit4]    >        at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
   [junit4]    >        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]    >        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 3851405 INFO  
(SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[B49F89C8FEBF92DB]-worker)
 [n:127.0.0.1:40698_ c:collection1 s:shard1 r:core_node2 x:collection1] 
o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_B49F89C8FEBF92DB-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene53): {}, 
docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, 
locale=it_IT, timezone=Singapore
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 
1.8.0_45 (64-bit)/cpus=4,threads=1,free=387846552,total=503840768
   [junit4]   2> NOTE: All tests run in this JVM: [StressHdfsTest, 
TestWriterPerf, AnalyticsMergeStrategyTest, 
DistributedFacetPivotSmallAdvancedTest, TestPerFieldSimilarity, 
DistributedQueryComponentCustomSortTest, CoreAdminHandlerTest, 
TestScoreJoinQPScore, SharedFSAutoReplicaFailoverUtilsTest, 
TestCursorMarkWithoutUniqueKey, CloudMLTQParserTest, 
TestDefaultSimilarityFactory, TestFieldSortValues, 
BigEndianAscendingWordDeserializerTest, TestSort, PluginInfoTest, 
TestExactSharedStatsCache, TestRequestStatusCollectionAPI, TestFieldResource, 
RegexBoostProcessorTest, DateRangeFieldTest, TestMissingGroups, 
TestDistributedStatsComponentCardinality, DocValuesMultiTest, 
TestImplicitCoreProperties, AddBlockUpdateTest, TestConfigReload, 
BadComponentTest, TestSmileRequest, TestFuzzyAnalyzedSuggestions, 
HLLSerializationTest, TestManagedStopFilterFactory, 
DistributedDebugComponentTest, OverriddenZkACLAndCredentialsProvidersTest, 
TermVectorComponentDistributedTest, HdfsDirectoryTest, SolrTestCaseJ4Test, 
QueryParsingTest, ChangedSchemaMergeTest, QueryResultKeyTest, 
TestSearcherReuse, TestReversedWildcardFilterFactory, RAMDirectoryFactoryTest, 
TestExtendedDismaxParser, TestSolrCLIRunExample, BasicAuthIntegrationTest, 
TestSolrCloudWithKerberosAlt, TestStressReorder, TestCloudManagedSchema, 
TestCollectionAPI, TestConfigSetProperties, TestCharFilters, BufferStoreTest, 
TestElisionMultitermQuery, TestSolrIndexConfig, TestShardHandlerFactory, 
PrimitiveFieldTypeTest, CdcrUpdateLogTest, ShowFileRequestHandlerTest, 
TestCoreDiscovery, TestSolrQueryParser, 
ConcurrentDeleteAndCreateCollectionTest, TestAddFieldRealTimeGet, 
TestClassNameShortening, ZkNodePropsTest, UUIDFieldTest, 
TestPostingsSolrHighlighter, TestXIncludeConfig, HdfsChaosMonkeySafeLeaderTest, 
HdfsDirectoryFactoryTest, AddSchemaFieldsUpdateProcessorFactoryTest, 
UpdateParamsTest, TestJsonFacets, TestRandomRequestDistribution, 
DistributedFacetPivotSmallTest, PKIAuthenticationIntegrationTest, 
TestFastWriter, AutoCommitTest, DeleteReplicaTest, 
TestLeaderInitiatedRecoveryThread, SearchHandlerTest, BJQParserTest, 
NoCacheHeaderTest, TestLRUCache, SuggesterWFSTTest, TestBulkSchemaConcurrent, 
TestTrieFacet, TestPHPSerializedResponseWriter, 
SuggestComponentContextFilterQueryTest, BasicDistributedZkTest, 
BasicDistributedZk2Test, UnloadDistributedZkTest, 
CollectionsAPIDistributedZkTest, TestBinaryResponseWriter, 
HighlighterConfigTest, TestQuerySenderListener, CopyFieldTest, 
ResponseLogComponentTest, SolrIndexConfigTest, TestStressRecovery, 
DistributedIntervalFacetingTest, CollectionTooManyReplicasTest, 
LeaderInitiatedRecoveryOnShardRestartTest]
   [junit4] Completed [443/552] on J2 in 361.83s, 1 test, 1 error <<< FAILURES!

[...truncated 364 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:782:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:719:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:59:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build.xml:233:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/common-build.xml:516:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:1432:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:987:
 There were test failures: 552 suites (8 ignored), 2170 tests, 3 errors, 1 
failure, 30 ignored (17 assumptions) [seed: B49F89C8FEBF92DB]

Total time: 194 minutes 28 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Reply via email to