Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-trunk/820/

4 tests failed.
FAILED:  org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest.test

Error Message:
The Monkey ran for over 30 seconds and no jetties were stopped - this is worth 
investigating!

Stack Trace:
java.lang.AssertionError: The Monkey ran for over 30 seconds and no jetties 
were stopped - this is worth investigating!
        at 
__randomizedtesting.SeedInfo.seed([79703BC618647C1C:F124041CB69811E4]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.apache.solr.cloud.ChaosMonkey.stopTheMonkey(ChaosMonkey.java:539)
        at 
org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.test(ChaosMonkeySafeLeaderTest.java:143)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:864)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:900)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:914)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
        at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
        at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:873)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:775)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:809)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:820)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=22328, name=collection2, 
state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught 
exception in thread: Thread[id=22328, name=collection2, state=RUNNABLE, 
group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: 
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at http://127.0.0.1:38108/oav: Could not find collection : 
awholynewstresscollection_collection2_1
        at __randomizedtesting.SeedInfo.seed([79703BC618647C1C]:0)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
        at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
        at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
        at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
        at 
org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=6584, name=collection4, 
state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught 
exception in thread: Thread[id=6584, name=collection4, state=RUNNABLE, 
group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: 
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at http://127.0.0.1:54364: Could not find collection : 
awholynewstresscollection_collection4_0
        at __randomizedtesting.SeedInfo.seed([79703BC618647C1C]:0)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
        at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
        at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:372)
        at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:325)
        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1099)
        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
        at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
        at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
        at 
org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:895)


FAILED:  
org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR

Error Message:
Captured an uncaught exception in thread: Thread[id=68378, 
name=coreZkRegister-6128-thread-2, state=RUNNABLE, 
group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught 
exception in thread: Thread[id=68378, name=coreZkRegister-6128-thread-2, 
state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
Caused by: java.lang.AssertionError
        at __randomizedtesting.SeedInfo.seed([79703BC618647C1C]:0)
        at 
org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2126)
        at 
org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:433)
        at 
org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
        at 
org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
        at 
org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
        at 
org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
        at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
        at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
        at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
        at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10261 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest
   [junit4]   2> Creating dataDir: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_79703BC618647C1C-001/init-core-data-001
   [junit4]   2> 1078488 INFO  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[79703BC618647C1C]-worker) [    ] 
o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true)
   [junit4]   2> 1078488 INFO  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[79703BC618647C1C]-worker) [    ] 
o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 1078603 WARN  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[79703BC618647C1C]-worker) [    ] 
o.a.h.m.i.MetricsConfig Cannot locate configuration: tried 
hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 1078624 WARN  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[79703BC618647C1C]-worker) [    ] 
o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 1078636 INFO  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[79703BC618647C1C]-worker) [    ] 
o.m.log jetty-6.1.26
   [junit4]   2> 1078648 INFO  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[79703BC618647C1C]-worker) [    ] 
o.m.log Extract 
jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/hdfs
 to ./temp/Jetty_localhost_36226_hdfs____u0pisd/webapp
   [junit4]   2> 1078763 INFO  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[79703BC618647C1C]-worker) [    ] 
o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 1079016 INFO  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[79703BC618647C1C]-worker) [    ] 
o.m.log Started 
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36226
   [junit4]   2> 1079283 WARN  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[79703BC618647C1C]-worker) [    ] 
o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 1079284 INFO  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[79703BC618647C1C]-worker) [    ] 
o.m.log jetty-6.1.26
   [junit4]   2> 1079301 INFO  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[79703BC618647C1C]-worker) [    ] 
o.m.log Extract 
jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode
 to ./temp/Jetty_localhost_40556_datanode____.45e0e7/webapp
   [junit4]   2> 1079402 INFO  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[79703BC618647C1C]-worker) [    ] 
o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 1079589 INFO  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[79703BC618647C1C]-worker) [    ] 
o.m.log Started 
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40556
   [junit4]   2> 1079634 WARN  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[79703BC618647C1C]-worker) [    ] 
o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 1079635 INFO  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[79703BC618647C1C]-worker) [    ] 
o.m.log jetty-6.1.26
   [junit4]   2> 1079658 INFO  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[79703BC618647C1C]-worker) [    ] 
o.m.log Extract 
jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.6.0-tests.jar!/webapps/datanode
 to ./temp/Jetty_localhost_39289_datanode____.3v4emk/webapp
   [junit4]   2> 1079798 INFO  (IPC Server handler 3 on 53937) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-7ba55d40-ca92-4118-b425-07ab32550d0c node DatanodeRegistration(127.0.0.1, 
datanodeUuid=2eb1b2b8-ef3f-445e-8693-e3e6b9e02fa9, infoPort=40556, 
ipcPort=36362, storageInfo=lv=-56;cid=testClusterID;nsid=31085310;c=0), blocks: 
0, hasStaleStorages: true, processing time: 1 msecs
   [junit4]   2> 1079798 INFO  (IPC Server handler 3 on 53937) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-037684ac-428a-4159-8b1c-6c2440624643 node DatanodeRegistration(127.0.0.1, 
datanodeUuid=2eb1b2b8-ef3f-445e-8693-e3e6b9e02fa9, infoPort=40556, 
ipcPort=36362, storageInfo=lv=-56;cid=testClusterID;nsid=31085310;c=0), blocks: 
0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 1079808 INFO  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[79703BC618647C1C]-worker) [    ] 
o.m.log NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 1080145 INFO  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[79703BC618647C1C]-worker) [    ] 
o.m.log Started 
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:39289
   [junit4]   2> 1080422 INFO  (IPC Server handler 5 on 53937) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-f01edb34-4ea8-4352-a408-decd4b2a44c6 node DatanodeRegistration(127.0.0.1, 
datanodeUuid=fe2fa4d9-28e9-4359-a499-e86d52ccf5e0, infoPort=39289, 
ipcPort=32938, storageInfo=lv=-56;cid=testClusterID;nsid=31085310;c=0), blocks: 
0, hasStaleStorages: true, processing time: 1 msecs
   [junit4]   2> 1080422 INFO  (IPC Server handler 5 on 53937) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-05be9c37-aa07-49e8-b1f7-ca758fd1a5fe node DatanodeRegistration(127.0.0.1, 
datanodeUuid=fe2fa4d9-28e9-4359-a499-e86d52ccf5e0, infoPort=39289, 
ipcPort=32938, storageInfo=lv=-56;cid=testClusterID;nsid=31085310;c=0), blocks: 
0, hasStaleStorages: false, processing time: 0 msecs
   [junit4]   2> 1080540 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1080540 INFO  (Thread-11828) [    ] o.a.s.c.ZkTestServer 
client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1080541 INFO  (Thread-11828) [    ] o.a.s.c.ZkTestServer 
Starting server
   [junit4]   2> 1080641 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.ZkTestServer start zk server on port:44893
   [junit4]   2> 1080641 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1080641 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1080644 INFO  (zkCallback-772-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@4877cdf0 
name:ZooKeeperConnection Watcher:127.0.0.1:44893 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1080644 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1080644 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1080644 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1080647 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1080647 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1080648 INFO  (zkCallback-773-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@54485bd1 
name:ZooKeeperConnection Watcher:127.0.0.1:44893/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1080648 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1080649 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1080649 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1080652 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1080654 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1080655 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1080657 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml
 to /configs/conf1/solrconfig.xml
   [junit4]   2> 1080657 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1080659 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/schema15.xml
 to /configs/conf1/schema.xml
   [junit4]   2> 1080659 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1080660 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml
 to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1080661 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.c.SolrZkClient makePath: 
/configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1080662 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/stopwords.txt
 to /configs/conf1/stopwords.txt
   [junit4]   2> 1080662 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1080663 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/protwords.txt
 to /configs/conf1/protwords.txt
   [junit4]   2> 1080664 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1080665 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/currency.xml
 to /configs/conf1/currency.xml
   [junit4]   2> 1080665 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1080667 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml
 to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1080667 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1080668 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json
 to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1080668 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1080670 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt
 to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1080670 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1080671 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt
 to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1080671 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1080673 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/core/src/test-files/solr/collection1/conf/synonyms.txt
 to /configs/conf1/synonyms.txt
   [junit4]   2> 1080673 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1080987 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.SolrTestCaseJ4 Writing core.properties file to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_79703BC618647C1C-001/control-001/cores/collection1
   [junit4]   2> 1080988 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1080990 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@21354ce0{/,null,AVAILABLE}
   [junit4]   2> 1080990 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.e.j.s.ServerConnector Started 
ServerConnector@371e207{HTTP/1.1}{127.0.0.1:49192}
   [junit4]   2> 1080990 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.e.j.s.Server Started @1084231ms
   [junit4]   2> 1080990 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: 
{solr.data.dir=hdfs://localhost:53937/hdfs__localhost_53937__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-trunk_solr_build_solr-core_test_J2_temp_solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_79703BC618647C1C-001_tempDir-002_control_data,
 hostContext=/, hostPort=49192, 
coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_79703BC618647C1C-001/control-001/cores}
   [junit4]   2> 1080991 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): 
sun.misc.Launcher$AppClassLoader@4e0e2f2a
   [junit4]   2> 1080991 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: 
'/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_79703BC618647C1C-001/control-001/'
   [junit4]   2> 1080991 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1080991 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find 
system property or JNDI)
   [junit4]   2> 1081005 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1081006 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1081007 INFO  (zkCallback-774-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@7e02154a 
name:ZooKeeperConnection Watcher:127.0.0.1:44893/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1081007 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1081007 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1081008 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in 
ZooKeeper)
   [junit4]   2> 1081008 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.SolrXmlConfig Loading container configuration from 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_79703BC618647C1C-001/control-001/solr.xml
   [junit4]   2> 1081013 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.CoresLocator Config-defined core root directory: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_79703BC618647C1C-001/control-001/cores
   [junit4]   2> 1081013 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.CoreContainer New CoreContainer 2110446700
   [junit4]   2> 1081013 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.CoreContainer Loading cores into CoreContainer 
[instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_79703BC618647C1C-001/control-001/]
   [junit4]   2> 1081013 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.CoreContainer loading shared library: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_79703BC618647C1C-001/control-001/lib
   [junit4]   2> 1081013 WARN  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to 
classloader: lib (resolved as: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_79703BC618647C1C-001/control-001/lib).
   [junit4]   2> 1081020 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme 
: ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 
10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 
5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1081021 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: 
socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1081021 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1081021 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.l.LogWatcher Registering Log Listener [Log4j 
(org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1081022 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44893/solr
   [junit4]   2> 1081022 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1081022 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1081022 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1081023 INFO  (zkCallback-776-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@1283bb0b 
name:ZooKeeperConnection Watcher:127.0.0.1:44893 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1081024 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1081024 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1081026 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) 
[n:127.0.0.1:49192_    ] o.a.s.c.c.ConnectionManager Waiting for client to 
connect to ZooKeeper
   [junit4]   2> 1081027 INFO  
(zkCallback-777-thread-1-processing-n:127.0.0.1:49192_) [n:127.0.0.1:49192_    
] o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@4cabb976 
name:ZooKeeperConnection Watcher:127.0.0.1:44893/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1081027 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) 
[n:127.0.0.1:49192_    ] o.a.s.c.c.ConnectionManager Client is connected to 
ZooKeeper
   [junit4]   2> 1081028 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) 
[n:127.0.0.1:49192_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1081031 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) 
[n:127.0.0.1:49192_    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/collection-queue-work
   [junit4]   2> 1081034 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) 
[n:127.0.0.1:49192_    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/collection-map-running
   [junit4]   2> 1081035 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) 
[n:127.0.0.1:49192_    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/collection-map-completed
   [junit4]   2> 1081037 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) 
[n:127.0.0.1:49192_    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/collection-map-failure
   [junit4]   2> 1081039 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) 
[n:127.0.0.1:49192_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1081040 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) 
[n:127.0.0.1:49192_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1081041 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) 
[n:127.0.0.1:49192_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1081043 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) 
[n:127.0.0.1:49192_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1081043 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) 
[n:127.0.0.1:49192_    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:49192_
   [junit4]   2> 1081044 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) 
[n:127.0.0.1:49192_    ] o.a.s.c.c.SolrZkClient makePath: 
/live_nodes/127.0.0.1:49192_
   [junit4]   2> 1081046 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) 
[n:127.0.0.1:49192_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1081047 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) 
[n:127.0.0.1:49192_    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer_elect/election
   [junit4]   2> 1081048 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) 
[n:127.0.0.1:49192_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1081048 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) 
[n:127.0.0.1:49192_    ] o.a.s.c.LeaderElector Joined leadership election with 
path: /overseer_elect/election/94682573230047236-127.0.0.1:49192_-n_0000000000
   [junit4]   2> 1081049 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) 
[n:127.0.0.1:49192_    ] o.a.s.c.OverseerElectionContext I am going to be the 
leader 127.0.0.1:49192_
   [junit4]   2> 1081049 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) 
[n:127.0.0.1:49192_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1081050 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) 
[n:127.0.0.1:49192_    ] o.a.s.c.Overseer Overseer 
(id=94682573230047236-127.0.0.1:49192_-n_0000000000) starting
   [junit4]   2> 1081052 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) 
[n:127.0.0.1:49192_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1081059 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) 
[n:127.0.0.1:49192_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting 
OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 
autoReplicaFailoverWaitAfterExpiration=30000 
autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1081059 INFO  
(OverseerCollectionConfigSetProcessor-94682573230047236-127.0.0.1:49192_-n_0000000000)
 [n:127.0.0.1:49192_    ] o.a.s.c.OverseerTaskProcessor Process current queue 
of overseer operations
   [junit4]   2> 1081059 INFO  
(OverseerStateUpdate-94682573230047236-127.0.0.1:49192_-n_0000000000) 
[n:127.0.0.1:49192_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1081060 INFO  
(OverseerStateUpdate-94682573230047236-127.0.0.1:49192_-n_0000000000) 
[n:127.0.0.1:49192_    ] o.a.s.c.c.ZkStateReader Updating cluster state from 
ZooKeeper... 
   [junit4]   2> 1081070 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) 
[n:127.0.0.1:49192_    ] o.a.s.c.c.ZkStateReader Updating cluster state from 
ZooKeeper... 
   [junit4]   2> 1081096 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) 
[n:127.0.0.1:49192_    ] o.a.s.c.CoreContainer Security conf doesn't exist. 
Skipping setup for authorization module.
   [junit4]   2> 1081096 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) 
[n:127.0.0.1:49192_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1081097 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) 
[n:127.0.0.1:49192_    ] o.a.s.c.CoresLocator Looking for core definitions 
underneath 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_79703BC618647C1C-001/control-001/cores
   [junit4]   2> 1081098 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) 
[n:127.0.0.1:49192_    ] o.a.s.c.SolrCore Created CoreDescriptor: 
{name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, 
loadOnStartup=true, configSetProperties=configsetprops.json, 
instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_79703BC618647C1C-001/control-001/cores/collection1,
 collection=control_collection, 
absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_79703BC618647C1C-001/control-001/cores/collection1/,
 coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 1081098 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) 
[n:127.0.0.1:49192_    ] o.a.s.c.CoresLocator Found core collection1 in 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_79703BC618647C1C-001/control-001/cores/collection1/
   [junit4]   2> 1081098 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) 
[n:127.0.0.1:49192_    ] o.a.s.c.CoresLocator Found 1 core definitions
   [junit4]   2> 1081100 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection   x:collection1] o.a.s.c.ZkController 
publishing state=down
   [junit4]   2> 1081100 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection   x:collection1] o.a.s.c.ZkController 
numShards not found on descriptor - reading it from system property
   [junit4]   2> 1081100 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) 
[n:127.0.0.1:49192_    ] o.a.s.s.SolrDispatchFilter 
user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2
   [junit4]   2> 1081100 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) 
[n:127.0.0.1:49192_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() 
done
   [junit4]   2> 1081101 INFO  
(OverseerStateUpdate-94682573230047236-127.0.0.1:49192_-n_0000000000) 
[n:127.0.0.1:49192_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message 
= {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:49192";,
   [junit4]   2>   "node_name":"127.0.0.1:49192_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 1081102 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection   x:collection1] o.a.s.c.ZkController 
look for our core node name
   [junit4]   2> 1081102 INFO  
(OverseerStateUpdate-94682573230047236-127.0.0.1:49192_-n_0000000000) 
[n:127.0.0.1:49192_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 
message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:49192";,
   [junit4]   2>   "node_name":"127.0.0.1:49192_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1081102 INFO  
(OverseerStateUpdate-94682573230047236-127.0.0.1:49192_-n_0000000000) 
[n:127.0.0.1:49192_    ] o.a.s.c.o.ClusterStateMutator building a new cName: 
control_collection
   [junit4]   2> 1081103 INFO  
(OverseerStateUpdate-94682573230047236-127.0.0.1:49192_-n_0000000000) 
[n:127.0.0.1:49192_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard 
shard=shard1
   [junit4]   2> 1081104 INFO  
(zkCallback-777-thread-1-processing-n:127.0.0.1:49192_) [n:127.0.0.1:49192_    
] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred 
- updating... (live nodes size: 1)
   [junit4]   2> 1081104 INFO  
(zkCallback-777-thread-2-processing-n:127.0.0.1:49192_) [n:127.0.0.1:49192_    
] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred 
- updating... (live nodes size: 1)
   [junit4]   2> 1082102 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection   x:collection1] o.a.s.c.ZkController 
waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1082102 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection   x:collection1] o.a.s.c.ZkController 
Check for collection zkNode:control_collection
   [junit4]   2> 1082103 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection   x:collection1] o.a.s.c.ZkController 
Collection zkNode exists
   [junit4]   2> 1082103 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection   x:collection1] 
o.a.s.c.c.ZkStateReader Load collection config 
from:/collections/control_collection
   [junit4]   2> 1082104 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection   x:collection1] 
o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 
specified config exists in ZooKeeper
   [junit4]   2> 1082104 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection   x:collection1] 
o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: 
'/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_79703BC618647C1C-001/control-001/cores/collection1/'
   [junit4]   2> 1082104 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection   x:collection1] 
o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1082104 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection   x:collection1] 
o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find 
system property or JNDI)
   [junit4]   2> 1082118 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection   x:collection1] o.a.s.c.Config loaded 
config solrconfig.xml with version 0 
   [junit4]   2> 1082122 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection   x:collection1] o.a.s.c.SolrConfig 
current version of requestparams : -1
   [junit4]   2> 1082126 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection   x:collection1] o.a.s.c.SolrConfig 
Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 1082133 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection   x:collection1] o.a.s.c.Config Loaded 
SolrConfig: solrconfig.xml
   [junit4]   2> 1082134 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection   x:collection1] o.a.s.s.IndexSchema 
Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 1082138 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection   x:collection1] o.a.s.s.IndexSchema 
[collection1] Schema name=test
   [junit4]   2> 1082221 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection   x:collection1] o.a.s.s.IndexSchema 
default search field in schema is text
   [junit4]   2> 1082222 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection   x:collection1] o.a.s.s.IndexSchema 
unique key field: id
   [junit4]   2> 1082222 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection   x:collection1] 
o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1082225 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection   x:collection1] 
o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1082239 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection   x:collection1] 
o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default 
properties: Can't find resource 'configsetprops.json' in classpath or 
'/configs/conf1', 
cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2
   [junit4]   2> 1082239 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection   x:collection1] o.a.s.c.CoreContainer 
Creating SolrCore 'collection1' using configuration from collection 
control_collection
   [junit4]   2> 1082239 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 1082240 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.HdfsDirectoryFactory 
solr.hdfs.home=hdfs://localhost:53937/solr_hdfs_home
   [junit4]   2> 1082240 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 1082240 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at 
[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_79703BC618647C1C-001/control-001/cores/collection1/],
 dataDir=[null]
   [junit4]   2> 1082240 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX 
Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1adee74e
   [junit4]   2> 1082240 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.HdfsDirectoryFactory creating directory factory for path 
hdfs://localhost:53937/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 1082251 WARN  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds 
and try again.
   [junit4]   2> 1087257 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.CachingDirectoryFactory return new directory for 
hdfs://localhost:53937/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 1087258 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore New index directory detected: old=null 
new=hdfs://localhost:53937/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 1087267 WARN  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore [collection1] Solr index directory 
'hdfs:/localhost:53937/solr_hdfs_home/control_collection/core_node1/data/index' 
doesn't exist. Creating new index...
   [junit4]   2> 1087267 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.HdfsDirectoryFactory creating directory factory for path 
hdfs://localhost:53937/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 1087275 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct 
memory allocation set to [true]
   [junit4]   2> 1087275 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of 
[8388608] will allocate [1] slabs and use ~[8388608] bytes
   [junit4]   2> 1087280 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 1087280 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.CachingDirectoryFactory return new directory for 
hdfs://localhost:53937/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 1087281 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: 
minMergeSize=0, mergeFactor=10, maxMergeSize=543362292, 
maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, 
maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.1]
   [junit4]   2> 1087308 INFO  (IPC Server handler 7 on 53937) [    ] 
BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:52933 is 
added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, 
primaryNodeIndex=-1, 
replicas=[ReplicaUnderConstruction[[DISK]DS-f01edb34-4ea8-4352-a408-decd4b2a44c6:NORMAL:127.0.0.1:56089|RBW],
 
ReplicaUnderConstruction[[DISK]DS-7ba55d40-ca92-4118-b425-07ab32550d0c:NORMAL:127.0.0.1:52933|RBW]]}
 size 0
   [junit4]   2> 1087310 INFO  (IPC Server handler 8 on 53937) [    ] 
BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:56089 is 
added to blk_1073741825_1001{blockUCState=UNDER_CONSTRUCTION, 
primaryNodeIndex=-1, 
replicas=[ReplicaUnderConstruction[[DISK]DS-7ba55d40-ca92-4118-b425-07ab32550d0c:NORMAL:127.0.0.1:52933|RBW],
 
ReplicaUnderConstruction[[DISK]DS-05be9c37-aa07-49e8-b1f7-ca758fd1a5fe:NORMAL:127.0.0.1:56089|FINALIZED]]}
 size 0
   [junit4]   2> 1087313 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2>        
commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@b86d5e03 
lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@58afe95e); 
maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 1087313 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1087315 INFO  
(OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:49192_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore 
Looking for old index directories to cleanup for core collection1 in 
hdfs://localhost:53937/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 1087318 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain 
"nodistrib"
   [junit4]   2> 1087319 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain 
"dedupe"
   [junit4]   2> 1087319 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.UpdateRequestProcessorChain inserting 
DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1087319 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain 
"stored_sig"
   [junit4]   2> 1087319 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.UpdateRequestProcessorChain inserting 
DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1087319 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain 
"distrib-dup-test-chain-explicit"
   [junit4]   2> 1087320 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain 
"distrib-dup-test-chain-implicit"
   [junit4]   2> 1087320 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.UpdateRequestProcessorChain inserting 
DistributedUpdateProcessorFactory into updateRequestProcessorChain 
"distrib-dup-test-chain-implicit"
   [junit4]   2> 1087320 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating 
implicit default
   [junit4]   2> 1087322 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1087323 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1087324 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1087325 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1087329 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.RequestHandlers Registered paths: 
/admin/mbeans,standard,/update/csv,/update/json/docs,/admin/luke,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/schema,/admin/plugins,/admin/logging,/update/json,/admin/threads,/admin/ping,/update,/admin/file
   [junit4]   2> 1087329 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore Using default statsCache cache: 
org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1087330 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.UpdateHandler Using UpdateLog implementation: 
org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 1087330 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.UpdateLog Initializing HdfsUpdateLog: dataDir= defaultSyncLevel=FLUSH 
numRecordsToKeep=100 maxNumLogsToKeep=10 tlogDfsReplication=2
   [junit4]   2> 1087341 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore Hard AutoCommit: disabled
   [junit4]   2> 1087341 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore Soft AutoCommit: disabled
   [junit4]   2> 1087342 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: 
minMergeSize=1677721, mergeFactor=30, maxMergeSize=2147483648, 
maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, 
maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.0]
   [junit4]   2> 1087350 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2>        
commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@b86d5e03 
lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@58afe95e); 
maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 1087350 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore newest commit generation = 1
   [junit4]   2> 1087350 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.s.SolrIndexSearcher Opening Searcher@2ef2ffcd[collection1] main
   [junit4]   2> 1087350 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.c.ZkStateReader Load collection config 
from:/collections/control_collection
   [junit4]   2> 1087351 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.c.ZkStateReader path=/collections/control_collection configName=conf1 
specified config exists in ZooKeeper
   [junit4]   2> 1087351 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the 
RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1087351 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: 
/configs/conf1
   [junit4]   2> 1087351 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1087351 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1087352 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.r.ManagedResourceStorage No data found for znode 
/configs/conf1/_rest_managed.json
   [junit4]   2> 1087352 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1087352 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1087352 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1087353 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1087353 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.VersionInfo Refreshing highest value of _version_ for 256 version 
buckets from index
   [junit4]   2> 1087353 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket 
highest value from index
   [junit4]   2> 1087353 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.UpdateLog Could not find max version in index or recent updates, using 
new clock 1514921178822606848
   [junit4]   2> 1087353 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.UpdateLog Took 0.0ms to seed version buckets with highest version 
1514921178822606848
   [junit4]   2> 1087353 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1087354 INFO  
(coreLoadExecutor-2476-thread-1-processing-n:127.0.0.1:49192_) 
[n:127.0.0.1:49192_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1087354 INFO  
(coreZkRegister-2470-thread-1-processing-n:127.0.0.1:49192_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:49192_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController 
Register replica - core:collection1 address:http://127.0.0.1:49192 
collection:control_collection shard:shard1
   [junit4]   2> 1087354 INFO  
(searcherExecutor-2477-thread-1-processing-n:127.0.0.1:49192_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:49192_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore 
[collection1] Registered new searcher Searcher@2ef2ffcd[collection1] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1087355 INFO  
(coreZkRegister-2470-thread-1-processing-n:127.0.0.1:49192_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:49192_ 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.c.SolrZkClient makePath: 
/collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1087358 INFO  
(coreZkRegister-2470-thread-1-processing-n:127.0.0.1:49192_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:49192_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector 
Joined leadership election with path: 
/collections/control_collection/leader_elect/shard1/election/94682573230047236-core_node1-n_0000000000
   [junit4]   2> 1087359 INFO  
(coreZkRegister-2470-thread-1-processing-n:127.0.0.1:49192_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:49192_ 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.ShardLeaderElectionContext Running the leader process for shard shard1
   [junit4]   2> 1087361 INFO  
(coreZkRegister-2470-thread-1-processing-n:127.0.0.1:49192_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:49192_ 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1087361 INFO  
(OverseerStateUpdate-94682573230047236-127.0.0.1:49192_-n_0000000000) 
[n:127.0.0.1:49192_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message 
= {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1087361 INFO  
(coreZkRegister-2470-thread-1-processing-n:127.0.0.1:49192_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:49192_ 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1087361 INFO  
(coreZkRegister-2470-thread-1-processing-n:127.0.0.1:49192_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:49192_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy 
Sync replicas to http://127.0.0.1:49192/collection1/
   [junit4]   2> 1087362 INFO  
(coreZkRegister-2470-thread-1-processing-n:127.0.0.1:49192_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:49192_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy 
Sync Success - now sync replicas to me
   [junit4]   2> 1087362 INFO  
(coreZkRegister-2470-thread-1-processing-n:127.0.0.1:49192_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:49192_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy 
http://127.0.0.1:49192/collection1/ has no replicas
   [junit4]   2> 1087362 INFO  
(coreZkRegister-2470-thread-1-processing-n:127.0.0.1:49192_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:49192_ 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.ShardLeaderElectionContext I am the new leader: 
http://127.0.0.1:49192/collection1/ shard1
   [junit4]   2> 1087362 INFO  
(coreZkRegister-2470-thread-1-processing-n:127.0.0.1:49192_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:49192_ 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1087362 INFO  
(zkCallback-777-thread-1-processing-n:127.0.0.1:49192_) [n:127.0.0.1:49192_    
] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred 
- updating... (live nodes size: 1)
   [junit4]   2> 1087363 INFO  
(zkCallback-777-thread-2-processing-n:127.0.0.1:49192_) [n:127.0.0.1:49192_    
] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred 
- updating... (live nodes size: 1)
   [junit4]   2> 1087366 INFO  
(coreZkRegister-2470-thread-1-processing-n:127.0.0.1:49192_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:49192_ 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node
   [junit4]   2> 1087368 INFO  
(OverseerStateUpdate-94682573230047236-127.0.0.1:49192_-n_0000000000) 
[n:127.0.0.1:49192_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message 
= {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:49192";,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 1087471 INFO  
(zkCallback-777-thread-1-processing-n:127.0.0.1:49192_) [n:127.0.0.1:49192_    
] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred 
- updating... (live nodes size: 1)
   [junit4]   2> 1087471 INFO  
(zkCallback-777-thread-2-processing-n:127.0.0.1:49192_) [n:127.0.0.1:49192_    
] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred 
- updating... (live nodes size: 1)
   [junit4]   2> 1087519 INFO  
(coreZkRegister-2470-thread-1-processing-n:127.0.0.1:49192_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:49192_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController 
We are http://127.0.0.1:49192/collection1/ and leader is 
http://127.0.0.1:49192/collection1/
   [junit4]   2> 1087520 INFO  
(coreZkRegister-2470-thread-1-processing-n:127.0.0.1:49192_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:49192_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController 
No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:49192
   [junit4]   2> 1087520 INFO  
(coreZkRegister-2470-thread-1-processing-n:127.0.0.1:49192_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:49192_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController 
I am the leader, no recovery necessary
   [junit4]   2> 1087520 INFO  
(coreZkRegister-2470-thread-1-processing-n:127.0.0.1:49192_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:49192_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController 
publishing state=active
   [junit4]   2> 1087520 INFO  
(coreZkRegister-2470-thread-1-processing-n:127.0.0.1:49192_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:49192_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController 
numShards not found on descriptor - reading it from system property
   [junit4]   2> 1087522 INFO  
(OverseerStateUpdate-94682573230047236-127.0.0.1:49192_-n_0000000000) 
[n:127.0.0.1:49192_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message 
= {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:49192";,
   [junit4]   2>   "node_name":"127.0.0.1:49192_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 3
   [junit4]   2> 1087522 INFO  
(OverseerStateUpdate-94682573230047236-127.0.0.1:49192_-n_0000000000) 
[n:127.0.0.1:49192_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 
message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:49192";,
   [junit4]   2>   "node_name":"127.0.0.1:49192_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1087604 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1087605 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1087606 INFO  (zkCallback-779-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@545a468d 
name:ZooKeeperConnection Watcher:127.0.0.1:44893/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1087606 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1087606 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1087606 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1087609 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection 
loss:false
   [junit4]   2> 1087625 INFO  
(zkCallback-777-thread-1-processing-n:127.0.0.1:49192_) [n:127.0.0.1:49192_    
] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred 
- updating... (live nodes size: 1)
   [junit4]   2> 1087625 INFO  
(zkCallback-777-thread-2-processing-n:127.0.0.1:49192_) [n:127.0.0.1:49192_    
] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred 
- updating... (live nodes size: 1)
   [junit4]   2> 1087625 INFO  (zkCallback-779-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred 
- updating... (live nodes size: 1)
   [junit4]   2> 1087930 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.SolrTestCaseJ4 Writing core.properties file to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_79703BC618647C1C-001/shard-1-001/cores/collection1
   [junit4]   2> 1087931 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_79703BC618647C1C-001/shard-1-001
   [junit4]   2> 1087932 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.e.j.s.Server jetty-9.2.13.v20150730
   [junit4]   2> 1087933 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@351b5edd{/,null,AVAILABLE}
   [junit4]   2> 1087934 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.e.j.s.ServerConnector Started 
ServerConnector@9096c93{HTTP/1.1}{127.0.0.1:50337}
   [junit4]   2> 1087934 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.e.j.s.Server Started @1091175ms
   [junit4]   2> 1087934 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: 
{solr.data.dir=hdfs://localhost:53937/hdfs__localhost_53937__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-trunk_solr_build_solr-core_test_J2_temp_solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_79703BC618647C1C-001_tempDir-002_jetty1,
 solrconfig=solrconfig.xml, hostContext=/, hostPort=50337, 
coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_79703BC618647C1C-001/shard-1-001/cores}
   [junit4]   2> 1087934 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): 
sun.misc.Launcher$AppClassLoader@4e0e2f2a
   [junit4]   2> 1087934 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: 
'/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_79703BC618647C1C-001/shard-1-001/'
   [junit4]   2> 1087934 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1087935 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find 
system property or JNDI)
   [junit4]   2> 1087949 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1087950 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1087951 INFO  (zkCallback-780-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@63b12ad9 
name:ZooKeeperConnection Watcher:127.0.0.1:44893/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1087951 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1087951 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1087952 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in 
ZooKeeper)
   [junit4]   2> 1087952 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.SolrXmlConfig Loading container configuration from 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_79703BC618647C1C-001/shard-1-001/solr.xml
   [junit4]   2> 1087957 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.CoresLocator Config-defined core root directory: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_79703BC618647C1C-001/shard-1-001/cores
   [junit4]   2> 1087957 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.CoreContainer New CoreContainer 936397902
   [junit4]   2> 1087957 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.CoreContainer Loading cores into CoreContainer 
[instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_79703BC618647C1C-001/shard-1-001/]
   [junit4]   2> 1087957 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.CoreContainer loading shared library: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_79703BC618647C1C-001/shard-1-001/lib
   [junit4]   2> 1087957 WARN  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.SolrResourceLoader Can't find (or read) directory to add to 
classloader: lib (resolved as: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_79703BC618647C1C-001/shard-1-001/lib).
   [junit4]   2> 1087965 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme 
: ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 
10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 
5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1087966 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: 
socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1087966 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1087966 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.l.LogWatcher Registering Log Listener [Log4j 
(org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1087967 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44893/solr
   [junit4]   2> 1087967 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1087967 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1087967 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1087968 INFO  (zkCallback-782-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@1284ea70 
name:ZooKeeperConnection Watcher:127.0.0.1:44893 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1087969 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1087970 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1087971 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) 
[n:127.0.0.1:50337_    ] o.a.s.c.c.ConnectionManager Waiting for client to 
connect to ZooKeeper
   [junit4]   2> 1087972 INFO  
(zkCallback-783-thread-1-processing-n:127.0.0.1:50337_) [n:127.0.0.1:50337_    
] o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@23c60952 
name:ZooKeeperConnection Watcher:127.0.0.1:44893/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1087972 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) 
[n:127.0.0.1:50337_    ] o.a.s.c.c.ConnectionManager Client is connected to 
ZooKeeper
   [junit4]   2> 1087977 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) 
[n:127.0.0.1:50337_    ] o.a.s.c.c.ZkStateReader Updating cluster state from 
ZooKeeper... 
   [junit4]   2> 1088982 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) 
[n:127.0.0.1:50337_    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:50337_
   [junit4]   2> 1088982 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) 
[n:127.0.0.1:50337_    ] o.a.s.c.c.SolrZkClient makePath: 
/live_nodes/127.0.0.1:50337_
   [junit4]   2> 1088983 INFO  
(zkCallback-777-thread-2-processing-n:127.0.0.1:49192_) [n:127.0.0.1:49192_    
] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected 
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live 
nodes size: 1)
   [junit4]   2> 1088983 INFO  
(zkCallback-777-thread-1-processing-n:127.0.0.1:49192_) [n:127.0.0.1:49192_    
] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected 
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live 
nodes size: 1)
   [junit4]   2> 1088984 INFO  (zkCallback-779-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected 
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live 
nodes size: 1)
   [junit4]   2> 1088983 INFO  
(zkCallback-783-thread-1-processing-n:127.0.0.1:50337_) [n:127.0.0.1:50337_    
] o.a.s.c.c.ZkStateReader A live node change: WatchedEvent state:SyncConnected 
type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live 
nodes size: 1)
   [junit4]   2> 1088985 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) 
[n:127.0.0.1:50337_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1088990 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) 
[n:127.0.0.1:50337_    ] o.a.s.c.LeaderElector Joined leadership election with 
path: /overseer_elect/election/94682573230047240-127.0.0.1:50337_-n_0000000001
   [junit4]   2> 1088991 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) 
[n:127.0.0.1:50337_    ] o.a.s.c.LeaderElector Watching path 
/overseer_elect/election/94682573230047236-127.0.0.1:49192_-n_0000000000 to 
know if I could be the leader
   [junit4]   2> 1089007 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) 
[n:127.0.0.1:50337_    ] o.a.s.c.CoreContainer Security conf doesn't exist. 
Skipping setup for authorization module.
   [junit4]   2> 1089007 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) 
[n:127.0.0.1:50337_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1089008 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) 
[n:127.0.0.1:50337_    ] o.a.s.c.CoresLocator Looking for core definitions 
underneath 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_79703BC618647C1C-001/shard-1-001/cores
   [junit4]   2> 1089009 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[79703BC618647C1C]) 
[n:127.0.0.1:50337_    ] o.a.s.c.SolrCore Created CoreDescriptor: 
{name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, 
loadOnStartup=true, configSetProperties=configsetprops.json, 
instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_79703BC618647C1C-001/shard-1-001/cores/collection1,
 collection=collection1, 
absoluteInstDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_79703BC618647C1C-001/shard-1-001/cores/collection1/,
 coreNodeName=, dataDir=data/, shard

[...truncated too long message...]

:415)
   [junit4]   2>        at 
com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546)
   [junit4]   2>        at 
org.apache.hadoop.metrics2.util.MBeans.unregister(MBeans.java:81)
   [junit4]   2>        at 
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stopMBeans(MetricsSourceAdapter.java:227)
   [junit4]   2>        at 
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stop(MetricsSourceAdapter.java:212)
   [junit4]   2>        at 
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stopSources(MetricsSystemImpl.java:461)
   [junit4]   2>        at 
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stop(MetricsSystemImpl.java:212)
   [junit4]   2>        at 
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:592)
   [junit4]   2>        at 
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72)
   [junit4]   2>        at 
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
   [junit4]   2>        at 
org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:145)
   [junit4]   2>        at 
org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:822)
   [junit4]   2>        at 
org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1720)
   [junit4]   2>        at 
org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1699)
   [junit4]   2>        at 
org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:197)
   [junit4]   2>        at 
org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.teardownClass(HdfsCollectionsAPIDistributedZkTest.java:53)
   [junit4]   2>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native 
Method)
   [junit4]   2>        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2>        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2>        at java.lang.reflect.Method.invoke(Method.java:497)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1665)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2>        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2>        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2>        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2>        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.NullPointerException
   [junit4]   2>        at 
org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:198)
   [junit4]   2>        at 
org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3291)
   [junit4]   2>        at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:6223)
   [junit4]   2>        ... 53 more
   [junit4]   2> 3731338 INFO  
(SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[79703BC618647C1C]-worker) 
[n:127.0.0.1:38108_oav c:addReplicaColl s:shard1 r:core_node4 
x:addReplicaColl_shard1_replica1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> Oct 13, 2015 9:47:44 AM 
com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 2 leaked 
thread(s).
   [junit4]   2> NOTE: test params are: codec=CheapBastard, 
sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=ro_RO, 
timezone=Etc/Greenwich
   [junit4]   2> NOTE: Linux 3.13.0-52-generic amd64/Oracle Corporation 
1.8.0_45 (64-bit)/cpus=4,threads=2,free=274685592,total=522715136
   [junit4]   2> NOTE: All tests run in this JVM: [TestBinaryResponseWriter, 
DistribDocExpirationUpdateProcessorTest, 
IgnoreCommitOptimizeUpdateProcessorFactoryTest, TestDownShardTolerantSearch, 
TestSolrQueryParserResource, AtomicUpdatesTest, TestRebalanceLeaders, 
DocExpirationUpdateProcessorFactoryTest, InfoHandlerTest, SuggestComponentTest, 
TermsComponentTest, SimplePostToolTest, TestFastOutputStream, 
TestHashPartitioner, TestFileDictionaryLookup, FastVectorHighlighterTest, 
ConnectionManagerTest, CachingDirectoryFactoryTest, FullHLLTest, 
TestJsonRequest, DocumentBuilderTest, TestCodecSupport, TestDistribIDF, 
TestReloadAndDeleteDocs, TestTrie, SoftAutoCommitTest, TestJettySolrRunner, 
TestUtils, RuleEngineTest, FacetPivotSmallTest, 
VMParamsZkACLAndCredentialsProvidersTest, TestStressVersions, 
MBeansHandlerTest, DirectSolrSpellCheckerTest, TestSolrDeletionPolicy2, 
CoreAdminCreateDiscoverTest, TestSurroundQueryParser, MultiTermTest, 
DistribCursorPagingTest, NotRequiredUniqueKeyTest, GraphQueryTest, 
TestNonDefinedSimilarityFactory, TestReloadDeadlock, RollingRestartTest, 
SaslZkACLProviderTest, ConvertedLegacyTest, DistributedQueueTest, 
BlockCacheTest, ChaosMonkeyNothingIsSafeTest, RequestLoggingTest, 
SolrIndexSplitterTest, DOMUtilTest, TestFiltering, TestRestoreCore, 
SortByFunctionTest, SpatialRPTFieldTypeTest, TestUpdate, 
TestRuleBasedAuthorizationPlugin, SpellCheckComponentTest, HdfsLockFactoryTest, 
SolrPluginUtilsTest, TermVectorComponentTest, SolrInfoMBeanTest, 
TestSortByMinMaxFunction, TestNRTOpen, BitVectorTest, TestMiniSolrCloudCluster, 
CircularListTest, TestManagedSchema, TestComplexPhraseQParserPlugin, 
TestLeaderElectionZkExpiry, TestCopyFieldCollectionResource, TestConfigOverlay, 
ZkControllerTest, CollectionsAPIDistributedZkTest, HdfsRecoverLeaseTest, 
TestJmxMonitoredMap, TestMacros, DistributedFacetPivotWhiteBoxTest, 
SpellCheckCollatorTest, TestManagedSchemaFieldTypeResource, 
MoreLikeThisHandlerTest, TestCryptoKeys, TestConfigSets, SolrCloudExampleTest, 
OpenCloseCoreStressTest, TestTolerantSearch, DistributedExpandComponentTest, 
UniqFieldsUpdateProcessorFactoryTest, TestSchemaManager, 
SignatureUpdateProcessorFactoryTest, CoreAdminHandlerTest, TestStressLucene, 
TestCSVLoader, WordBreakSolrSpellCheckerTest, TestPseudoReturnFields, 
TestWordDelimiterFilterFactory, TestRemoteStreaming, DocValuesMultiTest, 
DistanceFunctionTest, XsltUpdateRequestHandlerTest, LukeRequestHandlerTest, 
TestWriterPerf, TestOmitPositions, RequestHandlersTest, LoggingHandlerTest, 
IndexSchemaTest, TestCSVResponseWriter, TestPartialUpdateDeduplication, 
TestComponentsName, SOLR749Test, TestQuerySenderListener, TestSolrIndexConfig, 
TestQuerySenderNoQuery, CopyFieldTest, SolrIndexConfigTest, BadComponentTest, 
TestStressRecovery, TestRTGBase, DistributedIntervalFacetingTest, AssignTest, 
AsyncCallRequestStatusResponseTest, BaseCdcrDistributedZkTest, 
CdcrReplicationDistributedZkTest, RemoteQueryErrorTest, 
SharedFSAutoReplicaFailoverTest, SharedFSAutoReplicaFailoverUtilsTest, 
TestCloudInspectUtil, TestCloudPivotFacet, TestMiniSolrCloudClusterSSL, 
TestRandomRequestDistribution, TestSolrCloudWithKerberosAlt, 
HdfsCollectionsAPIDistributedZkTest]
   [junit4] Completed [461/552] on J1 in 290.94s, 1 test, 1 error <<< FAILURES!

[...truncated 297 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:782:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:719:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/build.xml:59:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/build.xml:233:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/solr/common-build.xml:516:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:1432:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-trunk/lucene/common-build.xml:987:
 There were test failures: 552 suites (8 ignored), 2170 tests, 3 errors, 1 
failure, 30 ignored (17 assumptions) [seed: 79703BC618647C1C]

Total time: 176 minutes 32 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Reply via email to