Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/117/

1 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest

Error Message:
ObjectTracker found 2 object(s) that were not released!!! [HdfsTransactionLog, 
HdfsTransactionLog]

Stack Trace:
java.lang.AssertionError: ObjectTracker found 2 object(s) that were not 
released!!! [HdfsTransactionLog, HdfsTransactionLog]
        at __randomizedtesting.SeedInfo.seed([88A15EA81E630D4B]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertNull(Assert.java:551)
        at 
org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:258)
        at sun.reflect.GeneratedMethodAccessor59.invoke(Unknown Source)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
        at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
        at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11666 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest
   [junit4]   2> Creating dataDir: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001/init-core-data-001
   [junit4]   2> 1219445 INFO  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[88A15EA81E630D4B]-worker) [    ] 
o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: 
@org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 1219445 INFO  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[88A15EA81E630D4B]-worker) [    ] 
o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 1219482 WARN  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[88A15EA81E630D4B]-worker) [    ] 
o.a.h.m.i.MetricsConfig Cannot locate configuration: tried 
hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 1219492 WARN  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[88A15EA81E630D4B]-worker) [    ] 
o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 1219494 INFO  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[88A15EA81E630D4B]-worker) [    ] 
o.m.log jetty-6.1.26
   [junit4]   2> 1219505 INFO  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[88A15EA81E630D4B]-worker) [    ] 
o.m.log Extract 
jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/hdfs
 to ./temp/Jetty_localhost_33211_hdfs____y9hezg/webapp
   [junit4]   2> 1219868 INFO  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[88A15EA81E630D4B]-worker) [    ] 
o.m.log Started 
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33211
   [junit4]   2> 1219940 WARN  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[88A15EA81E630D4B]-worker) [    ] 
o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 1219941 INFO  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[88A15EA81E630D4B]-worker) [    ] 
o.m.log jetty-6.1.26
   [junit4]   2> 1219954 INFO  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[88A15EA81E630D4B]-worker) [    ] 
o.m.log Extract 
jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode
 to ./temp/Jetty_localhost_33600_datanode____.r1gfk3/webapp
   [junit4]   2> 1220317 INFO  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[88A15EA81E630D4B]-worker) [    ] 
o.m.log Started 
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33600
   [junit4]   2> 1220407 WARN  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[88A15EA81E630D4B]-worker) [    ] 
o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 1220408 INFO  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[88A15EA81E630D4B]-worker) [    ] 
o.m.log jetty-6.1.26
   [junit4]   2> 1220427 INFO  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[88A15EA81E630D4B]-worker) [    ] 
o.m.log Extract 
jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode
 to ./temp/Jetty_localhost_35143_datanode____.3hv2vf/webapp
   [junit4]   2> 1220484 INFO  (IPC Server handler 3 on 36530) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-8d2db89d-66ff-4879-ae8c-007c303993f8 node 
DatanodeRegistration(127.0.0.1:49909, 
datanodeUuid=3912d198-a7b8-4049-a0aa-11c949a5c3b9, infoPort=40536, 
infoSecurePort=0, ipcPort=35703, 
storageInfo=lv=-56;cid=testClusterID;nsid=2044110516;c=0), blocks: 0, 
hasStaleStorage: true, processing time: 0 msecs
   [junit4]   2> 1220484 INFO  (IPC Server handler 3 on 36530) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-beeed90d-2972-4d08-9e93-f0dd794142f5 node 
DatanodeRegistration(127.0.0.1:49909, 
datanodeUuid=3912d198-a7b8-4049-a0aa-11c949a5c3b9, infoPort=40536, 
infoSecurePort=0, ipcPort=35703, 
storageInfo=lv=-56;cid=testClusterID;nsid=2044110516;c=0), blocks: 0, 
hasStaleStorage: false, processing time: 1 msecs
   [junit4]   2> 1220882 INFO  
(SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[88A15EA81E630D4B]-worker) [    ] 
o.m.log Started 
HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:35143
   [junit4]   2> 1221025 INFO  (IPC Server handler 3 on 36530) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-6fd1d7fc-8455-415e-9e87-614f38f1b4da node 
DatanodeRegistration(127.0.0.1:40002, 
datanodeUuid=4342edd8-7180-44f7-a734-2734af85847b, infoPort=40941, 
infoSecurePort=0, ipcPort=59249, 
storageInfo=lv=-56;cid=testClusterID;nsid=2044110516;c=0), blocks: 0, 
hasStaleStorage: true, processing time: 0 msecs
   [junit4]   2> 1221025 INFO  (IPC Server handler 3 on 36530) [    ] 
BlockStateChange BLOCK* processReport: from storage 
DS-e740d183-55a8-4adc-893d-b021faaeb4bb node 
DatanodeRegistration(127.0.0.1:40002, 
datanodeUuid=4342edd8-7180-44f7-a734-2734af85847b, infoPort=40941, 
infoSecurePort=0, ipcPort=59249, 
storageInfo=lv=-56;cid=testClusterID;nsid=2044110516;c=0), blocks: 0, 
hasStaleStorage: false, processing time: 0 msecs
   [junit4]   2> 1221173 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1221173 INFO  (Thread-14985) [    ] o.a.s.c.ZkTestServer 
client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1221173 INFO  (Thread-14985) [    ] o.a.s.c.ZkTestServer 
Starting server
   [junit4]   2> 1221273 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.ZkTestServer start zk server on port:49574
   [junit4]   2> 1221274 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1221274 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1221277 INFO  (zkCallback-1350-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@6e863bae 
name:ZooKeeperConnection Watcher:127.0.0.1:49574 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1221277 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1221277 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1221277 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1221280 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1221280 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1221281 INFO  (zkCallback-1351-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@55b3dd0 name:ZooKeeperConnection 
Watcher:127.0.0.1:49574/solr got event WatchedEvent state:SyncConnected 
type:None path:null path:null type:None
   [junit4]   2> 1221281 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1221281 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1221281 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1221282 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1221283 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1221284 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1221286 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml
 to /configs/conf1/solrconfig.xml
   [junit4]   2> 1221286 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1221293 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml
 to /configs/conf1/schema.xml
   [junit4]   2> 1221293 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1221295 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml
 to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1221295 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.SolrZkClient makePath: 
/configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1221296 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt
 to /configs/conf1/stopwords.txt
   [junit4]   2> 1221296 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1221297 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt
 to /configs/conf1/protwords.txt
   [junit4]   2> 1221297 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1221298 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/currency.xml
 to /configs/conf1/currency.xml
   [junit4]   2> 1221298 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1221299 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml
 to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1221300 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1221301 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json
 to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1221301 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1221302 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt
 to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1221302 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1221303 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt
 to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1221303 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1221305 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.AbstractZkTestCase put 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt
 to /configs/conf1/synonyms.txt
   [junit4]   2> 1221305 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1221307 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1221307 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1221308 INFO  (zkCallback-1353-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@773e0e51 
name:ZooKeeperConnection Watcher:127.0.0.1:49574/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1221308 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1221308 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1221397 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.SolrTestCaseJ4 Writing core.properties file to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001/control-001/cores/collection1
   [junit4]   2> 1221399 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 1221401 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@12e04d99{/,null,AVAILABLE}
   [junit4]   2> 1221402 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.e.j.s.ServerConnector Started ServerConnector@2d781c93{SSL,[ssl, 
http/1.1]}{127.0.0.1:48830}
   [junit4]   2> 1221402 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.e.j.s.Server Started @1226244ms
   [junit4]   2> 1221402 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: 
{solr.data.dir=hdfs://localhost:36530/hdfs__localhost_36530__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-6.x_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001_tempDir-002_control_data,
 hostContext=/, hostPort=48830, 
coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001/control-001/cores}
   [junit4]   2> 1221403 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): 
sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 1221403 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: 
'/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001/control-001'
   [junit4]   2> 1221403 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1221403 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find 
system property or JNDI)
   [junit4]   2> 1221403 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1221404 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1221405 INFO  (zkCallback-1354-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@401bd564 
name:ZooKeeperConnection Watcher:127.0.0.1:49574/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1221406 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1221406 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1221407 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in 
ZooKeeper)
   [junit4]   2> 1221407 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.SolrXmlConfig Loading container configuration from 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001/control-001/solr.xml
   [junit4]   2> 1221412 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.CorePropertiesLocator Config-defined core root directory: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001/control-001/cores
   [junit4]   2> 1221412 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.CoreContainer New CoreContainer 204203848
   [junit4]   2> 1221412 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.CoreContainer Loading cores into CoreContainer 
[instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001/control-001]
   [junit4]   2> 1221412 WARN  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.CoreContainer Couldn't add files from 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001/control-001/lib
 to classpath: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001/control-001/lib
   [junit4]   2> 1221412 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme 
: ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 
10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 
5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1221413 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: 
socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1221414 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1221414 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.l.LogWatcher Registering Log Listener [Log4j 
(org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1221414 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:49574/solr
   [junit4]   2> 1221414 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1221414 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1221415 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1221416 INFO  (zkCallback-1357-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@7c9121b6 
name:ZooKeeperConnection Watcher:127.0.0.1:49574 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1221416 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1221416 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1221417 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:48830_    ] o.a.s.c.ZkController Added new OnReconnect listener 
org.apache.solr.cloud.ZkController$$Lambda$47/319444693@d49f11c
   [junit4]   2> 1221418 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:48830_    ] o.a.s.c.c.ConnectionManager Waiting for client to 
connect to ZooKeeper
   [junit4]   2> 1221418 INFO  
(zkCallback-1358-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_    
] o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@18d4db16 
name:ZooKeeperConnection Watcher:127.0.0.1:49574/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1221418 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:48830_    ] o.a.s.c.c.ConnectionManager Client is connected to 
ZooKeeper
   [junit4]   2> 1221419 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:48830_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1221420 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:48830_    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/collection-queue-work
   [junit4]   2> 1221422 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:48830_    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/collection-map-running
   [junit4]   2> 1221423 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:48830_    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/collection-map-completed
   [junit4]   2> 1221424 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:48830_    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer/collection-map-failure
   [junit4]   2> 1221425 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:48830_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1221426 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:48830_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1221427 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:48830_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1221428 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:48830_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1221436 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:48830_    ] o.a.s.c.c.ZkStateReader Updating cluster state from 
ZooKeeper... 
   [junit4]   2> 1221437 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:48830_    ] o.a.s.c.c.ZkStateReader Loaded cluster properties: 
{urlScheme=https}
   [junit4]   2> 1221437 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:48830_    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (0)
   [junit4]   2> 1221457 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:48830_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1221458 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:48830_    ] o.a.s.c.c.SolrZkClient makePath: 
/overseer_elect/election
   [junit4]   2> 1221459 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:48830_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1221459 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:48830_    ] o.a.s.c.LeaderElector Joined leadership election with 
path: /overseer_elect/election/96224740103684101-127.0.0.1:48830_-n_0000000000
   [junit4]   2> 1221460 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:48830_    ] o.a.s.c.OverseerElectionContext I am going to be the 
leader 127.0.0.1:48830_
   [junit4]   2> 1221460 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:48830_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1221461 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:48830_    ] o.a.s.c.Overseer Overseer 
(id=96224740103684101-127.0.0.1:48830_-n_0000000000) starting
   [junit4]   2> 1221463 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:48830_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1221474 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:48830_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting 
OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 
autoReplicaFailoverWaitAfterExpiration=10000 
autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1221479 INFO  
(OverseerStateUpdate-96224740103684101-127.0.0.1:48830_-n_0000000000) 
[n:127.0.0.1:48830_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1221501 INFO  
(OverseerCollectionConfigSetProcessor-96224740103684101-127.0.0.1:48830_-n_0000000000)
 [n:127.0.0.1:48830_    ] o.a.s.c.OverseerTaskProcessor Process current queue 
of overseer operations
   [junit4]   2> 1221505 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:48830_    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:48830_
   [junit4]   2> 1221505 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:48830_    ] o.a.s.c.c.SolrZkClient makePath: 
/live_nodes/127.0.0.1:48830_
   [junit4]   2> 1221505 INFO  
(OverseerStateUpdate-96224740103684101-127.0.0.1:48830_-n_0000000000) 
[n:127.0.0.1:48830_    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (0)
   [junit4]   2> 1221520 INFO  
(zkCallback-1358-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_    
] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected 
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live 
nodes size: [0])
   [junit4]   2> 1221532 INFO  
(zkCallback-1358-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_    
] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1221561 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:48830_    ] o.a.s.c.CoreContainer Security conf doesn't exist. 
Skipping setup for authorization module.
   [junit4]   2> 1221561 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:48830_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1221561 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:48830_    ] o.a.s.c.CorePropertiesLocator Looking for core 
definitions underneath 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001/control-001/cores
   [junit4]   2> 1221562 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:48830_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: 
{name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, 
loadOnStartup=true, configSetProperties=configsetprops.json, 
collection=control_collection, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 1221562 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:48830_    ] o.a.s.c.CorePropertiesLocator Found core collection1 
in 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001/control-001/cores/collection1
   [junit4]   2> 1221562 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:48830_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1221569 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection   x:collection1] o.a.s.c.ZkController 
publishing state=down
   [junit4]   2> 1221569 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection   x:collection1] o.a.s.c.ZkController 
numShards not found on descriptor - reading it from system property
   [junit4]   2> 1221570 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection   x:collection1] o.a.s.c.ZkController 
Collection control_collection not visible yet, but flagging it so a watch is 
registered when it becomes visible
   [junit4]   2> 1221571 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection   x:collection1] 
o.a.s.c.c.ZkStateReader Deleting data for [control_collection]
   [junit4]   2> 1221573 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:48830_    ] o.a.s.s.SolrDispatchFilter 
user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1
   [junit4]   2> 1221573 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:48830_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() 
done
   [junit4]   2> 1221580 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection   x:collection1] o.a.s.c.ZkController 
look for our core node name
   [junit4]   2> 1221581 INFO  
(OverseerStateUpdate-96224740103684101-127.0.0.1:48830_-n_0000000000) 
[n:127.0.0.1:48830_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message 
= {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:48830";,
   [junit4]   2>   "node_name":"127.0.0.1:48830_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 1221581 INFO  
(OverseerStateUpdate-96224740103684101-127.0.0.1:48830_-n_0000000000) 
[n:127.0.0.1:48830_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 
message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:48830";,
   [junit4]   2>   "node_name":"127.0.0.1:48830_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1221581 INFO  
(OverseerStateUpdate-96224740103684101-127.0.0.1:48830_-n_0000000000) 
[n:127.0.0.1:48830_    ] o.a.s.c.o.ClusterStateMutator building a new cName: 
control_collection
   [junit4]   2> 1221582 INFO  
(OverseerStateUpdate-96224740103684101-127.0.0.1:48830_-n_0000000000) 
[n:127.0.0.1:48830_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard 
shard=shard1
   [junit4]   2> 1221583 INFO  
(zkCallback-1358-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_    
] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred 
- updating... (live nodes size: [1])
   [junit4]   2> 1222584 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection   x:collection1] o.a.s.c.ZkController 
waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1222584 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection   x:collection1] o.a.s.c.ZkController 
Check for collection zkNode:control_collection
   [junit4]   2> 1222585 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection   x:collection1] o.a.s.c.ZkController 
Collection zkNode exists
   [junit4]   2> 1222585 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection   x:collection1] 
o.a.s.c.c.ZkStateReader Load collection config from: 
[/collections/control_collection]
   [junit4]   2> 1222585 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection   x:collection1] 
o.a.s.c.c.ZkStateReader path=[/collections/control_collection] 
[configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 1222585 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection   x:collection1] 
o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: 
'/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001/control-001/cores/collection1'
   [junit4]   2> 1222586 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection   x:collection1] 
o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1222586 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection   x:collection1] 
o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find 
system property or JNDI)
   [junit4]   2> 1222587 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection   x:collection1] o.a.s.c.Config loaded 
config solrconfig.xml with version 0 
   [junit4]   2> 1222591 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection   x:collection1] o.a.s.c.SolrConfig 
current version of requestparams : -1
   [junit4]   2> 1222595 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection   x:collection1] o.a.s.c.SolrConfig 
Using Lucene MatchVersion: 6.2.0
   [junit4]   2> 1222603 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection   x:collection1] o.a.s.c.SolrConfig 
Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1222607 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection   x:collection1] o.a.s.s.IndexSchema 
[collection1] Schema name=test
   [junit4]   2> 1222667 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection   x:collection1] o.a.s.s.IndexSchema 
default search field in schema is text
   [junit4]   2> 1222667 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection   x:collection1] o.a.s.s.IndexSchema 
unique key field: id
   [junit4]   2> 1222668 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection   x:collection1] 
o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1222670 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection   x:collection1] 
o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1222691 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection   x:collection1] 
o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default 
properties: Can't find resource 'configsetprops.json' in classpath or 
'/configs/conf1', 
cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1
   [junit4]   2> 1222691 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection   x:collection1] o.a.s.c.CoreContainer 
Creating SolrCore 'collection1' using configuration from collection 
control_collection
   [junit4]   2> 1222691 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 1222691 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.HdfsDirectoryFactory 
solr.hdfs.home=hdfs://localhost:36530/solr_hdfs_home
   [junit4]   2> 1222691 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 1222691 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at 
[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001/control-001/cores/collection1],
 
dataDir=[hdfs://localhost:36530/solr_hdfs_home/control_collection/core_node1/data/]
   [junit4]   2> 1222691 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX 
Server: com.sun.jmx.mbeanserver.JmxMBeanServer@14aded07
   [junit4]   2> 1222692 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.HdfsDirectoryFactory creating directory factory for path 
hdfs://localhost:36530/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 1222704 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.CachingDirectoryFactory return new directory for 
hdfs://localhost:36530/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 1222704 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore New index directory detected: old=null 
new=hdfs://localhost:36530/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 1222711 WARN  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore [collection1] Solr index directory 
'hdfs:/localhost:36530/solr_hdfs_home/control_collection/core_node1/data/index' 
doesn't exist. Creating new index...
   [junit4]   2> 1222711 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.HdfsDirectoryFactory creating directory factory for path 
hdfs://localhost:36530/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 1222718 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct 
memory allocation set to [true]
   [junit4]   2> 1222718 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of 
[8388608] will allocate [1] slabs and use ~[8388608] bytes
   [junit4]   2> 1222722 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 1222722 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.CachingDirectoryFactory return new directory for 
hdfs://localhost:36530/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 1222722 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: 
minMergeSize=1000, mergeFactor=16, maxMergeSize=9223372036854775807, 
maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, 
maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.6165534828591587]
   [junit4]   2> 1222744 INFO  (IPC Server handler 3 on 36530) [    ] 
BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:40002 is 
added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, 
primaryNodeIndex=-1, 
replicas=[ReplicaUC[[DISK]DS-beeed90d-2972-4d08-9e93-f0dd794142f5:NORMAL:127.0.0.1:49909|RBW],
 
ReplicaUC[[DISK]DS-e740d183-55a8-4adc-893d-b021faaeb4bb:NORMAL:127.0.0.1:40002|FINALIZED]]}
 size 0
   [junit4]   2> 1222745 INFO  (IPC Server handler 4 on 36530) [    ] 
BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:49909 is 
added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, 
primaryNodeIndex=-1, 
replicas=[ReplicaUC[[DISK]DS-e740d183-55a8-4adc-893d-b021faaeb4bb:NORMAL:127.0.0.1:40002|FINALIZED],
 
ReplicaUC[[DISK]DS-8d2db89d-66ff-4879-ae8c-007c303993f8:NORMAL:127.0.0.1:49909|FINALIZED]]}
 size 0
   [junit4]   2> 1222748 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2>        
commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@hdfs://localhost:36530/solr_hdfs_home/control_collection/core_node1/data/index
 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@68f4dd0a); 
maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 1222748 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1222752 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain 
"nodistrib"
   [junit4]   2> 1222752 INFO  
(OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:48830_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore 
Looking for old index directories to cleanup for core collection1 in 
hdfs://localhost:36530/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 1222752 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain 
"dedupe"
   [junit4]   2> 1222752 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.UpdateRequestProcessorChain inserting 
DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1222752 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain 
"stored_sig"
   [junit4]   2> 1222752 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.UpdateRequestProcessorChain inserting 
DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1222753 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain 
"distrib-dup-test-chain-explicit"
   [junit4]   2> 1222753 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain 
"distrib-dup-test-chain-implicit"
   [junit4]   2> 1222753 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.p.UpdateRequestProcessorChain inserting 
DistributedUpdateProcessorFactory into updateRequestProcessorChain 
"distrib-dup-test-chain-implicit"
   [junit4]   2> 1222753 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating 
implicit default
   [junit4]   2> 1222753 WARN  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = 
requestHandler,name = /dump,class = DumpRequestHandler,args = 
{defaults={a=A,b=B}}}
   [junit4]   2> 1222755 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1222755 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1222756 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1222757 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1222760 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.RequestHandlers Registered paths: 
/admin/mbeans,standard,/update/json/docs,/admin/luke,/export,/get,/admin/properties,/update/json,/admin/threads,/dump,/update/csv,/sql,/graph,/admin/segments,/admin/system,/replication,/config,/stream,/schema,/admin/plugins,/admin/logging,/admin/ping,/update,/admin/file,/terms
   [junit4]   2> 1222760 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrCore Using default statsCache cache: 
org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1222760 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.UpdateHandler Using UpdateLog implementation: 
org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 1222760 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH 
numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1222760 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
   [junit4]   2> 1222772 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.CommitTracker Hard AutoCommit: if uncommited for 15000ms; 
   [junit4]   2> 1222772 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1222773 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class 
org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: 
minMergeSize=1000, mergeFactor=14, maxMergeSize=9223372036854775807, 
maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, 
maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, 
noCFSRatio=0.0]
   [junit4]   2> 1222778 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2>        
commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@hdfs://localhost:36530/solr_hdfs_home/control_collection/core_node1/data/index
 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@68f4dd0a); 
maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 1222778 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1222778 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.s.SolrIndexSearcher Opening [Searcher@33f06aa[collection1] main]
   [junit4]   2> 1222778 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.c.ZkStateReader Load collection config from: 
[/collections/control_collection]
   [junit4]   2> 1222779 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.c.ZkStateReader path=[/collections/control_collection] 
[configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 1222779 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the 
RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1222779 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: 
/configs/conf1
   [junit4]   2> 1222779 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1222779 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1222780 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.r.ManagedResourceStorage No data found for znode 
/configs/conf1/_rest_managed.json
   [junit4]   2> 1222780 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using 
ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1222780 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1222780 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1222781 INFO  
(searcherExecutor-2959-thread-1-processing-n:127.0.0.1:48830_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:48830_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore 
[collection1] Registered new searcher Searcher@33f06aa[collection1] 
main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1222781 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1222781 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version 
buckets from index
   [junit4]   2> 1222781 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket 
highest value from index
   [junit4]   2> 1222781 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.UpdateLog Could not find max version in index or recent updates, using 
new clock 1539595843342958592
   [junit4]   2> 1222783 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.u.UpdateLog Took 1.0ms to seed version buckets with highest version 
1539595843342958592
   [junit4]   2> 1222783 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1222783 INFO  
(coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) 
[n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1222784 INFO  
(coreZkRegister-2952-thread-1-processing-n:127.0.0.1:48830_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:48830_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController 
Register replica - core:collection1 address:https://127.0.0.1:48830 
collection:control_collection shard:shard1
   [junit4]   2> 1222784 INFO  
(coreZkRegister-2952-thread-1-processing-n:127.0.0.1:48830_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:48830_ 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.c.SolrZkClient makePath: 
/collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1222787 INFO  
(coreZkRegister-2952-thread-1-processing-n:127.0.0.1:48830_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:48830_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector 
Joined leadership election with path: 
/collections/control_collection/leader_elect/shard1/election/96224740103684101-core_node1-n_0000000000
   [junit4]   2> 1222787 INFO  
(coreZkRegister-2952-thread-1-processing-n:127.0.0.1:48830_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:48830_ 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 
and weAreReplacement=false and leaderVoteWait=10000
   [junit4]   2> 1222788 INFO  
(coreZkRegister-2952-thread-1-processing-n:127.0.0.1:48830_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:48830_ 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1222789 INFO  
(coreZkRegister-2952-thread-1-processing-n:127.0.0.1:48830_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:48830_ 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1222789 INFO  
(OverseerStateUpdate-96224740103684101-127.0.0.1:48830_-n_0000000000) 
[n:127.0.0.1:48830_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message 
= {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1222789 INFO  
(coreZkRegister-2952-thread-1-processing-n:127.0.0.1:48830_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:48830_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy 
Sync replicas to https://127.0.0.1:48830/collection1/
   [junit4]   2> 1222789 INFO  
(coreZkRegister-2952-thread-1-processing-n:127.0.0.1:48830_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:48830_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy 
Sync Success - now sync replicas to me
   [junit4]   2> 1222789 INFO  
(coreZkRegister-2952-thread-1-processing-n:127.0.0.1:48830_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:48830_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy 
https://127.0.0.1:48830/collection1/ has no replicas
   [junit4]   2> 1222790 INFO  
(coreZkRegister-2952-thread-1-processing-n:127.0.0.1:48830_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:48830_ 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1222792 INFO  
(coreZkRegister-2952-thread-1-processing-n:127.0.0.1:48830_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:48830_ 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node 
/collections/control_collection/leaders/shard1/leader after winning as 
/collections/control_collection/leader_elect/shard1/election/96224740103684101-core_node1-n_0000000000
   [junit4]   2> 1222793 INFO  
(coreZkRegister-2952-thread-1-processing-n:127.0.0.1:48830_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:48830_ 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.ShardLeaderElectionContext I am the new leader: 
https://127.0.0.1:48830/collection1/ shard1
   [junit4]   2> 1222794 INFO  
(OverseerStateUpdate-96224740103684101-127.0.0.1:48830_-n_0000000000) 
[n:127.0.0.1:48830_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message 
= {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"https://127.0.0.1:48830";,
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 1222895 INFO  
(zkCallback-1358-thread-2-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_    
] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred 
- updating... (live nodes size: [1])
   [junit4]   2> 1222945 INFO  
(coreZkRegister-2952-thread-1-processing-n:127.0.0.1:48830_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:48830_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController 
We are https://127.0.0.1:48830/collection1/ and leader is 
https://127.0.0.1:48830/collection1/
   [junit4]   2> 1222945 INFO  
(coreZkRegister-2952-thread-1-processing-n:127.0.0.1:48830_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:48830_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController 
No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:48830
   [junit4]   2> 1222945 INFO  
(coreZkRegister-2952-thread-1-processing-n:127.0.0.1:48830_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:48830_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController 
I am the leader, no recovery necessary
   [junit4]   2> 1222945 INFO  
(coreZkRegister-2952-thread-1-processing-n:127.0.0.1:48830_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:48830_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController 
publishing state=active
   [junit4]   2> 1222945 INFO  
(coreZkRegister-2952-thread-1-processing-n:127.0.0.1:48830_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:48830_ 
c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController 
numShards not found on descriptor - reading it from system property
   [junit4]   2> 1222946 INFO  
(coreZkRegister-2952-thread-1-processing-n:127.0.0.1:48830_ x:collection1 
s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:48830_ 
c:control_collection s:shard1 r:core_node1 x:collection1] 
o.a.s.c.c.ZkStateReader Checking legacy cluster state for collection 
control_collection
   [junit4]   2> 1222946 INFO  
(OverseerStateUpdate-96224740103684101-127.0.0.1:48830_-n_0000000000) 
[n:127.0.0.1:48830_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message 
= {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   
"dataDir":"hdfs://localhost:36530/solr_hdfs_home/control_collection/core_node1/data/",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:48830";,
   [junit4]   2>   "node_name":"127.0.0.1:48830_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   
"ulogDir":"hdfs://localhost:36530/solr_hdfs_home/control_collection/core_node1/data/tlog"}
 current state version: 2
   [junit4]   2> 1222946 INFO  
(OverseerStateUpdate-96224740103684101-127.0.0.1:48830_-n_0000000000) 
[n:127.0.0.1:48830_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 
message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   
"dataDir":"hdfs://localhost:36530/solr_hdfs_home/control_collection/core_node1/data/",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:48830";,
   [junit4]   2>   "node_name":"127.0.0.1:48830_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   
"ulogDir":"hdfs://localhost:36530/solr_hdfs_home/control_collection/core_node1/data/tlog"}
   [junit4]   2> 1223048 INFO  
(zkCallback-1358-thread-2-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_    
] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred 
- updating... (live nodes size: [1])
   [junit4]   2> 1223074 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1223075 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1223076 INFO  (zkCallback-1362-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@6d924483 
name:ZooKeeperConnection Watcher:127.0.0.1:49574/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1223076 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1223076 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1223076 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1223077 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.ZkStateReader Loaded cluster properties: {urlScheme=https}
   [junit4]   2> 1223077 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1223078 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection 
loss:false
   [junit4]   2> 1223171 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.SolrTestCaseJ4 Writing core.properties file to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001/shard-1-001/cores/collection1
   [junit4]   2> 1223172 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001/shard-1-001
   [junit4]   2> 1223175 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 1223177 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@6a59a88d{/,null,AVAILABLE}
   [junit4]   2> 1223178 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.e.j.s.ServerConnector Started ServerConnector@7a65bfe8{SSL,[ssl, 
http/1.1]}{127.0.0.1:51914}
   [junit4]   2> 1223178 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.e.j.s.Server Started @1228020ms
   [junit4]   2> 1223178 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: 
{solr.data.dir=hdfs://localhost:36530/hdfs__localhost_36530__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-6.x_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001_tempDir-002_jetty1,
 solrconfig=solrconfig.xml, hostContext=/, hostPort=51914, 
coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001/shard-1-001/cores}
   [junit4]   2> 1223179 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): 
sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 1223179 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: 
'/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001/shard-1-001'
   [junit4]   2> 1223179 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1223179 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find 
system property or JNDI)
   [junit4]   2> 1223179 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1223181 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1223184 INFO  (zkCallback-1363-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@4cedc334 
name:ZooKeeperConnection Watcher:127.0.0.1:49574/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1223184 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1223185 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1223185 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in 
ZooKeeper)
   [junit4]   2> 1223185 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.SolrXmlConfig Loading container configuration from 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001/shard-1-001/solr.xml
   [junit4]   2> 1223190 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.CorePropertiesLocator Config-defined core root directory: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001/shard-1-001/cores
   [junit4]   2> 1223190 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.CoreContainer New CoreContainer 1427659436
   [junit4]   2> 1223190 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.CoreContainer Loading cores into CoreContainer 
[instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001/shard-1-001]
   [junit4]   2> 1223191 WARN  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.CoreContainer Couldn't add files from 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001/shard-1-001/lib
 to classpath: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001/shard-1-001/lib
   [junit4]   2> 1223191 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme 
: ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 
10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 
5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1223192 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: 
socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1223192 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1223192 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.l.LogWatcher Registering Log Listener [Log4j 
(org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1223192 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:49574/solr
   [junit4]   2> 1223192 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1223193 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1223193 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1223194 INFO  (zkCallback-1366-thread-1) [    ] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@7598ead5 
name:ZooKeeperConnection Watcher:127.0.0.1:49574 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1223194 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1223194 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [    ] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1223195 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:51914_    ] o.a.s.c.ZkController Added new OnReconnect listener 
org.apache.solr.cloud.ZkController$$Lambda$47/319444693@59538ca0
   [junit4]   2> 1223196 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:51914_    ] o.a.s.c.c.ConnectionManager Waiting for client to 
connect to ZooKeeper
   [junit4]   2> 1223197 INFO  
(zkCallback-1367-thread-1-processing-n:127.0.0.1:51914_) [n:127.0.0.1:51914_    
] o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@63e478e6 
name:ZooKeeperConnection Watcher:127.0.0.1:49574/solr got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1223197 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:51914_    ] o.a.s.c.c.ConnectionManager Client is connected to 
ZooKeeper
   [junit4]   2> 1223201 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:51914_    ] o.a.s.c.c.ZkStateReader Updating cluster state from 
ZooKeeper... 
   [junit4]   2> 1223202 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:51914_    ] o.a.s.c.c.ZkStateReader Loaded cluster properties: 
{urlScheme=https}
   [junit4]   2> 1223202 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:51914_    ] o.a.s.c.c.ZkStateReader Updated live nodes from 
ZooKeeper... (0) -> (1)
   [junit4]   2> 1223204 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:51914_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1223204 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:51914_    ] o.a.s.c.LeaderElector Joined leadership election with 
path: /overseer_elect/election/96224740103684105-127.0.0.1:51914_-n_0000000001
   [junit4]   2> 1223205 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:51914_    ] o.a.s.c.LeaderElector Watching path 
/overseer_elect/election/96224740103684101-127.0.0.1:48830_-n_0000000000 to 
know if I could be the leader
   [junit4]   2> 1223205 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:51914_    ] o.a.s.c.ZkController Publish node=127.0.0.1:51914_ as 
DOWN
   [junit4]   2> 1223206 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:51914_    ] o.a.s.c.ZkController Register node as live in 
ZooKeeper:/live_nodes/127.0.0.1:51914_
   [junit4]   2> 1223206 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:51914_    ] o.a.s.c.c.SolrZkClient makePath: 
/live_nodes/127.0.0.1:51914_
   [junit4]   2> 1223207 INFO  (zkCallback-1362-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected 
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live 
nodes size: [1])
   [junit4]   2> 1223207 INFO  
(zkCallback-1358-thread-2-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_    
] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected 
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live 
nodes size: [1])
   [junit4]   2> 1223207 INFO  
(zkCallback-1367-thread-1-processing-n:127.0.0.1:51914_) [n:127.0.0.1:51914_    
] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected 
type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live 
nodes size: [1])
   [junit4]   2> 1223207 INFO  
(OverseerStateUpdate-96224740103684101-127.0.0.1:48830_-n_0000000000) 
[n:127.0.0.1:48830_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message 
= {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:51914_"} current state version: 3
   [junit4]   2> 1223207 INFO  
(OverseerStateUpdate-96224740103684101-127.0.0.1:48830_-n_0000000000) 
[n:127.0.0.1:48830_    ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 
127.0.0.1:51914_
   [junit4]   2> 1223208 INFO  (zkCallback-1362-thread-1) [    ] 
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1223209 INFO  
(zkCallback-1358-thread-2-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_    
] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1223215 INFO  
(zkCallback-1367-thread-1-processing-n:127.0.0.1:51914_) [n:127.0.0.1:51914_    
] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1223216 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:51914_    ] o.a.s.c.CoreContainer Security conf doesn't exist. 
Skipping setup for authorization module.
   [junit4]   2> 1223216 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:51914_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1223216 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:51914_    ] o.a.s.c.CorePropertiesLocator Looking for core 
definitions underneath 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001/shard-1-001/cores
   [junit4]   2> 1223217 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:51914_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: 
{name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, 
loadOnStartup=true, configSetProperties=configsetprops.json, 
collection=collection1, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 1223217 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:51914_    ] o.a.s.c.CorePropertiesLocator Found core collection1 
in 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001/shard-1-001/cores/collection1
   [junit4]   2> 1223217 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:51914_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1223218 INFO  
(coreLoadExecutor-2969-thread-1-processing-n:127.0.0.1:51914_) 
[n:127.0.0.1:51914_ c:collection1   x:collection1] o.a.s.c.ZkController 
publishing state=down
   [junit4]   2> 1223218 INFO  
(coreLoadExecutor-2969-thread-1-processing-n:127.0.0.1:51914_) 
[n:127.0.0.1:51914_ c:collection1   x:collection1] o.a.s.c.ZkController 
numShards not found on descriptor - reading it from system property
   [junit4]   2> 1223219 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:51914_    ] o.a.s.s.SolrDispatchFilter 
user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1
   [junit4]   2> 1223219 INFO  
(TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) 
[n:127.0.0.1:51914_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() 
done
   [junit4]   2> 1223219 INFO  
(coreLoadExecutor-2969-thread-1-processing-n:127.0.0.1:51914_) 
[n:127.0.0.1:51914_ c:collection1   x:collection1] o.a.s.c.ZkController 
Collection collection1 not visible yet, but flagging it so a watch is 
registered when it becomes visible
   [junit4]   2> 1223220 INFO  
(coreLoadExecutor-2969-thread-1-processing-n:127.0.0.1:51914_) 
[n:127.0.0.1:51914_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader 
Deleting data for [collection1]
   [junit4]   2> 1223220 INFO  
(OverseerStateUpdate-96224740103684101-127.0.0.1:48830_-n_0000000000) 
[n:127.0.0.1:48830_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message 
= {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:51914";,
   [junit4]   2>   "node_name":"127.0.0.1:51914_",
   [junit4]   2>   "numShards":"4",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "operation":"state"} current state version: 3
   [junit4]   2> 1223220 INFO  
(OverseerStateUpdate-96224740103684101-127.0.0.1:48830_-n_0000000000) 
[n:127.0.0.1:48830_    ] o.a.s.c.o.ReplicaMutator Update state numShards=4 
message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:51914";,
   [junit4]   2>   "node_name":"127.0.0.1:51914_",
   [junit4]   2>   "numShards":"4",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1223220 INFO  
(OverseerStateUpdate-96224740103684101-127.0.0.1:48830_-n_0000000000) 
[n:127.0.0.1:48830_    ] o.a.s.c.o.ClusterStateMutator building a new cName: 
collection1
   [junit4]   2> 1223220 INFO  
(OverseerStateUpdate-96224740103684101-127.0.0.1:48830_-n_0000000000) 
[n:127.0.0.1:48830_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard 
shard=shard2
   [junit4]   2> 1223221 INFO  
(coreLoadExecutor-2969-thread-1-processing-n:127.0.0.1:51914_) 
[n:127.0.0.1:51914_ c:collection1   x:collection1] o.a.s.c.ZkController look 
for our core node name
   [junit4]   2> 1223322 INFO  (zkCallback-1362-thread-1) [    ] 
o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred 
- updating... (live nodes size: [2])
   [junit4]   2> 1223322 INFO  
(zkCallback-1358-thread-2-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_    
] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred 
- updating... (live nodes size: [2])
   [junit4]   2> 1223322 INFO  
(zkCallback-1367-thread-1-processing-n:127.0.0.1:51914_) [n:127.0.0.1:51914_    
] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent 
state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred 
- updating... (live nodes size: [2])
   [junit4]   2> 1224221 INFO  (coreLoadExecutor-2969-thread-

[...truncated too long message...]

htlyTests-6.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-common-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-hdfs-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-minikdc-2.7.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/htrace-core-3.2.0-incubating.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-core-asl-1.9.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-mapper-asl-1.9.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mockito-core-1.9.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/netty-all-4.0.36.Final.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/presto-parser-0.122.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/slice-0.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/analysis/icu/lucene-analyzers-icu-6.2.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/queryparser/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/contrib/analysis-extras/lib/icu4j-56.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.3.4.jar
 com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J0-20160711_211625_763.events
 
@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J0-20160711_211625_763.suites
 -stdin
   [junit4] ERROR: JVM J0 ended with an exception: Quit event not received from 
the forked process? This may indicate JVM crash or runner bugs.
   [junit4]     at 
com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1501)
   [junit4]     at 
com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:121)
   [junit4]     at 
com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:955)
   [junit4]     at 
com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:952)
   [junit4]     at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]     at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]     at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]     at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:747: 
The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:684: 
The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:59: 
The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build.xml:233:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/common-build.xml:530:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:1443:
 The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:984:
 At least one slave process threw an exception, first: Quit event not received 
from the forked process? This may indicate JVM crash or runner bugs.

Total time: 546 minutes 18 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
No prior successful build to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Reply via email to