Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/117/
1 tests failed. FAILED: junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest Error Message: ObjectTracker found 2 object(s) that were not released!!! [HdfsTransactionLog, HdfsTransactionLog] Stack Trace: java.lang.AssertionError: ObjectTracker found 2 object(s) that were not released!!! [HdfsTransactionLog, HdfsTransactionLog] at __randomizedtesting.SeedInfo.seed([88A15EA81E630D4B]:0) at org.junit.Assert.fail(Assert.java:93) at org.junit.Assert.assertTrue(Assert.java:43) at org.junit.Assert.assertNull(Assert.java:551) at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:258) at sun.reflect.GeneratedMethodAccessor59.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367) at java.lang.Thread.run(Thread.java:745) Build Log: [...truncated 11666 lines...] [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest [junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001/init-core-data-001 [junit4] 2> 1219445 INFO (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[88A15EA81E630D4B]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN) [junit4] 2> 1219445 INFO (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[88A15EA81E630D4B]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: / [junit4] 1> Formatting using clusterid: testClusterID [junit4] 2> 1219482 WARN (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[88A15EA81E630D4B]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties [junit4] 2> 1219492 WARN (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[88A15EA81E630D4B]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j [junit4] 2> 1219494 INFO (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[88A15EA81E630D4B]-worker) [ ] o.m.log jetty-6.1.26 [junit4] 2> 1219505 INFO (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[88A15EA81E630D4B]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_33211_hdfs____y9hezg/webapp [junit4] 2> 1219868 INFO (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[88A15EA81E630D4B]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33211 [junit4] 2> 1219940 WARN (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[88A15EA81E630D4B]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j [junit4] 2> 1219941 INFO (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[88A15EA81E630D4B]-worker) [ ] o.m.log jetty-6.1.26 [junit4] 2> 1219954 INFO (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[88A15EA81E630D4B]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_33600_datanode____.r1gfk3/webapp [junit4] 2> 1220317 INFO (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[88A15EA81E630D4B]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33600 [junit4] 2> 1220407 WARN (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[88A15EA81E630D4B]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j [junit4] 2> 1220408 INFO (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[88A15EA81E630D4B]-worker) [ ] o.m.log jetty-6.1.26 [junit4] 2> 1220427 INFO (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[88A15EA81E630D4B]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_35143_datanode____.3hv2vf/webapp [junit4] 2> 1220484 INFO (IPC Server handler 3 on 36530) [ ] BlockStateChange BLOCK* processReport: from storage DS-8d2db89d-66ff-4879-ae8c-007c303993f8 node DatanodeRegistration(127.0.0.1:49909, datanodeUuid=3912d198-a7b8-4049-a0aa-11c949a5c3b9, infoPort=40536, infoSecurePort=0, ipcPort=35703, storageInfo=lv=-56;cid=testClusterID;nsid=2044110516;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs [junit4] 2> 1220484 INFO (IPC Server handler 3 on 36530) [ ] BlockStateChange BLOCK* processReport: from storage DS-beeed90d-2972-4d08-9e93-f0dd794142f5 node DatanodeRegistration(127.0.0.1:49909, datanodeUuid=3912d198-a7b8-4049-a0aa-11c949a5c3b9, infoPort=40536, infoSecurePort=0, ipcPort=35703, storageInfo=lv=-56;cid=testClusterID;nsid=2044110516;c=0), blocks: 0, hasStaleStorage: false, processing time: 1 msecs [junit4] 2> 1220882 INFO (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[88A15EA81E630D4B]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:35143 [junit4] 2> 1221025 INFO (IPC Server handler 3 on 36530) [ ] BlockStateChange BLOCK* processReport: from storage DS-6fd1d7fc-8455-415e-9e87-614f38f1b4da node DatanodeRegistration(127.0.0.1:40002, datanodeUuid=4342edd8-7180-44f7-a734-2734af85847b, infoPort=40941, infoSecurePort=0, ipcPort=59249, storageInfo=lv=-56;cid=testClusterID;nsid=2044110516;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs [junit4] 2> 1221025 INFO (IPC Server handler 3 on 36530) [ ] BlockStateChange BLOCK* processReport: from storage DS-e740d183-55a8-4adc-893d-b021faaeb4bb node DatanodeRegistration(127.0.0.1:40002, datanodeUuid=4342edd8-7180-44f7-a734-2734af85847b, infoPort=40941, infoSecurePort=0, ipcPort=59249, storageInfo=lv=-56;cid=testClusterID;nsid=2044110516;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs [junit4] 2> 1221173 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER [junit4] 2> 1221173 INFO (Thread-14985) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0 [junit4] 2> 1221173 INFO (Thread-14985) [ ] o.a.s.c.ZkTestServer Starting server [junit4] 2> 1221273 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.ZkTestServer start zk server on port:49574 [junit4] 2> 1221274 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider [junit4] 2> 1221274 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1221277 INFO (zkCallback-1350-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6e863bae name:ZooKeeperConnection Watcher:127.0.0.1:49574 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1221277 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1221277 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider [junit4] 2> 1221277 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.SolrZkClient makePath: /solr [junit4] 2> 1221280 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider [junit4] 2> 1221280 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1221281 INFO (zkCallback-1351-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@55b3dd0 name:ZooKeeperConnection Watcher:127.0.0.1:49574/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1221281 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1221281 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider [junit4] 2> 1221281 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1 [junit4] 2> 1221282 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards [junit4] 2> 1221283 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection [junit4] 2> 1221284 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards [junit4] 2> 1221286 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml [junit4] 2> 1221286 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml [junit4] 2> 1221293 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml [junit4] 2> 1221293 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml [junit4] 2> 1221295 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml [junit4] 2> 1221295 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml [junit4] 2> 1221296 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt [junit4] 2> 1221296 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt [junit4] 2> 1221297 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt [junit4] 2> 1221297 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt [junit4] 2> 1221298 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml [junit4] 2> 1221298 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml [junit4] 2> 1221299 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml [junit4] 2> 1221300 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml [junit4] 2> 1221301 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json [junit4] 2> 1221301 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json [junit4] 2> 1221302 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt [junit4] 2> 1221302 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt [junit4] 2> 1221303 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt [junit4] 2> 1221303 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt [junit4] 2> 1221305 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt [junit4] 2> 1221305 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt [junit4] 2> 1221307 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider [junit4] 2> 1221307 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1221308 INFO (zkCallback-1353-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@773e0e51 name:ZooKeeperConnection Watcher:127.0.0.1:49574/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1221308 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1221308 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider [junit4] 2> 1221397 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001/control-001/cores/collection1 [junit4] 2> 1221399 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314 [junit4] 2> 1221401 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@12e04d99{/,null,AVAILABLE} [junit4] 2> 1221402 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.e.j.s.ServerConnector Started ServerConnector@2d781c93{SSL,[ssl, http/1.1]}{127.0.0.1:48830} [junit4] 2> 1221402 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.e.j.s.Server Started @1226244ms [junit4] 2> 1221402 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:36530/hdfs__localhost_36530__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-6.x_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001_tempDir-002_control_data, hostContext=/, hostPort=48830, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001/control-001/cores} [junit4] 2> 1221403 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93 [junit4] 2> 1221403 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001/control-001' [junit4] 2> 1221403 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 1221403 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI) [junit4] 2> 1221403 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider [junit4] 2> 1221404 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1221405 INFO (zkCallback-1354-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@401bd564 name:ZooKeeperConnection Watcher:127.0.0.1:49574/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1221406 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1221406 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider [junit4] 2> 1221407 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper) [junit4] 2> 1221407 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001/control-001/solr.xml [junit4] 2> 1221412 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001/control-001/cores [junit4] 2> 1221412 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.CoreContainer New CoreContainer 204203848 [junit4] 2> 1221412 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001/control-001] [junit4] 2> 1221412 WARN (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001/control-001/lib [junit4] 2> 1221412 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false, [junit4] 2> 1221413 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true [junit4] 2> 1221414 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 1221414 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 1221414 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:49574/solr [junit4] 2> 1221414 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.ZkController zkHost includes chroot [junit4] 2> 1221414 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider [junit4] 2> 1221415 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1221416 INFO (zkCallback-1357-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7c9121b6 name:ZooKeeperConnection Watcher:127.0.0.1:49574 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1221416 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1221416 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider [junit4] 2> 1221417 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:48830_ ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$47/319444693@d49f11c [junit4] 2> 1221418 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:48830_ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1221418 INFO (zkCallback-1358-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@18d4db16 name:ZooKeeperConnection Watcher:127.0.0.1:49574/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1221418 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:48830_ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1221419 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:48830_ ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue [junit4] 2> 1221420 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:48830_ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work [junit4] 2> 1221422 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:48830_ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running [junit4] 2> 1221423 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:48830_ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed [junit4] 2> 1221424 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:48830_ ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure [junit4] 2> 1221425 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:48830_ ] o.a.s.c.c.SolrZkClient makePath: /live_nodes [junit4] 2> 1221426 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:48830_ ] o.a.s.c.c.SolrZkClient makePath: /aliases.json [junit4] 2> 1221427 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:48830_ ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json [junit4] 2> 1221428 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:48830_ ] o.a.s.c.c.SolrZkClient makePath: /security.json [junit4] 2> 1221436 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:48830_ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... [junit4] 2> 1221437 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:48830_ ] o.a.s.c.c.ZkStateReader Loaded cluster properties: {urlScheme=https} [junit4] 2> 1221437 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:48830_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0) [junit4] 2> 1221457 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:48830_ ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect [junit4] 2> 1221458 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:48830_ ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election [junit4] 2> 1221459 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:48830_ ] o.a.s.c.Overseer Overseer (id=null) closing [junit4] 2> 1221459 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:48830_ ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96224740103684101-127.0.0.1:48830_-n_0000000000 [junit4] 2> 1221460 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:48830_ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:48830_ [junit4] 2> 1221460 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:48830_ ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader [junit4] 2> 1221461 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:48830_ ] o.a.s.c.Overseer Overseer (id=96224740103684101-127.0.0.1:48830_-n_0000000000) starting [junit4] 2> 1221463 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:48830_ ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work [junit4] 2> 1221474 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:48830_ ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000 [junit4] 2> 1221479 INFO (OverseerStateUpdate-96224740103684101-127.0.0.1:48830_-n_0000000000) [n:127.0.0.1:48830_ ] o.a.s.c.Overseer Starting to work on the main queue [junit4] 2> 1221501 INFO (OverseerCollectionConfigSetProcessor-96224740103684101-127.0.0.1:48830_-n_0000000000) [n:127.0.0.1:48830_ ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations [junit4] 2> 1221505 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:48830_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48830_ [junit4] 2> 1221505 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:48830_ ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:48830_ [junit4] 2> 1221505 INFO (OverseerStateUpdate-96224740103684101-127.0.0.1:48830_-n_0000000000) [n:127.0.0.1:48830_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0) [junit4] 2> 1221520 INFO (zkCallback-1358-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0]) [junit4] 2> 1221532 INFO (zkCallback-1358-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 1221561 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:48830_ ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module. [junit4] 2> 1221561 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:48830_ ] o.a.s.c.CoreContainer No authentication plugin used. [junit4] 2> 1221561 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:48830_ ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001/control-001/cores [junit4] 2> 1221562 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:48830_ ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=control_collection, coreNodeName=, dataDir=data/, shard=} [junit4] 2> 1221562 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:48830_ ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001/control-001/cores/collection1 [junit4] 2> 1221562 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:48830_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions [junit4] 2> 1221569 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection x:collection1] o.a.s.c.ZkController publishing state=down [junit4] 2> 1221569 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property [junit4] 2> 1221570 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection x:collection1] o.a.s.c.ZkController Collection control_collection not visible yet, but flagging it so a watch is registered when it becomes visible [junit4] 2> 1221571 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection x:collection1] o.a.s.c.c.ZkStateReader Deleting data for [control_collection] [junit4] 2> 1221573 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:48830_ ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1 [junit4] 2> 1221573 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:48830_ ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done [junit4] 2> 1221580 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection x:collection1] o.a.s.c.ZkController look for our core node name [junit4] 2> 1221581 INFO (OverseerStateUpdate-96224740103684101-127.0.0.1:48830_-n_0000000000) [n:127.0.0.1:48830_ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = { [junit4] 2> "core":"collection1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"https://127.0.0.1:48830", [junit4] 2> "node_name":"127.0.0.1:48830_", [junit4] 2> "numShards":"1", [junit4] 2> "state":"down", [junit4] 2> "shard":null, [junit4] 2> "collection":"control_collection", [junit4] 2> "operation":"state"} current state version: 0 [junit4] 2> 1221581 INFO (OverseerStateUpdate-96224740103684101-127.0.0.1:48830_-n_0000000000) [n:127.0.0.1:48830_ ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={ [junit4] 2> "core":"collection1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"https://127.0.0.1:48830", [junit4] 2> "node_name":"127.0.0.1:48830_", [junit4] 2> "numShards":"1", [junit4] 2> "state":"down", [junit4] 2> "shard":null, [junit4] 2> "collection":"control_collection", [junit4] 2> "operation":"state"} [junit4] 2> 1221581 INFO (OverseerStateUpdate-96224740103684101-127.0.0.1:48830_-n_0000000000) [n:127.0.0.1:48830_ ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection [junit4] 2> 1221582 INFO (OverseerStateUpdate-96224740103684101-127.0.0.1:48830_-n_0000000000) [n:127.0.0.1:48830_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1 [junit4] 2> 1221583 INFO (zkCallback-1358-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1]) [junit4] 2> 1222584 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1 [junit4] 2> 1222584 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection [junit4] 2> 1222585 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection x:collection1] o.a.s.c.ZkController Collection zkNode exists [junit4] 2> 1222585 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection] [junit4] 2> 1222585 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper [junit4] 2> 1222585 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001/control-001/cores/collection1' [junit4] 2> 1222586 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 1222586 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI) [junit4] 2> 1222587 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 [junit4] 2> 1222591 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1 [junit4] 2> 1222595 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.2.0 [junit4] 2> 1222603 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml [junit4] 2> 1222607 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test [junit4] 2> 1222667 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection x:collection1] o.a.s.s.IndexSchema default search field in schema is text [junit4] 2> 1222667 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection x:collection1] o.a.s.s.IndexSchema unique key field: id [junit4] 2> 1222668 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml [junit4] 2> 1222670 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml [junit4] 2> 1222691 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1 [junit4] 2> 1222691 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection [junit4] 2> 1222691 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory [junit4] 2> 1222691 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:36530/solr_hdfs_home [junit4] 2> 1222691 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled [junit4] 2> 1222691 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001/control-001/cores/collection1], dataDir=[hdfs://localhost:36530/solr_hdfs_home/control_collection/core_node1/data/] [junit4] 2> 1222691 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@14aded07 [junit4] 2> 1222692 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:36530/solr_hdfs_home/control_collection/core_node1/data [junit4] 2> 1222704 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:36530/solr_hdfs_home/control_collection/core_node1/data [junit4] 2> 1222704 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:36530/solr_hdfs_home/control_collection/core_node1/data/index/ [junit4] 2> 1222711 WARN (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:36530/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index... [junit4] 2> 1222711 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:36530/solr_hdfs_home/control_collection/core_node1/data/index [junit4] 2> 1222718 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true] [junit4] 2> 1222718 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes [junit4] 2> 1222722 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled [junit4] 2> 1222722 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:36530/solr_hdfs_home/control_collection/core_node1/data/index [junit4] 2> 1222722 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=16, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6165534828591587] [junit4] 2> 1222744 INFO (IPC Server handler 3 on 36530) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:40002 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-beeed90d-2972-4d08-9e93-f0dd794142f5:NORMAL:127.0.0.1:49909|RBW], ReplicaUC[[DISK]DS-e740d183-55a8-4adc-893d-b021faaeb4bb:NORMAL:127.0.0.1:40002|FINALIZED]]} size 0 [junit4] 2> 1222745 INFO (IPC Server handler 4 on 36530) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:49909 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-e740d183-55a8-4adc-893d-b021faaeb4bb:NORMAL:127.0.0.1:40002|FINALIZED], ReplicaUC[[DISK]DS-8d2db89d-66ff-4879-ae8c-007c303993f8:NORMAL:127.0.0.1:49909|FINALIZED]]} size 0 [junit4] 2> 1222748 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1 [junit4] 2> commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@hdfs://localhost:36530/solr_hdfs_home/control_collection/core_node1/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@68f4dd0a); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1} [junit4] 2> 1222748 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1 [junit4] 2> 1222752 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib" [junit4] 2> 1222752 INFO (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:36530/solr_hdfs_home/control_collection/core_node1/data/ [junit4] 2> 1222752 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe" [junit4] 2> 1222752 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe" [junit4] 2> 1222752 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig" [junit4] 2> 1222752 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig" [junit4] 2> 1222753 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit" [junit4] 2> 1222753 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit" [junit4] 2> 1222753 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit" [junit4] 2> 1222753 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default [junit4] 2> 1222753 WARN (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}} [junit4] 2> 1222755 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60 [junit4] 2> 1222755 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60 [junit4] 2> 1222756 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60 [junit4] 2> 1222757 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60 [junit4] 2> 1222760 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,/update/json/docs,/admin/luke,/export,/get,/admin/properties,/update/json,/admin/threads,/dump,/update/csv,/sql,/graph,/admin/segments,/admin/system,/replication,/config,/stream,/schema,/admin/plugins,/admin/logging,/admin/ping,/update,/admin/file,/terms [junit4] 2> 1222760 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache [junit4] 2> 1222760 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog [junit4] 2> 1222760 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536 [junit4] 2> 1222760 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2 [junit4] 2> 1222772 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: if uncommited for 15000ms; [junit4] 2> 1222772 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled [junit4] 2> 1222773 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=14, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0] [junit4] 2> 1222778 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1 [junit4] 2> commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@hdfs://localhost:36530/solr_hdfs_home/control_collection/core_node1/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@68f4dd0a); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1} [junit4] 2> 1222778 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1 [junit4] 2> 1222778 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@33f06aa[collection1] main] [junit4] 2> 1222778 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection] [junit4] 2> 1222779 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper [junit4] 2> 1222779 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1 [junit4] 2> 1222779 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1 [junit4] 2> 1222779 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {} [junit4] 2> 1222779 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 [junit4] 2> 1222780 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json [junit4] 2> 1222780 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 [junit4] 2> 1222780 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources [junit4] 2> 1222780 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 [junit4] 2> 1222781 INFO (searcherExecutor-2959-thread-1-processing-n:127.0.0.1:48830_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@33f06aa[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())} [junit4] 2> 1222781 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets [junit4] 2> 1222781 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index [junit4] 2> 1222781 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index [junit4] 2> 1222781 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1539595843342958592 [junit4] 2> 1222783 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 1.0ms to seed version buckets with highest version 1539595843342958592 [junit4] 2> 1222783 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1 [junit4] 2> 1222783 INFO (coreLoadExecutor-2958-thread-1-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1 [junit4] 2> 1222784 INFO (coreZkRegister-2952-thread-1-processing-n:127.0.0.1:48830_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:https://127.0.0.1:48830 collection:control_collection shard:shard1 [junit4] 2> 1222784 INFO (coreZkRegister-2952-thread-1-processing-n:127.0.0.1:48830_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election [junit4] 2> 1222787 INFO (coreZkRegister-2952-thread-1-processing-n:127.0.0.1:48830_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/96224740103684101-core_node1-n_0000000000 [junit4] 2> 1222787 INFO (coreZkRegister-2952-thread-1-processing-n:127.0.0.1:48830_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=10000 [junit4] 2> 1222788 INFO (coreZkRegister-2952-thread-1-processing-n:127.0.0.1:48830_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue. [junit4] 2> 1222789 INFO (coreZkRegister-2952-thread-1-processing-n:127.0.0.1:48830_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync [junit4] 2> 1222789 INFO (OverseerStateUpdate-96224740103684101-127.0.0.1:48830_-n_0000000000) [n:127.0.0.1:48830_ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = { [junit4] 2> "operation":"leader", [junit4] 2> "shard":"shard1", [junit4] 2> "collection":"control_collection"} current state version: 1 [junit4] 2> 1222789 INFO (coreZkRegister-2952-thread-1-processing-n:127.0.0.1:48830_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:48830/collection1/ [junit4] 2> 1222789 INFO (coreZkRegister-2952-thread-1-processing-n:127.0.0.1:48830_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me [junit4] 2> 1222789 INFO (coreZkRegister-2952-thread-1-processing-n:127.0.0.1:48830_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy https://127.0.0.1:48830/collection1/ has no replicas [junit4] 2> 1222790 INFO (coreZkRegister-2952-thread-1-processing-n:127.0.0.1:48830_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1 [junit4] 2> 1222792 INFO (coreZkRegister-2952-thread-1-processing-n:127.0.0.1:48830_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/control_collection/leaders/shard1/leader after winning as /collections/control_collection/leader_elect/shard1/election/96224740103684101-core_node1-n_0000000000 [junit4] 2> 1222793 INFO (coreZkRegister-2952-thread-1-processing-n:127.0.0.1:48830_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:48830/collection1/ shard1 [junit4] 2> 1222794 INFO (OverseerStateUpdate-96224740103684101-127.0.0.1:48830_-n_0000000000) [n:127.0.0.1:48830_ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = { [junit4] 2> "operation":"leader", [junit4] 2> "shard":"shard1", [junit4] 2> "collection":"control_collection", [junit4] 2> "base_url":"https://127.0.0.1:48830", [junit4] 2> "core":"collection1", [junit4] 2> "state":"active"} current state version: 1 [junit4] 2> 1222895 INFO (zkCallback-1358-thread-2-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1]) [junit4] 2> 1222945 INFO (coreZkRegister-2952-thread-1-processing-n:127.0.0.1:48830_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are https://127.0.0.1:48830/collection1/ and leader is https://127.0.0.1:48830/collection1/ [junit4] 2> 1222945 INFO (coreZkRegister-2952-thread-1-processing-n:127.0.0.1:48830_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:48830 [junit4] 2> 1222945 INFO (coreZkRegister-2952-thread-1-processing-n:127.0.0.1:48830_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary [junit4] 2> 1222945 INFO (coreZkRegister-2952-thread-1-processing-n:127.0.0.1:48830_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active [junit4] 2> 1222945 INFO (coreZkRegister-2952-thread-1-processing-n:127.0.0.1:48830_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property [junit4] 2> 1222946 INFO (coreZkRegister-2952-thread-1-processing-n:127.0.0.1:48830_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:48830_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Checking legacy cluster state for collection control_collection [junit4] 2> 1222946 INFO (OverseerStateUpdate-96224740103684101-127.0.0.1:48830_-n_0000000000) [n:127.0.0.1:48830_ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = { [junit4] 2> "core":"collection1", [junit4] 2> "core_node_name":"core_node1", [junit4] 2> "dataDir":"hdfs://localhost:36530/solr_hdfs_home/control_collection/core_node1/data/", [junit4] 2> "roles":null, [junit4] 2> "base_url":"https://127.0.0.1:48830", [junit4] 2> "node_name":"127.0.0.1:48830_", [junit4] 2> "numShards":"1", [junit4] 2> "state":"active", [junit4] 2> "shard":"shard1", [junit4] 2> "collection":"control_collection", [junit4] 2> "operation":"state", [junit4] 2> "ulogDir":"hdfs://localhost:36530/solr_hdfs_home/control_collection/core_node1/data/tlog"} current state version: 2 [junit4] 2> 1222946 INFO (OverseerStateUpdate-96224740103684101-127.0.0.1:48830_-n_0000000000) [n:127.0.0.1:48830_ ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={ [junit4] 2> "core":"collection1", [junit4] 2> "core_node_name":"core_node1", [junit4] 2> "dataDir":"hdfs://localhost:36530/solr_hdfs_home/control_collection/core_node1/data/", [junit4] 2> "roles":null, [junit4] 2> "base_url":"https://127.0.0.1:48830", [junit4] 2> "node_name":"127.0.0.1:48830_", [junit4] 2> "numShards":"1", [junit4] 2> "state":"active", [junit4] 2> "shard":"shard1", [junit4] 2> "collection":"control_collection", [junit4] 2> "operation":"state", [junit4] 2> "ulogDir":"hdfs://localhost:36530/solr_hdfs_home/control_collection/core_node1/data/tlog"} [junit4] 2> 1223048 INFO (zkCallback-1358-thread-2-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1]) [junit4] 2> 1223074 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider [junit4] 2> 1223075 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1223076 INFO (zkCallback-1362-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6d924483 name:ZooKeeperConnection Watcher:127.0.0.1:49574/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1223076 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1223076 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider [junit4] 2> 1223076 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... [junit4] 2> 1223077 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.ZkStateReader Loaded cluster properties: {urlScheme=https} [junit4] 2> 1223077 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 1223078 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false [junit4] 2> 1223171 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001/shard-1-001/cores/collection1 [junit4] 2> 1223172 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001/shard-1-001 [junit4] 2> 1223175 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314 [junit4] 2> 1223177 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6a59a88d{/,null,AVAILABLE} [junit4] 2> 1223178 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.e.j.s.ServerConnector Started ServerConnector@7a65bfe8{SSL,[ssl, http/1.1]}{127.0.0.1:51914} [junit4] 2> 1223178 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.e.j.s.Server Started @1228020ms [junit4] 2> 1223178 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:36530/hdfs__localhost_36530__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-6.x_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001_tempDir-002_jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=51914, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001/shard-1-001/cores} [junit4] 2> 1223179 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93 [junit4] 2> 1223179 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001/shard-1-001' [junit4] 2> 1223179 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 1223179 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI) [junit4] 2> 1223179 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider [junit4] 2> 1223181 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1223184 INFO (zkCallback-1363-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4cedc334 name:ZooKeeperConnection Watcher:127.0.0.1:49574/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1223184 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1223185 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider [junit4] 2> 1223185 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper) [junit4] 2> 1223185 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001/shard-1-001/solr.xml [junit4] 2> 1223190 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001/shard-1-001/cores [junit4] 2> 1223190 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.CoreContainer New CoreContainer 1427659436 [junit4] 2> 1223190 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001/shard-1-001] [junit4] 2> 1223191 WARN (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001/shard-1-001/lib [junit4] 2> 1223191 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false, [junit4] 2> 1223192 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true [junit4] 2> 1223192 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 1223192 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 1223192 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:49574/solr [junit4] 2> 1223192 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.ZkController zkHost includes chroot [junit4] 2> 1223193 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider [junit4] 2> 1223193 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1223194 INFO (zkCallback-1366-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7598ead5 name:ZooKeeperConnection Watcher:127.0.0.1:49574 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1223194 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1223194 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider [junit4] 2> 1223195 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:51914_ ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$47/319444693@59538ca0 [junit4] 2> 1223196 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:51914_ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper [junit4] 2> 1223197 INFO (zkCallback-1367-thread-1-processing-n:127.0.0.1:51914_) [n:127.0.0.1:51914_ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@63e478e6 name:ZooKeeperConnection Watcher:127.0.0.1:49574/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1223197 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:51914_ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper [junit4] 2> 1223201 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:51914_ ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... [junit4] 2> 1223202 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:51914_ ] o.a.s.c.c.ZkStateReader Loaded cluster properties: {urlScheme=https} [junit4] 2> 1223202 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:51914_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1) [junit4] 2> 1223204 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:51914_ ] o.a.s.c.Overseer Overseer (id=null) closing [junit4] 2> 1223204 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:51914_ ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96224740103684105-127.0.0.1:51914_-n_0000000001 [junit4] 2> 1223205 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:51914_ ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/96224740103684101-127.0.0.1:48830_-n_0000000000 to know if I could be the leader [junit4] 2> 1223205 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:51914_ ] o.a.s.c.ZkController Publish node=127.0.0.1:51914_ as DOWN [junit4] 2> 1223206 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:51914_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51914_ [junit4] 2> 1223206 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:51914_ ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:51914_ [junit4] 2> 1223207 INFO (zkCallback-1362-thread-1) [ ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1]) [junit4] 2> 1223207 INFO (zkCallback-1358-thread-2-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1]) [junit4] 2> 1223207 INFO (zkCallback-1367-thread-1-processing-n:127.0.0.1:51914_) [n:127.0.0.1:51914_ ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1]) [junit4] 2> 1223207 INFO (OverseerStateUpdate-96224740103684101-127.0.0.1:48830_-n_0000000000) [n:127.0.0.1:48830_ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = { [junit4] 2> "operation":"downnode", [junit4] 2> "node_name":"127.0.0.1:51914_"} current state version: 3 [junit4] 2> 1223207 INFO (OverseerStateUpdate-96224740103684101-127.0.0.1:48830_-n_0000000000) [n:127.0.0.1:48830_ ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:51914_ [junit4] 2> 1223208 INFO (zkCallback-1362-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2) [junit4] 2> 1223209 INFO (zkCallback-1358-thread-2-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2) [junit4] 2> 1223215 INFO (zkCallback-1367-thread-1-processing-n:127.0.0.1:51914_) [n:127.0.0.1:51914_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2) [junit4] 2> 1223216 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:51914_ ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module. [junit4] 2> 1223216 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:51914_ ] o.a.s.c.CoreContainer No authentication plugin used. [junit4] 2> 1223216 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:51914_ ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001/shard-1-001/cores [junit4] 2> 1223217 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:51914_ ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=collection1, coreNodeName=, dataDir=data/, shard=} [junit4] 2> 1223217 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:51914_ ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_88A15EA81E630D4B-001/shard-1-001/cores/collection1 [junit4] 2> 1223217 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:51914_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions [junit4] 2> 1223218 INFO (coreLoadExecutor-2969-thread-1-processing-n:127.0.0.1:51914_) [n:127.0.0.1:51914_ c:collection1 x:collection1] o.a.s.c.ZkController publishing state=down [junit4] 2> 1223218 INFO (coreLoadExecutor-2969-thread-1-processing-n:127.0.0.1:51914_) [n:127.0.0.1:51914_ c:collection1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property [junit4] 2> 1223219 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:51914_ ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1 [junit4] 2> 1223219 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[88A15EA81E630D4B]) [n:127.0.0.1:51914_ ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done [junit4] 2> 1223219 INFO (coreLoadExecutor-2969-thread-1-processing-n:127.0.0.1:51914_) [n:127.0.0.1:51914_ c:collection1 x:collection1] o.a.s.c.ZkController Collection collection1 not visible yet, but flagging it so a watch is registered when it becomes visible [junit4] 2> 1223220 INFO (coreLoadExecutor-2969-thread-1-processing-n:127.0.0.1:51914_) [n:127.0.0.1:51914_ c:collection1 x:collection1] o.a.s.c.c.ZkStateReader Deleting data for [collection1] [junit4] 2> 1223220 INFO (OverseerStateUpdate-96224740103684101-127.0.0.1:48830_-n_0000000000) [n:127.0.0.1:48830_ ] o.a.s.c.Overseer processMessage: queueSize: 1, message = { [junit4] 2> "core":"collection1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"https://127.0.0.1:51914", [junit4] 2> "node_name":"127.0.0.1:51914_", [junit4] 2> "numShards":"4", [junit4] 2> "state":"down", [junit4] 2> "shard":null, [junit4] 2> "collection":"collection1", [junit4] 2> "operation":"state"} current state version: 3 [junit4] 2> 1223220 INFO (OverseerStateUpdate-96224740103684101-127.0.0.1:48830_-n_0000000000) [n:127.0.0.1:48830_ ] o.a.s.c.o.ReplicaMutator Update state numShards=4 message={ [junit4] 2> "core":"collection1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"https://127.0.0.1:51914", [junit4] 2> "node_name":"127.0.0.1:51914_", [junit4] 2> "numShards":"4", [junit4] 2> "state":"down", [junit4] 2> "shard":null, [junit4] 2> "collection":"collection1", [junit4] 2> "operation":"state"} [junit4] 2> 1223220 INFO (OverseerStateUpdate-96224740103684101-127.0.0.1:48830_-n_0000000000) [n:127.0.0.1:48830_ ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1 [junit4] 2> 1223220 INFO (OverseerStateUpdate-96224740103684101-127.0.0.1:48830_-n_0000000000) [n:127.0.0.1:48830_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2 [junit4] 2> 1223221 INFO (coreLoadExecutor-2969-thread-1-processing-n:127.0.0.1:51914_) [n:127.0.0.1:51914_ c:collection1 x:collection1] o.a.s.c.ZkController look for our core node name [junit4] 2> 1223322 INFO (zkCallback-1362-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [2]) [junit4] 2> 1223322 INFO (zkCallback-1358-thread-2-processing-n:127.0.0.1:48830_) [n:127.0.0.1:48830_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [2]) [junit4] 2> 1223322 INFO (zkCallback-1367-thread-1-processing-n:127.0.0.1:51914_) [n:127.0.0.1:51914_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [2]) [junit4] 2> 1224221 INFO (coreLoadExecutor-2969-thread- [...truncated too long message...] htlyTests-6.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-common-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-hdfs-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-minikdc-2.7.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/htrace-core-3.2.0-incubating.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-core-asl-1.9.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-mapper-asl-1.9.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mockito-core-1.9.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/netty-all-4.0.36.Final.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/presto-parser-0.122.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/slice-0.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/analysis/icu/lucene-analyzers-icu-6.2.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/queryparser/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/contrib/analysis-extras/lib/icu4j-56.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.3.4.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J0-20160711_211625_763.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J0-20160711_211625_763.suites -stdin [junit4] ERROR: JVM J0 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs. [junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1501) [junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:121) [junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:955) [junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:952) [junit4] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [junit4] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [junit4] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [junit4] at java.lang.Thread.run(Thread.java:745) BUILD FAILED /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:747: The following error occurred while executing this line: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:684: The following error occurred while executing this line: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:59: The following error occurred while executing this line: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build.xml:233: The following error occurred while executing this line: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/common-build.xml:530: The following error occurred while executing this line: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:1443: The following error occurred while executing this line: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:984: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs. Total time: 546 minutes 18 seconds Build step 'Invoke Ant' marked build as failure Archiving artifacts No prior successful build to compare, so performing full copy of artifacts Recording test results Email was triggered for: Failure - Any Sending email for trigger: Failure - Any
--------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org