Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/1869/ Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseSerialGC
1 tests failed. REGRESSION: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.testDistribSearch Error Message: expected:<0> but was:<1> Stack Trace: java.lang.AssertionError: expected:<0> but was:<1> at __randomizedtesting.SeedInfo.seed([958A190CF6D69FCA:146C97148189FFF6]:0) at org.junit.Assert.fail(Assert.java:93) at org.junit.Assert.failNotEquals(Assert.java:647) at org.junit.Assert.assertEquals(Assert.java:128) at org.junit.Assert.assertEquals(Assert.java:472) at org.junit.Assert.assertEquals(Assert.java:456) at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.doTest(ChaosMonkeySafeLeaderTest.java:153) at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869) at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at java.lang.Thread.run(Thread.java:745) Build Log: [...truncated 11568 lines...] [junit4] Suite: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest [junit4] 2> Creating dataDir: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/init-core-data-001 [junit4] 2> 3877040 T9906 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (false) [junit4] 2> 3877040 T9906 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: / [junit4] 2> 3877048 T9906 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch [junit4] 2> 3877050 T9906 oasc.ZkTestServer.run STARTING ZK TEST SERVER [junit4] 1> client port:0.0.0.0/0.0.0.0:0 [junit4] 2> 3877052 T9907 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server [junit4] 2> 3877152 T9906 oasc.ZkTestServer.run start zk server on port:52251 [junit4] 2> 3877153 T9906 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 3877155 T9906 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 3877164 T9913 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1d42c8b1 name:ZooKeeperConnection Watcher:127.0.0.1:52251 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 3877165 T9906 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 3877165 T9906 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 3877165 T9906 oascc.SolrZkClient.makePath makePath: /solr [junit4] 2> 3877177 T9906 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 3877179 T9906 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 3877182 T9915 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5766258c name:ZooKeeperConnection Watcher:127.0.0.1:52251/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 3877185 T9906 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 3877185 T9906 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 3877185 T9906 oascc.SolrZkClient.makePath makePath: /collections/collection1 [junit4] 2> 3877193 T9906 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards [junit4] 2> 3877201 T9906 oascc.SolrZkClient.makePath makePath: /collections/control_collection [junit4] 2> 3877209 T9906 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards [junit4] 2> 3877218 T9906 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml [junit4] 2> 3877219 T9906 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml [junit4] 2> 3877233 T9906 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml [junit4] 2> 3877234 T9906 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml [junit4] 2> 3877242 T9906 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml [junit4] 2> 3877243 T9906 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml [junit4] 2> 3877251 T9906 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt [junit4] 2> 3877252 T9906 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt [junit4] 2> 3877262 T9906 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt [junit4] 2> 3877263 T9906 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt [junit4] 2> 3877272 T9906 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml [junit4] 2> 3877273 T9906 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml [junit4] 2> 3877282 T9906 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml [junit4] 2> 3877283 T9906 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml [junit4] 2> 3877291 T9906 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json [junit4] 2> 3877293 T9906 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json [junit4] 2> 3877301 T9906 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt [junit4] 2> 3877302 T9906 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt [junit4] 2> 3877309 T9906 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt [junit4] 2> 3877310 T9906 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt [junit4] 2> 3877317 T9906 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt [junit4] 2> 3877318 T9906 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt [junit4] 2> 3877325 T9906 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 3877327 T9906 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 3877331 T9917 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@225c6c08 name:ZooKeeperConnection Watcher:127.0.0.1:52251/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 3877331 T9906 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 3877332 T9906 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 3879357 T9906 oejs.Server.doStart jetty-8.1.10.v20130312 [junit4] 2> 3879369 T9906 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] [junit4] 2> 3879377 T9906 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:52255 [junit4] 2> 3879379 T9906 oass.SolrDispatchFilter.init SolrDispatchFilter.init() [junit4] 2> 3879381 T9906 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 3879381 T9906 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-002 [junit4] 2> 3879381 T9906 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-002/' [junit4] 2> 3879427 T9906 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-002/solr.xml [junit4] 2> 3879445 T9906 oasc.CoreContainer.<init> New CoreContainer 76270373 [junit4] 2> 3879446 T9906 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-002/] [junit4] 2> 3879446 T9906 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000 [junit4] 2> 3879447 T9906 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: [junit4] 2> 3879447 T9906 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000 [junit4] 2> 3879447 T9906 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20 [junit4] 2> 3879447 T9906 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0 [junit4] 2> 3879448 T9906 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647 [junit4] 2> 3879449 T9906 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5 [junit4] 2> 3879449 T9906 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1 [junit4] 2> 3879449 T9906 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false [junit4] 2> 3879451 T9906 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false [junit4] 2> 3879454 T9906 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 3879454 T9906 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 3879455 T9906 oasc.CoreContainer.load Host Name: 127.0.0.1 [junit4] 2> 3879455 T9906 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:52251/solr [junit4] 2> 3879456 T9906 oasc.ZkController.checkChrootPath zkHost includes chroot [junit4] 2> 3879456 T9906 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 3879458 T9906 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 3879461 T9928 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6bc17b47 name:ZooKeeperConnection Watcher:127.0.0.1:52251 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 3879462 T9906 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 3879463 T9906 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 3879468 T9906 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 3879472 T9930 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@22d22078 name:ZooKeeperConnection Watcher:127.0.0.1:52251/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 3879473 T9906 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 3879479 T9906 oascc.SolrZkClient.makePath makePath: /overseer/queue [junit4] 2> 3879487 T9906 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work [junit4] 2> 3879499 T9906 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running [junit4] 2> 3879508 T9906 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed [junit4] 2> 3879518 T9906 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure [junit4] 2> 3879530 T9906 oascc.SolrZkClient.makePath makePath: /live_nodes [junit4] 2> 3879535 T9906 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52255_ [junit4] 2> 3879538 T9906 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:52255_ [junit4] 2> 3879548 T9906 oascc.SolrZkClient.makePath makePath: /overseer_elect [junit4] 2> 3879554 T9906 oascc.SolrZkClient.makePath makePath: /overseer_elect/election [junit4] 2> 3879560 T9906 oasc.Overseer.close Overseer (id=null) closing [junit4] 2> 3879564 T9906 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:52255_ [junit4] 2> 3879564 T9906 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader [junit4] 2> 3879570 T9906 oasc.Overseer.start Overseer (id=92572385875525636-127.0.0.1:52255_-n_0000000000) starting [junit4] 2> 3879575 T9906 oascc.SolrZkClient.makePath makePath: /overseer/queue-work [junit4] 2> 3879600 T9906 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000 [junit4] 2> 3879602 T9932 oasc.OverseerCollectionProcessor.run Process current queue of collection creations [junit4] 2> 3879606 T9906 oascc.SolrZkClient.makePath makePath: /clusterstate.json [junit4] 2> 3879613 T9906 oascc.SolrZkClient.makePath makePath: /aliases.json [junit4] 2> 3879617 T9906 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 3879629 T9931 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue [junit4] 2> 3879638 T9934 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection [junit4] 2> 3879638 T9934 oasc.ZkController.publish numShards not found on descriptor - reading it from system property [junit4] 2> 3879640 T9934 oasc.ZkController.waitForCoreNodeName look for our core node name [junit4] 2> 3879640 T9935 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 3879649 T9931 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={ [junit4] 2> "core":"collection1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"https://127.0.0.1:52255", [junit4] 2> "node_name":"127.0.0.1:52255_", [junit4] 2> "numShards":"1", [junit4] 2> "state":"down", [junit4] 2> "shard":null, [junit4] 2> "collection":"control_collection", [junit4] 2> "operation":"state"} [junit4] 2> 3879649 T9931 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1] [junit4] 2> 3879650 T9931 oasc.Overseer$ClusterStateUpdater.createCollection state version control_collection 1 [junit4] 2> 3879650 T9931 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1 [junit4] 2> 3879658 T9935 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1) [junit4] 2> 3880642 T9934 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1 [junit4] 2> 3880642 T9934 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection [junit4] 2> 3880645 T9934 oasc.ZkController.createCollectionZkNode Collection zkNode exists [junit4] 2> 3880645 T9934 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection [junit4] 2> 3880649 T9934 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper [junit4] 2> 3880650 T9934 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-002/collection1/' [junit4] 2> 3880651 T9934 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-002/collection1/lib/.svn/' to classloader [junit4] 2> 3880651 T9934 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-002/collection1/lib/classes/' to classloader [junit4] 2> 3880652 T9934 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-002/collection1/lib/README' to classloader [junit4] 2> 3880704 T9934 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0 [junit4] 2> 3880745 T9934 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml [junit4] 2> 3880749 T9934 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml [junit4] 2> 3880756 T9934 oass.IndexSchema.readSchema [collection1] Schema name=test [junit4] 2> 3880914 T9934 oass.IndexSchema.readSchema default search field in schema is text [junit4] 2> 3880915 T9934 oass.IndexSchema.readSchema unique key field: id [junit4] 2> 3880916 T9934 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 3880922 T9934 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 3880994 T9934 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection [junit4] 2> 3880994 T9934 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory [junit4] 2> 3880995 T9934 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-002/collection1/, dataDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-001/control/data/ [junit4] 2> 3880995 T9934 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2e5d23ff [junit4] 2> 3880999 T9934 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-001/control/data [junit4] 2> 3880999 T9934 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-001/control/data/index/ [junit4] 2> 3881000 T9934 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-001/control/data/index' doesn't exist. Creating new index... [junit4] 2> 3881002 T9934 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-001/control/data/index [junit4] 2> 3881003 T9934 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=15, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8457713178206444] [junit4] 2> 3881009 T9934 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1 [junit4] 2> commit{dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-001/control/data/index,segFN=segments_1,generation=1} [junit4] 2> 3881009 T9934 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1 [junit4] 2> 3881012 T9934 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib" [junit4] 2> 3881013 T9934 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe" [junit4] 2> 3881014 T9934 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe" [junit4] 2> 3881015 T9934 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig" [junit4] 2> 3881015 T9934 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig" [junit4] 2> 3881015 T9934 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit" [junit4] 2> 3881016 T9934 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit" [junit4] 2> 3881016 T9934 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit" [junit4] 2> 3881016 T9934 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default [junit4] 2> 3881017 T9934 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 3881017 T9934 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 3881018 T9934 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 3881020 T9934 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler [junit4] 2> 3881020 T9934 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler [junit4] 2> 3881021 T9934 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler [junit4] 2> 3881021 T9934 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler [junit4] 2> 3881021 T9934 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler [junit4] 2> 3881022 T9934 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers [junit4] 2> 3881022 T9934 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler [junit4] 2> 3881022 T9934 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler [junit4] 2> 3881034 T9934 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 3881036 T9934 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 3881038 T9934 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 3881041 T9934 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 3881043 T9934 oasu.CommitTracker.<init> Hard AutoCommit: disabled [junit4] 2> 3881043 T9934 oasu.CommitTracker.<init> Soft AutoCommit: disabled [junit4] 2> 3881044 T9934 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=25, maxMergeAtOnceExplicit=28, maxMergedSegmentMB=99.0126953125, floorSegmentMB=0.8857421875, forceMergeDeletesPctAllowed=15.487768845341378, segmentsPerTier=11.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5159467199713256 [junit4] 2> 3881045 T9934 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1 [junit4] 2> commit{dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-001/control/data/index,segFN=segments_1,generation=1} [junit4] 2> 3881045 T9934 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1 [junit4] 2> 3881046 T9934 oass.SolrIndexSearcher.<init> Opening Searcher@732bc5e3[collection1] main [junit4] 2> 3881046 T9934 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection [junit4] 2> 3881049 T9934 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper [junit4] 2> 3881049 T9934 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1 [junit4] 2> 3881051 T9934 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1 [junit4] 2> 3881052 T9934 oasr.RestManager.init Initializing RestManager with initArgs: {} [junit4] 2> 3881052 T9934 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 [junit4] 2> 3881054 T9934 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json [junit4] 2> 3881055 T9934 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 [junit4] 2> 3881056 T9934 oascc.SolrZkClient.makePath makePath: /configs/conf1/_rest_managed.json [junit4] 2> 3881064 T9934 oasr.ManagedResourceStorage$ZooKeeperStorageIO$1.close Wrote 38 bytes to new znode /configs/conf1/_rest_managed.json [junit4] 2> 3881064 T9934 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 [junit4] 2> 3881065 T9934 oasr.RestManager.init Initializing 0 registered ManagedResources [junit4] 2> 3881069 T9936 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@732bc5e3[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())} [junit4] 2> 3881071 T9934 oasc.CoreContainer.registerCore registering core: collection1 [junit4] 2> 3881073 T9939 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:52255 collection:control_collection shard:shard1 [junit4] 2> 3881074 T9906 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0 [junit4] 2> 3881075 T9906 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done [junit4] 2> 3881082 T9906 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 3881085 T9939 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election [junit4] 2> 3881086 T9906 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 3881093 T9941 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6fc0eb8 name:ZooKeeperConnection Watcher:127.0.0.1:52251/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 3881093 T9906 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 3881094 T9906 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 3881096 T9906 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 3881110 T9906 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false [junit4] 2> 3881116 T9939 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1 [junit4] 2> 3881123 T9935 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 3881126 T9939 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue. [junit4] 2> 3881126 T9939 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync [junit4] 2> ASYNC NEW_CORE C4477 name=collection1 org.apache.solr.core.SolrCore@4ffa5058 url=https://127.0.0.1:52255/collection1 node=127.0.0.1:52255_ C4477_STATE=coll:control_collection core:collection1 props:{core=collection1, base_url=https://127.0.0.1:52255, node_name=127.0.0.1:52255_, state=down} [junit4] 2> 3881127 T9939 C4477 P52255 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:52255/collection1/ [junit4] 2> 3881127 T9939 C4477 P52255 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me [junit4] 2> 3881128 T9939 C4477 P52255 oasc.SyncStrategy.syncToMe https://127.0.0.1:52255/collection1/ has no replicas [junit4] 2> 3881129 T9939 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:52255/collection1/ shard1 [junit4] 2> 3881129 T9939 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1 [junit4] 2> 3881161 T9942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1) [junit4] 2> 3881168 T9935 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1) [junit4] 2> 3881211 T9935 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 3881322 T9942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1) [junit4] 2> 3881323 T9935 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1) [junit4] 2> 3881377 T9939 oasc.ZkController.register We are https://127.0.0.1:52255/collection1/ and leader is https://127.0.0.1:52255/collection1/ [junit4] 2> 3881377 T9939 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:52255 [junit4] 2> 3881378 T9939 oasc.ZkController.checkRecovery I am the leader, no recovery necessary [junit4] 2> 3881379 T9939 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection [junit4] 2> 3881380 T9939 oasc.ZkController.publish numShards not found on descriptor - reading it from system property [junit4] 2> 3881385 T9935 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 3881404 T9931 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={ [junit4] 2> "core":"collection1", [junit4] 2> "core_node_name":"core_node1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"https://127.0.0.1:52255", [junit4] 2> "node_name":"127.0.0.1:52255_", [junit4] 2> "numShards":"3", [junit4] 2> "state":"active", [junit4] 2> "shard":"shard1", [junit4] 2> "collection":"control_collection", [junit4] 2> "operation":"state"} [junit4] 2> 3881519 T9935 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1) [junit4] 2> 3881519 T9942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1) [junit4] 2> 3883333 T9906 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1 [junit4] 2> 3883335 T9906 oejs.Server.doStart jetty-8.1.10.v20130312 [junit4] 2> 3883345 T9906 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] [junit4] 2> 3883348 T9906 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:52259 [junit4] 2> 3883355 T9906 oass.SolrDispatchFilter.init SolrDispatchFilter.init() [junit4] 2> 3883355 T9906 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 3883356 T9906 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-003 [junit4] 2> 3883357 T9906 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-003/' [junit4] 2> 3883426 T9906 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-003/solr.xml [junit4] 2> 3883471 T9906 oasc.CoreContainer.<init> New CoreContainer 72931794 [junit4] 2> 3883472 T9906 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-003/] [junit4] 2> 3883472 T9906 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000 [junit4] 2> 3883473 T9906 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: [junit4] 2> 3883473 T9906 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000 [junit4] 2> 3883474 T9906 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20 [junit4] 2> 3883474 T9906 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0 [junit4] 2> 3883474 T9906 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647 [junit4] 2> 3883475 T9906 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5 [junit4] 2> 3883475 T9906 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1 [junit4] 2> 3883475 T9906 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false [junit4] 2> 3883478 T9906 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false [junit4] 2> 3883481 T9906 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 3883481 T9906 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 3883481 T9906 oasc.CoreContainer.load Host Name: 127.0.0.1 [junit4] 2> 3883482 T9906 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:52251/solr [junit4] 2> 3883482 T9906 oasc.ZkController.checkChrootPath zkHost includes chroot [junit4] 2> 3883483 T9906 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 3883485 T9906 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 3883490 T9953 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@279ba633 name:ZooKeeperConnection Watcher:127.0.0.1:52251 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 3883490 T9906 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 3883492 T9906 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 3883496 T9906 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 3883501 T9955 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@28add7cd name:ZooKeeperConnection Watcher:127.0.0.1:52251/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 3883502 T9906 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 3883534 T9906 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 3884552 T9906 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52259_ [junit4] 2> 3884555 T9906 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:52259_ [junit4] 2> 3884571 T9906 oasc.Overseer.close Overseer (id=null) closing [junit4] 2> 3884585 T9957 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1 [junit4] 2> 3884586 T9957 oasc.ZkController.publish numShards not found on descriptor - reading it from system property [junit4] 2> 3884589 T9935 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 3884589 T9957 oasc.ZkController.waitForCoreNodeName look for our core node name [junit4] 2> 3884601 T9931 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={ [junit4] 2> "core":"collection1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"https://127.0.0.1:52259", [junit4] 2> "node_name":"127.0.0.1:52259_", [junit4] 2> "numShards":"3", [junit4] 2> "state":"down", [junit4] 2> "shard":null, [junit4] 2> "collection":"collection1", [junit4] 2> "operation":"state"} [junit4] 2> 3884601 T9931 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2, shard3] [junit4] 2> 3884601 T9931 oasc.Overseer$ClusterStateUpdater.createCollection state version collection1 1 [junit4] 2> 3884602 T9931 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2 [junit4] 2> 3884610 T9956 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2) [junit4] 2> 3884610 T9942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2) [junit4] 2> 3884611 T9935 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2) [junit4] 2> 3885594 T9957 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1 [junit4] 2> 3885595 T9957 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1 [junit4] 2> 3885597 T9957 oasc.ZkController.createCollectionZkNode Collection zkNode exists [junit4] 2> 3885597 T9957 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1 [junit4] 2> 3885601 T9957 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper [junit4] 2> 3885601 T9957 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-003/collection1/' [junit4] 2> 3885603 T9957 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-003/collection1/lib/.svn/' to classloader [junit4] 2> 3885603 T9957 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-003/collection1/lib/classes/' to classloader [junit4] 2> 3885604 T9957 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-003/collection1/lib/README' to classloader [junit4] 2> 3885659 T9957 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0 [junit4] 2> 3885675 T9957 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml [junit4] 2> 3885681 T9957 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml [junit4] 2> 3885688 T9957 oass.IndexSchema.readSchema [collection1] Schema name=test [junit4] 2> 3885858 T9957 oass.IndexSchema.readSchema default search field in schema is text [junit4] 2> 3885859 T9957 oass.IndexSchema.readSchema unique key field: id [junit4] 2> 3885860 T9957 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 3885865 T9957 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 3885938 T9957 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1 [junit4] 2> 3885939 T9957 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory [junit4] 2> 3885939 T9957 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-003/collection1/, dataDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-001/jetty1/ [junit4] 2> 3885939 T9957 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2e5d23ff [junit4] 2> 3885942 T9957 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-001/jetty1 [junit4] 2> 3885943 T9957 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-001/jetty1/index/ [junit4] 2> 3885944 T9957 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-001/jetty1/index' doesn't exist. Creating new index... [junit4] 2> 3885945 T9957 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-001/jetty1/index [junit4] 2> 3885945 T9957 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=15, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8457713178206444] [junit4] 2> 3885950 T9957 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1 [junit4] 2> commit{dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-001/jetty1/index,segFN=segments_1,generation=1} [junit4] 2> 3885951 T9957 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1 [junit4] 2> 3885957 T9957 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib" [junit4] 2> 3885957 T9957 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe" [junit4] 2> 3885957 T9957 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe" [junit4] 2> 3885957 T9957 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig" [junit4] 2> 3885958 T9957 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig" [junit4] 2> 3885958 T9957 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit" [junit4] 2> 3885959 T9957 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit" [junit4] 2> 3885959 T9957 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit" [junit4] 2> 3885959 T9957 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default [junit4] 2> 3885960 T9957 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 3885960 T9957 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 3885961 T9957 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 3885962 T9957 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler [junit4] 2> 3885963 T9957 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler [junit4] 2> 3885963 T9957 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler [junit4] 2> 3885963 T9957 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler [junit4] 2> 3885964 T9957 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler [junit4] 2> 3885964 T9957 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers [junit4] 2> 3885965 T9957 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler [junit4] 2> 3885965 T9957 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler [junit4] 2> 3885977 T9957 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 3885979 T9957 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 3885980 T9957 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 3885981 T9957 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 3885997 T9957 oasu.CommitTracker.<init> Hard AutoCommit: disabled [junit4] 2> 3885998 T9957 oasu.CommitTracker.<init> Soft AutoCommit: disabled [junit4] 2> 3885999 T9957 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=25, maxMergeAtOnceExplicit=28, maxMergedSegmentMB=99.0126953125, floorSegmentMB=0.8857421875, forceMergeDeletesPctAllowed=15.487768845341378, segmentsPerTier=11.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5159467199713256 [junit4] 2> 3886001 T9957 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1 [junit4] 2> commit{dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-001/jetty1/index,segFN=segments_1,generation=1} [junit4] 2> 3886001 T9957 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1 [junit4] 2> 3886001 T9957 oass.SolrIndexSearcher.<init> Opening Searcher@2ea97b3f[collection1] main [junit4] 2> 3886002 T9957 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1 [junit4] 2> 3886008 T9957 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper [junit4] 2> 3886010 T9957 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1 [junit4] 2> 3886011 T9957 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1 [junit4] 2> 3886011 T9957 oasr.RestManager.init Initializing RestManager with initArgs: {} [junit4] 2> 3886012 T9957 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 [junit4] 2> 3886016 T9957 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 38 bytes from znode /configs/conf1/_rest_managed.json [junit4] 2> 3886016 T9957 oasr.ManagedResourceStorage.load Loaded LinkedHashMap at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 [junit4] 2> 3886016 T9957 oasr.ManagedResource.reloadFromStorage Loaded initArgs {} for /rest/managed [junit4] 2> 3886017 T9957 oasr.RestManager.init Initializing 0 registered ManagedResources [junit4] 2> 3886027 T9958 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2ea97b3f[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())} [junit4] 2> 3886050 T9957 oasc.CoreContainer.registerCore registering core: collection1 [junit4] 2> 3886053 T9961 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:52259 collection:collection1 shard:shard2 [junit4] 2> 3886055 T9906 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0 [junit4] 2> 3886055 T9906 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done [junit4] 2> 3886056 T9961 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election [junit4] 2> 3886083 T9961 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2 [junit4] 2> 3886092 T9935 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 3886100 T9961 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue. [junit4] 2> 3886101 T9961 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync [junit4] 2> ASYNC NEW_CORE C4478 name=collection1 org.apache.solr.core.SolrCore@1578b5ef url=https://127.0.0.1:52259/collection1 node=127.0.0.1:52259_ C4478_STATE=coll:collection1 core:collection1 props:{core=collection1, base_url=https://127.0.0.1:52259, node_name=127.0.0.1:52259_, state=down} [junit4] 2> 3886101 T9961 C4478 P52259 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:52259/collection1/ [junit4] 2> 3886101 T9961 C4478 P52259 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me [junit4] 2> 3886102 T9961 C4478 P52259 oasc.SyncStrategy.syncToMe https://127.0.0.1:52259/collection1/ has no replicas [junit4] 2> 3886102 T9961 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:52259/collection1/ shard2 [junit4] 2> 3886102 T9961 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2 [junit4] 2> 3886148 T9942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2) [junit4] 2> 3886148 T9935 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2) [junit4] 2> 3886151 T9956 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2) [junit4] 2> 3886200 T9935 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 3886308 T9942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2) [junit4] 2> 3886308 T9935 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2) [junit4] 2> 3886311 T9956 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2) [junit4] 2> 3886366 T9961 oasc.ZkController.register We are https://127.0.0.1:52259/collection1/ and leader is https://127.0.0.1:52259/collection1/ [junit4] 2> 3886368 T9961 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:52259 [junit4] 2> 3886369 T9961 oasc.ZkController.checkRecovery I am the leader, no recovery necessary [junit4] 2> 3886369 T9961 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1 [junit4] 2> 3886369 T9961 oasc.ZkController.publish numShards not found on descriptor - reading it from system property [junit4] 2> 3886374 T9935 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 3886388 T9931 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={ [junit4] 2> "core":"collection1", [junit4] 2> "core_node_name":"core_node1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"https://127.0.0.1:52259", [junit4] 2> "node_name":"127.0.0.1:52259_", [junit4] 2> "numShards":"3", [junit4] 2> "state":"active", [junit4] 2> "shard":"shard2", [junit4] 2> "collection":"collection1", [junit4] 2> "operation":"state"} [junit4] 2> 3886499 T9956 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2) [junit4] 2> 3886500 T9942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2) [junit4] 2> 3886500 T9935 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2) [junit4] 2> 3888549 T9906 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2 [junit4] 2> 3888550 T9906 oejs.Server.doStart jetty-8.1.10.v20130312 [junit4] 2> 3888561 T9906 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] [junit4] 2> 3888565 T9906 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:52262 [junit4] 2> 3888570 T9906 oass.SolrDispatchFilter.init SolrDispatchFilter.init() [junit4] 2> 3888571 T9906 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 3888571 T9906 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-004 [junit4] 2> 3888572 T9906 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-004/' [junit4] 2> 3888615 T9906 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-004/solr.xml [junit4] 2> 3888635 T9906 oasc.CoreContainer.<init> New CoreContainer 981099462 [junit4] 2> 3888635 T9906 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-004/] [junit4] 2> 3888636 T9906 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000 [junit4] 2> 3888636 T9906 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: [junit4] 2> 3888637 T9906 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000 [junit4] 2> 3888637 T9906 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20 [junit4] 2> 3888638 T9906 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0 [junit4] 2> 3888638 T9906 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647 [junit4] 2> 3888639 T9906 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5 [junit4] 2> 3888639 T9906 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1 [junit4] 2> 3888639 T9906 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false [junit4] 2> 3888643 T9906 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false [junit4] 2> 3888644 T9906 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 3888645 T9906 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 3888645 T9906 oasc.CoreContainer.load Host Name: 127.0.0.1 [junit4] 2> 3888645 T9906 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:52251/solr [junit4] 2> 3888646 T9906 oasc.ZkController.checkChrootPath zkHost includes chroot [junit4] 2> 3888646 T9906 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 3888647 T9906 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 3888652 T9972 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@61b9e524 name:ZooKeeperConnection Watcher:127.0.0.1:52251 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 3888653 T9906 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 3888653 T9906 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 3888663 T9906 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 3888666 T9974 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1f3ceae7 name:ZooKeeperConnection Watcher:127.0.0.1:52251/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 3888666 T9906 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 3888688 T9906 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 3889703 T9906 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52262_ [junit4] 2> 3889706 T9906 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:52262_ [junit4] 2> 3889719 T9906 oasc.Overseer.close Overseer (id=null) closing [junit4] 2> 3889735 T9976 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1 [junit4] 2> 3889736 T9976 oasc.ZkController.publish numShards not found on descriptor - reading it from system property [junit4] 2> 3889738 T9935 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 3889739 T9976 oasc.ZkController.waitForCoreNodeName look for our core node name [junit4] 2> 3889751 T9931 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={ [junit4] 2> "core":"collection1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"https://127.0.0.1:52262", [junit4] 2> "node_name":"127.0.0.1:52262_", [junit4] 2> "numShards":"3", [junit4] 2> "state":"down", [junit4] 2> "shard":null, [junit4] 2> "collection":"collection1", [junit4] 2> "operation":"state"} [junit4] 2> 3889751 T9931 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3 [junit4] 2> 3889751 T9931 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard3 [junit4] 2> 3889761 T9956 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3) [junit4] 2> 3889761 T9975 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3) [junit4] 2> 3889762 T9942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3) [junit4] 2> 3889762 T9935 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3) [junit4] 2> 3890740 T9976 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1 [junit4] 2> 3890740 T9976 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1 [junit4] 2> 3890742 T9976 oasc.ZkController.createCollectionZkNode Collection zkNode exists [junit4] 2> 3890742 T9976 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1 [junit4] 2> 3890745 T9976 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper [junit4] 2> 3890745 T9976 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-004/collection1/' [junit4] 2> 3890746 T9976 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-004/collection1/lib/.svn/' to classloader [junit4] 2> 3890747 T9976 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-004/collection1/lib/classes/' to classloader [junit4] 2> 3890748 T9976 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-004/collection1/lib/README' to classloader [junit4] 2> 3890792 T9976 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0 [junit4] 2> 3890827 T9976 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml [junit4] 2> 3890833 T9976 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml [junit4] 2> 3890836 T9976 oass.IndexSchema.readSchema [collection1] Schema name=test [junit4] 2> 3890967 T9976 oass.IndexSchema.readSchema default search field in schema is text [junit4] 2> 3890971 T9976 oass.IndexSchema.readSchema unique key field: id [junit4] 2> 3890972 T9976 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 3890977 T9976 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 3891040 T9976 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1 [junit4] 2> 3891040 T9976 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory [junit4] 2> 3891041 T9976 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-004/collection1/, dataDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-001/jetty2/ [junit4] 2> 3891041 T9976 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2e5d23ff [junit4] 2> 3891042 T9976 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-001/jetty2 [junit4] 2> 3891043 T9976 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-001/jetty2/index/ [junit4] 2> 3891044 T9976 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-001/jetty2/index' doesn't exist. Creating new index... [junit4] 2> 3891044 T9976 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-001/jetty2/index [junit4] 2> 3891045 T9976 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=15, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8457713178206444] [junit4] 2> 3891049 T9976 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1 [junit4] 2> commit{dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-001/jetty2/index,segFN=segments_1,generation=1} [junit4] 2> 3891049 T9976 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1 [junit4] 2> 3891054 T9976 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib" [junit4] 2> 3891055 T9976 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe" [junit4] 2> 3891055 T9976 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe" [junit4] 2> 3891055 T9976 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig" [junit4] 2> 3891056 T9976 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig" [junit4] 2> 3891056 T9976 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit" [junit4] 2> 3891057 T9976 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit" [junit4] 2> 3891057 T9976 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit" [junit4] 2> 3891057 T9976 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default [junit4] 2> 3891058 T9976 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 3891058 T9976 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 3891059 T9976 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 3891059 T9976 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler [junit4] 2> 3891060 T9976 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler [junit4] 2> 3891060 T9976 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler [junit4] 2> 3891060 T9976 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler [junit4] 2> 3891061 T9976 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler [junit4] 2> 3891061 T9976 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers [junit4] 2> 3891062 T9976 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler [junit4] 2> 3891063 T9976 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler [junit4] 2> 3891074 T9976 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 3891076 T9976 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 3891078 T9976 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 3891079 T9976 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 3891082 T9976 oasu.CommitTracker.<init> Hard AutoCommit: disabled [junit4] 2> 3891082 T9976 oasu.CommitTracker.<init> Soft AutoCommit: disabled [junit4] 2> 3891084 T9976 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=25, maxMergeAtOnceExplicit=28, maxMergedSegmentMB=99.0126953125, floorSegmentMB=0.8857421875, forceMergeDeletesPctAllowed=15.487768845341378, segmentsPerTier=11.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5159467199713256 [junit4] 2> 3891085 T9976 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1 [junit4] 2> commit{dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-001/jetty2/index,segFN=segments_1,generation=1} [junit4] 2> 3891085 T9976 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1 [junit4] 2> 3891085 T9976 oass.SolrIndexSearcher.<init> Opening Searcher@4db12b91[collection1] main [junit4] 2> 3891086 T9976 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1 [junit4] 2> 3891089 T9976 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper [junit4] 2> 3891090 T9976 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1 [junit4] 2> 3891091 T9976 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1 [junit4] 2> 3891092 T9976 oasr.RestManager.init Initializing RestManager with initArgs: {} [junit4] 2> 3891092 T9976 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 [junit4] 2> 3891096 T9976 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 38 bytes from znode /configs/conf1/_rest_managed.json [junit4] 2> 3891096 T9976 oasr.ManagedResourceStorage.load Loaded LinkedHashMap at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 [junit4] 2> 3891097 T9976 oasr.ManagedResource.reloadFromStorage Loaded initArgs {} for /rest/managed [junit4] 2> 3891097 T9976 oasr.RestManager.init Initializing 0 registered ManagedResources [junit4] 2> 3891104 T9977 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4db12b91[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())} [junit4] 2> 3891107 T9976 oasc.CoreContainer.registerCore registering core: collection1 [junit4] 2> 3891109 T9980 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:52262 collection:collection1 shard:shard3 [junit4] 2> 3891110 T9906 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0 [junit4] 2> 3891111 T9906 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done [junit4] 2> 3891112 T9980 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard3/election [junit4] 2> 3891124 T9980 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard3 [junit4] 2> 3891127 T9935 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 3891133 T9980 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue. [junit4] 2> 3891133 T9980 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync [junit4] 2> ASYNC NEW_CORE C4479 name=collection1 org.apache.solr.core.SolrCore@75bc7264 url=https://127.0.0.1:52262/collection1 node=127.0.0.1:52262_ C4479_STATE=coll:collection1 core:collection1 props:{core=collection1, base_url=https://127.0.0.1:52262, node_name=127.0.0.1:52262_, state=down} [junit4] 2> 3891134 T9980 C4479 P52262 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:52262/collection1/ [junit4] 2> 3891134 T9980 C4479 P52262 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me [junit4] 2> 3891134 T9980 C4479 P52262 oasc.SyncStrategy.syncToMe https://127.0.0.1:52262/collection1/ has no replicas [junit4] 2> 3891135 T9980 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:52262/collection1/ shard3 [junit4] 2> 3891135 T9980 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard3 [junit4] 2> 3891150 T9935 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 3891273 T9956 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3) [junit4] 2> 3891274 T9942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3) [junit4] 2> 3891273 T9935 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3) [junit4] 2> 3891275 T9975 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3) [junit4] 2> 3891311 T9980 oasc.ZkController.register We are https://127.0.0.1:52262/collection1/ and leader is https://127.0.0.1:52262/collection1/ [junit4] 2> 3891311 T9980 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:52262 [junit4] 2> 3891312 T9980 oasc.ZkController.checkRecovery I am the leader, no recovery necessary [junit4] 2> 3891312 T9980 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1 [junit4] 2> 3891312 T9980 oasc.ZkController.publish numShards not found on descriptor - reading it from system property [junit4] 2> 3891316 T9935 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 3891326 T9931 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={ [junit4] 2> "core":"collection1", [junit4] 2> "core_node_name":"core_node2", [junit4] 2> "roles":null, [junit4] 2> "base_url":"https://127.0.0.1:52262", [junit4] 2> "node_name":"127.0.0.1:52262_", [junit4] 2> "numShards":"3", [junit4] 2> "state":"active", [junit4] 2> "shard":"shard3", [junit4] 2> "collection":"collection1", [junit4] 2> "operation":"state"} [junit4] 2> 3891438 T9942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3) [junit4] 2> 3891439 T9956 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3) [junit4] 2> 3891438 T9935 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3) [junit4] 2> 3891441 T9975 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3) [junit4] 2> 3893049 T9906 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3 [junit4] 2> 3893051 T9906 oejs.Server.doStart jetty-8.1.10.v20130312 [junit4] 2> 3893061 T9906 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] [junit4] 2> 3893063 T9906 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:52265 [junit4] 2> 3893066 T9906 oass.SolrDispatchFilter.init SolrDispatchFilter.init() [junit4] 2> 3893067 T9906 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 3893067 T9906 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-005 [junit4] 2> 3893067 T9906 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-005/' [junit4] 2> 3893111 T9906 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-005/solr.xml [junit4] 2> 3893146 T9906 oasc.CoreContainer.<init> New CoreContainer 1679949905 [junit4] 2> 3893146 T9906 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-005/] [junit4] 2> 3893147 T9906 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000 [junit4] 2> 3893147 T9906 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: [junit4] 2> 3893148 T9906 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000 [junit4] 2> 3893148 T9906 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20 [junit4] 2> 3893148 T9906 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0 [junit4] 2> 3893149 T9906 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647 [junit4] 2> 3893149 T9906 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5 [junit4] 2> 3893150 T9906 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1 [junit4] 2> 3893150 T9906 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false [junit4] 2> 3893153 T9906 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false [junit4] 2> 3893154 T9906 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 3893155 T9906 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 3893156 T9906 oasc.CoreContainer.load Host Name: 127.0.0.1 [junit4] 2> 3893156 T9906 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:52251/solr [junit4] 2> 3893157 T9906 oasc.ZkController.checkChrootPath zkHost includes chroot [junit4] 2> 3893157 T9906 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 3893159 T9906 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 3893163 T9991 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5066579f name:ZooKeeperConnection Watcher:127.0.0.1:52251 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 3893163 T9906 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 3893164 T9906 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 3893168 T9906 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 3893171 T9993 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3207e14a name:ZooKeeperConnection Watcher:127.0.0.1:52251/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 3893172 T9906 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 3893191 T9906 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 3894205 T9906 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52265_ [junit4] 2> 3894207 T9906 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:52265_ [junit4] 2> 3894219 T9906 oasc.Overseer.close Overseer (id=null) closing [junit4] 2> 3894233 T9995 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1 [junit4] 2> 3894233 T9995 oasc.ZkController.publish numShards not found on descriptor - reading it from system property [junit4] 2> 3894235 T9935 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 3894235 T9995 oasc.ZkController.waitForCoreNodeName look for our core node name [jun [...truncated too long message...] rkspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-001/jetty3/index;done=false>>] [junit4] 2> 3931429 T9906 oasc.CachingDirectoryFactory.close Closing directory: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001/tempDir-001/jetty3/index [junit4] 2> 3931431 T9906 oasc.Overseer.close Overseer (id=92572385875525643-127.0.0.1:52265_-n_0000000003) closing [junit4] 2> 3931431 T10047 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:52265_ [junit4] 2> 3931438 T10046 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK [junit4] 2> 3931453 T9906 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null} [junit4] 2> 3936444 T9906 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 52268 [junit4] 2> 3936445 T9906 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch [junit4] 2> 3936448 T9906 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:52251 52251 [junit4] 2> 3936459 T9907 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:52251 52251 [junit4] 2> 3936670 T9907 oasc.ZkTestServer$ZKServerMain.runFromConfig WARN Watch limit violations: [junit4] 2> Maximum concurrent create/delete watches above limit: [junit4] 2> [junit4] 2> 6 /solr/aliases.json [junit4] 2> 6 /solr/clusterstate.json [junit4] 2> [junit4] 2> Maximum concurrent children watches above limit: [junit4] 2> [junit4] 2> 6 /solr/live_nodes [junit4] 2> 4 /solr/overseer/queue [junit4] 2> 4 /solr/overseer/collection-queue-work [junit4] 2> [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=ChaosMonkeySafeLeaderTest -Dtests.method=testDistribSearch -Dtests.seed=958A190CF6D69FCA -Dtests.slow=true -Dtests.locale=fr_CA -Dtests.timezone=Asia/Baku -Dtests.file.encoding=ISO-8859-1 [junit4] FAILURE 59.6s | ChaosMonkeySafeLeaderTest.testDistribSearch <<< [junit4] > Throwable #1: java.lang.AssertionError: expected:<0> but was:<1> [junit4] > at __randomizedtesting.SeedInfo.seed([958A190CF6D69FCA:146C97148189FFF6]:0) [junit4] > at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.doTest(ChaosMonkeySafeLeaderTest.java:153) [junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869) [junit4] > at java.lang.Thread.run(Thread.java:745) [junit4] 2> 3936680 T9906 oas.SolrTestCaseJ4.deleteCore ###deleteCore [junit4] 2> NOTE: leaving temporary files on disk at: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.ChaosMonkeySafeLeaderTest-958A190CF6D69FCA-001 [junit4] 2> 59649 T9905 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s). [junit4] 2> NOTE: test params are: codec=Asserting(Lucene50): {rnd_b=Lucene41(blocksize=128), _version_=PostingsFormat(name=Memory doPackFST= true), a_t=PostingsFormat(name=Asserting), a_i=Lucene41(blocksize=128), id=Lucene41(blocksize=128)}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=fr_CA, timezone=Asia/Baku [junit4] 2> NOTE: Mac OS X 10.8.5 x86_64/Oracle Corporation 1.8.0_20 (64-bit)/cpus=2,threads=1,free=49074200,total=300253184 [junit4] 2> NOTE: All tests run in this JVM: [ExternalFileFieldSortTest, TestSimpleQParserPlugin, AsyncMigrateRouteKeyTest, AnalyticsQueryTest, CloudExitableDirectoryReaderTest, TermsComponentTest, TestCoreDiscovery, TestBinaryResponseWriter, SimpleFacetsTest, DocValuesTest, TestReversedWildcardFilterFactory, CoreAdminCreateDiscoverTest, TestOverriddenPrefixQueryForCustomFieldType, TestCloudManagedSchemaConcurrent, TestDocSet, TestSchemaSimilarityResource, SolrRequestParserTest, XsltUpdateRequestHandlerTest, HdfsBasicDistributedZk2Test, RequiredFieldsTest, FieldMutatingUpdateProcessorTest, TestCloudPivotFacet, ExitableDirectoryReaderTest, TestFastWriter, DistributedSuggestComponentTest, TestSolrDeletionPolicy2, DirectSolrConnectionTest, ShardRoutingTest, DirectSolrSpellCheckerTest, TestSchemaVersionResource, TestDistributedSearch, TestQuerySenderNoQuery, TestMissingGroups, ShardRoutingCustomTest, TestRangeQuery, TestRequestStatusCollectionAPI, TestExceedMaxTermLength, MigrateRouteKeyTest, LegacyHTMLStripCharFilterTest, SignatureUpdateProcessorFactoryTest, ModifyConfFileTest, TestHashPartitioner, TestJoin, CurrencyFieldXmlFileTest, DistributedSpellCheckComponentTest, TestIBSimilarityFactory, DistribDocExpirationUpdateProcessorTest, TestReplicationHandler, TestSolrJ, BadComponentTest, TestSuggestSpellingConverter, IndexBasedSpellCheckerTest, RankQueryTest, IndexSchemaTest, TestDynamicFieldCollectionResource, TestDocBasedVersionConstraints, TestSchemaManager, CurrencyFieldOpenExchangeTest, DocumentBuilderTest, TriLevelCompositeIdRoutingTest, WordBreakSolrSpellCheckerTest, TestQuerySenderListener, XmlUpdateRequestHandlerTest, RollingRestartTest, HdfsWriteToMultipleCollectionsTest, RemoteQueryErrorTest, StatsComponentTest, TestShardHandlerFactory, SyncSliceTest, TestValueSourceCache, QueryElevationComponentTest, TestCodecSupport, TestCSVLoader, TestSolr4Spatial2, HdfsRecoveryZkTest, TestStressRecovery, TestCSVResponseWriter, TestPostingsSolrHighlighter, TestNRTOpen, TestCollationField, BJQParserTest, TestInitParams, PreAnalyzedUpdateProcessorTest, TermVectorComponentTest, SolrXmlInZkTest, TestPerFieldSimilarity, HttpPartitionTest, TestBadConfig, TestReRankQParserPlugin, SOLR749Test, TestSearcherReuse, DirectUpdateHandlerTest, MultiTermTest, BasicDistributedZk2Test, ZkSolrClientTest, FieldAnalysisRequestHandlerTest, DistributedFacetPivotLargeTest, TermVectorComponentDistributedTest, TestBulkSchemaConcurrent, TestModifyConfFiles, ScriptEngineTest, TestSolrQueryParser, SharedFSAutoReplicaFailoverUtilsTest, TestManagedSchema, TestSolrXmlPersistor, TestXIncludeConfig, StressHdfsTest, TestOmitPositions, TestPseudoReturnFields, AddBlockUpdateTest, StatelessScriptUpdateProcessorFactoryTest, JSONWriterTest, TestDistribDocBasedVersion, TestRecoveryHdfs, BlockCacheTest, SolrIndexConfigTest, SolrCoreCheckLockOnStartupTest, PingRequestHandlerTest, SolrCoreTest, NumericFieldsTest, CSVRequestHandlerTest, TestSweetSpotSimilarityFactory, QueryResultKeyTest, SoftAutoCommitTest, TestCollationFieldDocValues, URLClassifyProcessorTest, TestLazyCores, BadIndexSchemaTest, NoCacheHeaderTest, LukeRequestHandlerTest, DOMUtilTest, TestJmxMonitoredMap, QueryEqualityTest, TestQueryUtils, UniqFieldsUpdateProcessorFactoryTest, UUIDFieldTest, ParsingFieldUpdateProcessorsTest, TestStandardQParsers, TestSolrIndexConfig, DateMathParserTest, TestIntervalFaceting, TestCollectionAPI, TestDistributedGrouping, UnloadDistributedZkTest, DirectUpdateHandlerOptimizeTest, TestSchemaNameResource, TestPivotHelperCode, TestSolr4Spatial, ChangedSchemaMergeTest, TestDefaultSimilarityFactory, TestSystemIdResolver, TestCharFilters, TestFoldingMultitermQuery, TestAddFieldRealTimeGet, TestSolrQueryParserDefaultOperatorResource, CoreContainerCoreInitFailuresTest, ReturnFieldsTest, TestFastOutputStream, TestArbitraryIndexDir, QueryParsingTest, MinimalSchemaTest, CursorMarkTest, AliasIntegrationTest, HdfsBasicDistributedZkTest, SearchHandlerTest, TestWriterPerf, DocExpirationUpdateProcessorFactoryTest, DistributedExpandComponentTest, TestManagedSynonymFilterFactory, TestRandomFaceting, TestInfoStreamLogging, RegexBoostProcessorTest, CoreMergeIndexesAdminHandlerTest, TestSolrDeletionPolicy1, BasicZkTest, TestStressUserVersions, TestDefaultSearchFieldResource, LoggingHandlerTest, DistributedQueryComponentCustomSortTest, RequestHandlersTest, BadCopyFieldTest, TestDistributedMissingSort, TestSurroundQueryParser, OverseerRolesTest, AddSchemaFieldsUpdateProcessorFactoryTest, SolrIndexSplitterTest, EchoParamsTest, SuggestComponentTest, PeerSyncTest, TestLuceneMatchVersion, TestFunctionQuery, TestFastLRUCache, PrimUtilsTest, HighlighterMaxOffsetTest, TestCloudInspectUtil, HdfsChaosMonkeySafeLeaderTest, SharedFSAutoReplicaFailoverTest, TestFiltering, TestExpandComponent, CircularListTest, DeleteShardTest, SpellingQueryConverterTest, TestSort, MultiThreadedOCPTest, TestNonNRTOpen, BasicFunctionalityTest, HighlighterConfigTest, SystemInfoHandlerTest, TestClassNameShortening, BasicDistributedZkTest, ChaosMonkeySafeLeaderTest] [junit4] Completed in 60.19s, 1 test, 1 failure <<< FAILURES! [...truncated 692 lines...] BUILD FAILED /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:524: The following error occurred while executing this line: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:472: The following error occurred while executing this line: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:61: The following error occurred while executing this line: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/extra-targets.xml:39: The following error occurred while executing this line: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build.xml:189: The following error occurred while executing this line: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/common-build.xml:496: The following error occurred while executing this line: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:1359: The following error occurred while executing this line: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:961: There were test failures: 436 suites, 1783 tests, 1 failure, 49 ignored (21 assumptions) Total time: 175 minutes 54 seconds Build step 'Invoke Ant' marked build as failure [description-setter] Description set: Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseSerialGC Archiving artifacts Recording test results Email was triggered for: Failure - Any Sending email for trigger: Failure - Any
--------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org