Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java7/4925/
1 tests failed. REGRESSION: org.apache.solr.cloud.DeleteReplicaTest.testDistribSearch Error Message: No live SolrServers available to handle this request:[https://127.0.0.1:11201/gllb/wr, https://127.0.0.1:11187/gllb/wr, https://127.0.0.1:11195/gllb/wr, https://127.0.0.1:11171/gllb/wr, https://127.0.0.1:11578/gllb/wr] Stack Trace: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[https://127.0.0.1:11201/gllb/wr, https://127.0.0.1:11187/gllb/wr, https://127.0.0.1:11195/gllb/wr, https://127.0.0.1:11171/gllb/wr, https://127.0.0.1:11578/gllb/wr] at __randomizedtesting.SeedInfo.seed([217FCCCCCF39A5F8:A09942D4B866C5C4]:0) at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:333) at org.apache.solr.client.solrj.impl.CloudSolrServer.sendRequest(CloudSolrServer.java:1015) at org.apache.solr.client.solrj.impl.CloudSolrServer.requestWithRetryOnStaleState(CloudSolrServer.java:793) at org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:736) at org.apache.solr.cloud.DeleteReplicaTest.removeAndWaitForReplicaGone(DeleteReplicaTest.java:172) at org.apache.solr.cloud.DeleteReplicaTest.deleteLiveReplicaTest(DeleteReplicaTest.java:145) at org.apache.solr.cloud.DeleteReplicaTest.doTest(DeleteReplicaTest.java:89) at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Could not remove replica : delLiveColl/shard1/core_node1 at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:569) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:215) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:211) at org.apache.solr.client.solrj.impl.LBHttpSolrServer.doRequest(LBHttpSolrServer.java:354) at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:313) ... 47 more Build Log: [...truncated 12194 lines...] [junit4] Suite: org.apache.solr.cloud.DeleteReplicaTest [junit4] 2> Creating dataDir: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-217FCCCCCF39A5F8-001/init-core-data-001 [junit4] 2> 1481160 T5086 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (true) [junit4] 2> 1481160 T5086 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /gllb/wr [junit4] 2> 1481165 T5086 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch [junit4] 2> 1481165 T5086 oasc.ZkTestServer.run STARTING ZK TEST SERVER [junit4] 1> client port:0.0.0.0/0.0.0.0:0 [junit4] 2> 1481166 T5087 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server [junit4] 2> 1481266 T5086 oasc.ZkTestServer.run start zk server on port:21570 [junit4] 2> 1481267 T5086 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 1481268 T5086 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 1481271 T5093 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7e1b1c75 name:ZooKeeperConnection Watcher:127.0.0.1:21570 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1481271 T5086 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 1481271 T5086 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 1481272 T5086 oascc.SolrZkClient.makePath makePath: /solr [junit4] 2> 1481274 T5086 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 1481275 T5086 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 1481276 T5095 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@38cede16 name:ZooKeeperConnection Watcher:127.0.0.1:21570/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1481276 T5086 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 1481277 T5086 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 1481277 T5086 oascc.SolrZkClient.makePath makePath: /collections/collection1 [junit4] 2> 1481279 T5086 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards [junit4] 2> 1481280 T5086 oascc.SolrZkClient.makePath makePath: /collections/control_collection [junit4] 2> 1481281 T5086 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards [junit4] 2> 1481283 T5086 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml [junit4] 2> 1481284 T5086 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml [junit4] 2> 1481286 T5086 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml [junit4] 2> 1481287 T5086 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml [junit4] 2> 1481389 T5086 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml [junit4] 2> 1481389 T5086 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml [junit4] 2> 1481391 T5086 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt [junit4] 2> 1481392 T5086 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt [junit4] 2> 1481394 T5086 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt [junit4] 2> 1481394 T5086 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt [junit4] 2> 1481396 T5086 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml [junit4] 2> 1481397 T5086 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml [junit4] 2> 1481398 T5086 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml [junit4] 2> 1481399 T5086 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml [junit4] 2> 1481401 T5086 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json [junit4] 2> 1481401 T5086 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json [junit4] 2> 1481403 T5086 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt [junit4] 2> 1481404 T5086 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt [junit4] 2> 1481405 T5086 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt [junit4] 2> 1481406 T5086 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt [junit4] 2> 1481408 T5086 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt [junit4] 2> 1481408 T5086 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt [junit4] 2> 1481410 T5086 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 1481411 T5086 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 1481412 T5097 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@574dc393 name:ZooKeeperConnection Watcher:127.0.0.1:21570/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1481412 T5086 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 1481412 T5086 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 1483568 T5086 oejs.Server.doStart jetty-8.1.10.v20130312 [junit4] 2> 1483576 T5086 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] [junit4] 2> 1483578 T5086 oejs.AbstractConnector.doStart Started SslSocketConnector@127.0.0.1:11578 [junit4] 2> 1483581 T5086 oass.SolrDispatchFilter.init SolrDispatchFilter.init() [junit4] 2> 1483582 T5086 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 1483582 T5086 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-217FCCCCCF39A5F8-001/tempDir-002 [junit4] 2> 1483582 T5086 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-217FCCCCCF39A5F8-001/tempDir-002/' [junit4] 2> 1483645 T5086 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-217FCCCCCF39A5F8-001/tempDir-002/solr.xml [junit4] 2> 1483731 T5086 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-217FCCCCCF39A5F8-001/tempDir-002 [junit4] 2> 1483732 T5086 oasc.CoreContainer.<init> New CoreContainer 1364411116 [junit4] 2> 1483732 T5086 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-217FCCCCCF39A5F8-001/tempDir-002/] [junit4] 2> 1483733 T5086 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000 [junit4] 2> 1483734 T5086 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: [junit4] 2> 1483734 T5086 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000 [junit4] 2> 1483734 T5086 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20 [junit4] 2> 1483735 T5086 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000 [junit4] 2> 1483735 T5086 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0 [junit4] 2> 1483735 T5086 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647 [junit4] 2> 1483736 T5086 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5 [junit4] 2> 1483736 T5086 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1 [junit4] 2> 1483737 T5086 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false [junit4] 2> 1483740 T5086 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false [junit4] 2> 1483742 T5086 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 1483742 T5086 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 1483743 T5086 oasc.CoreContainer.load Host Name: 127.0.0.1 [junit4] 2> 1483743 T5086 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:21570/solr [junit4] 2> 1483743 T5086 oasc.ZkController.checkChrootPath zkHost includes chroot [junit4] 2> 1483744 T5086 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 1483745 T5086 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 1483746 T5108 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7b9e81dc name:ZooKeeperConnection Watcher:127.0.0.1:21570 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1483747 T5086 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 1483748 T5086 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 1483749 T5086 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 1483750 T5110 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7420bc63 name:ZooKeeperConnection Watcher:127.0.0.1:21570/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1483751 T5086 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 1483752 T5086 oascc.SolrZkClient.makePath makePath: /overseer/queue [junit4] 2> 1483753 T5086 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work [junit4] 2> 1483755 T5086 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running [junit4] 2> 1483757 T5086 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed [junit4] 2> 1483759 T5086 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure [junit4] 2> 1483761 T5086 oascc.SolrZkClient.makePath makePath: /live_nodes [junit4] 2> 1483762 T5086 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:11578_gllb%2Fwr [junit4] 2> 1483763 T5086 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:11578_gllb%2Fwr [junit4] 2> 1483765 T5086 oascc.SolrZkClient.makePath makePath: /overseer_elect [junit4] 2> 1483766 T5086 oascc.SolrZkClient.makePath makePath: /overseer_elect/election [junit4] 2> 1483768 T5086 oasc.Overseer.close Overseer (id=null) closing [junit4] 2> 1483769 T5086 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:11578_gllb%2Fwr [junit4] 2> 1483770 T5086 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader [junit4] 2> 1483771 T5086 oasc.Overseer.start Overseer (id=92640996769595396-127.0.0.1:11578_gllb%2Fwr-n_0000000000) starting [junit4] 2> 1483773 T5086 oascc.SolrZkClient.makePath makePath: /overseer/queue-work [junit4] 2> 1483780 T5086 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000 [junit4] 2> 1483781 T5112 oasc.OverseerCollectionProcessor.run Process current queue of collection creations [junit4] 2> 1483782 T5086 oascc.SolrZkClient.makePath makePath: /clusterstate.json [junit4] 2> 1483783 T5086 oascc.SolrZkClient.makePath makePath: /aliases.json [junit4] 2> 1483784 T5086 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 1483787 T5111 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue [junit4] 2> 1483790 T5086 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-217FCCCCCF39A5F8-001/tempDir-002 [junit4] 2> 1485275 T5086 oasc.CorePropertiesLocator.discoverUnder Found core conf in /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-217FCCCCCF39A5F8-001/tempDir-002/conf/ [junit4] 2> 1485532 T5086 oasc.CorePropertiesLocator.discover Found 1 core definitions [junit4] 2> 1485533 T5086 oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0 [junit4] 2> 1485533 T5086 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done [junit4] 2> 1485537 T5086 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 1485538 T5086 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 1485539 T5116 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@37d1a75a name:ZooKeeperConnection Watcher:127.0.0.1:21570/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1485539 T5086 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 1485539 T5086 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 1485540 T5086 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 1485541 T5086 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false [junit4] 2> 1485542 T5086 oasc.AbstractFullDistribZkTestBase.createJettys Creating collection1 with stateFormat=2 [junit4] 2> 1485542 T5086 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 1485542 T5086 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 1485543 T5118 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@50fcea14 name:ZooKeeperConnection Watcher:127.0.0.1:21570/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1485543 T5086 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 1485543 T5086 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 1485566 T5119 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 1485568 T5111 oasc.Overseer$ClusterStateUpdater.buildCollection building a new collection: collection1 [junit4] 2> 1485568 T5111 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2] [junit4] 2> 1485569 T5111 oasc.Overseer$ClusterStateUpdater.createCollection state version collection1 2 [junit4] 2> 1485570 T5111 oasc.Overseer$ClusterStateUpdater.updateZkStates going to create_collection /collections/collection1/state.json [junit4] 2> 1485571 T5119 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1) [junit4] 2> 1485572 T5120 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1) [junit4] 2> 1487192 T5086 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1 [junit4] 2> 1487193 T5086 oejs.Server.doStart jetty-8.1.10.v20130312 [junit4] 2> 1487198 T5086 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] [junit4] 2> 1487201 T5086 oejs.AbstractConnector.doStart Started SslSocketConnector@127.0.0.1:11171 [junit4] 2> 1487204 T5086 oass.SolrDispatchFilter.init SolrDispatchFilter.init() [junit4] 2> 1487205 T5086 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 1487205 T5086 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-217FCCCCCF39A5F8-001/tempDir-003 [junit4] 2> 1487206 T5086 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-217FCCCCCF39A5F8-001/tempDir-003/' [junit4] 2> 1487258 T5086 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-217FCCCCCF39A5F8-001/tempDir-003/solr.xml [junit4] 2> 1487335 T5086 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-217FCCCCCF39A5F8-001/tempDir-003 [junit4] 2> 1487335 T5086 oasc.CoreContainer.<init> New CoreContainer 135732258 [junit4] 2> 1487336 T5086 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-217FCCCCCF39A5F8-001/tempDir-003/] [junit4] 2> 1487337 T5086 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000 [junit4] 2> 1487337 T5086 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: [junit4] 2> 1487338 T5086 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000 [junit4] 2> 1487338 T5086 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20 [junit4] 2> 1487338 T5086 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000 [junit4] 2> 1487339 T5086 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0 [junit4] 2> 1487339 T5086 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647 [junit4] 2> 1487339 T5086 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5 [junit4] 2> 1487340 T5086 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1 [junit4] 2> 1487340 T5086 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false [junit4] 2> 1487343 T5086 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false [junit4] 2> 1487345 T5086 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 1487345 T5086 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 1487346 T5086 oasc.CoreContainer.load Host Name: 127.0.0.1 [junit4] 2> 1487346 T5086 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:21570/solr [junit4] 2> 1487347 T5086 oasc.ZkController.checkChrootPath zkHost includes chroot [junit4] 2> 1487347 T5086 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 1487348 T5086 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 1487349 T5131 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@faa02a8 name:ZooKeeperConnection Watcher:127.0.0.1:21570 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1487350 T5086 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 1487351 T5086 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 1487352 T5086 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 1487353 T5133 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5017864a name:ZooKeeperConnection Watcher:127.0.0.1:21570/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1487353 T5086 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 1487357 T5086 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 1488361 T5086 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:11171_gllb%2Fwr [junit4] 2> 1488362 T5086 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:11171_gllb%2Fwr [junit4] 2> 1488365 T5086 oasc.Overseer.close Overseer (id=null) closing [junit4] 2> 1488369 T5086 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-217FCCCCCF39A5F8-001/tempDir-003 [junit4] 2> 1488373 T5086 oasc.CorePropertiesLocator.discoverUnder Found core conf in /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-217FCCCCCF39A5F8-001/tempDir-003/conf/ [junit4] 2> 1489301 T5086 oasc.CorePropertiesLocator.discover Found 1 core definitions [junit4] 2> 1489302 T5086 oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0 [junit4] 2> 1489302 T5086 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done [junit4] 2> 1490524 T5086 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2 [junit4] 2> 1490525 T5086 oejs.Server.doStart jetty-8.1.10.v20130312 [junit4] 2> 1490530 T5086 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] [junit4] 2> 1490531 T5086 oejs.AbstractConnector.doStart Started SslSocketConnector@127.0.0.1:11187 [junit4] 2> 1490533 T5086 oass.SolrDispatchFilter.init SolrDispatchFilter.init() [junit4] 2> 1490533 T5086 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 1490534 T5086 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-217FCCCCCF39A5F8-001/tempDir-004 [junit4] 2> 1490534 T5086 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-217FCCCCCF39A5F8-001/tempDir-004/' [junit4] 2> 1490576 T5086 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-217FCCCCCF39A5F8-001/tempDir-004/solr.xml [junit4] 2> 1490637 T5086 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-217FCCCCCF39A5F8-001/tempDir-004 [junit4] 2> 1490638 T5086 oasc.CoreContainer.<init> New CoreContainer 1972637437 [junit4] 2> 1490638 T5086 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-217FCCCCCF39A5F8-001/tempDir-004/] [junit4] 2> 1490639 T5086 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000 [junit4] 2> 1490639 T5086 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: [junit4] 2> 1490639 T5086 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000 [junit4] 2> 1490640 T5086 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20 [junit4] 2> 1490640 T5086 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000 [junit4] 2> 1490640 T5086 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0 [junit4] 2> 1490640 T5086 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647 [junit4] 2> 1490640 T5086 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5 [junit4] 2> 1490641 T5086 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1 [junit4] 2> 1490641 T5086 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false [junit4] 2> 1490643 T5086 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false [junit4] 2> 1490645 T5086 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 1490645 T5086 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 1490645 T5086 oasc.CoreContainer.load Host Name: 127.0.0.1 [junit4] 2> 1490645 T5086 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:21570/solr [junit4] 2> 1490646 T5086 oasc.ZkController.checkChrootPath zkHost includes chroot [junit4] 2> 1490646 T5086 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 1490646 T5086 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 1490648 T5146 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5cf7caaf name:ZooKeeperConnection Watcher:127.0.0.1:21570 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1490649 T5086 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 1490649 T5086 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 1490650 T5086 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 1490651 T5148 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@405449b9 name:ZooKeeperConnection Watcher:127.0.0.1:21570/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1490651 T5086 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 1490655 T5086 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 1491658 T5086 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:11187_gllb%2Fwr [junit4] 2> 1491659 T5086 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:11187_gllb%2Fwr [junit4] 2> 1491663 T5086 oasc.Overseer.close Overseer (id=null) closing [junit4] 2> 1491667 T5086 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-217FCCCCCF39A5F8-001/tempDir-004 [junit4] 2> 1493333 T5086 oasc.CorePropertiesLocator.discoverUnder Found core conf in /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-217FCCCCCF39A5F8-001/tempDir-004/conf/ [junit4] 2> 1493334 T5086 oasc.CorePropertiesLocator.discover Found 1 core definitions [junit4] 2> 1493335 T5086 oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0 [junit4] 2> 1493335 T5086 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done [junit4] 2> 1494928 T5086 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3 [junit4] 2> 1494929 T5086 oejs.Server.doStart jetty-8.1.10.v20130312 [junit4] 2> 1494933 T5086 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] [junit4] 2> 1494934 T5086 oejs.AbstractConnector.doStart Started SslSocketConnector@127.0.0.1:11195 [junit4] 2> 1494937 T5086 oass.SolrDispatchFilter.init SolrDispatchFilter.init() [junit4] 2> 1494937 T5086 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 1494937 T5086 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-217FCCCCCF39A5F8-001/tempDir-005 [junit4] 2> 1494938 T5086 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-217FCCCCCF39A5F8-001/tempDir-005/' [junit4] 2> 1494980 T5086 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-217FCCCCCF39A5F8-001/tempDir-005/solr.xml [junit4] 2> 1495040 T5086 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-217FCCCCCF39A5F8-001/tempDir-005 [junit4] 2> 1495041 T5086 oasc.CoreContainer.<init> New CoreContainer 1744263985 [junit4] 2> 1495041 T5086 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-217FCCCCCF39A5F8-001/tempDir-005/] [junit4] 2> 1495042 T5086 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000 [junit4] 2> 1495042 T5086 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: [junit4] 2> 1495043 T5086 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000 [junit4] 2> 1495043 T5086 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20 [junit4] 2> 1495043 T5086 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000 [junit4] 2> 1495043 T5086 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0 [junit4] 2> 1495044 T5086 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647 [junit4] 2> 1495044 T5086 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5 [junit4] 2> 1495044 T5086 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1 [junit4] 2> 1495044 T5086 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false [junit4] 2> 1495046 T5086 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false [junit4] 2> 1495048 T5086 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 1495048 T5086 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 1495048 T5086 oasc.CoreContainer.load Host Name: 127.0.0.1 [junit4] 2> 1495049 T5086 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:21570/solr [junit4] 2> 1495049 T5086 oasc.ZkController.checkChrootPath zkHost includes chroot [junit4] 2> 1495049 T5086 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 1495050 T5086 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 1495051 T5161 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4d912012 name:ZooKeeperConnection Watcher:127.0.0.1:21570 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1495052 T5086 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 1495052 T5086 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 1495054 T5086 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 1495055 T5163 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@23a59269 name:ZooKeeperConnection Watcher:127.0.0.1:21570/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1495055 T5086 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 1495058 T5086 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 1496061 T5086 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:11195_gllb%2Fwr [junit4] 2> 1496062 T5086 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:11195_gllb%2Fwr [junit4] 2> 1496065 T5086 oasc.Overseer.close Overseer (id=null) closing [junit4] 2> 1496068 T5086 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-217FCCCCCF39A5F8-001/tempDir-005 [junit4] 2> 1496081 T5086 oasc.CorePropertiesLocator.discoverUnder Found core conf in /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-217FCCCCCF39A5F8-001/tempDir-005/conf/ [junit4] 2> 1496844 T5086 oasc.CorePropertiesLocator.discover Found 1 core definitions [junit4] 2> 1496845 T5086 oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0 [junit4] 2> 1496845 T5086 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done [junit4] 2> 1497836 T5086 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 4 [junit4] 2> 1497836 T5086 oejs.Server.doStart jetty-8.1.10.v20130312 [junit4] 2> 1497841 T5086 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] [junit4] 2> 1497842 T5086 oejs.AbstractConnector.doStart Started SslSocketConnector@127.0.0.1:11201 [junit4] 2> 1497845 T5086 oass.SolrDispatchFilter.init SolrDispatchFilter.init() [junit4] 2> 1497845 T5086 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 1497845 T5086 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-217FCCCCCF39A5F8-001/tempDir-006 [junit4] 2> 1497846 T5086 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-217FCCCCCF39A5F8-001/tempDir-006/' [junit4] 2> 1497881 T5086 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-217FCCCCCF39A5F8-001/tempDir-006/solr.xml [junit4] 2> 1497931 T5086 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-217FCCCCCF39A5F8-001/tempDir-006 [junit4] 2> 1497932 T5086 oasc.CoreContainer.<init> New CoreContainer 845854624 [junit4] 2> 1497932 T5086 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-217FCCCCCF39A5F8-001/tempDir-006/] [junit4] 2> 1497933 T5086 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000 [junit4] 2> 1497933 T5086 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: [junit4] 2> 1497933 T5086 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000 [junit4] 2> 1497933 T5086 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20 [junit4] 2> 1497934 T5086 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000 [junit4] 2> 1497934 T5086 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0 [junit4] 2> 1497934 T5086 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647 [junit4] 2> 1497934 T5086 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5 [junit4] 2> 1497934 T5086 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1 [junit4] 2> 1497935 T5086 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false [junit4] 2> 1497937 T5086 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false [junit4] 2> 1497938 T5086 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 1497938 T5086 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 1497939 T5086 oasc.CoreContainer.load Host Name: 127.0.0.1 [junit4] 2> 1497939 T5086 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:21570/solr [junit4] 2> 1497939 T5086 oasc.ZkController.checkChrootPath zkHost includes chroot [junit4] 2> 1497939 T5086 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 1497940 T5086 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 1497942 T5176 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@23f0e5ef name:ZooKeeperConnection Watcher:127.0.0.1:21570 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1497943 T5086 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 1497943 T5086 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 1497945 T5086 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 1497946 T5178 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@468a31a3 name:ZooKeeperConnection Watcher:127.0.0.1:21570/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1497946 T5086 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 1497949 T5086 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 1498953 T5086 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:11201_gllb%2Fwr [junit4] 2> 1498954 T5086 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:11201_gllb%2Fwr [junit4] 2> 1498957 T5086 oasc.Overseer.close Overseer (id=null) closing [junit4] 2> 1498962 T5086 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-217FCCCCCF39A5F8-001/tempDir-006 [junit4] 2> 1500412 T5086 oasc.CorePropertiesLocator.discoverUnder Found core conf in /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-217FCCCCCF39A5F8-001/tempDir-006/conf/ [junit4] 2> 1500581 T5086 oasc.CorePropertiesLocator.discover Found 1 core definitions [junit4] 2> 1500582 T5086 oass.SolrDispatchFilter.init user.dir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0 [junit4] 2> 1500582 T5086 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done [junit4] 2> 1500588 T5086 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 1500589 T5086 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 1500590 T5182 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@49f25609 name:ZooKeeperConnection Watcher:127.0.0.1:21570/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1500591 T5086 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 1500591 T5086 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 1500592 T5086 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 1500594 T5086 oasc.AbstractFullDistribZkTestBase.createCollection Creating collection with stateFormat=2: delLiveColl [junit4] 2> 1500594 T5086 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 1500595 T5086 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 1500596 T5184 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@51a4016 name:ZooKeeperConnection Watcher:127.0.0.1:21570/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1500596 T5086 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 1500597 T5086 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 1500598 T5086 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 1500734 T5106 oasha.CollectionsHandler.handleCreateAction Creating Collection : maxShardsPerNode=2&name=delLiveColl&action=CREATE&wt=javabin&numShards=2&replicationFactor=2&stateFormat=2&version=2 [junit4] 2> 1500735 T5119 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged [junit4] 2> 1500736 T5112 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{ [junit4] 2> "operation":"create", [junit4] 2> "fromApi":"true", [junit4] 2> "name":"delLiveColl", [junit4] 2> "replicationFactor":"2", [junit4] 2> "numShards":"2", [junit4] 2> "maxShardsPerNode":"2", [junit4] 2> "stateFormat":"2"} [junit4] 2> 1500737 T5186 oasc.OverseerCollectionProcessor.processMessage WARN OverseerCollectionProcessor.processMessage : create , { [junit4] 2> "operation":"create", [junit4] 2> "fromApi":"true", [junit4] 2> "name":"delLiveColl", [junit4] 2> "replicationFactor":"2", [junit4] 2> "numShards":"2", [junit4] 2> "maxShardsPerNode":"2", [junit4] 2> "stateFormat":"2"} [junit4] 2> 1500738 T5186 oasc.OverseerCollectionProcessor.createConfNode Only one config set found in zk - using it:conf1 [junit4] 2> 1500738 T5186 oasc.OverseerCollectionProcessor.createConfNode creating collections conf node /collections/delLiveColl [junit4] 2> 1500738 T5186 oascc.SolrZkClient.makePath makePath: /collections/delLiveColl [junit4] 2> 1500740 T5119 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 1500742 T5111 oasc.Overseer$ClusterStateUpdater.buildCollection building a new collection: delLiveColl [junit4] 2> 1500742 T5111 oasc.Overseer$ClusterStateUpdater.createCollection Create collection delLiveColl with shards [shard1, shard2] [junit4] 2> 1500742 T5111 oasc.Overseer$ClusterStateUpdater.createCollection state version delLiveColl 2 [junit4] 2> 1500744 T5111 oasc.Overseer$ClusterStateUpdater.updateZkStates going to create_collection /collections/delLiveColl/state.json [junit4] 2> 1500745 T5134 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4] 2> 1500745 T5164 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4] 2> 1500745 T5188 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4] 2> 1500745 T5179 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4] 2> 1500745 T5187 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4] 2> 1500745 T5120 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4] 2> 1500745 T5119 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4] 2> 1500745 T5149 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4] 2> 1500840 T5186 oasc.OverseerCollectionProcessor.createCollection Creating SolrCores for new collection delLiveColl, shardNames [shard1, shard2] , replicationFactor : 2 [junit4] 2> 1500841 T5186 oasc.OverseerCollectionProcessor.createCollection Creating shard delLiveColl_shard1_replica1 as part of slice shard1 of collection delLiveColl on 127.0.0.1:11578_gllb%2Fwr [junit4] 2> 1500841 T5186 oasc.OverseerCollectionProcessor.createCollection Creating shard delLiveColl_shard1_replica2 as part of slice shard1 of collection delLiveColl on 127.0.0.1:11171_gllb%2Fwr [junit4] 2> 1500843 T5186 oasc.OverseerCollectionProcessor.createCollection Creating shard delLiveColl_shard2_replica1 as part of slice shard2 of collection delLiveColl on 127.0.0.1:11187_gllb%2Fwr [junit4] 2> 1500844 T5186 oasc.OverseerCollectionProcessor.createCollection Creating shard delLiveColl_shard2_replica2 as part of slice shard2 of collection delLiveColl on 127.0.0.1:11195_gllb%2Fwr [junit4] 2> 1500966 T5105 oasha.CoreAdminHandler.handleCreateAction core create command collection.configName=conf1&collection=delLiveColl&name=delLiveColl_shard1_replica1&action=CREATE&wt=javabin&numShards=2&qt=/admin/cores&version=2&shard=shard1 [junit4] 2> 1500968 T5156 oasha.CoreAdminHandler.handleCreateAction core create command collection.configName=conf1&collection=delLiveColl&name=delLiveColl_shard2_replica2&action=CREATE&wt=javabin&numShards=2&qt=/admin/cores&version=2&shard=shard2 [junit4] 2> 1500969 T5141 oasha.CoreAdminHandler.handleCreateAction core create command collection.configName=conf1&collection=delLiveColl&name=delLiveColl_shard2_replica1&action=CREATE&wt=javabin&numShards=2&qt=/admin/cores&version=2&shard=shard2 [junit4] 2> 1500971 T5105 oasc.ZkController.publish publishing core=delLiveColl_shard1_replica1 state=down collection=delLiveColl [junit4] 2> 1500971 T5156 oasc.ZkController.publish publishing core=delLiveColl_shard2_replica2 state=down collection=delLiveColl [junit4] 2> 1500972 T5119 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 1500972 T5141 oasc.ZkController.publish publishing core=delLiveColl_shard2_replica1 state=down collection=delLiveColl [junit4] 2> 1500972 T5126 oasha.CoreAdminHandler.handleCreateAction core create command collection.configName=conf1&collection=delLiveColl&name=delLiveColl_shard1_replica2&action=CREATE&wt=javabin&numShards=2&qt=/admin/cores&version=2&shard=shard1 [junit4] 2> 1500972 T5105 oasc.ZkController.preRegister Registering watch for external collection delLiveColl [junit4] 2> 1500974 T5156 oasc.ZkController.preRegister Registering watch for external collection delLiveColl [junit4] 2> 1500974 T5141 oasc.ZkController.preRegister Registering watch for external collection delLiveColl [junit4] 2> 1500974 T5111 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={ [junit4] 2> "node_name":"127.0.0.1:11578_gllb%2Fwr", [junit4] 2> "operation":"state", [junit4] 2> "collection":"delLiveColl", [junit4] 2> "roles":null, [junit4] 2> "base_url":"https://127.0.0.1:11578/gllb/wr";, [junit4] 2> "state":"down", [junit4] 2> "core":"delLiveColl_shard1_replica1", [junit4] 2> "numShards":"2", [junit4] 2> "shard":"shard1"} [junit4] 2> 1500974 T5156 oascc.ZkStateReader.addZkWatch addZkWatch delLiveColl [junit4] 2> 1500975 T5141 oascc.ZkStateReader.addZkWatch addZkWatch delLiveColl [junit4] 2> 1500976 T5156 oascc.ZkStateReader.addZkWatch Updating collection state at /collections/delLiveColl/state.json from ZooKeeper... [junit4] 2> 1500976 T5126 oasc.ZkController.publish publishing core=delLiveColl_shard1_replica2 state=down collection=delLiveColl [junit4] 2> 1500976 T5141 oascc.ZkStateReader.addZkWatch Updating collection state at /collections/delLiveColl/state.json from ZooKeeper... [junit4] 2> 1500977 T5156 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 0 [junit4] 2> 1500977 T5156 oasc.ZkController.waitForCoreNodeName look for our core node name [junit4] 2> 1500977 T5141 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 0 [junit4] 2> 1500978 T5141 oasc.ZkController.waitForCoreNodeName look for our core node name [junit4] 2> 1500978 T5126 oasc.ZkController.preRegister Registering watch for external collection delLiveColl [junit4] 2> 1500978 T5126 oascc.ZkStateReader.addZkWatch addZkWatch delLiveColl [junit4] 2> 1500979 T5126 oascc.ZkStateReader.addZkWatch Updating collection state at /collections/delLiveColl/state.json from ZooKeeper... [junit4] 2> 1500979 T5111 oasc.Overseer$ClusterStateUpdater.updateZkStates going to update_collection /collections/delLiveColl/state.json [junit4] 2> 1500980 T5149 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... [junit4] 2> 1500980 T5126 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 1 [junit4] 2> 1500980 T5164 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... [junit4] 2> 1500980 T5134 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... [junit4] 2> 1500981 T5149 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 1 [junit4] 2> 1500981 T5126 oasc.ZkController.waitForCoreNodeName look for our core node name [junit4] 2> 1500981 T5134 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 1 [junit4] 2> 1500981 T5164 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 1 [junit4] 2> 1500981 T5105 oascc.ZkStateReader.addZkWatch addZkWatch delLiveColl [junit4] 2> 1500982 T5105 oascc.ZkStateReader.addZkWatch Updating collection state at /collections/delLiveColl/state.json from ZooKeeper... [junit4] 2> 1500983 T5105 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 1 [junit4] 2> 1500984 T5105 oasc.ZkController.waitForCoreNodeName look for our core node name [junit4] 2> 1500984 T5105 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for delLiveColl_shard1_replica1 [junit4] 2> 1500984 T5105 oasc.ZkController.createCollectionZkNode Check for collection zkNode:delLiveColl [junit4] 2> 1500985 T5105 oasc.ZkController.createCollectionZkNode Collection zkNode exists [junit4] 2> 1500985 T5105 oascc.ZkStateReader.readConfigName Load collection config from:/collections/delLiveColl [junit4] 2> 1500985 T5111 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 1 [junit4] 2> 1500986 T5111 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={ [junit4] 2> "node_name":"127.0.0.1:11195_gllb%2Fwr", [junit4] 2> "operation":"state", [junit4] 2> "collection":"delLiveColl", [junit4] 2> "roles":null, [junit4] 2> "base_url":"https://127.0.0.1:11195/gllb/wr";, [junit4] 2> "state":"down", [junit4] 2> "core":"delLiveColl_shard2_replica2", [junit4] 2> "numShards":"2", [junit4] 2> "shard":"shard2"} [junit4] 2> 1500986 T5105 oascc.ZkStateReader.readConfigName path=/collections/delLiveColl configName=conf1 specified config exists in ZooKeeper [junit4] 2> 1500986 T5105 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-217FCCCCCF39A5F8-001/tempDir-002/delLiveColl_shard1_replica1/' [junit4] 2> 1500987 T5119 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 1500987 T5111 oasc.Overseer$ClusterStateUpdater.updateZkStates going to update_collection /collections/delLiveColl/state.json [junit4] 2> 1500988 T5119 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... [junit4] 2> 1500988 T5164 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... [junit4] 2> 1500988 T5149 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... [junit4] 2> 1500988 T5134 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... [junit4] 2> 1501027 T5164 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 2 [junit4] 2> 1501028 T5134 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 2 [junit4] 2> 1501028 T5119 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 2 [junit4] 2> 1501027 T5149 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 2 [junit4] 2> 1501030 T5111 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 2 [junit4] 2> 1501030 T5111 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={ [junit4] 2> "node_name":"127.0.0.1:11187_gllb%2Fwr", [junit4] 2> "operation":"state", [junit4] 2> "collection":"delLiveColl", [junit4] 2> "roles":null, [junit4] 2> "base_url":"https://127.0.0.1:11187/gllb/wr";, [junit4] 2> "state":"down", [junit4] 2> "core":"delLiveColl_shard2_replica1", [junit4] 2> "numShards":"2", [junit4] 2> "shard":"shard2"} [junit4] 2> 1501031 T5119 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 1501032 T5111 oasc.Overseer$ClusterStateUpdater.updateZkStates going to update_collection /collections/delLiveColl/state.json [junit4] 2> 1501032 T5119 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... [junit4] 2> 1501032 T5134 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... [junit4] 2> 1501032 T5149 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... [junit4] 2> 1501032 T5164 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... [junit4] 2> 1501033 T5134 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 3 [junit4] 2> 1501033 T5164 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 3 [junit4] 2> 1501033 T5149 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 3 [junit4] 2> 1501034 T5119 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 3 [junit4] 2> 1501036 T5111 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 3 [junit4] 2> 1501037 T5111 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={ [junit4] 2> "node_name":"127.0.0.1:11171_gllb%2Fwr", [junit4] 2> "operation":"state", [junit4] 2> "collection":"delLiveColl", [junit4] 2> "roles":null, [junit4] 2> "base_url":"https://127.0.0.1:11171/gllb/wr";, [junit4] 2> "state":"down", [junit4] 2> "core":"delLiveColl_shard1_replica2", [junit4] 2> "numShards":"2", [junit4] 2> "shard":"shard1"} [junit4] 2> 1501038 T5119 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 1501038 T5111 oasc.Overseer$ClusterStateUpdater.updateZkStates going to update_collection /collections/delLiveColl/state.json [junit4] 2> 1501039 T5119 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... [junit4] 2> 1501039 T5149 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... [junit4] 2> 1501039 T5164 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... [junit4] 2> 1501039 T5134 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... [junit4] 2> 1501040 T5149 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 4 [junit4] 2> 1501040 T5164 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 4 [junit4] 2> 1501040 T5134 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 4 [junit4] 2> 1501040 T5119 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 4 [junit4] 2> 1501136 T5105 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0 [junit4] 2> 1501182 T5105 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml [junit4] 2> 1501183 T5105 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml [junit4] 2> 1501194 T5105 oass.IndexSchema.readSchema [delLiveColl_shard1_replica1] Schema name=test [junit4] 2> 1501555 T5105 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440. [junit4] 2> 1501565 T5105 oass.IndexSchema.readSchema default search field in schema is text [junit4] 2> 1501568 T5105 oass.IndexSchema.readSchema unique key field: id [junit4] 2> 1501577 T5105 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 1501582 T5105 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 1501586 T5105 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4] 2> 1501587 T5105 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE [junit4] 2> 1501589 T5105 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING [junit4] 2> 1501589 T5105 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4] 2> 1501590 T5105 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE [junit4] 2> 1501591 T5105 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING [junit4] 2> 1501591 T5105 oasc.CoreContainer.create Creating SolrCore 'delLiveColl_shard1_replica1' using configuration from collection delLiveColl [junit4] 2> 1501591 T5105 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory [junit4] 2> 1501592 T5105 oasc.SolrCore.<init> [delLiveColl_shard1_replica1] Opening new SolrCore at ../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-217FCCCCCF39A5F8-001/tempDir-002/delLiveColl_shard1_replica1/, dataDir=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-217FCCCCCF39A5F8-001/tempDir-002/delLiveColl_shard1_replica1/data/ [junit4] 2> 1501592 T5105 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@62d962c [junit4] 2> 1501593 T5105 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-217FCCCCCF39A5F8-001/tempDir-002/delLiveColl_shard1_replica1/data [junit4] 2> 1501594 T5105 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-217FCCCCCF39A5F8-001/tempDir-002/delLiveColl_shard1_replica1/data/index/ [junit4] 2> 1501594 T5105 oasc.SolrCore.initIndex WARN [delLiveColl_shard1_replica1] Solr index directory '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-217FCCCCCF39A5F8-001/tempDir-002/delLiveColl_shard1_replica1/data/index' doesn't exist. Creating new index... [junit4] 2> 1501595 T5105 oasc.CachingDirectoryFactory.get return new directory for /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-217FCCCCCF39A5F8-001/tempDir-002/delLiveColl_shard1_replica1/data/index [junit4] 2> 1501595 T5105 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=10, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0] [junit4] 2> 1501596 T5105 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1 [junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory(id=lucene-1e61dd79)),segFN=segments_1,generation=1} [junit4] 2> 1501597 T5105 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1 [junit4] 2> 1501601 T5105 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib" [junit4] 2> 1501601 T5105 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe" [junit4] 2> 1501602 T5105 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe" [junit4] 2> 1501602 T5105 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig" [junit4] 2> 1501602 T5105 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig" [junit4] 2> 1501603 T5105 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit" [junit4] 2> 1501603 T5105 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit" [junit4] 2> 1501604 T5105 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit" [junit4] 2> 1501604 T5105 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default [junit4] 2> 1501605 T5105 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 1501605 T5105 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 1501606 T5105 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 1501606 T5105 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler [junit4] 2> 1501607 T5105 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler [junit4] 2> 1501607 T5105 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler [junit4] 2> 1501607 T5105 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler [junit4] 2> 1501608 T5105 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler [junit4] 2> 1501608 T5105 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers [junit4] 2> 1501609 T5105 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler [junit4] 2> 1501610 T5105 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler [junit4] 2> 1501629 T5105 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 1501632 T5105 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 1501634 T5105 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 1501636 T5105 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 1501643 T5105 oasu.CommitTracker.<init> Hard AutoCommit: disabled [junit4] 2> 1501643 T5105 oasu.CommitTracker.<init> Soft AutoCommit: disabled [junit4] 2> 1501644 T5105 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=10, maxMergeAtOnceExplicit=43, maxMergedSegmentMB=51.8427734375, floorSegmentMB=1.451171875, forceMergeDeletesPctAllowed=19.5665253061794, segmentsPerTier=6.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0 [junit4] 2> 1501645 T5105 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1 [junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory(id=lucene-1e61dd79)),segFN=segments_1,generation=1} [junit4] 2> 1501645 T5105 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1 [junit4] 2> 1501645 T5105 oass.SolrIndexSearcher.<init> Opening Searcher@6285fb97[delLiveColl_shard1_replica1] main [junit4] 2> 1501646 T5105 oascc.ZkStateReader.readConfigName Load collection config from:/collections/delLiveColl [junit4] 2> 1501647 T5105 oascc.ZkStateReader.readConfigName path=/collections/delLiveColl configName=conf1 specified config exists in ZooKeeper [junit4] 2> 1501647 T5105 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1 [junit4] 2> 1501648 T5105 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1 [junit4] 2> 1501648 T5105 oasr.RestManager.init Initializing RestManager with initArgs: {} [junit4] 2> 1501648 T5105 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 [junit4] 2> 1501649 T5105 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json [junit4] 2> 1501649 T5105 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 [junit4] 2> 1501650 T5105 oascc.SolrZkClient.makePath makePath: /configs/conf1/_rest_managed.json [junit4] 2> 1501651 T5105 oasr.ManagedResourceStorage$ZooKeeperStorageIO$1.close Wrote 38 bytes to new znode /configs/conf1/_rest_managed.json [junit4] 2> 1501652 T5105 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 [junit4] 2> 1501652 T5105 oasr.RestManager.init Initializing 0 registered ManagedResources [junit4] 2> 1501655 T5197 oasc.SolrCore.registerSearcher [delLiveColl_shard1_replica1] Registered new searcher Searcher@6285fb97[delLiveColl_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())} [junit4] 2> 1501658 T5105 oasc.CoreContainer.registerCore registering core: delLiveColl_shard1_replica1 [junit4] 2> 1501659 T5105 oasc.ZkController.register Register replica - core:delLiveColl_shard1_replica1 address:https://127.0.0.1:11578/gllb/wr collection:delLiveColl shard:shard1 [junit4] 2> 1501660 T5105 oascc.SolrZkClient.makePath makePath: /collections/delLiveColl/leader_elect/shard1/election [junit4] 2> 1501664 T5105 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1 [junit4] 2> 1501665 T5119 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged [junit4] 2> 1501665 T5105 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=7.4973518E15ms [junit4] 2> 1501667 T5111 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 4 [junit4] 2> 1501669 T5111 oasc.Overseer$ClusterStateUpdater.updateZkStates going to update_collection /collections/delLiveColl/state.json [junit4] 2> 1501670 T5119 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... [junit4] 2> 1501670 T5134 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... [junit4] 2> 1501670 T5164 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... [junit4] 2> 1501670 T5149 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... [junit4] 2> 1501671 T5134 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 5 [junit4] 2> 1501671 T5149 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 5 [junit4] 2> 1501671 T5164 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 5 [junit4] 2> 1501671 T5119 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 5 [junit4] 2> 1501978 T5156 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for delLiveColl_shard2_replica2 [junit4] 2> 1501978 T5141 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for delLiveColl_shard2_replica1 [junit4] 2> 1501979 T5156 oasc.ZkController.createCollectionZkNode Check for collection zkNode:delLiveColl [junit4] 2> 1501979 T5141 oasc.ZkController.createCollectionZkNode Check for collection zkNode:delLiveColl [junit4] 2> 1501980 T5156 oasc.ZkController.createCollectionZkNode Collection zkNode exists [junit4] 2> 1501980 T5141 oasc.ZkController.createCollectionZkNode Collection zkNode exists [junit4] 2> 1501980 T5156 oascc.ZkStateReader.readConfigName Load collection config from:/collections/delLiveColl [junit4] 2> 1501981 T5141 oascc.ZkStateReader.readConfigName Load collection config from:/collections/delLiveColl [junit4] 2> 1501982 T5156 oascc.ZkStateReader.readConfigName path=/collections/delLiveColl configName=conf1 specified config exists in ZooKeeper [junit4] 2> 1501982 T5156 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-217FCCCCCF39A5F8-001/tempDir-005/delLiveColl_shard2_replica2/' [junit4] 2> 1501982 T5141 oascc.ZkStateReader.readConfigName path=/collections/delLiveColl configName=conf1 specified config exists in ZooKeeper [junit4] 2> 1501982 T5126 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for delLiveColl_shard1_replica2 [junit4] 2> 1501983 T5141 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-217FCCCCCF39A5F8-001/tempDir-004/delLiveColl_shard2_replica1/' [junit4] 2> 1501983 T5126 oasc.ZkController.createCollectionZkNode Check for collection zkNode:delLiveColl [junit4] 2> 1501984 T5126 oasc.ZkController.createCollectionZkNode Collection zkNode exists [junit4] 2> 1501984 T5126 oascc.ZkStateReader.readConfigName Load collection config from:/collections/delLiveColl [junit4] 2> 1501985 T5126 oascc.ZkStateReader.readConfigName path=/collections/delLiveColl configName=conf1 specified config exists in ZooKeeper [junit4] 2> 1501985 T5126 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-217FCCCCCF39A5F8-001/tempDir-003/delLiveColl_shard1_replica2/' [junit4] 2> 1502049 T5156 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0 [junit4] 2> 1502054 T5141 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0 [junit4] 2> 1502058 T5126 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0 [junit4] 2> 1502101 T5156 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml [junit4] 2> 1502105 T5141 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml [junit4] 2> 1502109 T5126 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml [junit4] 2> 1502202 T5156 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml [junit4] 2> 1502206 T5141 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml [junit4] 2> 1502210 T5126 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml [junit4] 2> 1502213 T5156 oass.IndexSchema.readSchema [delLiveColl_shard2_replica2] Schema name=test [junit4] 2> 1502218 T5141 oass.IndexSchema.readSchema [delLiveColl_shard2_replica1] Schema name=test [junit4] 2> 1502221 T5126 oass.IndexSchema.readSchema [delLiveColl_shard1_replica2] Schema name=test [junit4] 2> 1502652 T5156 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440. [junit4] 2> 1502656 T5141 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440. [junit4] 2> 1502659 T5156 oass.IndexSchema.readSchema default search field in schema is text [junit4] 2> 1502662 T5156 oass.IndexSchema.readSchema unique key field: id [junit4] 2> 1502662 T5141 oass.IndexSchema.readSchema default search field in schema is text [junit4] 2> 1502666 T5141 oass.IndexSchema.readSchema uniq [...truncated too long message...] 2> 1699734 T5271 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:11195_gllb%2Fwr [junit4] 2> 1699738 T5258 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:11201_gllb%2Fwr [junit4] 2> 1699738 T5258 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader [junit4] 2> 1699739 T5258 oasc.Overseer.start Overseer (id=92640996769595406-127.0.0.1:11201_gllb%2Fwr-n_0000000004) starting [junit4] 2> 1699745 T5258 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000 [junit4] 2> 1699745 T5277 oasc.OverseerCollectionProcessor.run Process current queue of collection creations [junit4] 2> 1699746 T5276 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue [junit4] 2> 1701236 T5164 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK [junit4] 2> 1701238 T5086 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/gllb/wr,null} [junit4] 2> 1706096 T5086 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 11201 [junit4] 2> 1706097 T5086 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=845854624 [junit4] 2> 1706099 T5086 oasc.Overseer.close Overseer (id=92640996769595406-127.0.0.1:11201_gllb%2Fwr-n_0000000004) closing [junit4] 2> 1706099 T5276 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:11201_gllb%2Fwr [junit4] 2> 1706102 T5258 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK [junit4] 2> 1706103 T5086 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/gllb/wr,null} [junit4] 2> 1708185 T5086 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch [junit4] 2> 1708187 T5086 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:21570 21570 [junit4] 2> 1708449 T5087 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:21570 21570 [junit4] 2> 1708451 T5087 oasc.ZkTestServer$ZKServerMain.runFromConfig WARN Watch limit violations: [junit4] 2> Maximum concurrent create/delete watches above limit: [junit4] 2> [junit4] 2> 8 /solr/aliases.json [junit4] 2> 8 /solr/clusterstate.json [junit4] 2> 4 /solr/collections/delLiveColl/state.json [junit4] 2> [junit4] 2> Maximum concurrent children watches above limit: [junit4] 2> [junit4] 2> 8 /solr/live_nodes [junit4] 2> 5 /solr/overseer/collection-queue-work [junit4] 2> 5 /solr/overseer/queue [junit4] 2> [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=DeleteReplicaTest -Dtests.method=testDistribSearch -Dtests.seed=217FCCCCCF39A5F8 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=lv -Dtests.timezone=US/Hawaii -Dtests.file.encoding=UTF-8 [junit4] ERROR 227s J0 | DeleteReplicaTest.testDistribSearch <<< [junit4] > Throwable #1: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[https://127.0.0.1:11201/gllb/wr, https://127.0.0.1:11187/gllb/wr, https://127.0.0.1:11195/gllb/wr, https://127.0.0.1:11171/gllb/wr, https://127.0.0.1:11578/gllb/wr] [junit4] > at __randomizedtesting.SeedInfo.seed([217FCCCCCF39A5F8:A09942D4B866C5C4]:0) [junit4] > at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:333) [junit4] > at org.apache.solr.client.solrj.impl.CloudSolrServer.sendRequest(CloudSolrServer.java:1015) [junit4] > at org.apache.solr.client.solrj.impl.CloudSolrServer.requestWithRetryOnStaleState(CloudSolrServer.java:793) [junit4] > at org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:736) [junit4] > at org.apache.solr.cloud.DeleteReplicaTest.removeAndWaitForReplicaGone(DeleteReplicaTest.java:172) [junit4] > at org.apache.solr.cloud.DeleteReplicaTest.deleteLiveReplicaTest(DeleteReplicaTest.java:145) [junit4] > at org.apache.solr.cloud.DeleteReplicaTest.doTest(DeleteReplicaTest.java:89) [junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869) [junit4] > at java.lang.Thread.run(Thread.java:745) [junit4] > Caused by: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Could not remove replica : delLiveColl/shard1/core_node1 [junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:569) [junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:215) [junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:211) [junit4] > at org.apache.solr.client.solrj.impl.LBHttpSolrServer.doRequest(LBHttpSolrServer.java:354) [junit4] > at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:313) [junit4] > ... 47 more [junit4] 2> 1708474 T5086 oas.SolrTestCaseJ4.deleteCore ###deleteCore [junit4] 2> NOTE: leaving temporary files on disk at: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-217FCCCCCF39A5F8-001 [junit4] 2> 227319 T5085 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s). [junit4] 2> NOTE: test params are: codec=Asserting(Lucene50): {}, docValues:{}, sim=DefaultSimilarity, locale=lv, timezone=US/Hawaii [junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_65 (64-bit)/cpus=16,threads=1,free=127362992,total=299892736 [junit4] 2> NOTE: All tests run in this JVM: [TestCollectionAPI, TestCustomSort, BadComponentTest, TestRawResponseWriter, TestCloudManagedSchema, MinimalSchemaTest, TestArbitraryIndexDir, TestCoreContainer, ConnectionManagerTest, TestStressReorder, TestSolrJ, SuggesterWFSTTest, TestDefaultSimilarityFactory, ZkSolrClientTest, TestLazyCores, ShardRoutingCustomTest, BlockDirectoryTest, CacheHeaderTest, TestSolrXml, MultiThreadedOCPTest, TestUtils, TestBinaryResponseWriter, AlternateDirectoryTest, SchemaVersionSpecificBehaviorTest, TestIBSimilarityFactory, TestOrdValues, TestFaceting, DistributedFacetPivotSmallTest, TestTrie, HdfsBasicDistributedZkTest, BasicDistributedZkTest, DirectSolrConnectionTest, ResourceLoaderTest, ChangedSchemaMergeTest, ConvertedLegacyTest, TestRequestStatusCollectionAPI, DistributedQueryElevationComponentTest, TestAnalyzedSuggestions, TestSolrDeletionPolicy1, DistanceFunctionTest, BlockCacheTest, TestRangeQuery, TestFoldingMultitermQuery, TestWriterPerf, TestReloadAndDeleteDocs, TestPivotHelperCode, TestSolrIndexConfig, TestNRTOpen, SpellingQueryConverterTest, TestExpandComponent, SoftAutoCommitTest, ChaosMonkeyNothingIsSafeTest, TestFieldTypeCollectionResource, TestDefaultSearchFieldResource, CollectionsAPIDistributedZkTest, OpenCloseCoreStressTest, LeaderElectionIntegrationTest, RecoveryZkTest, TestRandomFaceting, TestZkChroot, TestRandomDVFaceting, ZkCLITest, DistributedSpellCheckComponentTest, SpellCheckComponentTest, TestGroupingSearch, TestSort, TestFiltering, BasicFunctionalityTest, OverseerCollectionProcessorTest, AnalysisAfterCoreReloadTest, TestExtendedDismaxParser, SuggesterTest, TestUpdate, TestAtomicUpdateErrorCases, QueryEqualityTest, TestQueryUtils, DirectSolrSpellCheckerTest, PrimitiveFieldTypeTest, DocumentBuilderTest, TermVectorComponentTest, FastVectorHighlighterTest, TestJmxIntegration, MBeansHandlerTest, JsonLoaderTest, PingRequestHandlerTest, CoreContainerCoreInitFailuresTest, TestQuerySenderListener, ResponseLogComponentTest, TestSolrDeletionPolicy2, TestSearchPerf, TestElisionMultitermQuery, TestFuzzyAnalyzedSuggestions, TestPostingsSolrHighlighter, TestLuceneMatchVersion, TestCodecSupport, SynonymTokenizerTest, EchoParamsTest, TestPerFieldSimilarity, TestFastWriter, OpenExchangeRatesOrgProviderTest, TestFastLRUCache, PreAnalyzedFieldTest, DateFieldTest, TestCursorMarkWithoutUniqueKey, TestDistributedMissingSort, CollectionsAPIAsyncDistributedZkTest, DeleteReplicaTest] [junit4] Completed on J0 in 227.86s, 1 test, 1 error <<< FAILURES! [...truncated 300 lines...] BUILD FAILED /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:524: The following error occurred while executing this line: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:472: The following error occurred while executing this line: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:61: The following error occurred while executing this line: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/extra-targets.xml:39: The following error occurred while executing this line: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build.xml:189: The following error occurred while executing this line: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/common-build.xml:496: The following error occurred while executing this line: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:1359: The following error occurred while executing this line: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:961: There were test failures: 437 suites, 1794 tests, 1 error, 58 ignored (22 assumptions) Total time: 81 minutes 46 seconds Build step 'Invoke Ant' marked build as failure Archiving artifacts Sending artifact delta relative to Lucene-Solr-Tests-trunk-Java7 #4924 Archived 5 artifacts Archive block size is 32768 Received 0 blocks and 156714285 bytes Compression is 0.0% Took 29 sec Recording test results Email was triggered for: Failure Sending email for trigger: Failure
--------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
