Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/1992/ Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseSerialGC (asserts: true)
1 tests failed. FAILED: org.apache.solr.cloud.DeleteReplicaTest.testDistribSearch Error Message: Should have had a good message here Stack Trace: java.lang.AssertionError: Should have had a good message here at __randomizedtesting.SeedInfo.seed([D2DD35915246F37B:533BBB8925199347]:0) at org.junit.Assert.fail(Assert.java:93) at org.junit.Assert.assertTrue(Assert.java:43) at org.apache.solr.cloud.DeleteReplicaTest.deleteLiveReplicaTest(DeleteReplicaTest.java:138) at org.apache.solr.cloud.DeleteReplicaTest.doTest(DeleteReplicaTest.java:89) at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869) at sun.reflect.GeneratedMethodAccessor48.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at java.lang.Thread.run(Thread.java:745) Build Log: [...truncated 9133 lines...] [junit4] Suite: org.apache.solr.cloud.DeleteReplicaTest [junit4] 2> Creating dataDir: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/init-core-data-001 [junit4] 2> 2349462 T7794 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (false) [junit4] 2> 2349463 T7794 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /u_ksd/q [junit4] 2> 2349471 T7794 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch [junit4] 2> 2349474 T7794 oasc.ZkTestServer.run STARTING ZK TEST SERVER [junit4] 1> client port:0.0.0.0/0.0.0.0:0 [junit4] 2> 2349476 T7795 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server [junit4] 2> 2349578 T7794 oasc.ZkTestServer.run start zk server on port:51526 [junit4] 2> 2349580 T7794 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 2349583 T7794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 2349597 T7802 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5db7158e name:ZooKeeperConnection Watcher:127.0.0.1:51526 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 2349598 T7794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 2349598 T7794 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 2349599 T7794 oascc.SolrZkClient.makePath makePath: /solr [junit4] 2> 2349608 T7796 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x14a4308b7540000, likely client has closed socket [junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228) [junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208) [junit4] 2> at java.lang.Thread.run(Thread.java:745) [junit4] 2> [junit4] 2> 2349611 T7794 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 2349612 T7794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 2349616 T7805 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@63d3c48f name:ZooKeeperConnection Watcher:127.0.0.1:51526/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 2349616 T7794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 2349617 T7794 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 2349617 T7794 oascc.SolrZkClient.makePath makePath: /collections/collection1 [junit4] 2> 2349623 T7794 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards [junit4] 2> 2349629 T7794 oascc.SolrZkClient.makePath makePath: /collections/control_collection [junit4] 2> 2349635 T7794 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards [junit4] 2> 2349641 T7794 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml [junit4] 2> 2349642 T7794 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml [junit4] 2> 2349652 T7794 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml [junit4] 2> 2349653 T7794 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml [junit4] 2> 2349660 T7794 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml [junit4] 2> 2349661 T7794 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml [junit4] 2> 2349666 T7794 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt [junit4] 2> 2349667 T7794 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt [junit4] 2> 2349673 T7794 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt [junit4] 2> 2349673 T7794 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt [junit4] 2> 2349679 T7794 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml [junit4] 2> 2349680 T7794 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml [junit4] 2> 2349687 T7794 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml [junit4] 2> 2349688 T7794 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml [junit4] 2> 2349696 T7794 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json [junit4] 2> 2349697 T7794 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json [junit4] 2> 2349703 T7794 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt [junit4] 2> 2349704 T7794 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt [junit4] 2> 2349712 T7794 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt [junit4] 2> 2349712 T7794 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt [junit4] 2> 2349720 T7794 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt [junit4] 2> 2349721 T7794 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt [junit4] 2> 2351605 T7794 oejs.Server.doStart jetty-8.1.10.v20130312 [junit4] 2> 2351643 T7794 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:51529 [junit4] 2> 2351644 T7794 oass.SolrDispatchFilter.init SolrDispatchFilter.init() [junit4] 2> 2351644 T7794 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 2351644 T7794 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-002 [junit4] 2> 2351645 T7794 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-002/' [junit4] 2> 2351692 T7794 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-002/solr.xml [junit4] 2> 2351716 T7794 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-002 [junit4] 2> 2351717 T7794 oasc.CoreContainer.<init> New CoreContainer 1704627795 [junit4] 2> 2351717 T7794 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-002/] [junit4] 2> 2351718 T7794 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000 [junit4] 2> 2351719 T7794 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: [junit4] 2> 2351720 T7794 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000 [junit4] 2> 2351720 T7794 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20 [junit4] 2> 2351721 T7794 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000 [junit4] 2> 2351721 T7794 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0 [junit4] 2> 2351722 T7794 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647 [junit4] 2> 2351722 T7794 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5 [junit4] 2> 2351723 T7794 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1 [junit4] 2> 2351723 T7794 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false [junit4] 2> 2351724 T7794 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false [junit4] 2> 2351725 T7794 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 2351728 T7794 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 2351728 T7794 oasc.CoreContainer.load Host Name: 127.0.0.1 [junit4] 2> 2351729 T7794 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51526/solr [junit4] 2> 2351730 T7794 oasc.ZkController.checkChrootPath zkHost includes chroot [junit4] 2> 2351730 T7794 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 2351733 T7794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 2351739 T7817 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4355027c name:ZooKeeperConnection Watcher:127.0.0.1:51526 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 2351739 T7794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 2351739 T7794 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 2351751 T7794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 2351766 T7820 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@62686eca name:ZooKeeperConnection Watcher:127.0.0.1:51526/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 2351767 T7794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 2351772 T7794 oascc.SolrZkClient.makePath makePath: /overseer/queue [junit4] 2> 2351781 T7794 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work [junit4] 2> 2351790 T7794 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running [junit4] 2> 2351800 T7794 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed [junit4] 2> 2351811 T7794 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure [junit4] 2> 2351821 T7794 oascc.SolrZkClient.makePath makePath: /live_nodes [junit4] 2> 2351827 T7794 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51529_u_ksd%2Fq [junit4] 2> 2351831 T7794 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51529_u_ksd%2Fq [junit4] 2> 2351839 T7794 oascc.SolrZkClient.makePath makePath: /overseer_elect [junit4] 2> 2351846 T7794 oascc.SolrZkClient.makePath makePath: /overseer_elect/election [junit4] 2> 2351852 T7794 oasc.Overseer.close Overseer (id=null) closing [junit4] 2> 2351858 T7794 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:51529_u_ksd%2Fq [junit4] 2> 2351859 T7794 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader [junit4] 2> 2351865 T7794 oasc.Overseer.start Overseer (id=92960447029051395-127.0.0.1:51529_u_ksd%2Fq-n_0000000000) starting [junit4] 2> 2351873 T7794 oascc.SolrZkClient.makePath makePath: /overseer/queue-work [junit4] 2> 2351902 T7794 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000 [junit4] 2> 2351904 T7822 oasc.OverseerCollectionProcessor.run Process current queue of collection creations [junit4] 2> 2351908 T7794 oascc.SolrZkClient.makePath makePath: /clusterstate.json [junit4] 2> 2351914 T7794 oascc.SolrZkClient.makePath makePath: /aliases.json [junit4] 2> 2351919 T7794 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 2351929 T7821 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue [junit4] 2> 2351932 T7794 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-002 [junit4] 2> 2352803 T7794 oasc.CorePropertiesLocator.discoverUnder Found core conf in /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-002/conf/ [junit4] 2> 2352922 T7794 oasc.CorePropertiesLocator.discover Found 1 core definitions [junit4] 2> 2352925 T7794 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0 [junit4] 2> 2352925 T7794 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done [junit4] 2> 2352926 T7794 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 2352927 T7794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 2352935 T7827 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@96dddff name:ZooKeeperConnection Watcher:127.0.0.1:51526/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 2352936 T7794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 2352936 T7794 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 2352940 T7794 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 2352952 T7794 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false [junit4] 2> 2352953 T7794 oasc.AbstractFullDistribZkTestBase.createJettys Creating collection1 with stateFormat=2 [junit4] 2> 2352953 T7794 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 2352955 T7794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 2352964 T7830 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@da60823 name:ZooKeeperConnection Watcher:127.0.0.1:51526/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 2352964 T7794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 2352964 T7794 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 2352971 T7820 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 2352973 T7796 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x14a4308b7540005, likely client has closed socket [junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228) [junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208) [junit4] 2> at java.lang.Thread.run(Thread.java:745) [junit4] 2> [junit4] 2> 2353011 T7821 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 0, message = { [junit4] 2> "operation":"create", [junit4] 2> "name":"collection1", [junit4] 2> "numShards":"2", [junit4] 2> "stateFormat":2} current state version: 0 [junit4] 2> 2353012 T7821 oasco.ClusterStateMutator.createCollection building a new cName: collection1 [junit4] 2> 2353013 T7821 oasco.ZkStateWriter.writePendingUpdates going to create_collection /collections/collection1/state.json [junit4] 2> 2353019 T7820 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1) [junit4] 2> 2353019 T7827 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1) [junit4] 2> 2354899 T7794 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1 [junit4] 2> 2354900 T7794 oejs.Server.doStart jetty-8.1.10.v20130312 [junit4] 2> 2354908 T7794 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:51534 [junit4] 2> 2354910 T7794 oass.SolrDispatchFilter.init SolrDispatchFilter.init() [junit4] 2> 2354910 T7794 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 2354911 T7794 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-003 [junit4] 2> 2354911 T7794 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-003/' [junit4] 2> 2354955 T7794 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-003/solr.xml [junit4] 2> 2354968 T7794 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-003 [junit4] 2> 2354969 T7794 oasc.CoreContainer.<init> New CoreContainer 1474863892 [junit4] 2> 2354969 T7794 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-003/] [junit4] 2> 2354970 T7794 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000 [junit4] 2> 2354970 T7794 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: [junit4] 2> 2354971 T7794 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000 [junit4] 2> 2354971 T7794 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20 [junit4] 2> 2354971 T7794 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000 [junit4] 2> 2354972 T7794 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0 [junit4] 2> 2354972 T7794 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647 [junit4] 2> 2354972 T7794 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5 [junit4] 2> 2354973 T7794 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1 [junit4] 2> 2354973 T7794 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false [junit4] 2> 2354973 T7794 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false [junit4] 2> 2354974 T7794 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 2354975 T7794 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 2354975 T7794 oasc.CoreContainer.load Host Name: 127.0.0.1 [junit4] 2> 2354975 T7794 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51526/solr [junit4] 2> 2354975 T7794 oasc.ZkController.checkChrootPath zkHost includes chroot [junit4] 2> 2354976 T7794 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 2354978 T7794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 2354983 T7842 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@40414610 name:ZooKeeperConnection Watcher:127.0.0.1:51526 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 2354983 T7794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 2354984 T7794 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 2354991 T7794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 2354996 T7845 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@45351664 name:ZooKeeperConnection Watcher:127.0.0.1:51526/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 2354997 T7794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 2355016 T7794 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 2356034 T7794 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51534_u_ksd%2Fq [junit4] 2> 2356039 T7794 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51534_u_ksd%2Fq [junit4] 2> 2356057 T7794 oasc.Overseer.close Overseer (id=null) closing [junit4] 2> 2356064 T7794 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-003 [junit4] 2> 2356701 T7794 oasc.CorePropertiesLocator.discoverUnder Found core conf in /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-003/conf/ [junit4] 2> 2356806 T7794 oasc.CorePropertiesLocator.discover Found 1 core definitions [junit4] 2> 2356809 T7794 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0 [junit4] 2> 2356810 T7794 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done [junit4] 2> 2358880 T7794 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2 [junit4] 2> 2358880 T7794 oejs.Server.doStart jetty-8.1.10.v20130312 [junit4] 2> 2358888 T7794 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:51537 [junit4] 2> 2358892 T7794 oass.SolrDispatchFilter.init SolrDispatchFilter.init() [junit4] 2> 2358893 T7794 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 2358894 T7794 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-004 [junit4] 2> 2358895 T7794 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-004/' [junit4] 2> 2358938 T7794 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-004/solr.xml [junit4] 2> 2358955 T7794 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-004 [junit4] 2> 2358955 T7794 oasc.CoreContainer.<init> New CoreContainer 1886158987 [junit4] 2> 2358956 T7794 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-004/] [junit4] 2> 2358956 T7794 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000 [junit4] 2> 2358957 T7794 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: [junit4] 2> 2358958 T7794 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000 [junit4] 2> 2358959 T7794 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20 [junit4] 2> 2358959 T7794 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000 [junit4] 2> 2358959 T7794 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0 [junit4] 2> 2358960 T7794 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647 [junit4] 2> 2358960 T7794 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5 [junit4] 2> 2358961 T7794 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1 [junit4] 2> 2358961 T7794 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false [junit4] 2> 2358962 T7794 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false [junit4] 2> 2358963 T7794 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 2358963 T7794 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 2358963 T7794 oasc.CoreContainer.load Host Name: 127.0.0.1 [junit4] 2> 2358964 T7794 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51526/solr [junit4] 2> 2358964 T7794 oasc.ZkController.checkChrootPath zkHost includes chroot [junit4] 2> 2358964 T7794 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 2358966 T7794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 2358971 T7858 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@16529c7a name:ZooKeeperConnection Watcher:127.0.0.1:51526 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 2358971 T7794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 2358972 T7794 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 2358985 T7794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 2358989 T7861 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5ff258ba name:ZooKeeperConnection Watcher:127.0.0.1:51526/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 2358990 T7794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 2359065 T7794 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 2360080 T7794 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51537_u_ksd%2Fq [junit4] 2> 2360082 T7794 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51537_u_ksd%2Fq [junit4] 2> 2360094 T7794 oasc.Overseer.close Overseer (id=null) closing [junit4] 2> 2360103 T7794 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-004 [junit4] 2> 2360873 T7794 oasc.CorePropertiesLocator.discoverUnder Found core conf in /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-004/conf/ [junit4] 2> 2360987 T7794 oasc.CorePropertiesLocator.discover Found 1 core definitions [junit4] 2> 2360990 T7794 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0 [junit4] 2> 2360991 T7794 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done [junit4] 2> 2362993 T7794 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3 [junit4] 2> 2362995 T7794 oejs.Server.doStart jetty-8.1.10.v20130312 [junit4] 2> 2363007 T7794 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:51540 [junit4] 2> 2363009 T7794 oass.SolrDispatchFilter.init SolrDispatchFilter.init() [junit4] 2> 2363011 T7794 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 2363012 T7794 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-005 [junit4] 2> 2363012 T7794 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-005/' [junit4] 2> 2363066 T7794 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-005/solr.xml [junit4] 2> 2363078 T7794 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-005 [junit4] 2> 2363079 T7794 oasc.CoreContainer.<init> New CoreContainer 1251588988 [junit4] 2> 2363079 T7794 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-005/] [junit4] 2> 2363080 T7794 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000 [junit4] 2> 2363080 T7794 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: [junit4] 2> 2363081 T7794 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000 [junit4] 2> 2363081 T7794 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20 [junit4] 2> 2363082 T7794 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000 [junit4] 2> 2363082 T7794 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0 [junit4] 2> 2363082 T7794 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647 [junit4] 2> 2363082 T7794 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5 [junit4] 2> 2363083 T7794 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1 [junit4] 2> 2363084 T7794 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false [junit4] 2> 2363084 T7794 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false [junit4] 2> 2363085 T7794 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 2363085 T7794 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 2363086 T7794 oasc.CoreContainer.load Host Name: 127.0.0.1 [junit4] 2> 2363086 T7794 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51526/solr [junit4] 2> 2363086 T7794 oasc.ZkController.checkChrootPath zkHost includes chroot [junit4] 2> 2363087 T7794 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 2363088 T7794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 2363095 T7874 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@f519a4f name:ZooKeeperConnection Watcher:127.0.0.1:51526 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 2363096 T7794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 2363096 T7794 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 2363103 T7794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 2363112 T7877 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@10e4893c name:ZooKeeperConnection Watcher:127.0.0.1:51526/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 2363112 T7794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 2363147 T7794 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 2364169 T7794 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51540_u_ksd%2Fq [junit4] 2> 2364173 T7794 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51540_u_ksd%2Fq [junit4] 2> 2364193 T7794 oasc.Overseer.close Overseer (id=null) closing [junit4] 2> 2364203 T7794 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-005 [junit4] 2> 2365068 T7794 oasc.CorePropertiesLocator.discoverUnder Found core conf in /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-005/conf/ [junit4] 2> 2365190 T7794 oasc.CorePropertiesLocator.discover Found 1 core definitions [junit4] 2> 2365192 T7794 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0 [junit4] 2> 2365193 T7794 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done [junit4] 2> 2367105 T7794 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 4 [junit4] 2> 2367107 T7794 oejs.Server.doStart jetty-8.1.10.v20130312 [junit4] 2> 2367114 T7794 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:51543 [junit4] 2> 2367116 T7794 oass.SolrDispatchFilter.init SolrDispatchFilter.init() [junit4] 2> 2367116 T7794 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 2367117 T7794 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-006 [junit4] 2> 2367117 T7794 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-006/' [junit4] 2> 2367166 T7794 oasc.ConfigSolr.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-006/solr.xml [junit4] 2> 2367179 T7794 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-006 [junit4] 2> 2367180 T7794 oasc.CoreContainer.<init> New CoreContainer 1777461615 [junit4] 2> 2367180 T7794 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-006/] [junit4] 2> 2367181 T7794 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000 [junit4] 2> 2367181 T7794 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: [junit4] 2> 2367181 T7794 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000 [junit4] 2> 2367182 T7794 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20 [junit4] 2> 2367182 T7794 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000 [junit4] 2> 2367183 T7794 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0 [junit4] 2> 2367183 T7794 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647 [junit4] 2> 2367183 T7794 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5 [junit4] 2> 2367184 T7794 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1 [junit4] 2> 2367184 T7794 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false [junit4] 2> 2367185 T7794 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false [junit4] 2> 2367185 T7794 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 2367186 T7794 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 2367186 T7794 oasc.CoreContainer.load Host Name: 127.0.0.1 [junit4] 2> 2367187 T7794 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51526/solr [junit4] 2> 2367187 T7794 oasc.ZkController.checkChrootPath zkHost includes chroot [junit4] 2> 2367187 T7794 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 2367191 T7794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 2367194 T7890 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6b051b46 name:ZooKeeperConnection Watcher:127.0.0.1:51526 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 2367194 T7794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 2367194 T7794 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 2367201 T7794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 2367205 T7893 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@679495f2 name:ZooKeeperConnection Watcher:127.0.0.1:51526/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 2367206 T7794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 2367234 T7794 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 2368260 T7794 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51543_u_ksd%2Fq [junit4] 2> 2368263 T7794 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51543_u_ksd%2Fq [junit4] 2> 2368281 T7794 oasc.Overseer.close Overseer (id=null) closing [junit4] 2> 2368805 T7794 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-006 [junit4] 2> 2369621 T7794 oasc.CorePropertiesLocator.discoverUnder Found core conf in /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-006/conf/ [junit4] 2> 2369742 T7794 oasc.CorePropertiesLocator.discover Found 1 core definitions [junit4] 2> 2369745 T7794 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0 [junit4] 2> 2369745 T7794 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done [junit4] 2> 2369750 T7794 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 2369751 T7794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 2369756 T7897 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4b82547b name:ZooKeeperConnection Watcher:127.0.0.1:51526/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 2369756 T7794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 2369756 T7794 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 2369760 T7794 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 2369771 T7794 oasc.AbstractFullDistribZkTestBase.createCollection Creating collection with stateFormat=2: delLiveColl [junit4] 2> 2369771 T7794 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 2369773 T7794 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 2369777 T7900 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4f250410 name:ZooKeeperConnection Watcher:127.0.0.1:51526/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 2369777 T7794 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 2369777 T7794 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 2369781 T7794 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 2369804 T7882 oasha.CollectionsHandler.handleCreateAction Creating Collection : replicationFactor=2&maxShardsPerNode=2&name=delLiveColl&action=CREATE&numShards=2&stateFormat=2&wt=javabin&version=2 [junit4] 2> 2369806 T7820 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/collection-queue-work state SyncConnected [junit4] 2> 2369808 T7822 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{ [junit4] 2> "operation":"create", [junit4] 2> "fromApi":"true", [junit4] 2> "name":"delLiveColl", [junit4] 2> "replicationFactor":"2", [junit4] 2> "numShards":"2", [junit4] 2> "maxShardsPerNode":"2", [junit4] 2> "stateFormat":"2"} [junit4] 2> 2369814 T7901 oasc.OverseerCollectionProcessor.processMessage WARN OverseerCollectionProcessor.processMessage : create , { [junit4] 2> "operation":"create", [junit4] 2> "fromApi":"true", [junit4] 2> "name":"delLiveColl", [junit4] 2> "replicationFactor":"2", [junit4] 2> "numShards":"2", [junit4] 2> "maxShardsPerNode":"2", [junit4] 2> "stateFormat":"2"} [junit4] 2> 2369818 T7901 oasc.OverseerCollectionProcessor.createConfNode Only one config set found in zk - using it:conf1 [junit4] 2> 2369819 T7901 oasc.OverseerCollectionProcessor.createConfNode creating collections conf node /collections/delLiveColl [junit4] 2> 2369820 T7901 oascc.SolrZkClient.makePath makePath: /collections/delLiveColl [junit4] 2> 2369828 T7820 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 2369831 T7821 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = { [junit4] 2> "operation":"create", [junit4] 2> "fromApi":"true", [junit4] 2> "name":"delLiveColl", [junit4] 2> "replicationFactor":"2", [junit4] 2> "numShards":"2", [junit4] 2> "maxShardsPerNode":"2", [junit4] 2> "stateFormat":"2"} current state version: 1 [junit4] 2> 2369831 T7821 oasco.ClusterStateMutator.createCollection building a new cName: delLiveColl [junit4] 2> 2369832 T7821 oasco.ZkStateWriter.writePendingUpdates going to create_collection /collections/delLiveColl/state.json [junit4] 2> 2369840 T7893 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4] 2> 2369840 T7897 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4] 2> 2369840 T7900 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4] 2> 2369840 T7877 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4] 2> 2369841 T7820 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4] 2> 2369840 T7861 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4] 2> 2369840 T7827 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4] 2> 2369840 T7845 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5) [junit4] 2> 2370034 T7901 oasc.OverseerCollectionProcessor.createCollection Creating SolrCores for new collection delLiveColl, shardNames [shard1, shard2] , replicationFactor : 2 [junit4] 2> 2370034 T7901 oasc.OverseerCollectionProcessor.createCollection Creating shard delLiveColl_shard1_replica1 as part of slice shard1 of collection delLiveColl on 127.0.0.1:51540_u_ksd%2Fq [junit4] 2> 2370036 T7901 oasc.OverseerCollectionProcessor.createCollection Creating shard delLiveColl_shard1_replica2 as part of slice shard1 of collection delLiveColl on 127.0.0.1:51529_u_ksd%2Fq [junit4] 2> 2370040 T7901 oasc.OverseerCollectionProcessor.createCollection Creating shard delLiveColl_shard2_replica1 as part of slice shard2 of collection delLiveColl on 127.0.0.1:51537_u_ksd%2Fq [junit4] 2> 2370052 T7871 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=delLiveColl_shard1_replica1&action=CREATE&numShards=2&collection=delLiveColl&shard=shard1&wt=javabin&version=2 [junit4] 2> 2370056 T7809 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=delLiveColl_shard1_replica2&action=CREATE&numShards=2&collection=delLiveColl&shard=shard1&wt=javabin&version=2 [junit4] 2> 2370058 T7901 oasc.OverseerCollectionProcessor.createCollection Creating shard delLiveColl_shard2_replica2 as part of slice shard2 of collection delLiveColl on 127.0.0.1:51534_u_ksd%2Fq [junit4] 2> 2370062 T7871 oasc.ZkController.publish publishing core=delLiveColl_shard1_replica1 state=down collection=delLiveColl [junit4] 2> 2370069 T7871 oasc.ZkController.preRegister Registering watch for external collection delLiveColl [junit4] 2> 2370070 T7871 oascc.ZkStateReader.addZkWatch addZkWatch delLiveColl [junit4] 2> 2370070 T7820 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 2370073 T7853 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=delLiveColl_shard2_replica1&action=CREATE&numShards=2&collection=delLiveColl&shard=shard2&wt=javabin&version=2 [junit4] 2> 2370075 T7871 oascc.ZkStateReader.addZkWatch Updating collection state at /collections/delLiveColl/state.json from ZooKeeper... [junit4] 2> 2370079 T7853 oasc.ZkController.publish publishing core=delLiveColl_shard2_replica1 state=down collection=delLiveColl [junit4] 2> 2370085 T7821 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = { [junit4] 2> "core":"delLiveColl_shard1_replica1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"http://127.0.0.1:51540/u_ksd/q", [junit4] 2> "node_name":"127.0.0.1:51540_u_ksd%2Fq", [junit4] 2> "numShards":"2", [junit4] 2> "state":"down", [junit4] 2> "shard":"shard1", [junit4] 2> "collection":"delLiveColl", [junit4] 2> "operation":"state"} current state version: 2 [junit4] 2> 2370090 T7834 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=delLiveColl_shard2_replica2&action=CREATE&numShards=2&collection=delLiveColl&shard=shard2&wt=javabin&version=2 [junit4] 2> 2370092 T7871 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 0 [junit4] 2> 2370092 T7871 oasc.ZkController.waitForCoreNodeName look for our core node name [junit4] 2> 2370096 T7809 oasc.ZkController.publish publishing core=delLiveColl_shard1_replica2 state=down collection=delLiveColl [junit4] 2> 2370097 T7821 oasco.ReplicaMutator.updateState Update state numShards=2 message={ [junit4] 2> "core":"delLiveColl_shard1_replica1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"http://127.0.0.1:51540/u_ksd/q", [junit4] 2> "node_name":"127.0.0.1:51540_u_ksd%2Fq", [junit4] 2> "numShards":"2", [junit4] 2> "state":"down", [junit4] 2> "shard":"shard1", [junit4] 2> "collection":"delLiveColl", [junit4] 2> "operation":"state"} [junit4] 2> 2370098 T7834 oasc.ZkController.publish publishing core=delLiveColl_shard2_replica2 state=down collection=delLiveColl [junit4] 2> 2370101 T7853 oasc.ZkController.preRegister Registering watch for external collection delLiveColl [junit4] 2> 2370101 T7853 oascc.ZkStateReader.addZkWatch addZkWatch delLiveColl [junit4] 2> 2370104 T7853 oascc.ZkStateReader.addZkWatch Updating collection state at /collections/delLiveColl/state.json from ZooKeeper... [junit4] 2> 2370107 T7834 oasc.ZkController.preRegister Registering watch for external collection delLiveColl [junit4] 2> 2370108 T7834 oascc.ZkStateReader.addZkWatch addZkWatch delLiveColl [junit4] 2> 2370108 T7853 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 0 [junit4] 2> 2370110 T7853 oasc.ZkController.waitForCoreNodeName look for our core node name [junit4] 2> 2370110 T7834 oascc.ZkStateReader.addZkWatch Updating collection state at /collections/delLiveColl/state.json from ZooKeeper... [junit4] 2> 2370112 T7809 oasc.ZkController.preRegister Registering watch for external collection delLiveColl [junit4] 2> 2370113 T7821 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 3, message = { [junit4] 2> "core":"delLiveColl_shard2_replica1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"http://127.0.0.1:51537/u_ksd/q", [junit4] 2> "node_name":"127.0.0.1:51537_u_ksd%2Fq", [junit4] 2> "numShards":"2", [junit4] 2> "state":"down", [junit4] 2> "shard":"shard2", [junit4] 2> "collection":"delLiveColl", [junit4] 2> "operation":"state"} current state version: 2 [junit4] 2> 2370115 T7821 oasco.ReplicaMutator.updateState Update state numShards=2 message={ [junit4] 2> "core":"delLiveColl_shard2_replica1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"http://127.0.0.1:51537/u_ksd/q", [junit4] 2> "node_name":"127.0.0.1:51537_u_ksd%2Fq", [junit4] 2> "numShards":"2", [junit4] 2> "state":"down", [junit4] 2> "shard":"shard2", [junit4] 2> "collection":"delLiveColl", [junit4] 2> "operation":"state"} [junit4] 2> 2370116 T7834 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 0 [junit4] 2> 2370116 T7834 oasc.ZkController.waitForCoreNodeName look for our core node name [junit4] 2> 2370121 T7820 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 2370123 T7821 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 3, message = { [junit4] 2> "core":"delLiveColl_shard2_replica2", [junit4] 2> "roles":null, [junit4] 2> "base_url":"http://127.0.0.1:51534/u_ksd/q", [junit4] 2> "node_name":"127.0.0.1:51534_u_ksd%2Fq", [junit4] 2> "numShards":"2", [junit4] 2> "state":"down", [junit4] 2> "shard":"shard2", [junit4] 2> "collection":"delLiveColl", [junit4] 2> "operation":"state"} current state version: 2 [junit4] 2> 2370125 T7821 oasco.ReplicaMutator.updateState Update state numShards=2 message={ [junit4] 2> "core":"delLiveColl_shard2_replica2", [junit4] 2> "roles":null, [junit4] 2> "base_url":"http://127.0.0.1:51534/u_ksd/q", [junit4] 2> "node_name":"127.0.0.1:51534_u_ksd%2Fq", [junit4] 2> "numShards":"2", [junit4] 2> "state":"down", [junit4] 2> "shard":"shard2", [junit4] 2> "collection":"delLiveColl", [junit4] 2> "operation":"state"} [junit4] 2> 2370134 T7821 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = { [junit4] 2> "core":"delLiveColl_shard1_replica2", [junit4] 2> "roles":null, [junit4] 2> "base_url":"http://127.0.0.1:51529/u_ksd/q", [junit4] 2> "node_name":"127.0.0.1:51529_u_ksd%2Fq", [junit4] 2> "numShards":"2", [junit4] 2> "state":"down", [junit4] 2> "shard":"shard1", [junit4] 2> "collection":"delLiveColl", [junit4] 2> "operation":"state"} current state version: 2 [junit4] 2> 2370135 T7821 oasco.ReplicaMutator.updateState Update state numShards=2 message={ [junit4] 2> "core":"delLiveColl_shard1_replica2", [junit4] 2> "roles":null, [junit4] 2> "base_url":"http://127.0.0.1:51529/u_ksd/q", [junit4] 2> "node_name":"127.0.0.1:51529_u_ksd%2Fq", [junit4] 2> "numShards":"2", [junit4] 2> "state":"down", [junit4] 2> "shard":"shard1", [junit4] 2> "collection":"delLiveColl", [junit4] 2> "operation":"state"} [junit4] 2> 2370141 T7820 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 2370144 T7821 oasco.ZkStateWriter.writePendingUpdates going to update_collection /collections/delLiveColl/state.json version: 0 [junit4] 2> 2370147 T7845 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... [junit4] 2> 2370147 T7861 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... [junit4] 2> 2370147 T7877 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... [junit4] 2> 2370151 T7861 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 1 [junit4] 2> 2370151 T7877 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 1 [junit4] 2> 2370152 T7845 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 1 [junit4] 2> 2370166 T7809 oascc.ZkStateReader.addZkWatch addZkWatch delLiveColl [junit4] 2> 2370168 T7809 oascc.ZkStateReader.addZkWatch Updating collection state at /collections/delLiveColl/state.json from ZooKeeper... [junit4] 2> 2370173 T7809 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 1 [junit4] 2> 2370173 T7809 oasc.ZkController.waitForCoreNodeName look for our core node name [junit4] 2> 2370174 T7809 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for delLiveColl_shard1_replica2 [junit4] 2> 2370174 T7809 oasc.ZkController.createCollectionZkNode Check for collection zkNode:delLiveColl [junit4] 2> 2370175 T7809 oasc.ZkController.createCollectionZkNode Collection zkNode exists [junit4] 2> 2370176 T7809 oascc.ZkStateReader.readConfigName Load collection config from:/collections/delLiveColl [junit4] 2> 2370178 T7809 oascc.ZkStateReader.readConfigName path=/collections/delLiveColl configName=conf1 specified config exists in ZooKeeper [junit4] 2> 2370178 T7809 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-002/delLiveColl_shard1_replica2/' [junit4] 2> 2370262 T7809 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1 [junit4] 2> 2370273 T7809 oasc.Config.<init> loaded config solrconfig.xml with version 0 [junit4] 2> 2370299 T7809 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0 [junit4] 2> 2370320 T7809 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml [junit4] 2> 2370324 T7809 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml [junit4] 2> 2370333 T7809 oass.IndexSchema.readSchema [delLiveColl_shard1_replica2] Schema name=test [junit4] 2> 2370501 T7809 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440. [junit4] 2> 2370510 T7809 oass.IndexSchema.readSchema default search field in schema is text [junit4] 2> 2370511 T7809 oass.IndexSchema.readSchema unique key field: id [junit4] 2> 2370549 T7809 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 2370553 T7809 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 2370558 T7809 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4] 2> 2370560 T7809 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE [junit4] 2> 2370564 T7809 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING [junit4] 2> 2370565 T7809 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4] 2> 2370567 T7809 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE [junit4] 2> 2370567 T7809 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING [junit4] 2> 2370568 T7809 oasc.CoreContainer.create Creating SolrCore 'delLiveColl_shard1_replica2' using configuration from collection delLiveColl [junit4] 2> 2370568 T7809 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory [junit4] 2> 2370569 T7809 oasc.SolrCore.<init> [delLiveColl_shard1_replica2] Opening new SolrCore at ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-002/delLiveColl_shard1_replica2/, dataDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-002/delLiveColl_shard1_replica2/data/ [junit4] 2> 2370570 T7809 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4b62dc70 [junit4] 2> 2370571 T7809 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-002/delLiveColl_shard1_replica2/data [junit4] 2> 2370572 T7809 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-002/delLiveColl_shard1_replica2/data/index/ [junit4] 2> 2370572 T7809 oasc.SolrCore.initIndex WARN [delLiveColl_shard1_replica2] Solr index directory '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-002/delLiveColl_shard1_replica2/data/index' doesn't exist. Creating new index... [junit4] 2> 2370572 T7809 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-002/delLiveColl_shard1_replica2/data/index [junit4] 2> 2370573 T7809 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=41, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8412472851871099] [junit4] 2> 2370574 T7809 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1 [junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1625cc00 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@16777a6d),segFN=segments_1,generation=1} [junit4] 2> 2370574 T7809 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1 [junit4] 2> 2370581 T7809 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib" [junit4] 2> 2370582 T7809 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe" [junit4] 2> 2370582 T7809 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe" [junit4] 2> 2370583 T7809 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig" [junit4] 2> 2370583 T7809 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig" [junit4] 2> 2370584 T7809 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit" [junit4] 2> 2370585 T7809 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit" [junit4] 2> 2370586 T7809 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit" [junit4] 2> 2370586 T7809 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default [junit4] 2> 2370587 T7809 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 2370587 T7809 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 2370588 T7809 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 2370589 T7809 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 2370590 T7809 oasc.RequestHandlers.initHandlersFromConfig created /config: org.apache.solr.handler.SolrConfigHandler [junit4] 2> 2370591 T7809 oasc.RequestHandlers.initHandlersFromConfig created /schema: org.apache.solr.handler.SchemaHandler [junit4] 2> 2370591 T7809 oasc.RequestHandlers.initHandlersFromConfig created /replication: org.apache.solr.handler.ReplicationHandler [junit4] 2> 2370592 T7809 oasc.RequestHandlers.initHandlersFromConfig created /get: org.apache.solr.handler.RealTimeGetHandler [junit4] 2> 2370592 T7809 oasc.RequestHandlers.initHandlersFromConfig created /admin/luke: org.apache.solr.handler.admin.LukeRequestHandler [junit4] 2> 2370593 T7809 oasc.RequestHandlers.initHandlersFromConfig created /admin/system: org.apache.solr.handler.admin.SystemInfoHandler [junit4] 2> 2370594 T7809 oasc.RequestHandlers.initHandlersFromConfig created /admin/mbeans: org.apache.solr.handler.admin.SolrInfoMBeanHandler [junit4] 2> 2370594 T7809 oasc.RequestHandlers.initHandlersFromConfig created /admin/plugins: org.apache.solr.handler.admin.PluginInfoHandler [junit4] 2> 2370595 T7809 oasc.RequestHandlers.initHandlersFromConfig created /admin/threads: org.apache.solr.handler.admin.ThreadDumpHandler [junit4] 2> 2370596 T7809 oasc.RequestHandlers.initHandlersFromConfig created /admin/properties: org.apache.solr.handler.admin.PropertiesRequestHandler [junit4] 2> 2370596 T7809 oasc.RequestHandlers.initHandlersFromConfig created /admin/logging: org.apache.solr.handler.admin.LoggingHandler [junit4] 2> 2370597 T7809 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: org.apache.solr.handler.PingRequestHandler [junit4] 2> 2370597 T7809 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler [junit4] 2> 2370598 T7809 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler [junit4] 2> 2370598 T7809 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler [junit4] 2> 2370608 T7809 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 2370609 T7809 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 2370610 T7809 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 2370612 T7809 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 2370618 T7809 oasu.CommitTracker.<init> Hard AutoCommit: disabled [junit4] 2> 2370618 T7809 oasu.CommitTracker.<init> Soft AutoCommit: disabled [junit4] 2> 2370620 T7809 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=42, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0] [junit4] 2> 2370621 T7809 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1 [junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1625cc00 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@16777a6d),segFN=segments_1,generation=1} [junit4] 2> 2370622 T7809 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1 [junit4] 2> 2370623 T7809 oass.SolrIndexSearcher.<init> Opening Searcher@580ad4b8[delLiveColl_shard1_replica2] main [junit4] 2> 2370623 T7809 oascc.ZkStateReader.readConfigName Load collection config from:/collections/delLiveColl [junit4] 2> 2370626 T7809 oascc.ZkStateReader.readConfigName path=/collections/delLiveColl configName=conf1 specified config exists in ZooKeeper [junit4] 2> 2370627 T7809 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1 [junit4] 2> 2370628 T7809 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1 [junit4] 2> 2370629 T7809 oasr.RestManager.init Initializing RestManager with initArgs: {} [junit4] 2> 2370629 T7809 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 [junit4] 2> 2370630 T7809 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json [junit4] 2> 2370631 T7809 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 [junit4] 2> 2370632 T7809 oascc.SolrZkClient.makePath makePath: /configs/conf1/_rest_managed.json [junit4] 2> 2370638 T7809 oasr.ManagedResourceStorage$ZooKeeperStorageIO$1.close Wrote 38 bytes to new znode /configs/conf1/_rest_managed.json [junit4] 2> 2370639 T7809 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 [junit4] 2> 2370639 T7809 oasr.RestManager.init Initializing 0 registered ManagedResources [junit4] 2> 2370639 T7809 oash.ReplicationHandler.inform Commits will be reserved for 10000 [junit4] 2> 2370640 T7809 oasc.CoreContainer.registerCore registering core: delLiveColl_shard1_replica2 [junit4] 2> 2370640 T7906 oasc.SolrCore.registerSearcher [delLiveColl_shard1_replica2] Registered new searcher Searcher@580ad4b8[delLiveColl_shard1_replica2] main{ExitableDirectoryReader(UninvertingDirectoryReader())} [junit4] 2> 2370640 T7809 oasc.ZkController.register Register replica - core:delLiveColl_shard1_replica2 address:http://127.0.0.1:51529/u_ksd/q collection:delLiveColl shard:shard1 [junit4] 2> 2370644 T7809 oascc.SolrZkClient.makePath makePath: /collections/delLiveColl/leader_elect/shard1/election [junit4] 2> 2370659 T7809 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1 [junit4] 2> 2370663 T7820 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 2370665 T7809 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=179998ms [junit4] 2> 2370667 T7821 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = { [junit4] 2> "operation":"leader", [junit4] 2> "shard":"shard1", [junit4] 2> "collection":"delLiveColl"} current state version: 2 [junit4] 2> 2370777 T7821 oasco.ZkStateWriter.writePendingUpdates going to update_collection /collections/delLiveColl/state.json version: 1 [junit4] 2> 2370781 T7877 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... [junit4] 2> 2370781 T7845 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... [junit4] 2> 2370781 T7861 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... [junit4] 2> 2370783 T7820 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... [junit4] 2> 2370785 T7877 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 2 [junit4] 2> 2370786 T7845 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 2 [junit4] 2> 2370786 T7861 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 2 [junit4] 2> 2370793 T7820 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 2 [junit4] 2> 2371094 T7871 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for delLiveColl_shard1_replica1 [junit4] 2> 2371094 T7871 oasc.ZkController.createCollectionZkNode Check for collection zkNode:delLiveColl [junit4] 2> 2371095 T7871 oasc.ZkController.createCollectionZkNode Collection zkNode exists [junit4] 2> 2371096 T7871 oascc.ZkStateReader.readConfigName Load collection config from:/collections/delLiveColl [junit4] 2> 2371098 T7871 oascc.ZkStateReader.readConfigName path=/collections/delLiveColl configName=conf1 specified config exists in ZooKeeper [junit4] 2> 2371098 T7871 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-005/delLiveColl_shard1_replica1/' [junit4] 2> 2371111 T7853 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for delLiveColl_shard2_replica1 [junit4] 2> 2371112 T7853 oasc.ZkController.createCollectionZkNode Check for collection zkNode:delLiveColl [junit4] 2> 2371114 T7853 oasc.ZkController.createCollectionZkNode Collection zkNode exists [junit4] 2> 2371114 T7853 oascc.ZkStateReader.readConfigName Load collection config from:/collections/delLiveColl [junit4] 2> 2371116 T7853 oascc.ZkStateReader.readConfigName path=/collections/delLiveColl configName=conf1 specified config exists in ZooKeeper [junit4] 2> 2371116 T7853 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-004/delLiveColl_shard2_replica1/' [junit4] 2> 2371129 T7834 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for delLiveColl_shard2_replica2 [junit4] 2> 2371129 T7834 oasc.ZkController.createCollectionZkNode Check for collection zkNode:delLiveColl [junit4] 2> 2371129 T7871 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1 [junit4] 2> 2371132 T7871 oasc.Config.<init> loaded config solrconfig.xml with version 0 [junit4] 2> 2371134 T7834 oasc.ZkController.createCollectionZkNode Collection zkNode exists [junit4] 2> 2371135 T7834 oascc.ZkStateReader.readConfigName Load collection config from:/collections/delLiveColl [junit4] 2> 2371142 T7853 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1 [junit4] 2> 2371142 T7834 oascc.ZkStateReader.readConfigName path=/collections/delLiveColl configName=conf1 specified config exists in ZooKeeper [junit4] 2> 2371144 T7834 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-003/delLiveColl_shard2_replica2/' [junit4] 2> 2371156 T7871 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0 [junit4] 2> 2371178 T7871 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml [junit4] 2> 2371179 T7834 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1 [junit4] 2> 2371183 T7853 oasc.Config.<init> loaded config solrconfig.xml with version 0 [junit4] 2> 2371190 T7834 oasc.Config.<init> loaded config solrconfig.xml with version 0 [junit4] 2> 2371191 T7871 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml [junit4] 2> 2371206 T7871 oass.IndexSchema.readSchema [delLiveColl_shard1_replica1] Schema name=test [junit4] 2> 2371209 T7834 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0 [junit4] 2> 2371200 T7853 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0 [junit4] 2> 2371248 T7853 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml [junit4] 2> 2371252 T7853 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml [junit4] 2> 2371257 T7853 oass.IndexSchema.readSchema [delLiveColl_shard2_replica1] Schema name=test [junit4] 2> 2371281 T7834 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml [junit4] 2> 2371296 T7834 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml [junit4] 2> 2371305 T7834 oass.IndexSchema.readSchema [delLiveColl_shard2_replica2] Schema name=test [junit4] 2> 2371529 T7871 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440. [junit4] 2> 2371533 T7871 oass.IndexSchema.readSchema default search field in schema is text [junit4] 2> 2371535 T7871 oass.IndexSchema.readSchema unique key field: id [junit4] 2> 2371549 T7834 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440. [junit4] 2> 2371550 T7853 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440. [junit4] 2> 2371556 T7834 oass.IndexSchema.readSchema default search field in schema is text [junit4] 2> 2371557 T7853 oass.IndexSchema.readSchema default search field in schema is text [junit4] 2> 2371557 T7834 oass.IndexSchema.readSchema unique key field: id [junit4] 2> 2371561 T7853 oass.IndexSchema.readSchema unique key field: id [junit4] 2> 2371584 T7871 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 2371593 T7871 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 2371598 T7871 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4] 2> 2371600 T7871 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE [junit4] 2> 2371601 T7871 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING [junit4] 2> 2371601 T7871 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4] 2> 2371604 T7871 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE [junit4] 2> 2371604 T7871 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING [junit4] 2> 2371605 T7871 oasc.CoreContainer.create Creating SolrCore 'delLiveColl_shard1_replica1' using configuration from collection delLiveColl [junit4] 2> 2371605 T7871 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory [junit4] 2> 2371605 T7871 oasc.SolrCore.<init> [delLiveColl_shard1_replica1] Opening new SolrCore at ../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-005/delLiveColl_shard1_replica1/, dataDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-005/delLiveColl_shard1_replica1/data/ [junit4] 2> 2371606 T7871 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4b62dc70 [junit4] 2> 2371607 T7871 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-005/delLiveColl_shard1_replica1/data [junit4] 2> 2371607 T7871 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-005/delLiveColl_shard1_replica1/data/index/ [junit4] 2> 2371607 T7871 oasc.SolrCore.initIndex WARN [delLiveColl_shard1_replica1] Solr index directory '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-005/delLiveColl_shard1_replica1/data/index' doesn't exist. Creating new index... [junit4] 2> 2371608 T7871 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001/tempDir-005/delLiveColl_shard1_replica1/data/index [junit4] 2> 2371608 T7871 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=41, maxMergeSize=2147483648, maxMergeSi [...truncated too long message...] :51540_u_ksd%2Fq", [junit4] 2> "numShards":"2", [junit4] 2> "state":"down", [junit4] 2> "shard":"shard1", [junit4] 2> "collection":"delLiveColl", [junit4] 2> "operation":"state"} [junit4] 2> 2394368 T7941 oasco.ReplicaMutator.updateState Update state numShards=2 message={ [junit4] 2> "core":"delLiveColl_shard1_replica1", [junit4] 2> "core_node_name":"core_node1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"http://127.0.0.1:51540/u_ksd/q", [junit4] 2> "node_name":"127.0.0.1:51540_u_ksd%2Fq", [junit4] 2> "numShards":"2", [junit4] 2> "state":"down", [junit4] 2> "shard":"shard1", [junit4] 2> "collection":"delLiveColl", [junit4] 2> "operation":"state"} [junit4] 2> 2394378 T7941 oasco.ZkStateWriter.writePendingUpdates going to update_collection /collections/delLiveColl/state.json version: 22 [junit4] 2> 2394387 T7941 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue [junit4] 2> 2395788 T7926 oascc.ZkStateReader$7.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK [junit4] 2> 2395807 T7794 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/u_ksd/q,null} [junit4] 2> 2400792 T7794 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 51543 [junit4] 2> 2400792 T7794 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1777461615 [junit4] 2> 2400793 T7794 oasc.Overseer.close Overseer (id=92960447029051405-127.0.0.1:51543_u_ksd%2Fq-n_0000000004) closing [junit4] 2> 2400794 T7941 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:51543_u_ksd%2Fq [junit4] 2> 2400797 T7940 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK [junit4] 2> 2400797 T7796 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x14a4308b754000d, likely client has closed socket [junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228) [junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208) [junit4] 2> at java.lang.Thread.run(Thread.java:745) [junit4] 2> [junit4] 2> 2400813 T7794 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/u_ksd/q,null} [junit4] 2> 2405802 T7794 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch [junit4] 2> 2405805 T7794 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:51526 51526 [junit4] 2> 2405902 T7795 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:51526 51526 [junit4] 2> 2406142 T7795 oasc.ZkTestServer$ZKServerMain.runFromConfig WARN Watch limit violations: [junit4] 2> Maximum concurrent create/delete watches above limit: [junit4] 2> [junit4] 2> 8 /solr/aliases.json [junit4] 2> 8 /solr/clusterstate.json [junit4] 2> 4 /solr/collections/delLiveColl/state.json [junit4] 2> 4 /solr/configs/conf1 [junit4] 2> [junit4] 2> Maximum concurrent children watches above limit: [junit4] 2> [junit4] 2> 8 /solr/live_nodes [junit4] 2> 5 /solr/overseer/queue [junit4] 2> 5 /solr/overseer/collection-queue-work [junit4] 2> [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=DeleteReplicaTest -Dtests.method=testDistribSearch -Dtests.seed=D2DD35915246F37B -Dtests.slow=true -Dtests.locale=sr_BA_#Latn -Dtests.timezone=Europe/Vienna -Dtests.asserts=true -Dtests.file.encoding=US-ASCII [junit4] FAILURE 56.7s | DeleteReplicaTest.testDistribSearch <<< [junit4] > Throwable #1: java.lang.AssertionError: Should have had a good message here [junit4] > at __randomizedtesting.SeedInfo.seed([D2DD35915246F37B:533BBB8925199347]:0) [junit4] > at org.apache.solr.cloud.DeleteReplicaTest.deleteLiveReplicaTest(DeleteReplicaTest.java:138) [junit4] > at org.apache.solr.cloud.DeleteReplicaTest.doTest(DeleteReplicaTest.java:89) [junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869) [junit4] > at java.lang.Thread.run(Thread.java:745) [junit4] 2> 2406163 T7794 oas.SolrTestCaseJ4.deleteCore ###deleteCore [junit4] 2> NOTE: leaving temporary files on disk at: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.DeleteReplicaTest-D2DD35915246F37B-001 [junit4] 2> NOTE: test params are: codec=CheapBastard, sim=DefaultSimilarity, locale=sr_BA_#Latn, timezone=Europe/Vienna [junit4] 2> NOTE: Mac OS X 10.8.5 x86_64/Oracle Corporation 1.8.0_20 (64-bit)/cpus=2,threads=1,free=185254816,total=368599040 [junit4] 2> NOTE: All tests run in this JVM: [TestDefaultSearchFieldResource, TestDistributedGrouping, DistributedFacetPivotSmallTest, TestGroupingSearch, FileUtilsTest, StatelessScriptUpdateProcessorFactoryTest, TestCloudSchemaless, EchoParamsTest, OutputWriterTest, OverseerRolesTest, TestDocSet, TestInitParams, CoreAdminCreateDiscoverTest, AnalysisErrorHandlingTest, TestBM25SimilarityFactory, TestCursorMarkWithoutUniqueKey, QueryParsingTest, TestSolrJ, TestFieldTypeCollectionResource, DistributedFacetPivotLongTailTest, TestManagedResource, JSONWriterTest, LeaderElectionIntegrationTest, UpdateRequestProcessorFactoryTest, TestSolrDeletionPolicy2, TimeZoneUtilsTest, InfoHandlerTest, TestModifyConfFiles, SortByFunctionTest, DocValuesMissingTest, LukeRequestHandlerTest, ConnectionManagerTest, TestComponentsName, HdfsBasicDistributedZk2Test, TestFunctionQuery, ReplicationFactorTest, TestManagedResourceStorage, ResourceLoaderTest, ZkNodePropsTest, CoreContainerCoreInitFailuresTest, TestSearcherReuse, MultiThreadedOCPTest, TestFieldResource, TestLRUCache, TestPHPSerializedResponseWriter, TestStressReorder, ShardRoutingTest, TestFieldCollectionResource, BinaryUpdateRequestHandlerTest, LegacyHTMLStripCharFilterTest, TestChildDocTransformer, DirectSolrConnectionTest, SignatureUpdateProcessorFactoryTest, TestBlobHandler, SolrCoreCheckLockOnStartupTest, QueryResultKeyTest, RankQueryTest, BlockDirectoryTest, TestPseudoReturnFields, DistributedSuggestComponentTest, TestPhraseSuggestions, JsonLoaderTest, TestSolrXmlPersistor, TestStandardQParsers, ModifyConfFileTest, MigrateRouteKeyTest, SimpleMLTQParserTest, TestReload, AtomicUpdatesTest, CursorPagingTest, TestSolrIndexConfig, TestReplicationHandlerBackup, SpellCheckComponentTest, TestHighFrequencyDictionaryFactory, TestFreeTextSuggestions, HdfsCollectionsAPIDistributedZkTest, TestRandomMergePolicy, ReturnFieldsTest, TestSort, SharedFSAutoReplicaFailoverUtilsTest, DisMaxRequestHandlerTest, MBeansHandlerTest, TestSolrConfigHandler, TestInitQParser, TestCloudPivotFacet, CursorMarkTest, CopyFieldTest, TestJmxIntegration, ChaosMonkeyNothingIsSafeTest, HdfsSyncSliceTest, TestCollationFieldDocValues, SolrCmdDistributorTest, TestXIncludeConfig, TestAtomicUpdateErrorCases, UpdateParamsTest, TestLeaderElectionZkExpiry, TestPerFieldSimilarity, BasicDistributedZkTest, CurrencyFieldXmlFileTest, TestCollapseQParserPlugin, TestIndexSearcher, TestCloudInspectUtil, HdfsBasicDistributedZkTest, TestSearchPerf, WordBreakSolrSpellCheckerTest, CoreMergeIndexesAdminHandlerTest, CircularListTest, TestSolrDeletionPolicy1, TestCopyFieldCollectionResource, TestSolrQueryParserResource, OpenCloseCoreStressTest, DocValuesTest, SpellPossibilityIteratorTest, CloudExitableDirectoryReaderTest, TestManagedSchema, AssignTest, TestSchemaManager, DirectSolrSpellCheckerTest, TestRTGBase, TestStressVersions, PingRequestHandlerTest, SharedFSAutoReplicaFailoverTest, TestAddFieldRealTimeGet, SOLR749Test, UnloadDistributedZkTest, TestSolrConfigHandlerCloud, HttpPartitionTest, PeerSyncTest, AddBlockUpdateTest, TestRandomFaceting, TestRecoveryHdfs, TestNoOpRegenerator, PrimitiveFieldTypeTest, TestQuerySenderNoQuery, SuggestComponentTest, TestSolrConfigHandlerConcurrent, TestShardHandlerFactory, DistributedFacetPivotLargeTest, FastVectorHighlighterTest, TestPivotHelperCode, ClusterStateTest, SpatialFilterTest, StandardRequestHandlerTest, SimplePostToolTest, TestSolr4Spatial, DocumentBuilderTest, TestFastWriter, TestSerializedLuceneMatchVersion, HdfsLockFactoryTest, SearchHandlerTest, SolrIndexSplitterTest, ShardRoutingCustomTest, TestSystemIdResolver, TestReloadAndDeleteDocs, TestValueSourceCache, TestManagedStopFilterFactory, DeleteReplicaTest] [junit4] Completed in 56.76s, 1 test, 1 failure <<< FAILURES! [...truncated 908 lines...] BUILD FAILED /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:519: The following error occurred while executing this line: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:467: The following error occurred while executing this line: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:61: The following error occurred while executing this line: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/extra-targets.xml:39: The following error occurred while executing this line: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build.xml:187: The following error occurred while executing this line: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/common-build.xml:510: The following error occurred while executing this line: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:1349: The following error occurred while executing this line: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:956: There were test failures: 452 suites, 1843 tests, 1 failure, 37 ignored (21 assumptions) Total time: 164 minutes 2 seconds Build step 'Invoke Ant' marked build as failure [description-setter] Description set: Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseSerialGC (asserts: true) Archiving artifacts Recording test results Email was triggered for: Failure - Any Sending email for trigger: Failure - Any
--------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org