Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/11681/ Java: 64bit/jdk1.8.0_31 -XX:-UseCompressedOops -XX:+UseG1GC
2 tests failed. FAILED: junit.framework.TestSuite.org.apache.solr.cloud.DistribCursorPagingTest Error Message: Suite timeout exceeded (>= 7200000 msec). Stack Trace: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec). at __randomizedtesting.SeedInfo.seed([251A2C2425DBB840]:0) FAILED: org.apache.solr.cloud.DistribCursorPagingTest.test Error Message: Test abandoned because suite timeout was reached. Stack Trace: java.lang.Exception: Test abandoned because suite timeout was reached. at __randomizedtesting.SeedInfo.seed([251A2C2425DBB840]:0) Build Log: [...truncated 9998 lines...] [junit4] Suite: org.apache.solr.cloud.DistribCursorPagingTest [junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/init-core-data-001 [junit4] 2> 198335 T1394 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (false) [junit4] 2> 198336 T1394 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: / [junit4] 2> 198339 T1394 oasc.ZkTestServer.run STARTING ZK TEST SERVER [junit4] 1> client port:0.0.0.0/0.0.0.0:0 [junit4] 2> 198339 T1395 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server [junit4] 2> 198439 T1394 oasc.ZkTestServer.run start zk server on port:55006 [junit4] 2> 198440 T1394 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 198440 T1394 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 198444 T1402 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@53bb1e8e name:ZooKeeperConnection Watcher:127.0.0.1:55006 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 198444 T1394 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 198444 T1394 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 198445 T1394 oascc.SolrZkClient.makePath makePath: /solr [junit4] 2> 198449 T1394 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 198449 T1394 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 198451 T1405 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5ae0ad02 name:ZooKeeperConnection Watcher:127.0.0.1:55006/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 198451 T1394 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 198451 T1394 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 198451 T1394 oascc.SolrZkClient.makePath makePath: /collections/collection1 [junit4] 2> 198453 T1394 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards [junit4] 2> 198454 T1394 oascc.SolrZkClient.makePath makePath: /collections/control_collection [junit4] 2> 198455 T1394 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards [junit4] 2> 198459 T1394 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-deeppaging.xml to /configs/conf1/solrconfig.xml [junit4] 2> 198459 T1394 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml [junit4] 2> 198463 T1394 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/schema-sorts.xml to /configs/conf1/schema.xml [junit4] 2> 198464 T1394 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml [junit4] 2> 198465 T1394 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml [junit4] 2> 198465 T1394 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml [junit4] 2> 198467 T1394 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt [junit4] 2> 198467 T1394 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt [junit4] 2> 198469 T1394 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt [junit4] 2> 198469 T1394 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt [junit4] 2> 198470 T1394 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml [junit4] 2> 198470 T1394 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml [junit4] 2> 198471 T1394 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml [junit4] 2> 198471 T1394 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml [junit4] 2> 198472 T1394 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json [junit4] 2> 198473 T1394 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json [junit4] 2> 198474 T1394 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt [junit4] 2> 198474 T1394 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt [junit4] 2> 198476 T1394 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt [junit4] 2> 198476 T1394 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt [junit4] 2> 198478 T1394 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt [junit4] 2> 198478 T1394 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt [junit4] 2> 198480 T1394 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 198481 T1394 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 198488 T1408 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3ee1351c name:ZooKeeperConnection Watcher:127.0.0.1:55006/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 198489 T1394 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 198489 T1394 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 198728 T1394 oas.SolrTestCaseJ4.writeCoreProperties Writing core.properties file to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/control-001/cores/collection1 [junit4] 2> 198730 T1394 oejs.Server.doStart jetty-9.2.6.v20141205 [junit4] 2> 198731 T1394 oejsh.ContextHandler.doStart Started o.e.j.s.ServletContextHandler@123d20f0{/,null,AVAILABLE} [junit4] 2> 198744 T1394 oejs.AbstractConnector.doStart Started ServerConnector@73a40b86{SSL-http/1.1}{127.0.0.1:32946} [junit4] 2> 198744 T1394 oejs.Server.doStart Started @200395ms [junit4] 2> 198746 T1394 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@c387f44 [junit4] 2> 198747 T1394 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 198747 T1394 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/control-001 [junit4] 2> 198747 T1394 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/control-001/' [junit4] 2> 198765 T1394 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/control-001/solr.xml [junit4] 2> 198772 T1394 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/control-001/cores [junit4] 2> 198772 T1394 oasc.CoreContainer.<init> New CoreContainer 1626639358 [junit4] 2> 198773 T1394 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/control-001/] [junit4] 2> 198774 T1394 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000 [junit4] 2> 198774 T1394 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: [junit4] 2> 198774 T1394 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000 [junit4] 2> 198774 T1394 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20 [junit4] 2> 198775 T1394 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000 [junit4] 2> 198775 T1394 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0 [junit4] 2> 198775 T1394 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647 [junit4] 2> 198775 T1394 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5 [junit4] 2> 198775 T1394 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1 [junit4] 2> 198775 T1394 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false [junit4] 2> 198776 T1394 oashc.HttpShardHandlerFactory.getParameter Setting useRetries to: false [junit4] 2> 198777 T1394 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true [junit4] 2> 198778 T1394 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 198778 T1394 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 198779 T1394 oasc.CoreContainer.load Host Name: 127.0.0.1 [junit4] 2> 198779 T1394 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:55006/solr [junit4] 2> 198779 T1394 oasc.ZkController.checkChrootPath zkHost includes chroot [junit4] 2> 198779 T1394 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 198780 T1394 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 198781 T1422 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@50659858 name:ZooKeeperConnection Watcher:127.0.0.1:55006 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 198782 T1394 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 198782 T1394 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 198784 T1394 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 198785 T1425 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@69608fb2 name:ZooKeeperConnection Watcher:127.0.0.1:55006/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 198786 T1394 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 198787 T1394 oascc.SolrZkClient.makePath makePath: /overseer/queue [junit4] 2> 198788 T1394 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work [junit4] 2> 198790 T1394 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running [junit4] 2> 198791 T1394 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed [junit4] 2> 198792 T1394 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure [junit4] 2> 198793 T1394 oascc.SolrZkClient.makePath makePath: /live_nodes [junit4] 2> 198794 T1394 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:32946_ [junit4] 2> 198794 T1394 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:32946_ [junit4] 2> 198795 T1394 oascc.SolrZkClient.makePath makePath: /overseer_elect [junit4] 2> 198796 T1394 oascc.SolrZkClient.makePath makePath: /overseer_elect/election [junit4] 2> 198797 T1394 oasc.Overseer.close Overseer (id=null) closing [junit4] 2> 198798 T1394 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:32946_ [junit4] 2> 198798 T1394 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader [junit4] 2> 198799 T1394 oasc.Overseer.start Overseer (id=93204509224665092-127.0.0.1:32946_-n_0000000000) starting [junit4] 2> 198800 T1394 oascc.SolrZkClient.makePath makePath: /overseer/queue-work [junit4] 2> 198803 T1394 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000 [junit4] 2> 198804 T1427 oasc.OverseerCollectionProcessor.run Process current queue of collection creations [junit4] 2> 198804 T1394 oascc.SolrZkClient.makePath makePath: /clusterstate.json [junit4] 2> 198806 T1394 oascc.SolrZkClient.makePath makePath: /aliases.json [junit4] 2> 198806 T1394 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 198808 T1426 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue [junit4] 2> 198808 T1394 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/control-001/cores [junit4] 2> 198809 T1394 oasc.CorePropertiesLocator.discoverUnder Found core collection1 in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/control-001/cores/collection1/ [junit4] 2> 198809 T1394 oasc.CorePropertiesLocator.discover Found 1 core definitions [junit4] 2> 198810 T1429 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection [junit4] 2> 198810 T1429 oasc.ZkController.publish numShards not found on descriptor - reading it from system property [junit4] 2> 198810 T1425 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 198810 T1429 oasc.ZkController.waitForCoreNodeName look for our core node name [junit4] 2> 198812 T1426 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 0, message = { [junit4] 2> "core":"collection1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"https://127.0.0.1:32946", [junit4] 2> "node_name":"127.0.0.1:32946_", [junit4] 2> "numShards":"1", [junit4] 2> "state":"down", [junit4] 2> "shard":null, [junit4] 2> "collection":"control_collection", [junit4] 2> "operation":"state"} current state version: 0 [junit4] 2> 198813 T1426 oasco.ReplicaMutator.updateState Update state numShards=1 message={ [junit4] 2> "core":"collection1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"https://127.0.0.1:32946", [junit4] 2> "node_name":"127.0.0.1:32946_", [junit4] 2> "numShards":"1", [junit4] 2> "state":"down", [junit4] 2> "shard":null, [junit4] 2> "collection":"control_collection", [junit4] 2> "operation":"state"} [junit4] 2> 198813 T1426 oasco.ClusterStateMutator.createCollection building a new cName: control_collection [junit4] 2> 198813 T1426 oasco.ReplicaMutator.updateState Assigning new node to shard shard=shard1 [junit4] 2> 198814 T1425 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1) [junit4] 2> 199811 T1429 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1 [junit4] 2> 199811 T1429 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection [junit4] 2> 199812 T1429 oasc.ZkController.createCollectionZkNode Collection zkNode exists [junit4] 2> 199812 T1429 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection [junit4] 2> 199812 T1429 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper [junit4] 2> 199812 T1429 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/control-001/cores/collection1/' [junit4] 2> 199821 T1429 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1 [junit4] 2> 199823 T1429 oasc.Config.<init> loaded config solrconfig.xml with version 0 [junit4] 2> 199826 T1429 oasc.SolrConfig.refreshRequestParams current version of requestparams : -1 [junit4] 2> 199830 T1429 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0 [junit4] 2> 199850 T1429 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml [junit4] 2> 199852 T1429 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml [junit4] 2> 199855 T1429 oass.IndexSchema.readSchema [collection1] Schema name=sortable [junit4] 2> 199945 T1429 oass.IndexSchema.readSchema unique key field: id [junit4] 2> 199945 T1429 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection [junit4] 2> 199946 T1429 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory [junit4] 2> 199946 T1429 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/control-001/cores/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/tempDir-001/control/data/ [junit4] 2> 199946 T1429 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1 [junit4] 2> 199946 T1429 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/tempDir-001/control/data [junit4] 2> 199947 T1429 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/tempDir-001/control/data/index/ [junit4] 2> 199947 T1429 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/tempDir-001/control/data/index' doesn't exist. Creating new index... [junit4] 2> 199947 T1429 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/tempDir-001/control/data/index [junit4] 2> 199947 T1429 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=16, maxMergeAtOnceExplicit=36, maxMergedSegmentMB=95.1484375, floorSegmentMB=1.859375, forceMergeDeletesPctAllowed=10.942055862729083, segmentsPerTier=18.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0 [junit4] 2> 199948 T1429 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1 [junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@2a684d27 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@526ae8e7),segFN=segments_1,generation=1} [junit4] 2> 199948 T1429 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1 [junit4] 2> 199949 T1429 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default [junit4] 2> 199949 T1429 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 199949 T1429 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 199949 T1429 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 199949 T1429 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 199949 T1429 oasc.RequestHandlers.initHandlersFromConfig created /config: org.apache.solr.handler.SolrConfigHandler [junit4] 2> 199950 T1429 oasc.RequestHandlers.initHandlersFromConfig created /schema: org.apache.solr.handler.SchemaHandler [junit4] 2> 199950 T1429 oasc.RequestHandlers.initHandlersFromConfig created /replication: org.apache.solr.handler.ReplicationHandler [junit4] 2> 199950 T1429 oasc.RequestHandlers.initHandlersFromConfig created /get: org.apache.solr.handler.RealTimeGetHandler [junit4] 2> 199950 T1429 oasc.RequestHandlers.initHandlersFromConfig created /admin/luke: org.apache.solr.handler.admin.LukeRequestHandler [junit4] 2> 199951 T1429 oasc.RequestHandlers.initHandlersFromConfig created /admin/system: org.apache.solr.handler.admin.SystemInfoHandler [junit4] 2> 199951 T1429 oasc.RequestHandlers.initHandlersFromConfig created /admin/mbeans: org.apache.solr.handler.admin.SolrInfoMBeanHandler [junit4] 2> 199951 T1429 oasc.RequestHandlers.initHandlersFromConfig created /admin/plugins: org.apache.solr.handler.admin.PluginInfoHandler [junit4] 2> 199951 T1429 oasc.RequestHandlers.initHandlersFromConfig created /admin/threads: org.apache.solr.handler.admin.ThreadDumpHandler [junit4] 2> 199952 T1429 oasc.RequestHandlers.initHandlersFromConfig created /admin/properties: org.apache.solr.handler.admin.PropertiesRequestHandler [junit4] 2> 199952 T1429 oasc.RequestHandlers.initHandlersFromConfig created /admin/logging: org.apache.solr.handler.admin.LoggingHandler [junit4] 2> 199952 T1429 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: org.apache.solr.handler.admin.ShowFileRequestHandler [junit4] 2> 199952 T1429 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: org.apache.solr.handler.PingRequestHandler [junit4] 2> 199952 T1429 oasc.RequestHandlers.initHandlersFromConfig created /select: solr.SearchHandler [junit4] 2> 199954 T1429 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 199954 T1429 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 199955 T1429 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 199955 T1429 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 199956 T1429 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache [junit4] 2> 199957 T1429 oasu.CommitTracker.<init> Hard AutoCommit: disabled [junit4] 2> 199957 T1429 oasu.CommitTracker.<init> Soft AutoCommit: disabled [junit4] 2> 199957 T1429 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=39, maxMergeAtOnceExplicit=49, maxMergedSegmentMB=66.1259765625, floorSegmentMB=1.71875, forceMergeDeletesPctAllowed=4.463875612395901, segmentsPerTier=14.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0 [junit4] 2> 199958 T1429 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1 [junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@2a684d27 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@526ae8e7),segFN=segments_1,generation=1} [junit4] 2> 199958 T1429 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1 [junit4] 2> 199958 T1429 oass.SolrIndexSearcher.<init> Opening Searcher@6893820[collection1] main [junit4] 2> 199958 T1429 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection [junit4] 2> 199959 T1429 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper [junit4] 2> 199959 T1429 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1 [junit4] 2> 199959 T1429 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1 [junit4] 2> 199959 T1429 oasr.RestManager.init Initializing RestManager with initArgs: {} [junit4] 2> 199959 T1429 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 [junit4] 2> 199960 T1429 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json [junit4] 2> 199960 T1429 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 [junit4] 2> 199960 T1429 oasr.RestManager.init Initializing 0 registered ManagedResources [junit4] 2> 199960 T1429 oash.ReplicationHandler.inform Commits will be reserved for 10000 [junit4] 2> 199960 T1430 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6893820[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())} [junit4] 2> 199960 T1429 oasc.CoreContainer.registerCore registering core: collection1 [junit4] 2> 199961 T1433 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:32946 collection:control_collection shard:shard1 [junit4] 2> 199961 T1394 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0 [junit4] 2> 199961 T1394 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done [junit4] 2> 199961 T1433 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election [junit4] 2> 199964 T1433 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1 [junit4] 2> 199965 T1394 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 199965 T1433 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue. [junit4] 2> 199966 T1433 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync [junit4] 2> 199966 T1394 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 199966 T1425 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 199968 T1426 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = { [junit4] 2> "operation":"leader", [junit4] 2> "shard":"shard1", [junit4] 2> "collection":"control_collection"} current state version: 1 [junit4] 2> 199968 T1436 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@38cb1ea5 name:ZooKeeperConnection Watcher:127.0.0.1:55006/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 199968 T1394 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 199969 T1394 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> ASYNC NEW_CORE C518 name=collection1 org.apache.solr.core.SolrCore@28457b5e url=https://127.0.0.1:32946/collection1 node=127.0.0.1:32946_ C518_STATE=coll:control_collection core:collection1 props:{core=collection1, base_url=https://127.0.0.1:32946, node_name=127.0.0.1:32946_, state=down} [junit4] 2> 199966 T1433 C518 P32946 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:32946/collection1/ [junit4] 2> 199970 T1433 C518 P32946 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me [junit4] 2> 199970 T1433 C518 P32946 oasc.SyncStrategy.syncToMe https://127.0.0.1:32946/collection1/ has no replicas [junit4] 2> 199971 T1433 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:32946/collection1/ shard1 [junit4] 2> 199971 T1433 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1 [junit4] 2> 199975 T1394 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 199977 T1425 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 199977 T1426 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = { [junit4] 2> "operation":"leader", [junit4] 2> "shard":"shard1", [junit4] 2> "collection":"control_collection", [junit4] 2> "base_url":"https://127.0.0.1:32946", [junit4] 2> "core":"collection1", [junit4] 2> "state":"active"} current state version: 1 [junit4] 2> 199979 T1394 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false [junit4] 2> 199979 T1394 oasc.AbstractFullDistribZkTestBase.createJettys Creating collection1 with stateFormat=2 [junit4] 2> 199980 T1394 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 199982 T1394 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 199985 T1439 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@258ebaed name:ZooKeeperConnection Watcher:127.0.0.1:55006/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 199985 T1394 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 199986 T1394 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 199988 T1425 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 199993 T1426 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = { [junit4] 2> "operation":"create", [junit4] 2> "name":"collection1", [junit4] 2> "numShards":"2", [junit4] 2> "stateFormat":"2"} current state version: 1 [junit4] 2> 199993 T1426 oasco.ClusterStateMutator.createCollection building a new cName: collection1 [junit4] 2> 199994 T1425 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1) [junit4] 2> 199994 T1436 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1) [junit4] 2> 200097 T1426 oasco.ZkStateWriter.writePendingUpdates going to create_collection /collections/collection1/state.json [junit4] 2> 200098 T1436 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1) [junit4] 2> 200128 T1433 oasc.ZkController.register We are https://127.0.0.1:32946/collection1/ and leader is https://127.0.0.1:32946/collection1/ [junit4] 2> 200129 T1433 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:32946 [junit4] 2> 200129 T1433 oasc.ZkController.checkRecovery I am the leader, no recovery necessary [junit4] 2> 200129 T1433 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection [junit4] 2> 200129 T1433 oasc.ZkController.publish numShards not found on descriptor - reading it from system property [junit4] 2> 200130 T1425 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 200131 T1426 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = { [junit4] 2> "core":"collection1", [junit4] 2> "core_node_name":"core_node1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"https://127.0.0.1:32946", [junit4] 2> "node_name":"127.0.0.1:32946_", [junit4] 2> "numShards":"2", [junit4] 2> "state":"active", [junit4] 2> "shard":"shard1", [junit4] 2> "collection":"control_collection", [junit4] 2> "operation":"state"} current state version: 3 [junit4] 2> 200132 T1426 oasco.ReplicaMutator.updateState Update state numShards=2 message={ [junit4] 2> "core":"collection1", [junit4] 2> "core_node_name":"core_node1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"https://127.0.0.1:32946", [junit4] 2> "node_name":"127.0.0.1:32946_", [junit4] 2> "numShards":"2", [junit4] 2> "state":"active", [junit4] 2> "shard":"shard1", [junit4] 2> "collection":"control_collection", [junit4] 2> "operation":"state"} [junit4] 2> 200234 T1425 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1) [junit4] 2> 200234 T1436 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1) [junit4] 2> 200389 T1394 oas.SolrTestCaseJ4.writeCoreProperties Writing core.properties file to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/shard-1-001/cores/collection1 [junit4] 2> 200390 T1394 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1 [junit4] 2> 200391 T1394 oejs.Server.doStart jetty-9.2.6.v20141205 [junit4] 2> 200393 T1394 oejsh.ContextHandler.doStart Started o.e.j.s.ServletContextHandler@521fb526{/,null,AVAILABLE} [junit4] 2> 200404 T1394 oejs.AbstractConnector.doStart Started ServerConnector@434b2e0f{SSL-http/1.1}{127.0.0.1:46807} [junit4] 2> 200405 T1394 oejs.Server.doStart Started @202056ms [junit4] 2> 200407 T1394 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@c387f44 [junit4] 2> 200408 T1394 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 200408 T1394 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/shard-1-001 [junit4] 2> 200408 T1394 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/shard-1-001/' [junit4] 2> 200427 T1394 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/shard-1-001/solr.xml [junit4] 2> 200433 T1394 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/shard-1-001/cores [junit4] 2> 200434 T1394 oasc.CoreContainer.<init> New CoreContainer 909648518 [junit4] 2> 200434 T1394 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/shard-1-001/] [junit4] 2> 200435 T1394 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000 [junit4] 2> 200435 T1394 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: [junit4] 2> 200436 T1394 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000 [junit4] 2> 200436 T1394 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20 [junit4] 2> 200436 T1394 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000 [junit4] 2> 200437 T1394 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0 [junit4] 2> 200437 T1394 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647 [junit4] 2> 200437 T1394 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5 [junit4] 2> 200437 T1394 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1 [junit4] 2> 200438 T1394 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false [junit4] 2> 200438 T1394 oashc.HttpShardHandlerFactory.getParameter Setting useRetries to: false [junit4] 2> 200440 T1394 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true [junit4] 2> 200442 T1394 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 200443 T1394 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 200443 T1394 oasc.CoreContainer.load Host Name: 127.0.0.1 [junit4] 2> 200443 T1394 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:55006/solr [junit4] 2> 200444 T1394 oasc.ZkController.checkChrootPath zkHost includes chroot [junit4] 2> 200444 T1394 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 200445 T1394 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 200447 T1453 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4a80156b name:ZooKeeperConnection Watcher:127.0.0.1:55006 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 200448 T1394 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 200449 T1394 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 200452 T1394 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 200453 T1456 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@71e7a4a4 name:ZooKeeperConnection Watcher:127.0.0.1:55006/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 200454 T1394 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 200458 T1394 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 201460 T1394 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46807_ [junit4] 2> 201462 T1394 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:46807_ [junit4] 2> 201466 T1394 oasc.Overseer.close Overseer (id=null) closing [junit4] 2> 201468 T1394 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/shard-1-001/cores [junit4] 2> 201469 T1394 oasc.CorePropertiesLocator.discoverUnder Found core collection1 in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/shard-1-001/cores/collection1/ [junit4] 2> 201470 T1394 oasc.CorePropertiesLocator.discover Found 1 core definitions [junit4] 2> 201473 T1457 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1 [junit4] 2> 201473 T1457 oasc.ZkController.publish numShards not found on descriptor - reading it from system property [junit4] 2> 201475 T1425 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 201476 T1457 oasc.ZkController.preRegister Registering watch for external collection collection1 [junit4] 2> 201476 T1457 oascc.ZkStateReader.addZkWatch addZkWatch collection1 [junit4] 2> 201476 T1426 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = { [junit4] 2> "core":"collection1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"https://127.0.0.1:46807", [junit4] 2> "node_name":"127.0.0.1:46807_", [junit4] 2> "numShards":"2", [junit4] 2> "state":"down", [junit4] 2> "shard":null, [junit4] 2> "collection":"collection1", [junit4] 2> "operation":"state"} current state version: 4 [junit4] 2> 201476 T1457 oascc.ZkStateReader.addZkWatch Updating collection state at /collections/collection1/state.json from ZooKeeper... [junit4] 2> 201477 T1426 oasco.ReplicaMutator.updateState Update state numShards=2 message={ [junit4] 2> "core":"collection1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"https://127.0.0.1:46807", [junit4] 2> "node_name":"127.0.0.1:46807_", [junit4] 2> "numShards":"2", [junit4] 2> "state":"down", [junit4] 2> "shard":null, [junit4] 2> "collection":"collection1", [junit4] 2> "operation":"state"} [junit4] 2> 201477 T1426 oasco.ReplicaMutator.updateState Collection already exists with numShards=2 [junit4] 2> 201477 T1457 oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 0 [junit4] 2> 201477 T1457 oasc.ZkController.waitForCoreNodeName look for our core node name [junit4] 2> 201477 T1426 oasco.ReplicaMutator.updateState Assigning new node to shard shard=shard2 [junit4] 2> 201580 T1426 oasco.ZkStateWriter.writePendingUpdates going to update_collection /collections/collection1/state.json version: 0 [junit4] 2> 201581 T1456 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json, has occurred - updating... [junit4] 2> 201582 T1456 oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 1 [junit4] 2> 202478 T1457 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1 [junit4] 2> 202478 T1457 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1 [junit4] 2> 202479 T1457 oasc.ZkController.createCollectionZkNode Collection zkNode exists [junit4] 2> 202479 T1457 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1 [junit4] 2> 202481 T1457 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper [junit4] 2> 202481 T1457 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/shard-1-001/cores/collection1/' [junit4] 2> 202498 T1457 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1 [junit4] 2> 202500 T1457 oasc.Config.<init> loaded config solrconfig.xml with version 0 [junit4] 2> 202505 T1457 oasc.SolrConfig.refreshRequestParams current version of requestparams : -1 [junit4] 2> 202510 T1457 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0 [junit4] 2> 202532 T1457 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml [junit4] 2> 202534 T1457 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml [junit4] 2> 202539 T1457 oass.IndexSchema.readSchema [collection1] Schema name=sortable [junit4] 2> 202659 T1457 oass.IndexSchema.readSchema unique key field: id [junit4] 2> 202661 T1457 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1 [junit4] 2> 202661 T1457 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory [junit4] 2> 202661 T1457 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/shard-1-001/cores/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/tempDir-001/jetty1/ [junit4] 2> 202662 T1457 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1 [junit4] 2> 202662 T1457 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/tempDir-001/jetty1 [junit4] 2> 202662 T1457 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/tempDir-001/jetty1/index/ [junit4] 2> 202662 T1457 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/tempDir-001/jetty1/index' doesn't exist. Creating new index... [junit4] 2> 202664 T1457 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/tempDir-001/jetty1/index [junit4] 2> 202664 T1457 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=16, maxMergeAtOnceExplicit=36, maxMergedSegmentMB=95.1484375, floorSegmentMB=1.859375, forceMergeDeletesPctAllowed=10.942055862729083, segmentsPerTier=18.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0 [junit4] 2> 202665 T1457 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1 [junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@26df2c98 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@38dd26c1),segFN=segments_1,generation=1} [junit4] 2> 202665 T1457 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1 [junit4] 2> 202666 T1457 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default [junit4] 2> 202667 T1457 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 202667 T1457 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 202667 T1457 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 202667 T1457 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 202667 T1457 oasc.RequestHandlers.initHandlersFromConfig created /config: org.apache.solr.handler.SolrConfigHandler [junit4] 2> 202668 T1457 oasc.RequestHandlers.initHandlersFromConfig created /schema: org.apache.solr.handler.SchemaHandler [junit4] 2> 202668 T1457 oasc.RequestHandlers.initHandlersFromConfig created /replication: org.apache.solr.handler.ReplicationHandler [junit4] 2> 202668 T1457 oasc.RequestHandlers.initHandlersFromConfig created /get: org.apache.solr.handler.RealTimeGetHandler [junit4] 2> 202668 T1457 oasc.RequestHandlers.initHandlersFromConfig created /admin/luke: org.apache.solr.handler.admin.LukeRequestHandler [junit4] 2> 202669 T1457 oasc.RequestHandlers.initHandlersFromConfig created /admin/system: org.apache.solr.handler.admin.SystemInfoHandler [junit4] 2> 202669 T1457 oasc.RequestHandlers.initHandlersFromConfig created /admin/mbeans: org.apache.solr.handler.admin.SolrInfoMBeanHandler [junit4] 2> 202669 T1457 oasc.RequestHandlers.initHandlersFromConfig created /admin/plugins: org.apache.solr.handler.admin.PluginInfoHandler [junit4] 2> 202670 T1457 oasc.RequestHandlers.initHandlersFromConfig created /admin/threads: org.apache.solr.handler.admin.ThreadDumpHandler [junit4] 2> 202670 T1457 oasc.RequestHandlers.initHandlersFromConfig created /admin/properties: org.apache.solr.handler.admin.PropertiesRequestHandler [junit4] 2> 202670 T1457 oasc.RequestHandlers.initHandlersFromConfig created /admin/logging: org.apache.solr.handler.admin.LoggingHandler [junit4] 2> 202670 T1457 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: org.apache.solr.handler.admin.ShowFileRequestHandler [junit4] 2> 202671 T1457 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: org.apache.solr.handler.PingRequestHandler [junit4] 2> 202671 T1457 oasc.RequestHandlers.initHandlersFromConfig created /select: solr.SearchHandler [junit4] 2> 202673 T1457 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 202674 T1457 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 202675 T1457 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 202676 T1457 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 202677 T1457 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache [junit4] 2> 202678 T1457 oasu.CommitTracker.<init> Hard AutoCommit: disabled [junit4] 2> 202678 T1457 oasu.CommitTracker.<init> Soft AutoCommit: disabled [junit4] 2> 202678 T1457 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=39, maxMergeAtOnceExplicit=49, maxMergedSegmentMB=66.1259765625, floorSegmentMB=1.71875, forceMergeDeletesPctAllowed=4.463875612395901, segmentsPerTier=14.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0 [junit4] 2> 202679 T1457 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1 [junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@26df2c98 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@38dd26c1),segFN=segments_1,generation=1} [junit4] 2> 202679 T1457 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1 [junit4] 2> 202680 T1457 oass.SolrIndexSearcher.<init> Opening Searcher@5fae84aa[collection1] main [junit4] 2> 202680 T1457 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1 [junit4] 2> 202681 T1457 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper [junit4] 2> 202681 T1457 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1 [junit4] 2> 202681 T1457 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1 [junit4] 2> 202681 T1457 oasr.RestManager.init Initializing RestManager with initArgs: {} [junit4] 2> 202681 T1457 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 [junit4] 2> 202682 T1457 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json [junit4] 2> 202682 T1457 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 [junit4] 2> 202682 T1457 oasr.RestManager.init Initializing 0 registered ManagedResources [junit4] 2> 202683 T1457 oash.ReplicationHandler.inform Commits will be reserved for 10000 [junit4] 2> 202683 T1458 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5fae84aa[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())} [junit4] 2> 202683 T1457 oasc.CoreContainer.registerCore registering core: collection1 [junit4] 2> 202684 T1461 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:46807 collection:collection1 shard:shard2 [junit4] 2> 202684 T1394 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0 [junit4] 2> 202684 T1394 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done [junit4] 2> 202698 T1461 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election [junit4] 2> 202701 T1461 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2 [junit4] 2> 202702 T1425 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 202703 T1461 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue. [junit4] 2> 202703 T1461 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync [junit4] 2> ASYNC NEW_CORE C519 name=collection1 org.apache.solr.core.SolrCore@3747ff1 url=https://127.0.0.1:46807/collection1 node=127.0.0.1:46807_ C519_STATE=coll:collection1 core:collection1 props:{core=collection1, base_url=https://127.0.0.1:46807, node_name=127.0.0.1:46807_, state=down} [junit4] 2> 202703 T1461 C519 P46807 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:46807/collection1/ [junit4] 2> 202703 T1426 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = { [junit4] 2> "operation":"leader", [junit4] 2> "shard":"shard2", [junit4] 2> "collection":"collection1"} current state version: 4 [junit4] 2> 202703 T1461 C519 P46807 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me [junit4] 2> 202703 T1461 C519 P46807 oasc.SyncStrategy.syncToMe https://127.0.0.1:46807/collection1/ has no replicas [junit4] 2> 202703 T1461 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:46807/collection1/ shard2 [junit4] 2> 202704 T1461 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2 [junit4] 2> 202704 T1426 oasco.ZkStateWriter.writePendingUpdates going to update_collection /collections/collection1/state.json version: 1 [junit4] 2> 202704 T1456 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json, has occurred - updating... [junit4] 2> 202705 T1456 oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 2 [junit4] 2> 202707 T1426 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = { [junit4] 2> "operation":"leader", [junit4] 2> "shard":"shard2", [junit4] 2> "collection":"collection1", [junit4] 2> "base_url":"https://127.0.0.1:46807", [junit4] 2> "core":"collection1", [junit4] 2> "state":"active"} current state version: 4 [junit4] 2> 202708 T1426 oasco.ZkStateWriter.writePendingUpdates going to update_collection /collections/collection1/state.json version: 2 [junit4] 2> 202708 T1456 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json, has occurred - updating... [junit4] 2> 202709 T1456 oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 3 [junit4] 2> 202709 T1425 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 202757 T1461 oasc.ZkController.register We are https://127.0.0.1:46807/collection1/ and leader is https://127.0.0.1:46807/collection1/ [junit4] 2> 202758 T1461 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:46807 [junit4] 2> 202758 T1461 oasc.ZkController.checkRecovery I am the leader, no recovery necessary [junit4] 2> 202758 T1461 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1 [junit4] 2> 202758 T1461 oasc.ZkController.publish numShards not found on descriptor - reading it from system property [junit4] 2> 202759 T1425 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 202759 T1426 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = { [junit4] 2> "core":"collection1", [junit4] 2> "core_node_name":"core_node1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"https://127.0.0.1:46807", [junit4] 2> "node_name":"127.0.0.1:46807_", [junit4] 2> "numShards":"2", [junit4] 2> "state":"active", [junit4] 2> "shard":"shard2", [junit4] 2> "collection":"collection1", [junit4] 2> "operation":"state"} current state version: 4 [junit4] 2> 202760 T1426 oasco.ReplicaMutator.updateState Update state numShards=2 message={ [junit4] 2> "core":"collection1", [junit4] 2> "core_node_name":"core_node1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"https://127.0.0.1:46807", [junit4] 2> "node_name":"127.0.0.1:46807_", [junit4] 2> "numShards":"2", [junit4] 2> "state":"active", [junit4] 2> "shard":"shard2", [junit4] 2> "collection":"collection1", [junit4] 2> "operation":"state"} [junit4] 2> 202761 T1426 oasco.ZkStateWriter.writePendingUpdates going to update_collection /collections/collection1/state.json version: 3 [junit4] 2> 202762 T1456 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json, has occurred - updating... [junit4] 2> 202762 T1461 oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 4 [junit4] 2> 202762 T1456 oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 4 [junit4] 2> 202939 T1394 oas.SolrTestCaseJ4.writeCoreProperties Writing core.properties file to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/shard-2-001/cores/collection1 [junit4] 2> 202940 T1394 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2 [junit4] 2> 202941 T1394 oejs.Server.doStart jetty-9.2.6.v20141205 [junit4] 2> 202942 T1394 oejsh.ContextHandler.doStart Started o.e.j.s.ServletContextHandler@22ca5116{/,null,AVAILABLE} [junit4] 2> 202948 T1394 oejs.AbstractConnector.doStart Started ServerConnector@79a87df0{SSL-http/1.1}{127.0.0.1:49075} [junit4] 2> 202948 T1394 oejs.Server.doStart Started @204599ms [junit4] 2> 202950 T1394 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@c387f44 [junit4] 2> 202950 T1394 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 202950 T1394 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/shard-2-001 [junit4] 2> 202951 T1394 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/shard-2-001/' [junit4] 2> 202962 T1394 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/shard-2-001/solr.xml [junit4] 2> 202966 T1394 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/shard-2-001/cores [junit4] 2> 202967 T1394 oasc.CoreContainer.<init> New CoreContainer 1280157244 [junit4] 2> 202967 T1394 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/shard-2-001/] [junit4] 2> 202968 T1394 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000 [junit4] 2> 202968 T1394 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: [junit4] 2> 202968 T1394 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000 [junit4] 2> 202968 T1394 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20 [junit4] 2> 202969 T1394 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000 [junit4] 2> 202969 T1394 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0 [junit4] 2> 202969 T1394 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647 [junit4] 2> 202969 T1394 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5 [junit4] 2> 202969 T1394 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1 [junit4] 2> 202969 T1394 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false [junit4] 2> 202970 T1394 oashc.HttpShardHandlerFactory.getParameter Setting useRetries to: false [junit4] 2> 202971 T1394 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true [junit4] 2> 202972 T1394 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 202973 T1394 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 202973 T1394 oasc.CoreContainer.load Host Name: 127.0.0.1 [junit4] 2> 202973 T1394 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:55006/solr [junit4] 2> 202973 T1394 oasc.ZkController.checkChrootPath zkHost includes chroot [junit4] 2> 202973 T1394 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 202974 T1394 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 202976 T1475 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2b898c0c name:ZooKeeperConnection Watcher:127.0.0.1:55006 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 202977 T1394 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 202977 T1394 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 202979 T1394 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 202980 T1478 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@63866008 name:ZooKeeperConnection Watcher:127.0.0.1:55006/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 202981 T1394 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 202984 T1394 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 203987 T1394 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49075_ [junit4] 2> 203988 T1394 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49075_ [junit4] 2> 203991 T1394 oasc.Overseer.close Overseer (id=null) closing [junit4] 2> 203992 T1394 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/shard-2-001/cores [junit4] 2> 203993 T1394 oasc.CorePropertiesLocator.discoverUnder Found core collection1 in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/shard-2-001/cores/collection1/ [junit4] 2> 203993 T1394 oasc.CorePropertiesLocator.discover Found 1 core definitions [junit4] 2> 203995 T1479 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1 [junit4] 2> 203995 T1479 oasc.ZkController.publish numShards not found on descriptor - reading it from system property [junit4] 2> 203996 T1425 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 203997 T1479 oasc.ZkController.preRegister Registering watch for external collection collection1 [junit4] 2> 203997 T1479 oascc.ZkStateReader.addZkWatch addZkWatch collection1 [junit4] 2> 203997 T1426 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = { [junit4] 2> "core":"collection1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"https://127.0.0.1:49075", [junit4] 2> "node_name":"127.0.0.1:49075_", [junit4] 2> "numShards":"2", [junit4] 2> "state":"down", [junit4] 2> "shard":null, [junit4] 2> "collection":"collection1", [junit4] 2> "operation":"state"} current state version: 4 [junit4] 2> 203997 T1479 oascc.ZkStateReader.addZkWatch Updating collection state at /collections/collection1/state.json from ZooKeeper... [junit4] 2> 203998 T1426 oasco.ReplicaMutator.updateState Update state numShards=2 message={ [junit4] 2> "core":"collection1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"https://127.0.0.1:49075", [junit4] 2> "node_name":"127.0.0.1:49075_", [junit4] 2> "numShards":"2", [junit4] 2> "state":"down", [junit4] 2> "shard":null, [junit4] 2> "collection":"collection1", [junit4] 2> "operation":"state"} [junit4] 2> 203998 T1426 oasco.ReplicaMutator.updateState Collection already exists with numShards=2 [junit4] 2> 203998 T1426 oasco.ReplicaMutator.updateState Assigning new node to shard shard=shard1 [junit4] 2> 203999 T1479 oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 4 [junit4] 2> 203999 T1426 oasco.ZkStateWriter.writePendingUpdates going to update_collection /collections/collection1/state.json version: 4 [junit4] 2> 203999 T1479 oasc.ZkController.waitForCoreNodeName look for our core node name [junit4] 2> 203999 T1456 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json, has occurred - updating... [junit4] 2> 203999 T1478 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json, has occurred - updating... [junit4] 2> 204000 T1456 oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 5 [junit4] 2> 204000 T1478 oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 5 [junit4] 2> 204999 T1479 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1 [junit4] 2> 204999 T1479 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1 [junit4] 2> 205000 T1479 oasc.ZkController.createCollectionZkNode Collection zkNode exists [junit4] 2> 205000 T1479 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1 [junit4] 2> 205001 T1479 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper [junit4] 2> 205001 T1479 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/shard-2-001/cores/collection1/' [junit4] 2> 205010 T1479 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1 [junit4] 2> 205011 T1479 oasc.Config.<init> loaded config solrconfig.xml with version 0 [junit4] 2> 205014 T1479 oasc.SolrConfig.refreshRequestParams current version of requestparams : -1 [junit4] 2> 205018 T1479 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0 [junit4] 2> 205039 T1479 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml [junit4] 2> 205042 T1479 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml [junit4] 2> 205045 T1479 oass.IndexSchema.readSchema [collection1] Schema name=sortable [junit4] 2> 205134 T1479 oass.IndexSchema.readSchema unique key field: id [junit4] 2> 205135 T1479 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1 [junit4] 2> 205135 T1479 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory [junit4] 2> 205136 T1479 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/shard-2-001/cores/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/tempDir-001/jetty2/ [junit4] 2> 205136 T1479 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1 [junit4] 2> 205136 T1479 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/tempDir-001/jetty2 [junit4] 2> 205136 T1479 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/tempDir-001/jetty2/index/ [junit4] 2> 205136 T1479 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/tempDir-001/jetty2/index' doesn't exist. Creating new index... [junit4] 2> 205137 T1479 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/tempDir-001/jetty2/index [junit4] 2> 205138 T1479 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=16, maxMergeAtOnceExplicit=36, maxMergedSegmentMB=95.1484375, floorSegmentMB=1.859375, forceMergeDeletesPctAllowed=10.942055862729083, segmentsPerTier=18.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0 [junit4] 2> 205138 T1479 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1 [junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1562f5f0 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@73aea0ae),segFN=segments_1,generation=1} [junit4] 2> 205138 T1479 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1 [junit4] 2> 205139 T1479 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default [junit4] 2> 205139 T1479 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 205139 T1479 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 205140 T1479 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 205140 T1479 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 205140 T1479 oasc.RequestHandlers.initHandlersFromConfig created /config: org.apache.solr.handler.SolrConfigHandler [junit4] 2> 205140 T1479 oasc.RequestHandlers.initHandlersFromConfig created /schema: org.apache.solr.handler.SchemaHandler [junit4] 2> 205140 T1479 oasc.RequestHandlers.initHandlersFromConfig created /replication: org.apache.solr.handler.ReplicationHandler [junit4] 2> 205140 T1479 oasc.RequestHandlers.initHandlersFromConfig created /get: org.apache.solr.handler.RealTimeGetHandler [junit4] 2> 205141 T1479 oasc.RequestHandlers.initHandlersFromConfig created /admin/luke: org.apache.solr.handler.admin.LukeRequestHandler [junit4] 2> 205141 T1479 oasc.RequestHandlers.initHandlersFromConfig created /admin/system: org.apache.solr.handler.admin.SystemInfoHandler [junit4] 2> 205142 T1479 oasc.RequestHandlers.initHandlersFromConfig created /admin/mbeans: org.apache.solr.handler.admin.SolrInfoMBeanHandler [junit4] 2> 205142 T1479 oasc.RequestHandlers.initHandlersFromConfig created /admin/plugins: org.apache.solr.handler.admin.PluginInfoHandler [junit4] 2> 205142 T1479 oasc.RequestHandlers.initHandlersFromConfig created /admin/threads: org.apache.solr.handler.admin.ThreadDumpHandler [junit4] 2> 205142 T1479 oasc.RequestHandlers.initHandlersFromConfig created /admin/properties: org.apache.solr.handler.admin.PropertiesRequestHandler [junit4] 2> 205142 T1479 oasc.RequestHandlers.initHandlersFromConfig created /admin/logging: org.apache.solr.handler.admin.LoggingHandler [junit4] 2> 205142 T1479 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: org.apache.solr.handler.admin.ShowFileRequestHandler [junit4] 2> 205143 T1479 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: org.apache.solr.handler.PingRequestHandler [junit4] 2> 205143 T1479 oasc.RequestHandlers.initHandlersFromConfig created /select: solr.SearchHandler [junit4] 2> 205144 T1479 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 205145 T1479 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 205145 T1479 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 205146 T1479 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 205146 T1479 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache [junit4] 2> 205147 T1479 oasu.CommitTracker.<init> Hard AutoCommit: disabled [junit4] 2> 205147 T1479 oasu.CommitTracker.<init> Soft AutoCommit: disabled [junit4] 2> 205148 T1479 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=39, maxMergeAtOnceExplicit=49, maxMergedSegmentMB=66.1259765625, floorSegmentMB=1.71875, forceMergeDeletesPctAllowed=4.463875612395901, segmentsPerTier=14.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0 [junit4] 2> 205148 T1479 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1 [junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1562f5f0 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@73aea0ae),segFN=segments_1,generation=1} [junit4] 2> 205148 T1479 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1 [junit4] 2> 205148 T1479 oass.SolrIndexSearcher.<init> Opening Searcher@2da2ea2d[collection1] main [junit4] 2> 205148 T1479 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1 [junit4] 2> 205149 T1479 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper [junit4] 2> 205149 T1479 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1 [junit4] 2> 205149 T1479 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1 [junit4] 2> 205150 T1479 oasr.RestManager.init Initializing RestManager with initArgs: {} [junit4] 2> 205150 T1479 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 [junit4] 2> 205150 T1479 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json [junit4] 2> 205150 T1479 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 [junit4] 2> 205150 T1479 oasr.RestManager.init Initializing 0 registered ManagedResources [junit4] 2> 205151 T1479 oash.ReplicationHandler.inform Commits will be reserved for 10000 [junit4] 2> 205151 T1479 oasc.CoreContainer.registerCore registering core: collection1 [junit4] 2> 205151 T1480 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2da2ea2d[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())} [junit4] 2> 205151 T1483 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:49075 collection:collection1 shard:shard1 [junit4] 2> 205152 T1394 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0 [junit4] 2> 205152 T1394 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done [junit4] 2> 205152 T1483 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election [junit4] 2> 205154 T1483 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1 [junit4] 2> 205155 T1425 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 205155 T1483 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue. [junit4] 2> 205156 T1483 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync [junit4] 2> ASYNC NEW_CORE C520 name=collection1 org.apache.solr.core.SolrCore@21b53e4e url=https://127.0.0.1:49075/collection1 node=127.0.0.1:49075_ C520_STATE=coll:collection1 core:collection1 props:{core=collection1, base_url=https://127.0.0.1:49075, node_name=127.0.0.1:49075_, state=down} [junit4] 2> 205156 T1483 C520 P49075 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:49075/collection1/ [junit4] 2> 205156 T1483 C520 P49075 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me [junit4] 2> 205156 T1426 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = { [junit4] 2> "operation":"leader", [junit4] 2> "shard":"shard1", [junit4] 2> "collection":"collection1"} current state version: 4 [junit4] 2> 205156 T1483 C520 P49075 oasc.SyncStrategy.syncToMe https://127.0.0.1:49075/collection1/ has no replicas [junit4] 2> 205156 T1483 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:49075/collection1/ shard1 [junit4] 2> 205157 T1483 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1 [junit4] 2> 205158 T1426 oasco.ZkStateWriter.writePendingUpdates going to update_collection /collections/collection1/state.json version: 5 [junit4] 2> 205158 T1478 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json, has occurred - updating... [junit4] 2> 205158 T1456 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json, has occurred - updating... [junit4] 2> 205159 T1456 oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 6 [junit4] 2> 205160 T1478 oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 6 [junit4] 2> 205161 T1425 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 205162 T1426 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = { [junit4] 2> "operation":"leader", [junit4] 2> "shard":"shard1", [junit4] 2> "collection":"collection1", [junit4] 2> "base_url":"https://127.0.0.1:49075", [junit4] 2> "core":"collection1", [junit4] 2> "state":"active"} current state version: 4 [junit4] 2> 205162 T1426 oasco.ZkStateWriter.writePendingUpdates going to update_collection /collections/collection1/state.json version: 6 [junit4] 2> 205163 T1478 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json, has occurred - updating... [junit4] 2> 205163 T1456 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json, has occurred - updating... [junit4] 2> 205164 T1478 oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 7 [junit4] 2> 205164 T1456 oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 7 [junit4] 2> 205212 T1483 oasc.ZkController.register We are https://127.0.0.1:49075/collection1/ and leader is https://127.0.0.1:49075/collection1/ [junit4] 2> 205212 T1483 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:49075 [junit4] 2> 205212 T1483 oasc.ZkController.checkRecovery I am the leader, no recovery necessary [junit4] 2> 205212 T1483 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1 [junit4] 2> 205212 T1483 oasc.ZkController.publish numShards not found on descriptor - reading it from system property [junit4] 2> 205213 T1425 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 205214 T1426 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = { [junit4] 2> "core":"collection1", [junit4] 2> "core_node_name":"core_node2", [junit4] 2> "roles":null, [junit4] 2> "base_url":"https://127.0.0.1:49075", [junit4] 2> "node_name":"127.0.0.1:49075_", [junit4] 2> "numShards":"2", [junit4] 2> "state":"active", [junit4] 2> "shard":"shard1", [junit4] 2> "collection":"collection1", [junit4] 2> "operation":"state"} current state version: 4 [junit4] 2> 205215 T1426 oasco.ReplicaMutator.updateState Update state numShards=2 message={ [junit4] 2> "core":"collection1", [junit4] 2> "core_node_name":"core_node2", [junit4] 2> "roles":null, [junit4] 2> "base_url":"https://127.0.0.1:49075", [junit4] 2> "node_name":"127.0.0.1:49075_", [junit4] 2> "numShards":"2", [junit4] 2> "state":"active", [junit4] 2> "shard":"shard1", [junit4] 2> "collection":"collection1", [junit4] 2> "operation":"state"} [junit4] 2> 205216 T1483 oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 7 [junit4] 2> 205216 T1426 oasco.ZkStateWriter.writePendingUpdates going to update_collection /collections/collection1/state.json version: 7 [junit4] 2> 205216 T1456 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json, has occurred - updating... [junit4] 2> 205216 T1478 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json, has occurred - updating... [junit4] 2> 205217 T1478 oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 8 [junit4] 2> 205217 T1456 oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 8 [junit4] 2> 205393 T1394 oas.SolrTestCaseJ4.writeCoreProperties Writing core.properties file to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/shard-3-001/cores/collection1 [junit4] 2> 205394 T1394 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3 [junit4] 2> 205395 T1394 oejs.Server.doStart jetty-9.2.6.v20141205 [junit4] 2> 205397 T1394 oejsh.ContextHandler.doStart Started o.e.j.s.ServletContextHandler@2d95b382{/,null,AVAILABLE} [junit4] 2> 205403 T1394 oejs.AbstractConnector.doStart Started ServerConnector@1ca5e57c{SSL-http/1.1}{127.0.0.1:33670} [junit4] 2> 205404 T1394 oejs.Server.doStart Started @207055ms [junit4] 2> 205406 T1394 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@c387f44 [junit4] 2> 205407 T1394 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 205407 T1394 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/shard-3-001 [junit4] 2> 205407 T1394 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/shard-3-001/' [junit4] 2> 205426 T1394 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/shard-3-001/solr.xml [junit4] 2> 205433 T1394 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.DistribCursorPagingTest 251A2C2425DBB840-001/sha [...truncated too long message...] 2> at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:943) [junit4] 2> at org.apache.solr.update.VersionInfo.blockUpdates(VersionInfo.java:107) [junit4] 2> at org.apache.solr.update.UpdateLog.bufferUpdates(UpdateLog.java:1073) [junit4] 2> at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:442) [junit4] 2> at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:235) [junit4] 2> 13) Thread[id=1515, name=qtp1044711881-1515, state=TIMED_WAITING, group=TGRP-DistribCursorPagingTest] [junit4] 2> at java.lang.Object.wait(Native Method) [junit4] 2> at org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:89) [junit4] 2> at org.apache.solr.update.DirectUpdateHandler2.deleteByQuery(DirectUpdateHandler2.java:409) [junit4] 2> at org.apache.solr.update.processor.RunUpdateProcessor.processDelete(RunUpdateProcessorFactory.java:80) [junit4] 2> at org.apache.solr.update.processor.UpdateRequestProcessor.processDelete(UpdateRequestProcessor.java:55) [junit4] 2> at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalDelete(DistributedUpdateProcessor.java:940) [junit4] 2> at org.apache.solr.update.processor.DistributedUpdateProcessor.doDeleteByQuery(DistributedUpdateProcessor.java:1351) [junit4] 2> at org.apache.solr.update.processor.DistributedUpdateProcessor.processDelete(DistributedUpdateProcessor.java:1133) [junit4] 2> at org.apache.solr.update.processor.LogUpdateProcessor.processDelete(LogUpdateProcessorFactory.java:125) [junit4] 2> at org.apache.solr.handler.loader.JavabinLoader.delete(JavabinLoader.java:150) [junit4] 2> at org.apache.solr.handler.loader.JavabinLoader.parseAndLoadDocs(JavabinLoader.java:111) [junit4] 2> at org.apache.solr.handler.loader.JavabinLoader.load(JavabinLoader.java:58) [junit4] 2> at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:103) [junit4] 2> at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74) [junit4] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:144) [junit4] 2> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2006) [junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:777) [junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:413) [junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:204) [junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) [junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:143) [junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) [junit4] 2> at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83) [junit4] 2> at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300) [junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) [junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585) [junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221) [junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1125) [junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) [junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) [junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1059) [junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) [junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) [junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:497) [junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:313) [junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:248) [junit4] 2> at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540) [junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:626) [junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:546) [junit4] 2> at java.lang.Thread.run(Thread.java:745) [junit4] 2> 14) Thread[id=1519, name=org.eclipse.jetty.server.session.HashSessionManager@1bb2343bTimer, state=TIMED_WAITING, group=TGRP-DistribCursorPagingTest] [junit4] 2> at sun.misc.Unsafe.park(Native Method) [junit4] 2> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) [junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) [junit4] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) [junit4] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) [junit4] 2> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) [junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) [junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [junit4] 2> at java.lang.Thread.run(Thread.java:745) [junit4] 2> 15) Thread[id=1518, name=Scheduler-1389002646, state=TIMED_WAITING, group=TGRP-DistribCursorPagingTest] [junit4] 2> at sun.misc.Unsafe.park(Native Method) [junit4] 2> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) [junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) [junit4] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) [junit4] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) [junit4] 2> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) [junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) [junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [junit4] 2> at java.lang.Thread.run(Thread.java:745) [junit4] 2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, locale=lv, timezone=Europe/Ljubljana [junit4] 2> NOTE: Linux 3.13.0-43-generic amd64/Oracle Corporation 1.8.0_31 (64-bit)/cpus=8,threads=20,free=159858832,total=527433728 [junit4] 2> NOTE: All tests run in this JVM: [CustomCollectionTest, TestSerializedLuceneMatchVersion, TestComponentsName, TestCoreContainer, TestBlobHandler, TestSolrIndexConfig, TestExtendedDismaxParser, DistributedIntervalFacetingTest, CursorPagingTest, SortByFunctionTest, DistributedQueryComponentOptimizationTest, TestManagedStopFilterFactory, DistribCursorPagingTest] [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=DistribCursorPagingTest -Dtests.seed=251A2C2425DBB840 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=lv -Dtests.timezone=Europe/Ljubljana -Dtests.asserts=true -Dtests.file.encoding=UTF-8 [junit4] ERROR 0.00s J0 | DistribCursorPagingTest (suite) <<< [junit4] > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec). [junit4] > at __randomizedtesting.SeedInfo.seed([251A2C2425DBB840]:0) [junit4] Completed on J0 in 7223.37s, 1 test, 2 errors <<< FAILURES! [...truncated 314 lines...] BUILD FAILED /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:519: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:61: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:39: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:198: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:510: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1348: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:956: There were test failures: 461 suites, 1844 tests, 1 suite-level error, 1 error, 399 ignored (19 assumptions) Total time: 153 minutes 22 seconds Build step 'Invoke Ant' marked build as failure [description-setter] Description set: Java: 64bit/jdk1.8.0_31 -XX:-UseCompressedOops -XX:+UseG1GC Archiving artifacts Recording test results Email was triggered for: Failure - Any Sending email for trigger: Failure - Any
--------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org