Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/11698/ Java: 64bit/jdk1.8.0_31 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC
1 tests failed. FAILED: junit.framework.TestSuite.org.apache.solr.cloud.MultiThreadedOCPTest Error Message: 1 thread leaked from SUITE scope at org.apache.solr.cloud.MultiThreadedOCPTest: 1) Thread[id=8976, name=OverseerThreadFactory-4615-thread-5, state=TIMED_WAITING, group=Overseer collection creation process.] at java.lang.Thread.sleep(Native Method) at org.apache.solr.cloud.OverseerCollectionProcessor.waitForCoreNodeName(OverseerCollectionProcessor.java:1841) at org.apache.solr.cloud.OverseerCollectionProcessor.splitShard(OverseerCollectionProcessor.java:1723) at org.apache.solr.cloud.OverseerCollectionProcessor.processMessage(OverseerCollectionProcessor.java:622) at org.apache.solr.cloud.OverseerCollectionProcessor$Runner.run(OverseerCollectionProcessor.java:2880) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Stack Trace: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.MultiThreadedOCPTest: 1) Thread[id=8976, name=OverseerThreadFactory-4615-thread-5, state=TIMED_WAITING, group=Overseer collection creation process.] at java.lang.Thread.sleep(Native Method) at org.apache.solr.cloud.OverseerCollectionProcessor.waitForCoreNodeName(OverseerCollectionProcessor.java:1841) at org.apache.solr.cloud.OverseerCollectionProcessor.splitShard(OverseerCollectionProcessor.java:1723) at org.apache.solr.cloud.OverseerCollectionProcessor.processMessage(OverseerCollectionProcessor.java:622) at org.apache.solr.cloud.OverseerCollectionProcessor$Runner.run(OverseerCollectionProcessor.java:2880) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) at __randomizedtesting.SeedInfo.seed([F9C450863DE111ED]:0) Build Log: [...truncated 9930 lines...] [junit4] Suite: org.apache.solr.cloud.MultiThreadedOCPTest [junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest F9C450863DE111ED-001/init-core-data-001 [junit4] 2> 1518183 T8748 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (true) [junit4] 2> 1518184 T8748 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: / [junit4] 2> 1518187 T8748 oasc.ZkTestServer.run STARTING ZK TEST SERVER [junit4] 1> client port:0.0.0.0/0.0.0.0:0 [junit4] 2> 1518188 T8749 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server [junit4] 2> 1518288 T8748 oasc.ZkTestServer.run start zk server on port:37606 [junit4] 2> 1518288 T8748 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 1518290 T8748 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 1518292 T8756 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2af01b1d name:ZooKeeperConnection Watcher:127.0.0.1:37606 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1518292 T8748 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 1518293 T8748 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 1518293 T8748 oascc.SolrZkClient.makePath makePath: /solr [junit4] 2> 1518297 T8748 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 1518298 T8748 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 1518299 T8759 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@b49fdc9 name:ZooKeeperConnection Watcher:127.0.0.1:37606/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1518299 T8748 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 1518300 T8748 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 1518300 T8748 oascc.SolrZkClient.makePath makePath: /collections/collection1 [junit4] 2> 1518301 T8748 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards [junit4] 2> 1518302 T8748 oascc.SolrZkClient.makePath makePath: /collections/control_collection [junit4] 2> 1518303 T8748 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards [junit4] 2> 1518304 T8748 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml [junit4] 2> 1518305 T8748 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml [junit4] 2> 1518306 T8748 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml [junit4] 2> 1518306 T8748 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml [junit4] 2> 1518308 T8748 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml [junit4] 2> 1518308 T8748 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml [junit4] 2> 1518309 T8748 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt [junit4] 2> 1518309 T8748 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt [junit4] 2> 1518310 T8748 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt [junit4] 2> 1518310 T8748 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt [junit4] 2> 1518312 T8748 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml [junit4] 2> 1518312 T8748 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml [junit4] 2> 1518314 T8748 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml [junit4] 2> 1518314 T8748 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml [junit4] 2> 1518316 T8748 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json [junit4] 2> 1518316 T8748 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json [junit4] 2> 1518317 T8748 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt [junit4] 2> 1518318 T8748 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt [junit4] 2> 1518319 T8748 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt [junit4] 2> 1518320 T8748 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt [junit4] 2> 1518321 T8748 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt [junit4] 2> 1518321 T8748 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt [junit4] 2> 1518327 T8748 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 1518328 T8748 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 1518330 T8762 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@188c9d03 name:ZooKeeperConnection Watcher:127.0.0.1:37606/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1518330 T8748 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 1518330 T8748 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 1518584 T8748 oas.SolrTestCaseJ4.writeCoreProperties Writing core.properties file to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest F9C450863DE111ED-001/control-001/cores/collection1 [junit4] 2> 1518586 T8748 oejs.Server.doStart jetty-9.2.6.v20141205 [junit4] 2> 1518587 T8748 oejsh.ContextHandler.doStart Started o.e.j.s.ServletContextHandler@539d64c1{/,null,AVAILABLE} [junit4] 2> 1518596 T8748 oejs.AbstractConnector.doStart Started ServerConnector@12847daf{SSL-http/1.1}{127.0.0.1:36513} [junit4] 2> 1518597 T8748 oejs.Server.doStart Started @1519934ms [junit4] 2> 1518598 T8748 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@c387f44 [junit4] 2> 1518599 T8748 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 1518599 T8748 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest F9C450863DE111ED-001/control-001 [junit4] 2> 1518599 T8748 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest F9C450863DE111ED-001/control-001/' [junit4] 2> 1518612 T8748 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest F9C450863DE111ED-001/control-001/solr.xml [junit4] 2> 1518617 T8748 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest F9C450863DE111ED-001/control-001/cores [junit4] 2> 1518618 T8748 oasc.CoreContainer.<init> New CoreContainer 1624880096 [junit4] 2> 1518618 T8748 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest F9C450863DE111ED-001/control-001/] [junit4] 2> 1518619 T8748 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000 [junit4] 2> 1518619 T8748 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: [junit4] 2> 1518619 T8748 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000 [junit4] 2> 1518619 T8748 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20 [junit4] 2> 1518620 T8748 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000 [junit4] 2> 1518620 T8748 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0 [junit4] 2> 1518620 T8748 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647 [junit4] 2> 1518620 T8748 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5 [junit4] 2> 1518620 T8748 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1 [junit4] 2> 1518621 T8748 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false [junit4] 2> 1518621 T8748 oashc.HttpShardHandlerFactory.getParameter Setting useRetries to: false [junit4] 2> 1518622 T8748 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true [junit4] 2> 1518623 T8748 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 1518623 T8748 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 1518623 T8748 oasc.CoreContainer.load Host Name: 127.0.0.1 [junit4] 2> 1518624 T8748 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:37606/solr [junit4] 2> 1518624 T8748 oasc.ZkController.checkChrootPath zkHost includes chroot [junit4] 2> 1518624 T8748 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 1518625 T8748 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 1518626 T8776 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@40d1f3d5 name:ZooKeeperConnection Watcher:127.0.0.1:37606 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1518626 T8748 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 1518626 T8748 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 1518628 T8748 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 1518629 T8779 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@92c85b5 name:ZooKeeperConnection Watcher:127.0.0.1:37606/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1518629 T8748 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 1518630 T8748 oascc.SolrZkClient.makePath makePath: /overseer/queue [junit4] 2> 1518632 T8748 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work [junit4] 2> 1518633 T8748 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running [junit4] 2> 1518635 T8748 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed [junit4] 2> 1518636 T8748 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure [junit4] 2> 1518638 T8748 oascc.SolrZkClient.makePath makePath: /live_nodes [junit4] 2> 1518639 T8748 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36513_ [junit4] 2> 1518640 T8748 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:36513_ [junit4] 2> 1518641 T8748 oascc.SolrZkClient.makePath makePath: /overseer_elect [junit4] 2> 1518642 T8748 oascc.SolrZkClient.makePath makePath: /overseer_elect/election [junit4] 2> 1518643 T8748 oasc.Overseer.close Overseer (id=null) closing [junit4] 2> 1518644 T8748 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:36513_ [junit4] 2> 1518645 T8748 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader [junit4] 2> 1518646 T8748 oasc.Overseer.start Overseer (id=93218998535979012-127.0.0.1:36513_-n_0000000000) starting [junit4] 2> 1518647 T8748 oascc.SolrZkClient.makePath makePath: /overseer/queue-work [junit4] 2> 1518652 T8748 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000 [junit4] 2> 1518653 T8781 oasc.OverseerCollectionProcessor.run Process current queue of collection creations [junit4] 2> 1518653 T8748 oascc.SolrZkClient.makePath makePath: /clusterstate.json [junit4] 2> 1518656 T8748 oascc.SolrZkClient.makePath makePath: /aliases.json [junit4] 2> 1518657 T8748 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 1518659 T8780 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue [junit4] 2> 1518659 T8748 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest F9C450863DE111ED-001/control-001/cores [junit4] 2> 1518660 T8748 oasc.CorePropertiesLocator.discoverUnder Found core collection1 in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest F9C450863DE111ED-001/control-001/cores/collection1/ [junit4] 2> 1518660 T8748 oasc.CorePropertiesLocator.discover Found 1 core definitions [junit4] 2> 1518661 T8783 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection [junit4] 2> 1518661 T8783 oasc.ZkController.publish numShards not found on descriptor - reading it from system property [junit4] 2> 1518662 T8779 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 1518662 T8783 oasc.ZkController.waitForCoreNodeName look for our core node name [junit4] 2> 1518663 T8780 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 0, message = { [junit4] 2> "core":"collection1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"https://127.0.0.1:36513", [junit4] 2> "node_name":"127.0.0.1:36513_", [junit4] 2> "numShards":"1", [junit4] 2> "state":"down", [junit4] 2> "shard":null, [junit4] 2> "collection":"control_collection", [junit4] 2> "operation":"state"} current state version: 0 [junit4] 2> 1518664 T8780 oasco.ReplicaMutator.updateState Update state numShards=1 message={ [junit4] 2> "core":"collection1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"https://127.0.0.1:36513", [junit4] 2> "node_name":"127.0.0.1:36513_", [junit4] 2> "numShards":"1", [junit4] 2> "state":"down", [junit4] 2> "shard":null, [junit4] 2> "collection":"control_collection", [junit4] 2> "operation":"state"} [junit4] 2> 1518664 T8780 oasco.ClusterStateMutator.createCollection building a new cName: control_collection [junit4] 2> 1518664 T8780 oasco.ReplicaMutator.updateState Assigning new node to shard shard=shard1 [junit4] 2> 1518665 T8779 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1) [junit4] 2> 1519662 T8783 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1 [junit4] 2> 1519663 T8783 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection [junit4] 2> 1519663 T8783 oasc.ZkController.createCollectionZkNode Collection zkNode exists [junit4] 2> 1519663 T8783 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection [junit4] 2> 1519665 T8783 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper [junit4] 2> 1519665 T8783 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest F9C450863DE111ED-001/control-001/cores/collection1/' [junit4] 2> 1519673 T8783 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1 [junit4] 2> 1519675 T8783 oasc.Config.<init> loaded config solrconfig.xml with version 0 [junit4] 2> 1519679 T8783 oasc.SolrConfig.refreshRequestParams current version of requestparams : -1 [junit4] 2> 1519684 T8783 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0 [junit4] 2> 1519706 T8783 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml [junit4] 2> 1519708 T8783 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml [junit4] 2> 1519712 T8783 oass.IndexSchema.readSchema [collection1] Schema name=test [junit4] 2> 1519837 T8783 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440. [junit4] 2> 1519840 T8783 oass.IndexSchema.readSchema default search field in schema is text [junit4] 2> 1519841 T8783 oass.IndexSchema.readSchema unique key field: id [junit4] 2> 1519849 T8783 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 1519851 T8783 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 1519853 T8783 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4] 2> 1519854 T8783 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE [junit4] 2> 1519855 T8783 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING [junit4] 2> 1519855 T8783 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4] 2> 1519856 T8783 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE [junit4] 2> 1519856 T8783 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING [junit4] 2> 1519856 T8783 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection [junit4] 2> 1519856 T8783 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory [junit4] 2> 1519857 T8783 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest F9C450863DE111ED-001/control-001/cores/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest F9C450863DE111ED-001/control-001/cores/collection1/data/ [junit4] 2> 1519857 T8783 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1dfcd914 [junit4] 2> 1519857 T8783 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest F9C450863DE111ED-001/control-001/cores/collection1/data [junit4] 2> 1519858 T8783 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest F9C450863DE111ED-001/control-001/cores/collection1/data/index/ [junit4] 2> 1519858 T8783 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest F9C450863DE111ED-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index... [junit4] 2> 1519858 T8783 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest F9C450863DE111ED-001/control-001/cores/collection1/data/index [junit4] 2> 1519858 T8783 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=38, maxMergeAtOnceExplicit=33, maxMergedSegmentMB=48.4775390625, floorSegmentMB=1.2421875, forceMergeDeletesPctAllowed=17.980402849398626, segmentsPerTier=47.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0 [junit4] 2> 1519859 T8783 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1 [junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@18ed8612 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4a65fdd5),segFN=segments_1,generation=1} [junit4] 2> 1519860 T8783 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1 [junit4] 2> 1519861 T8783 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib" [junit4] 2> 1519862 T8783 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe" [junit4] 2> 1519862 T8783 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe" [junit4] 2> 1519862 T8783 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig" [junit4] 2> 1519862 T8783 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig" [junit4] 2> 1519862 T8783 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit" [junit4] 2> 1519863 T8783 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit" [junit4] 2> 1519863 T8783 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit" [junit4] 2> 1519863 T8783 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default [junit4] 2> 1519863 T8783 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 1519864 T8783 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 1519864 T8783 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 1519864 T8783 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 1519865 T8783 oasc.RequestHandlers.initHandlersFromConfig created /config: org.apache.solr.handler.SolrConfigHandler [junit4] 2> 1519865 T8783 oasc.RequestHandlers.initHandlersFromConfig created /schema: org.apache.solr.handler.SchemaHandler [junit4] 2> 1519865 T8783 oasc.RequestHandlers.initHandlersFromConfig created /replication: org.apache.solr.handler.ReplicationHandler [junit4] 2> 1519865 T8783 oasc.RequestHandlers.initHandlersFromConfig created /get: org.apache.solr.handler.RealTimeGetHandler [junit4] 2> 1519866 T8783 oasc.RequestHandlers.initHandlersFromConfig created /admin/luke: org.apache.solr.handler.admin.LukeRequestHandler [junit4] 2> 1519866 T8783 oasc.RequestHandlers.initHandlersFromConfig created /admin/system: org.apache.solr.handler.admin.SystemInfoHandler [junit4] 2> 1519866 T8783 oasc.RequestHandlers.initHandlersFromConfig created /admin/mbeans: org.apache.solr.handler.admin.SolrInfoMBeanHandler [junit4] 2> 1519867 T8783 oasc.RequestHandlers.initHandlersFromConfig created /admin/plugins: org.apache.solr.handler.admin.PluginInfoHandler [junit4] 2> 1519867 T8783 oasc.RequestHandlers.initHandlersFromConfig created /admin/threads: org.apache.solr.handler.admin.ThreadDumpHandler [junit4] 2> 1519867 T8783 oasc.RequestHandlers.initHandlersFromConfig created /admin/properties: org.apache.solr.handler.admin.PropertiesRequestHandler [junit4] 2> 1519867 T8783 oasc.RequestHandlers.initHandlersFromConfig created /admin/logging: org.apache.solr.handler.admin.LoggingHandler [junit4] 2> 1519868 T8783 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: org.apache.solr.handler.PingRequestHandler [junit4] 2> 1519868 T8783 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler [junit4] 2> 1519868 T8783 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler [junit4] 2> 1519869 T8783 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 1519870 T8783 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 1519871 T8783 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 1519871 T8783 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 1519872 T8783 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache [junit4] 2> 1519872 T8783 oasu.CommitTracker.<init> Hard AutoCommit: disabled [junit4] 2> 1519873 T8783 oasu.CommitTracker.<init> Soft AutoCommit: disabled [junit4] 2> 1519873 T8783 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=16, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7119256815174956] [junit4] 2> 1519873 T8783 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1 [junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@18ed8612 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4a65fdd5),segFN=segments_1,generation=1} [junit4] 2> 1519873 T8783 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1 [junit4] 2> 1519874 T8783 oass.SolrIndexSearcher.<init> Opening Searcher@335cd933[collection1] main [junit4] 2> 1519874 T8783 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection [junit4] 2> 1519874 T8783 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper [junit4] 2> 1519875 T8783 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1 [junit4] 2> 1519875 T8783 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1 [junit4] 2> 1519875 T8783 oasr.RestManager.init Initializing RestManager with initArgs: {} [junit4] 2> 1519875 T8783 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 [junit4] 2> 1519875 T8783 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json [junit4] 2> 1519876 T8783 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 [junit4] 2> 1519876 T8783 oasr.RestManager.init Initializing 0 registered ManagedResources [junit4] 2> 1519876 T8783 oash.ReplicationHandler.inform Commits will be reserved for 10000 [junit4] 2> 1519876 T8783 oasc.CoreContainer.registerCore registering core: collection1 [junit4] 2> 1519876 T8784 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@335cd933[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())} [junit4] 2> 1519877 T8787 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:36513 collection:control_collection shard:shard1 [junit4] 2> 1519877 T8748 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1 [junit4] 2> 1519877 T8748 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done [junit4] 2> 1519877 T8787 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election [junit4] 2> 1519879 T8748 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 1519880 T8787 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1 [junit4] 2> 1519880 T8748 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 1519881 T8790 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7b9906aa name:ZooKeeperConnection Watcher:127.0.0.1:37606/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1519881 T8779 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 1519881 T8748 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 1519882 T8780 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = { [junit4] 2> "operation":"leader", [junit4] 2> "shard":"shard1", [junit4] 2> "collection":"control_collection"} current state version: 1 [junit4] 2> 1519882 T8787 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue. [junit4] 2> 1519882 T8748 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 1519882 T8787 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync [junit4] 2> ASYNC NEW_CORE C4683 name=collection1 org.apache.solr.core.SolrCore@743fee01 url=https://127.0.0.1:36513/collection1 node=127.0.0.1:36513_ C4683_STATE=coll:control_collection core:collection1 props:{core=collection1, base_url=https://127.0.0.1:36513, node_name=127.0.0.1:36513_, state=down} [junit4] 2> 1519883 T8787 C4683 P36513 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:36513/collection1/ [junit4] 2> 1519883 T8748 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 1519883 T8787 C4683 P36513 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me [junit4] 2> 1519884 T8787 C4683 P36513 oasc.SyncStrategy.syncToMe https://127.0.0.1:36513/collection1/ has no replicas [junit4] 2> 1519884 T8787 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:36513/collection1/ shard1 [junit4] 2> 1519884 T8787 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1 [junit4] 2> 1519885 T8748 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false [junit4] 2> 1519886 T8779 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 1519887 T8780 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = { [junit4] 2> "operation":"leader", [junit4] 2> "shard":"shard1", [junit4] 2> "collection":"control_collection", [junit4] 2> "base_url":"https://127.0.0.1:36513", [junit4] 2> "core":"collection1", [junit4] 2> "state":"active"} current state version: 1 [junit4] 2> 1519989 T8779 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1) [junit4] 2> 1519989 T8790 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1) [junit4] 2> 1520037 T8787 oasc.ZkController.register We are https://127.0.0.1:36513/collection1/ and leader is https://127.0.0.1:36513/collection1/ [junit4] 2> 1520038 T8787 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:36513 [junit4] 2> 1520038 T8787 oasc.ZkController.checkRecovery I am the leader, no recovery necessary [junit4] 2> 1520038 T8787 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection [junit4] 2> 1520038 T8787 oasc.ZkController.publish numShards not found on descriptor - reading it from system property [junit4] 2> 1520039 T8779 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 1520041 T8780 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = { [junit4] 2> "core":"collection1", [junit4] 2> "core_node_name":"core_node1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"https://127.0.0.1:36513", [junit4] 2> "node_name":"127.0.0.1:36513_", [junit4] 2> "numShards":"2", [junit4] 2> "state":"active", [junit4] 2> "shard":"shard1", [junit4] 2> "collection":"control_collection", [junit4] 2> "operation":"state"} current state version: 2 [junit4] 2> 1520041 T8780 oasco.ReplicaMutator.updateState Update state numShards=2 message={ [junit4] 2> "core":"collection1", [junit4] 2> "core_node_name":"core_node1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"https://127.0.0.1:36513", [junit4] 2> "node_name":"127.0.0.1:36513_", [junit4] 2> "numShards":"2", [junit4] 2> "state":"active", [junit4] 2> "shard":"shard1", [junit4] 2> "collection":"control_collection", [junit4] 2> "operation":"state"} [junit4] 2> 1520143 T8748 oas.SolrTestCaseJ4.writeCoreProperties Writing core.properties file to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest F9C450863DE111ED-001/shard-1-001/cores/collection1 [junit4] 2> 1520144 T8779 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1) [junit4] 2> 1520144 T8790 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1) [junit4] 2> 1520144 T8748 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1 [junit4] 2> 1520146 T8748 oejs.Server.doStart jetty-9.2.6.v20141205 [junit4] 2> 1520147 T8748 oejsh.ContextHandler.doStart Started o.e.j.s.ServletContextHandler@515a0b74{/,null,AVAILABLE} [junit4] 2> 1520151 T8748 oejs.AbstractConnector.doStart Started ServerConnector@748c80b2{SSL-http/1.1}{127.0.0.1:33971} [junit4] 2> 1520152 T8748 oejs.Server.doStart Started @1521490ms [junit4] 2> 1520153 T8748 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@c387f44 [junit4] 2> 1520154 T8748 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 1520154 T8748 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest F9C450863DE111ED-001/shard-1-001 [junit4] 2> 1520154 T8748 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest F9C450863DE111ED-001/shard-1-001/' [junit4] 2> 1520164 T8748 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest F9C450863DE111ED-001/shard-1-001/solr.xml [junit4] 2> 1520170 T8748 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest F9C450863DE111ED-001/shard-1-001/cores [junit4] 2> 1520170 T8748 oasc.CoreContainer.<init> New CoreContainer 1058599099 [junit4] 2> 1520171 T8748 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest F9C450863DE111ED-001/shard-1-001/] [junit4] 2> 1520171 T8748 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000 [junit4] 2> 1520172 T8748 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: [junit4] 2> 1520172 T8748 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000 [junit4] 2> 1520172 T8748 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20 [junit4] 2> 1520172 T8748 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000 [junit4] 2> 1520172 T8748 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0 [junit4] 2> 1520173 T8748 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647 [junit4] 2> 1520173 T8748 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5 [junit4] 2> 1520173 T8748 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1 [junit4] 2> 1520173 T8748 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false [junit4] 2> 1520173 T8748 oashc.HttpShardHandlerFactory.getParameter Setting useRetries to: false [junit4] 2> 1520175 T8748 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true [junit4] 2> 1520176 T8748 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 1520176 T8748 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 1520176 T8748 oasc.CoreContainer.load Host Name: 127.0.0.1 [junit4] 2> 1520177 T8748 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:37606/solr [junit4] 2> 1520177 T8748 oasc.ZkController.checkChrootPath zkHost includes chroot [junit4] 2> 1520177 T8748 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 1520178 T8748 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 1520180 T8804 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2a4cf89b name:ZooKeeperConnection Watcher:127.0.0.1:37606 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1520180 T8748 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 1520180 T8748 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 1520183 T8748 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 1520184 T8807 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@41d869b7 name:ZooKeeperConnection Watcher:127.0.0.1:37606/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1520184 T8748 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 1520188 T8748 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 1521191 T8748 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33971_ [junit4] 2> 1521192 T8748 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:33971_ [junit4] 2> 1521194 T8748 oasc.Overseer.close Overseer (id=null) closing [junit4] 2> 1521196 T8748 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest F9C450863DE111ED-001/shard-1-001/cores [junit4] 2> 1521196 T8748 oasc.CorePropertiesLocator.discoverUnder Found core collection1 in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest F9C450863DE111ED-001/shard-1-001/cores/collection1/ [junit4] 2> 1521196 T8748 oasc.CorePropertiesLocator.discover Found 1 core definitions [junit4] 2> 1521198 T8808 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1 [junit4] 2> 1521198 T8808 oasc.ZkController.publish numShards not found on descriptor - reading it from system property [junit4] 2> 1521198 T8808 oasc.ZkController.waitForCoreNodeName look for our core node name [junit4] 2> 1521198 T8779 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 1521199 T8780 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = { [junit4] 2> "core":"collection1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"https://127.0.0.1:33971", [junit4] 2> "node_name":"127.0.0.1:33971_", [junit4] 2> "numShards":"2", [junit4] 2> "state":"down", [junit4] 2> "shard":null, [junit4] 2> "collection":"collection1", [junit4] 2> "operation":"state"} current state version: 3 [junit4] 2> 1521200 T8780 oasco.ReplicaMutator.updateState Update state numShards=2 message={ [junit4] 2> "core":"collection1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"https://127.0.0.1:33971", [junit4] 2> "node_name":"127.0.0.1:33971_", [junit4] 2> "numShards":"2", [junit4] 2> "state":"down", [junit4] 2> "shard":null, [junit4] 2> "collection":"collection1", [junit4] 2> "operation":"state"} [junit4] 2> 1521200 T8780 oasco.ClusterStateMutator.createCollection building a new cName: collection1 [junit4] 2> 1521200 T8780 oasco.ReplicaMutator.updateState Assigning new node to shard shard=shard2 [junit4] 2> 1521303 T8790 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2) [junit4] 2> 1521303 T8807 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2) [junit4] 2> 1521303 T8779 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2) [junit4] 2> 1522199 T8808 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1 [junit4] 2> 1522199 T8808 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1 [junit4] 2> 1522200 T8808 oasc.ZkController.createCollectionZkNode Collection zkNode exists [junit4] 2> 1522200 T8808 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1 [junit4] 2> 1522201 T8808 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper [junit4] 2> 1522201 T8808 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest F9C450863DE111ED-001/shard-1-001/cores/collection1/' [junit4] 2> 1522213 T8808 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1 [junit4] 2> 1522215 T8808 oasc.Config.<init> loaded config solrconfig.xml with version 0 [junit4] 2> 1522219 T8808 oasc.SolrConfig.refreshRequestParams current version of requestparams : -1 [junit4] 2> 1522223 T8808 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0 [junit4] 2> 1522257 T8808 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml [junit4] 2> 1522258 T8808 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml [junit4] 2> 1522263 T8808 oass.IndexSchema.readSchema [collection1] Schema name=test [junit4] 2> 1522407 T8808 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440. [junit4] 2> 1522410 T8808 oass.IndexSchema.readSchema default search field in schema is text [junit4] 2> 1522411 T8808 oass.IndexSchema.readSchema unique key field: id [junit4] 2> 1522417 T8808 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 1522419 T8808 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 1522421 T8808 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4] 2> 1522422 T8808 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE [junit4] 2> 1522422 T8808 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING [junit4] 2> 1522423 T8808 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4] 2> 1522423 T8808 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE [junit4] 2> 1522424 T8808 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING [junit4] 2> 1522424 T8808 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1 [junit4] 2> 1522424 T8808 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory [junit4] 2> 1522424 T8808 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest F9C450863DE111ED-001/shard-1-001/cores/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest F9C450863DE111ED-001/shard-1-001/cores/collection1/data/ [junit4] 2> 1522424 T8808 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1dfcd914 [junit4] 2> 1522425 T8808 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest F9C450863DE111ED-001/shard-1-001/cores/collection1/data [junit4] 2> 1522425 T8808 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest F9C450863DE111ED-001/shard-1-001/cores/collection1/data/index/ [junit4] 2> 1522425 T8808 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest F9C450863DE111ED-001/shard-1-001/cores/collection1/data/index' doesn't exist. Creating new index... [junit4] 2> 1522426 T8808 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest F9C450863DE111ED-001/shard-1-001/cores/collection1/data/index [junit4] 2> 1522426 T8808 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=38, maxMergeAtOnceExplicit=33, maxMergedSegmentMB=48.4775390625, floorSegmentMB=1.2421875, forceMergeDeletesPctAllowed=17.980402849398626, segmentsPerTier=47.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0 [junit4] 2> 1522426 T8808 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1 [junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1d6572de lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@671128cd),segFN=segments_1,generation=1} [junit4] 2> 1522427 T8808 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1 [junit4] 2> 1522429 T8808 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib" [junit4] 2> 1522429 T8808 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe" [junit4] 2> 1522429 T8808 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe" [junit4] 2> 1522430 T8808 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig" [junit4] 2> 1522430 T8808 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig" [junit4] 2> 1522430 T8808 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit" [junit4] 2> 1522430 T8808 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit" [junit4] 2> 1522430 T8808 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit" [junit4] 2> 1522430 T8808 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default [junit4] 2> 1522431 T8808 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 1522431 T8808 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 1522432 T8808 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 1522432 T8808 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 1522432 T8808 oasc.RequestHandlers.initHandlersFromConfig created /config: org.apache.solr.handler.SolrConfigHandler [junit4] 2> 1522432 T8808 oasc.RequestHandlers.initHandlersFromConfig created /schema: org.apache.solr.handler.SchemaHandler [junit4] 2> 1522433 T8808 oasc.RequestHandlers.initHandlersFromConfig created /replication: org.apache.solr.handler.ReplicationHandler [junit4] 2> 1522433 T8808 oasc.RequestHandlers.initHandlersFromConfig created /get: org.apache.solr.handler.RealTimeGetHandler [junit4] 2> 1522433 T8808 oasc.RequestHandlers.initHandlersFromConfig created /admin/luke: org.apache.solr.handler.admin.LukeRequestHandler [junit4] 2> 1522434 T8808 oasc.RequestHandlers.initHandlersFromConfig created /admin/system: org.apache.solr.handler.admin.SystemInfoHandler [junit4] 2> 1522434 T8808 oasc.RequestHandlers.initHandlersFromConfig created /admin/mbeans: org.apache.solr.handler.admin.SolrInfoMBeanHandler [junit4] 2> 1522435 T8808 oasc.RequestHandlers.initHandlersFromConfig created /admin/plugins: org.apache.solr.handler.admin.PluginInfoHandler [junit4] 2> 1522435 T8808 oasc.RequestHandlers.initHandlersFromConfig created /admin/threads: org.apache.solr.handler.admin.ThreadDumpHandler [junit4] 2> 1522435 T8808 oasc.RequestHandlers.initHandlersFromConfig created /admin/properties: org.apache.solr.handler.admin.PropertiesRequestHandler [junit4] 2> 1522436 T8808 oasc.RequestHandlers.initHandlersFromConfig created /admin/logging: org.apache.solr.handler.admin.LoggingHandler [junit4] 2> 1522436 T8808 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: org.apache.solr.handler.PingRequestHandler [junit4] 2> 1522436 T8808 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler [junit4] 2> 1522437 T8808 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler [junit4] 2> 1522438 T8808 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 1522439 T8808 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 1522440 T8808 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 1522441 T8808 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 1522442 T8808 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache [junit4] 2> 1522443 T8808 oasu.CommitTracker.<init> Hard AutoCommit: disabled [junit4] 2> 1522443 T8808 oasu.CommitTracker.<init> Soft AutoCommit: disabled [junit4] 2> 1522443 T8808 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=16, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7119256815174956] [junit4] 2> 1522444 T8808 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1 [junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1d6572de lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@671128cd),segFN=segments_1,generation=1} [junit4] 2> 1522444 T8808 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1 [junit4] 2> 1522444 T8808 oass.SolrIndexSearcher.<init> Opening Searcher@2e8f8bf4[collection1] main [junit4] 2> 1522444 T8808 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1 [junit4] 2> 1522445 T8808 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper [junit4] 2> 1522445 T8808 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1 [junit4] 2> 1522445 T8808 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1 [junit4] 2> 1522446 T8808 oasr.RestManager.init Initializing RestManager with initArgs: {} [junit4] 2> 1522446 T8808 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 [junit4] 2> 1522446 T8808 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json [junit4] 2> 1522446 T8808 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 [junit4] 2> 1522446 T8808 oasr.RestManager.init Initializing 0 registered ManagedResources [junit4] 2> 1522447 T8808 oash.ReplicationHandler.inform Commits will be reserved for 10000 [junit4] 2> 1522447 T8809 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2e8f8bf4[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())} [junit4] 2> 1522448 T8808 oasc.CoreContainer.registerCore registering core: collection1 [junit4] 2> 1522448 T8812 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:33971 collection:collection1 shard:shard2 [junit4] 2> 1522449 T8748 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1 [junit4] 2> 1522449 T8812 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election [junit4] 2> 1522449 T8748 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done [junit4] 2> 1522451 T8812 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2 [junit4] 2> 1522452 T8779 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 1522452 T8812 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue. [junit4] 2> 1522452 T8812 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync [junit4] 2> ASYNC NEW_CORE C4684 name=collection1 org.apache.solr.core.SolrCore@469a2c64 url=https://127.0.0.1:33971/collection1 node=127.0.0.1:33971_ C4684_STATE=coll:collection1 core:collection1 props:{core=collection1, base_url=https://127.0.0.1:33971, node_name=127.0.0.1:33971_, state=down} [junit4] 2> 1522453 T8812 C4684 P33971 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:33971/collection1/ [junit4] 2> 1522453 T8780 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = { [junit4] 2> "operation":"leader", [junit4] 2> "shard":"shard2", [junit4] 2> "collection":"collection1"} current state version: 4 [junit4] 2> 1522453 T8812 C4684 P33971 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me [junit4] 2> 1522453 T8812 C4684 P33971 oasc.SyncStrategy.syncToMe https://127.0.0.1:33971/collection1/ has no replicas [junit4] 2> 1522453 T8812 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:33971/collection1/ shard2 [junit4] 2> 1522453 T8812 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2 [junit4] 2> 1522455 T8779 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 1522456 T8780 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = { [junit4] 2> "operation":"leader", [junit4] 2> "shard":"shard2", [junit4] 2> "collection":"collection1", [junit4] 2> "base_url":"https://127.0.0.1:33971", [junit4] 2> "core":"collection1", [junit4] 2> "state":"active"} current state version: 4 [junit4] 2> 1522558 T8779 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2) [junit4] 2> 1522558 T8807 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2) [junit4] 2> 1522558 T8790 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2) [junit4] 2> 1522606 T8812 oasc.ZkController.register We are https://127.0.0.1:33971/collection1/ and leader is https://127.0.0.1:33971/collection1/ [junit4] 2> 1522607 T8812 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:33971 [junit4] 2> 1522607 T8812 oasc.ZkController.checkRecovery I am the leader, no recovery necessary [junit4] 2> 1522607 T8812 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1 [junit4] 2> 1522607 T8812 oasc.ZkController.publish numShards not found on descriptor - reading it from system property [junit4] 2> 1522608 T8779 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 1522609 T8780 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = { [junit4] 2> "core":"collection1", [junit4] 2> "core_node_name":"core_node1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"https://127.0.0.1:33971", [junit4] 2> "node_name":"127.0.0.1:33971_", [junit4] 2> "numShards":"2", [junit4] 2> "state":"active", [junit4] 2> "shard":"shard2", [junit4] 2> "collection":"collection1", [junit4] 2> "operation":"state"} current state version: 5 [junit4] 2> 1522610 T8780 oasco.ReplicaMutator.updateState Update state numShards=2 message={ [junit4] 2> "core":"collection1", [junit4] 2> "core_node_name":"core_node1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"https://127.0.0.1:33971", [junit4] 2> "node_name":"127.0.0.1:33971_", [junit4] 2> "numShards":"2", [junit4] 2> "state":"active", [junit4] 2> "shard":"shard2", [junit4] 2> "collection":"collection1", [junit4] 2> "operation":"state"} [junit4] 2> 1522704 T8748 oas.SolrTestCaseJ4.writeCoreProperties Writing core.properties file to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest F9C450863DE111ED-001/shard-2-001/cores/collection1 [junit4] 2> 1522705 T8748 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2 [junit4] 2> 1522706 T8748 oejs.Server.doStart jetty-9.2.6.v20141205 [junit4] 2> 1522708 T8748 oejsh.ContextHandler.doStart Started o.e.j.s.ServletContextHandler@7075bb29{/,null,AVAILABLE} [junit4] 2> 1522712 T8748 oejs.AbstractConnector.doStart Started ServerConnector@61110337{SSL-http/1.1}{127.0.0.1:38973} [junit4] 2> 1522712 T8748 oejs.Server.doStart Started @1524050ms [junit4] 2> 1522713 T8807 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2) [junit4] 2> 1522713 T8790 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2) [junit4] 2> 1522713 T8779 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2) [junit4] 2> 1522714 T8748 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@c387f44 [junit4] 2> 1522714 T8748 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx) [junit4] 2> 1522714 T8748 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest F9C450863DE111ED-001/shard-2-001 [junit4] 2> 1522715 T8748 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest F9C450863DE111ED-001/shard-2-001/' [junit4] 2> 1522725 T8748 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest F9C450863DE111ED-001/shard-2-001/solr.xml [junit4] 2> 1522729 T8748 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest F9C450863DE111ED-001/shard-2-001/cores [junit4] 2> 1522729 T8748 oasc.CoreContainer.<init> New CoreContainer 129757061 [junit4] 2> 1522730 T8748 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest F9C450863DE111ED-001/shard-2-001/] [junit4] 2> 1522730 T8748 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000 [junit4] 2> 1522730 T8748 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: [junit4] 2> 1522730 T8748 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000 [junit4] 2> 1522731 T8748 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20 [junit4] 2> 1522731 T8748 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000 [junit4] 2> 1522731 T8748 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0 [junit4] 2> 1522731 T8748 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647 [junit4] 2> 1522732 T8748 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5 [junit4] 2> 1522732 T8748 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1 [junit4] 2> 1522732 T8748 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false [junit4] 2> 1522732 T8748 oashc.HttpShardHandlerFactory.getParameter Setting useRetries to: false [junit4] 2> 1522733 T8748 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true [junit4] 2> 1522735 T8748 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory [junit4] 2> 1522735 T8748 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] [junit4] 2> 1522735 T8748 oasc.CoreContainer.load Host Name: 127.0.0.1 [junit4] 2> 1522735 T8748 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:37606/solr [junit4] 2> 1522736 T8748 oasc.ZkController.checkChrootPath zkHost includes chroot [junit4] 2> 1522736 T8748 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2> 1522736 T8748 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 1522738 T8826 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@337b484b name:ZooKeeperConnection Watcher:127.0.0.1:37606 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1522738 T8748 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 1522739 T8748 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2> 1522741 T8748 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2> 1522743 T8829 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@26957931 name:ZooKeeperConnection Watcher:127.0.0.1:37606/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2> 1522743 T8748 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2> 1522746 T8748 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... [junit4] 2> 1523749 T8748 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38973_ [junit4] 2> 1523750 T8748 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:38973_ [junit4] 2> 1523753 T8748 oasc.Overseer.close Overseer (id=null) closing [junit4] 2> 1523756 T8748 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest F9C450863DE111ED-001/shard-2-001/cores [junit4] 2> 1523757 T8748 oasc.CorePropertiesLocator.discoverUnder Found core collection1 in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest F9C450863DE111ED-001/shard-2-001/cores/collection1/ [junit4] 2> 1523757 T8748 oasc.CorePropertiesLocator.discover Found 1 core definitions [junit4] 2> 1523758 T8830 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1 [junit4] 2> 1523758 T8830 oasc.ZkController.publish numShards not found on descriptor - reading it from system property [junit4] 2> 1523759 T8779 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 1523759 T8830 oasc.ZkController.waitForCoreNodeName look for our core node name [junit4] 2> 1523760 T8780 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = { [junit4] 2> "core":"collection1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"https://127.0.0.1:38973", [junit4] 2> "node_name":"127.0.0.1:38973_", [junit4] 2> "numShards":"2", [junit4] 2> "state":"down", [junit4] 2> "shard":null, [junit4] 2> "collection":"collection1", [junit4] 2> "operation":"state"} current state version: 6 [junit4] 2> 1523760 T8780 oasco.ReplicaMutator.updateState Update state numShards=2 message={ [junit4] 2> "core":"collection1", [junit4] 2> "roles":null, [junit4] 2> "base_url":"https://127.0.0.1:38973", [junit4] 2> "node_name":"127.0.0.1:38973_", [junit4] 2> "numShards":"2", [junit4] 2> "state":"down", [junit4] 2> "shard":null, [junit4] 2> "collection":"collection1", [junit4] 2> "operation":"state"} [junit4] 2> 1523761 T8780 oasco.ReplicaMutator.updateState Collection already exists with numShards=2 [junit4] 2> 1523761 T8780 oasco.ReplicaMutator.updateState Assigning new node to shard shard=shard1 [junit4] 2> 1523864 T8829 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3) [junit4] 2> 1523864 T8779 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3) [junit4] 2> 1523864 T8790 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3) [junit4] 2> 1523864 T8807 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3) [junit4] 2> 1524759 T8830 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1 [junit4] 2> 1524760 T8830 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1 [junit4] 2> 1524760 T8830 oasc.ZkController.createCollectionZkNode Collection zkNode exists [junit4] 2> 1524761 T8830 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1 [junit4] 2> 1524762 T8830 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper [junit4] 2> 1524762 T8830 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest F9C450863DE111ED-001/shard-2-001/cores/collection1/' [junit4] 2> 1524770 T8830 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1 [junit4] 2> 1524772 T8830 oasc.Config.<init> loaded config solrconfig.xml with version 0 [junit4] 2> 1524775 T8830 oasc.SolrConfig.refreshRequestParams current version of requestparams : -1 [junit4] 2> 1524789 T8830 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0 [junit4] 2> 1524801 T8830 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml [junit4] 2> 1524802 T8830 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml [junit4] 2> 1524806 T8830 oass.IndexSchema.readSchema [collection1] Schema name=test [junit4] 2> 1524932 T8830 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440. [junit4] 2> 1524935 T8830 oass.IndexSchema.readSchema default search field in schema is text [junit4] 2> 1524936 T8830 oass.IndexSchema.readSchema unique key field: id [junit4] 2> 1524943 T8830 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 1524945 T8830 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml [junit4] 2> 1524947 T8830 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4] 2> 1524948 T8830 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE [junit4] 2> 1524948 T8830 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING [junit4] 2> 1524949 T8830 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json [junit4] 2> 1524949 T8830 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE [junit4] 2> 1524950 T8830 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING [junit4] 2> 1524950 T8830 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1 [junit4] 2> 1524950 T8830 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory [junit4] 2> 1524950 T8830 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest F9C450863DE111ED-001/shard-2-001/cores/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest F9C450863DE111ED-001/shard-2-001/cores/collection1/data/ [junit4] 2> 1524950 T8830 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1dfcd914 [junit4] 2> 1524951 T8830 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest F9C450863DE111ED-001/shard-2-001/cores/collection1/data [junit4] 2> 1524951 T8830 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest F9C450863DE111ED-001/shard-2-001/cores/collection1/data/index/ [junit4] 2> 1524951 T8830 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest F9C450863DE111ED-001/shard-2-001/cores/collection1/data/index' doesn't exist. Creating new index... [junit4] 2> 1524952 T8830 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest F9C450863DE111ED-001/shard-2-001/cores/collection1/data/index [junit4] 2> 1524952 T8830 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=38, maxMergeAtOnceExplicit=33, maxMergedSegmentMB=48.4775390625, floorSegmentMB=1.2421875, forceMergeDeletesPctAllowed=17.980402849398626, segmentsPerTier=47.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0 [junit4] 2> 1524952 T8830 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1 [junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@2dca7fe2 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2c63d53d),segFN=segments_1,generation=1} [junit4] 2> 1524953 T8830 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1 [junit4] 2> 1524955 T8830 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib" [junit4] 2> 1524955 T8830 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe" [junit4] 2> 1524955 T8830 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe" [junit4] 2> 1524956 T8830 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig" [junit4] 2> 1524956 T8830 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig" [junit4] 2> 1524956 T8830 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit" [junit4] 2> 1524956 T8830 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit" [junit4] 2> 1524956 T8830 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit" [junit4] 2> 1524956 T8830 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default [junit4] 2> 1524957 T8830 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 1524957 T8830 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 1524958 T8830 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 1524958 T8830 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler [junit4] 2> 1524958 T8830 oasc.RequestHandlers.initHandlersFromConfig created /config: org.apache.solr.handler.SolrConfigHandler [junit4] 2> 1524959 T8830 oasc.RequestHandlers.initHandlersFromConfig created /schema: org.apache.solr.handler.SchemaHandler [junit4] 2> 1524959 T8830 oasc.RequestHandlers.initHandlersFromConfig created /replication: org.apache.solr.handler.ReplicationHandler [junit4] 2> 1524959 T8830 oasc.RequestHandlers.initHandlersFromConfig created /get: org.apache.solr.handler.RealTimeGetHandler [junit4] 2> 1524960 T8830 oasc.RequestHandlers.initHandlersFromConfig created /admin/luke: org.apache.solr.handler.admin.LukeRequestHandler [junit4] 2> 1524960 T8830 oasc.RequestHandlers.initHandlersFromConfig created /admin/system: org.apache.solr.handler.admin.SystemInfoHandler [junit4] 2> 1524961 T8830 oasc.RequestHandlers.initHandlersFromConfig created /admin/mbeans: org.apache.solr.handler.admin.SolrInfoMBeanHandler [junit4] 2> 1524961 T8830 oasc.RequestHandlers.initHandlersFromConfig created /admin/plugins: org.apache.solr.handler.admin.PluginInfoHandler [junit4] 2> 1524961 T8830 oasc.RequestHandlers.initHandlersFromConfig created /admin/threads: org.apache.solr.handler.admin.ThreadDumpHandler [junit4] 2> 1524961 T8830 oasc.RequestHandlers.initHandlersFromConfig created /admin/properties: org.apache.solr.handler.admin.PropertiesRequestHandler [junit4] 2> 1524962 T8830 oasc.RequestHandlers.initHandlersFromConfig created /admin/logging: org.apache.solr.handler.admin.LoggingHandler [junit4] 2> 1524962 T8830 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: org.apache.solr.handler.PingRequestHandler [junit4] 2> 1524962 T8830 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler [junit4] 2> 1524963 T8830 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler [junit4] 2> 1524964 T8830 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 1524965 T8830 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 1524966 T8830 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 1524966 T8830 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60 [junit4] 2> 1524967 T8830 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache [junit4] 2> 1524967 T8830 oasu.CommitTracker.<init> Hard AutoCommit: disabled [junit4] 2> 1524967 T8830 oasu.CommitTracker.<init> Soft AutoCommit: disabled [junit4] 2> 1524968 T8830 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=16, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7119256815174956] [junit4] 2> 1524968 T8830 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1 [junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@2dca7fe2 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2c63d53d),segFN=segments_1,generation=1} [junit4] 2> 1524968 T8830 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1 [junit4] 2> 1524968 T8830 oass.SolrIndexSearcher.<init> Opening Searcher@741c9b38[collection1] main [junit4] 2> 1524969 T8830 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1 [junit4] 2> 1524969 T8830 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper [junit4] 2> 1524970 T8830 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1 [junit4] 2> 1524970 T8830 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1 [junit4] 2> 1524970 T8830 oasr.RestManager.init Initializing RestManager with initArgs: {} [junit4] 2> 1524970 T8830 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 [junit4] 2> 1524971 T8830 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json [junit4] 2> 1524971 T8830 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1 [junit4] 2> 1524971 T8830 oasr.RestManager.init Initializing 0 registered ManagedResources [junit4] 2> 1524971 T8830 oash.ReplicationHandler.inform Commits will be reserved for 10000 [junit4] 2> 1524972 T8831 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@741c9b38[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())} [junit4] 2> 1524972 T8830 oasc.CoreContainer.registerCore registering core: collection1 [junit4] 2> 1524972 T8834 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:38973 collection:collection1 shard:shard1 [junit4] 2> 1524973 T8748 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1 [junit4] 2> 1524973 T8748 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done [junit4] 2> 1524973 T8834 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election [junit4] 2> 1524980 T8834 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1 [junit4] 2> 1524981 T8779 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected [junit4] 2> 1524982 T8834 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue. [junit4] 2> 1524982 T8834 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync [junit4] 2> ASYNC NEW_CORE C4685 name=collection1 org.apache.solr.core.SolrCore@7e8af269 url=https://127.0.0.1:38973/collection1 node=127.0.0.1:38973_ C4685_STATE=coll:collection1 core:collection1 props:{core=collection1, base_url=https://127.0.0.1:38973, node_name=127.0.0.1:38973_, state=down} [junit4] 2> 1524983 T8834 C4685 P38973 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:38973/collection1/ [junit4] 2> 1524982 T8780 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = { [junit4] 2> "operation":"leader", [junit4] 2> "shard":"shard1", [junit4] 2> "collection":"collection1"} current state version: 7 [junit4] 2> 1524984 T8834 C4685 P38973 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me [junit4] 2> 1524984 T8834 C4685 P38973 oasc.SyncStrategy.syncToMe https://127.0.0.1:38973/collection1/ has no replicas [junit4] 2> 1524984 T8834 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:38973/collection1/ shard1 [junit4] 2> 1524985 T8834 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1 [junit4] 2> 1524988 T8779 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /ov [...truncated too long message...] junit4] 2> at org.apache.solr.cloud.DistributedMap.createData(DistributedMap.java:112) [junit4] 2> at org.apache.solr.cloud.DistributedMap.put(DistributedMap.java:125) [junit4] 2> at org.apache.solr.cloud.OverseerCollectionProcessor$Runner.run(OverseerCollectionProcessor.java:2888) [junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [junit4] 2> at java.lang.Thread.run(Thread.java:745) [junit4] 2> [junit4] 2> 1672147 T8976 oasc.OverseerCollectionProcessor$Runner.resetTaskWithException WARN Resetting task: /overseer/collection-queue-work/qn-0000000010, requestid: 2000, collectionName: collection1 [junit4] 2> 1672147 T8976 oasc.SolrException.log ERROR :org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer/collection-map-running/mn-2000 [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127) [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) [junit4] 2> at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873) [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:243) [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:240) [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61) [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.delete(SolrZkClient.java:240) [junit4] 2> at org.apache.solr.cloud.DistributedMap.remove(DistributedMap.java:167) [junit4] 2> at org.apache.solr.cloud.OverseerCollectionProcessor$Runner.resetTaskWithException(OverseerCollectionProcessor.java:2948) [junit4] 2> at org.apache.solr.cloud.OverseerCollectionProcessor$Runner.run(OverseerCollectionProcessor.java:2916) [junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [junit4] 2> at java.lang.Thread.run(Thread.java:745) [junit4] 2> [junit4] 2> 153964 T8747 ccr.ThreadLeakControl.tryToInterruptAll All leaked threads terminated. [junit4] 2> NOTE: test params are: codec=Asserting(Lucene50): {_version_=Lucene50(blocksize=128), multiDefault=FSTOrd50, intDefault=Lucene50(blocksize=128), id=PostingsFormat(name=LuceneFixedGap), range_facet_l=FSTOrd50, timestamp=FSTOrd50}, docValues:{timestamp=DocValuesFormat(name=Direct)}, sim=DefaultSimilarity, locale=hr_HR, timezone=America/Asuncion [junit4] 2> NOTE: Linux 3.13.0-44-generic amd64/Oracle Corporation 1.8.0_31 (64-bit)/cpus=8,threads=1,free=121002656,total=456409088 [junit4] 2> NOTE: All tests run in this JVM: [SolrRequestParserTest, TestConfig, XsltUpdateRequestHandlerTest, MBeansHandlerTest, TestRawResponseWriter, TestConfigSets, TestCodecSupport, DirectSolrSpellCheckerTest, MinimalSchemaTest, TestReplicaProperties, TestFileDictionaryLookup, TestSearchPerf, TestDynamicLoading, TestRecovery, AtomicUpdatesTest, TestLMJelinekMercerSimilarityFactory, PreAnalyzedUpdateProcessorTest, SuggestComponentTest, FieldMutatingUpdateProcessorTest, DistanceUnitsTest, BasicFunctionalityTest, LoggingHandlerTest, TestCloudPivotFacet, TestNRTOpen, FacetPivotSmallTest, TestSolrQueryParserResource, RegexBoostProcessorTest, TestConfigOverlay, QueryEqualityTest, TestLMDirichletSimilarityFactory, DocumentBuilderTest, TestSolrDeletionPolicy2, TestHighlightDedupGrouping, SpatialFilterTest, TestRangeQuery, HdfsUnloadDistributedZkTest, BadIndexSchemaTest, ShardRoutingCustomTest, TestDocSet, TestCollationFieldDocValues, TestCloudManagedSchema, MigrateRouteKeyTest, TestFreeTextSuggestions, SchemaVersionSpecificBehaviorTest, TestQuerySenderNoQuery, TestSolrConfigHandler, DateRangeFieldTest, BasicZkTest, QueryParsingTest, TestIntervalFaceting, TestMergePolicyConfig, RankQueryTest, SignatureUpdateProcessorFactoryTest, TestPivotHelperCode, TestFastWriter, CurrencyFieldXmlFileTest, TestZkChroot, TestPHPSerializedResponseWriter, TestMaxScoreQueryParser, StatelessScriptUpdateProcessorFactoryTest, TestSchemaNameResource, IndexSchemaTest, TestExactSharedStatsCache, SpellCheckComponentTest, TestSolr4Spatial2, TestManagedSynonymFilterFactory, TestRemoteStreaming, TestExactStatsCache, CSVRequestHandlerTest, TestDynamicFieldResource, TestDefaultSearchFieldResource, TestHighFrequencyDictionaryFactory, DateFieldTest, TestSimpleQParserPlugin, TestRTGBase, DirectUpdateHandlerOptimizeTest, OpenExchangeRatesOrgProviderTest, SpatialRPTFieldTypeTest, ResourceLoaderTest, AddBlockUpdateTest, HdfsRecoveryZkTest, SharedFSAutoReplicaFailoverUtilsTest, SolrXmlInZkTest, TestShardHandlerFactory, TestFuzzyAnalyzedSuggestions, EnumFieldTest, TestInitQParser, PolyFieldTest, TestSystemIdResolver, TestReload, TestSchemaSimilarityResource, LeaderElectionIntegrationTest, UpdateRequestProcessorFactoryTest, TestExpandComponent, DistributedTermsComponentTest, TestRequestStatusCollectionAPI, TestSolrCoreProperties, TestRecoveryHdfs, ZkNodePropsTest, MergeStrategyTest, StressHdfsTest, TestUtils, FullSolrCloudDistribCmdsTest, SimplePostToolTest, TestRandomFaceting, JsonLoaderTest, NotRequiredUniqueKeyTest, CurrencyFieldOpenExchangeTest, TestDefaultSimilarityFactory, HdfsDirectoryTest, BasicDistributedZkTest, ChaosMonkeySafeLeaderTest, BasicDistributedZk2Test, UnloadDistributedZkTest, CollectionsAPIDistributedZkTest, OpenCloseCoreStressTest, ShardRoutingTest, TestReplicationHandler, AutoCommitTest, TestFaceting, DistributedSpellCheckComponentTest, TermVectorComponentDistributedTest, TestRealTimeGet, TestJoin, SolrCmdDistributorTest, QueryElevationComponentTest, TestFiltering, SolrCoreCheckLockOnStartupTest, TestWordDelimiterFilterFactory, DocValuesMultiTest, SolrInfoMBeanTest, CacheHeaderTest, LukeRequestHandlerTest, TestQueryUtils, TestWriterPerf, PrimitiveFieldTypeTest, TermsComponentTest, TestValueSourceCache, TestIndexingPerformance, TestSolrQueryParser, ReturnFieldsTest, BinaryUpdateRequestHandlerTest, SearchHandlerTest, HighlighterConfigTest, SOLR749Test, TestSolrIndexConfig, ResponseLogComponentTest, SolrIndexConfigTest, TestBinaryField, TestElisionMultitermQuery, TestLuceneMatchVersion, SpellPossibilityIteratorTest, TestXIncludeConfig, TestDFRSimilarityFactory, TestPerFieldSimilarity, TestJmxMonitoredMap, URLClassifyProcessorTest, PluginInfoTest, TestFastLRUCache, PrimUtilsTest, DOMUtilTest, ClusterStateTest, TestLRUCache, SliceStateTest, UUIDFieldTest, DistributedIntervalFacetingTest, CloudExitableDirectoryReaderTest, ConnectionManagerTest, CustomCollectionTest, DeleteReplicaTest, DistribDocExpirationUpdateProcessorTest, LeaderInitiatedRecoveryOnCommitTest, MultiThreadedOCPTest] [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=MultiThreadedOCPTest -Dtests.seed=F9C450863DE111ED -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=hr_HR -Dtests.timezone=America/Asuncion -Dtests.asserts=true -Dtests.file.encoding=US-ASCII [junit4] ERROR 0.00s J1 | MultiThreadedOCPTest (suite) <<< [junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.MultiThreadedOCPTest: [junit4] > 1) Thread[id=8976, name=OverseerThreadFactory-4615-thread-5, state=TIMED_WAITING, group=Overseer collection creation process.] [junit4] > at java.lang.Thread.sleep(Native Method) [junit4] > at org.apache.solr.cloud.OverseerCollectionProcessor.waitForCoreNodeName(OverseerCollectionProcessor.java:1841) [junit4] > at org.apache.solr.cloud.OverseerCollectionProcessor.splitShard(OverseerCollectionProcessor.java:1723) [junit4] > at org.apache.solr.cloud.OverseerCollectionProcessor.processMessage(OverseerCollectionProcessor.java:622) [junit4] > at org.apache.solr.cloud.OverseerCollectionProcessor$Runner.run(OverseerCollectionProcessor.java:2880) [junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [junit4] > at java.lang.Thread.run(Thread.java:745) [junit4] > at __randomizedtesting.SeedInfo.seed([F9C450863DE111ED]:0) [junit4] Completed on J1 in 154.00s, 1 test, 1 error <<< FAILURES! [...truncated 281 lines...] BUILD FAILED /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:519: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:61: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:39: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:191: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:510: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1348: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:956: There were test failures: 461 suites, 1845 tests, 1 suite-level error, 45 ignored (20 assumptions) Total time: 61 minutes 53 seconds Build step 'Invoke Ant' marked build as failure [description-setter] Description set: Java: 64bit/jdk1.8.0_31 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC Archiving artifacts Recording test results Email was triggered for: Failure - Any Sending email for trigger: Failure - Any
--------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org