Hi All, I am running a Solr cloud 7.4 with 4 shards and 4 nodes (JVM "-Xms20g -Xmx40g”), each shard has 32 million documents and 32Gbytes in size.
For a given query (I use complexphrase query), typically, the first time it took a couple of seconds to return the first 20 docs. However, for the following page, or sorting by a field, even run the same query again took a lot longer to return results. I can see my 4 solr nodes running crazy with more than 100%CPU. My understanding is that Solr has query cache, run same query should be faster. What could be wrong here? How do I debug? I checked solr.log in all nodes and didn’t see anything unusual. Most frequent log entry looks like this. INFO - 2018-11-02 19:32:55.189; [ ] org.apache.solr.servlet.HttpSolrCall; [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.patternmatch.shard3.replica_n8:UPDATE./update.requests&key=solr.core.patternmatch.shard3.replica_n8:INDEX.sizeInBytes&key=solr.core.patternmatch.shard1.replica_n1:QUERY./select.requests&key=solr.core.patternmatch.shard1.replica_n1:INDEX.sizeInBytes&key=solr.core.patternmatch.shard1.replica_n1:UPDATE./update.requests&key=solr.core.patternmatch.shard3.replica_n8:QUERY./select.requests} status=0 QTime=7 INFO - 2018-11-02 19:32:55.192; [ ] org.apache.solr.servlet.HttpSolrCall; [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.jvm:os.processCpuLoad&key=solr.node:CONTAINER.fs.coreRoot.usableSpace&key=solr.jvm:os.systemLoadAverage&key=solr.jvm:memory.heap.used} status=0 QTime=1 Thank you for your kind help. Chuming