On 7/22/16 8:34 AM, Erick Erickson wrote:
Mostly this sounds like a problem that could be cured with
autowarming. But two things are conflicting here:
1> you say "We have a requirement to have updates available immediately (NRT)"
2> your docs aren't available for 120 seconds given your autoSoftCommit
settings unless you're specifying
-Dsolr.autoSoftCommit.maxTime=some_other_interval
as a startup parameter.
Yes. We have 120 seconds available.
So assuming you really do have a 120 second autocommit time, you should be
able to smooth out the spikes by appropriate autowarming. You also haven't
indicated what your filterCache and queryResultCache settings are. They
come with a default of 0 for autowarm. But what is their size? And do you
see a correlation between longer queries every on 2 minute intervals? And
do you have some test harness in place (jmeter works well) to demonstrate
that differences in your configuration help or hurt? I can't over-emphasize the
importance of this, otherwise if you rely on somebody simply saying "it's slow"
you have no way to know what effect changes have.
Here is the cache configuration.
<filterCache class="solr.FastLRUCache"
size="5000"
initialSize="5000"
autowarmCount="500"/>
<!-- Query Result Cache
Caches results of searches - ordered lists of document ids
(DocList) based on a query, a sort, and the range of documents
requested.
-->
<queryResultCache class="solr.LRUCache"
size="20000"
initialSize="20000"
autowarmCount="500"/>
<!-- Document Cache
Caches Lucene Document objects (the stored fields for each
document). Since Lucene internal document ids are transient,
this cache will not be autowarmed.
-->
<documentCache class="solr.LRUCache"
size="100000"
initialSize="100000"
autowarmCount="0"/>
We have run load tests using JMeter with directory pointing to Solr and
also tests that are pointing to the application that queries Solr. In
both cases, we have noticed the results being slower.
Thanks
Best,
Erick
On Thu, Jul 21, 2016 at 11:22 PM, Shawn Heisey <apa...@elyograg.org> wrote:
On 7/21/2016 11:25 PM, Rallavagu wrote:
There is no other software running on the system and it is completely
dedicated to Solr. It is running on Linux. Here is the full version.
Linux version 3.8.13-55.1.6.el7uek.x86_64
(mockbu...@ca-build56.us.oracle.com) (gcc version 4.8.3 20140911 (Red
Hat 4.8.3-9) (GCC) ) #2 SMP Wed Feb 11 14:18:22 PST 2015
Run the top program, press shift-M to sort by memory usage, and then
grab a screenshot of the terminal window. Share it with a site like
dropbox, imgur, or something similar, and send the URL. You'll end up
with something like this:
https://www.dropbox.com/s/zlvpvd0rrr14yit/linux-solr-top.png?dl=0
If you know what to look for, you can figure out all the relevant memory
details from that.
Thanks,
Shawn
