On Mar 6, 2007, at 1:55 PM, Yonik Seeley wrote:
On 3/6/07, Kaan Erdener <[EMAIL PROTECTED]> wrote:
I'm looking for some tips / suggestions around reducing the query
time for Solr after I've post'ed a commit request. My Lucene index
contains around 2,000,000 documents, and I have a job that
periodically removes artibrary documents from Lucene and replaces
them with fresh copies from a database. Whenever that cycle occurs, I
send a commit to Solr to expose the updates. The problem is that
immediately after the commit, a Solr query that previously took
5-20ms now takes 20-25 seconds. Ouch.
If this is a normal query (no faceting) then most likely the time
is spent
populating a lucene FieldCache entry used for sorting results.
Put a static warming entry in solrconfig.xml that queries for a small
number of documents and sorts that query by all the fields you
commonly sort by.
-Yonik
I'm not exactly sure this is what you meant, but I did some more
research and it looks close. I added the following to my solrconfig.xml:
<listener event="newSearcher" class="solr.QuerySenderListener">
<arr name="queries">
<lst> <str name="q">allMessageContent:test</str> <str
name="start">0</str> <str name="rows">10</str> </lst>
</arr>
</listener>
and also:
<listener event="firstSearcher" class="solr.QuerySenderListener">
<arr name="queries">
<lst> <str name="q">allMessageContent:trying</str> <str
name="start">0</str> <str name="rows">10</str> </lst>
</arr>
</listener>
From what I can see in the logs, these are both invoked after the
commit. However, the query times after a commit are still slow
(around 20 seconds). I'm guessing I didn't set up the warming
correctly? I had some sorting parameters in there, but the syntax was
wrong, produced errors on startup, so I took them out for now.
Mar 6, 2007 4:51:52 PM org.apache.solr.update.DirectUpdateHandler2
commit
INFO: end_commit_flush
Mar 6, 2007 4:51:52 PM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming [EMAIL PROTECTED] main from [EMAIL PROTECTED] main
documentCache
{lookups=10,hits=0,hitratio=0.00,inserts=20,evictions=0,size=20,cumulati
ve_lookups=120,cumulative_hits=68,cumulative_hitratio=0.56,cumulative_in
serts=52,cumulative_evictions=0}
Mar 6, 2007 4:51:52 PM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming result for [EMAIL PROTECTED] main
documentCache
{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,cumulative_
lookups=120,cumulative_hits=68,cumulative_hitratio=0.56,cumulative_inser
ts=52,cumulative_evictions=0}
Mar 6, 2007 4:51:52 PM org.apache.solr.core.QuerySenderListener
newSearcher
INFO: QuerySenderListener sending requests to [EMAIL PROTECTED] main
Mar 6, 2007 4:51:52 PM org.apache.solr.core.SolrCore execute
INFO: rows=10&start=0&q=allMessageContent:trying 0 410
Mar 6, 2007 4:51:52 PM org.apache.solr.core.QuerySenderListener
newSearcher
Confidentiality Notice: This e-mail message (including any attached or
embedded documents) is intended for the exclusive and confidential use of the
individual or entity to which this message is addressed, and unless otherwise
expressly indicated, is confidential and privileged information of Rackspace
Managed Hosting. Any dissemination, distribution or copying of the enclosed
material is prohibited. If you receive this transmission in error, please
notify us immediately by e-mail at [EMAIL PROTECTED], and delete the
original message. Your cooperation is appreciated.
