I've got a query that takes 15 seconds to return whenever I have the term
"book" in a query that isn't cached. That's a pretty common term in our
search index. We're indexing about 120 GB of text data. We only store terms
and IDs, no document data, and the disk is virtually unused, it's all CPU
time.
I haven't done much yet to optimizing and scale solr, as we're only trying
to support a small number of users in a private beta. I currently only have
a couple of gigs of ram dedicated to Solr (we've ordered more hardware for
it, but it's not in yet).
I wonder if there's something I can do in the short term to alleviate the
problem. Many searches work great, but these ones that take 15+ sec are a
black eye. I'd be happy with a short term fix followed in the near future by
a more proper long-term fix.
If I were to take a stab at this I'd say the following two are the short and
long term solutions:
. Short: Configure solr to short-circuit quries, thus reducing query
quality, but guaranteeing a certain response time (I'm ok with this
tradeoff, but is this possible or contains risks I need to consider more?)
. Long: Implement sharding, get more hardware resources for these
boxes and split up the index across multiple servers.
Am I on track in my thinking here?
Thanks,
David
My long query:
<?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">15464</int>
<lst name="params">
<str name="debugQuery">true</str>
<str name="indent">true</str>
<str name="q">cook book fourth baby
</str>
<str name="wt">xml</str>
</lst>
</lst>
<result name="response" numFound="91695127" start="0">
<doc>
<str name="id">1004505170125</str>
<long name="facetime">0</long>
<long name="_version_">1428753459934396420</long></doc>
<doc>
<str name="id">1005401904542</str>
<long name="facetime">0</long>
<long name="_version_">1428760446538612739</long></doc>
<doc>
<str name="id">1003707566177</str>
<long name="facetime">0</long>
<long name="_version_">1428772178357125123</long></doc>
<doc>
<str name="id">1000610053924</str>
<long name="facetime">0</long>
<long name="_version_">1428787238421921794</long></doc>
<doc>
<str name="id">1000611651986</str>
<long name="facetime">0</long>
<long name="_version_">1428796273825153026</long></doc>
<doc>
<str name="id">1001419625706</str>
<long name="facetime">0</long>
<long name="_version_">1428823418682212355</long></doc>
<doc>
<str name="id">1004804435353</str>
<long name="facetime">0</long>
<long name="_version_">1428823070202658818</long></doc>
<doc>
<str name="id">1000514089336</str>
<long name="facetime">0</long>
<long name="_version_">1428755943804370945</long></doc>
<doc>
<str name="id">1000329540261</str>
<long name="facetime">0</long>
<long name="_version_">1428805063523958786</long></doc>
<doc>
<str name="id">1001607999738</str>
<long name="facetime">0</long>
<long name="_version_">1428757460650295298</long></doc>
</result>
<lst name="debug">
<str name="rawquerystring">cook book fourth baby
</str>
<str name="querystring">cook book fourth baby
</str>
<str name="parsedquery">all:cook all:book all:fourth all:babi</str>
<str name="parsedquery_toString">all:cook all:book all:fourth
all:babi</str>
<lst name="explain">
<str name="1004505170125">
10.476225 = (MATCH) product of:
13.9683 = (MATCH) sum of:
7.988946 = (MATCH) weight(all:cook in 3428426) [DefaultSimilarity],
result of:
7.988946 = score(doc=3428426,freq=2.0 = termFreq=2.0
), product of:
0.5447923 = queryWeight, product of:
5.925234 = idf(docFreq=931212, maxDocs=128248074)
0.091944434 = queryNorm
14.664206 = fieldWeight in 3428426, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
5.925234 = idf(docFreq=931212, maxDocs=128248074)
1.75 = fieldNorm(doc=3428426)
0.29064935 = (MATCH) weight(all:book in 3428426) [DefaultSimilarity],
result of:
0.29064935 = score(doc=3428426,freq=1.0 = termFreq=1.0
), product of:
0.12357436 = queryWeight, product of:
1.3440113 = idf(docFreq=90917737, maxDocs=128248074)
0.091944434 = queryNorm
2.3520198 = fieldWeight in 3428426, product of:
1.0 = tf(freq=1.0), with freq of:
1.0 = termFreq=1.0
1.3440113 = idf(docFreq=90917737, maxDocs=128248074)
1.75 = fieldNorm(doc=3428426)
5.688705 = (MATCH) weight(all:babi in 3428426) [DefaultSimilarity],
result of:
5.688705 = score(doc=3428426,freq=1.0 = termFreq=1.0
), product of:
0.54670167 = queryWeight, product of:
5.9460006 = idf(docFreq=912073, maxDocs=128248074)
0.091944434 = queryNorm
10.405501 = fieldWeight in 3428426, product of:
1.0 = tf(freq=1.0), with freq of:
1.0 = termFreq=1.0
5.9460006 = idf(docFreq=912073, maxDocs=128248074)
1.75 = fieldNorm(doc=3428426)
0.75 = coord(3/4)
</str>
<str name="1005401904542">
10.476225 = (MATCH) product of:
13.9683 = (MATCH) sum of:
7.988946 = (MATCH) weight(all:cook in 4132020) [DefaultSimilarity],
result of:
7.988946 = score(doc=4132020,freq=2.0 = termFreq=2.0
), product of:
0.5447923 = queryWeight, product of:
5.925234 = idf(docFreq=931212, maxDocs=128248074)
0.091944434 = queryNorm
14.664206 = fieldWeight in 4132020, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
5.925234 = idf(docFreq=931212, maxDocs=128248074)
1.75 = fieldNorm(doc=4132020)
0.29064935 = (MATCH) weight(all:book in 4132020) [DefaultSimilarity],
result of:
0.29064935 = score(doc=4132020,freq=1.0 = termFreq=1.0
), product of:
0.12357436 = queryWeight, product of:
1.3440113 = idf(docFreq=90917737, maxDocs=128248074)
0.091944434 = queryNorm
2.3520198 = fieldWeight in 4132020, product of:
1.0 = tf(freq=1.0), with freq of:
1.0 = termFreq=1.0
1.3440113 = idf(docFreq=90917737, maxDocs=128248074)
1.75 = fieldNorm(doc=4132020)
5.688705 = (MATCH) weight(all:babi in 4132020) [DefaultSimilarity],
result of:
5.688705 = score(doc=4132020,freq=1.0 = termFreq=1.0
), product of:
0.54670167 = queryWeight, product of:
5.9460006 = idf(docFreq=912073, maxDocs=128248074)
0.091944434 = queryNorm
10.405501 = fieldWeight in 4132020, product of:
1.0 = tf(freq=1.0), with freq of:
1.0 = termFreq=1.0
5.9460006 = idf(docFreq=912073, maxDocs=128248074)
1.75 = fieldNorm(doc=4132020)
0.75 = coord(3/4)
</str>
<str name="1003707566177">
10.476225 = (MATCH) product of:
13.9683 = (MATCH) sum of:
7.988946 = (MATCH) weight(all:cook in 2139457) [DefaultSimilarity],
result of:
7.988946 = score(doc=2139457,freq=2.0 = termFreq=2.0
), product of:
0.5447923 = queryWeight, product of:
5.925234 = idf(docFreq=931212, maxDocs=128248074)
0.091944434 = queryNorm
14.664206 = fieldWeight in 2139457, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
5.925234 = idf(docFreq=931212, maxDocs=128248074)
1.75 = fieldNorm(doc=2139457)
0.29064935 = (MATCH) weight(all:book in 2139457) [DefaultSimilarity],
result of:
0.29064935 = score(doc=2139457,freq=1.0 = termFreq=1.0
), product of:
0.12357436 = queryWeight, product of:
1.3440113 = idf(docFreq=90917737, maxDocs=128248074)
0.091944434 = queryNorm
2.3520198 = fieldWeight in 2139457, product of:
1.0 = tf(freq=1.0), with freq of:
1.0 = termFreq=1.0
1.3440113 = idf(docFreq=90917737, maxDocs=128248074)
1.75 = fieldNorm(doc=2139457)
5.688705 = (MATCH) weight(all:babi in 2139457) [DefaultSimilarity],
result of:
5.688705 = score(doc=2139457,freq=1.0 = termFreq=1.0
), product of:
0.54670167 = queryWeight, product of:
5.9460006 = idf(docFreq=912073, maxDocs=128248074)
0.091944434 = queryNorm
10.405501 = fieldWeight in 2139457, product of:
1.0 = tf(freq=1.0), with freq of:
1.0 = termFreq=1.0
5.9460006 = idf(docFreq=912073, maxDocs=128248074)
1.75 = fieldNorm(doc=2139457)
0.75 = coord(3/4)
</str>
<str name="1000610053924">
10.476225 = (MATCH) product of:
13.9683 = (MATCH) sum of:
7.988946 = (MATCH) weight(all:cook in 1862392) [DefaultSimilarity],
result of:
7.988946 = score(doc=1862392,freq=2.0 = termFreq=2.0
), product of:
0.5447923 = queryWeight, product of:
5.925234 = idf(docFreq=931212, maxDocs=128248074)
0.091944434 = queryNorm
14.664206 = fieldWeight in 1862392, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
5.925234 = idf(docFreq=931212, maxDocs=128248074)
1.75 = fieldNorm(doc=1862392)
0.29064935 = (MATCH) weight(all:book in 1862392) [DefaultSimilarity],
result of:
0.29064935 = score(doc=1862392,freq=1.0 = termFreq=1.0
), product of:
0.12357436 = queryWeight, product of:
1.3440113 = idf(docFreq=90917737, maxDocs=128248074)
0.091944434 = queryNorm
2.3520198 = fieldWeight in 1862392, product of:
1.0 = tf(freq=1.0), with freq of:
1.0 = termFreq=1.0
1.3440113 = idf(docFreq=90917737, maxDocs=128248074)
1.75 = fieldNorm(doc=1862392)
5.688705 = (MATCH) weight(all:babi in 1862392) [DefaultSimilarity],
result of:
5.688705 = score(doc=1862392,freq=1.0 = termFreq=1.0
), product of:
0.54670167 = queryWeight, product of:
5.9460006 = idf(docFreq=912073, maxDocs=128248074)
0.091944434 = queryNorm
10.405501 = fieldWeight in 1862392, product of:
1.0 = tf(freq=1.0), with freq of:
1.0 = termFreq=1.0
5.9460006 = idf(docFreq=912073, maxDocs=128248074)
1.75 = fieldNorm(doc=1862392)
0.75 = coord(3/4)
</str>
<str name="1000611651986">
10.476225 = (MATCH) product of:
13.9683 = (MATCH) sum of:
7.988946 = (MATCH) weight(all:cook in 2009559) [DefaultSimilarity],
result of:
7.988946 = score(doc=2009559,freq=2.0 = termFreq=2.0
), product of:
0.5447923 = queryWeight, product of:
5.925234 = idf(docFreq=931212, maxDocs=128248074)
0.091944434 = queryNorm
14.664206 = fieldWeight in 2009559, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
5.925234 = idf(docFreq=931212, maxDocs=128248074)
1.75 = fieldNorm(doc=2009559)
0.29064935 = (MATCH) weight(all:book in 2009559) [DefaultSimilarity],
result of:
0.29064935 = score(doc=2009559,freq=1.0 = termFreq=1.0
), product of:
0.12357436 = queryWeight, product of:
1.3440113 = idf(docFreq=90917737, maxDocs=128248074)
0.091944434 = queryNorm
2.3520198 = fieldWeight in 2009559, product of:
1.0 = tf(freq=1.0), with freq of:
1.0 = termFreq=1.0
1.3440113 = idf(docFreq=90917737, maxDocs=128248074)
1.75 = fieldNorm(doc=2009559)
5.688705 = (MATCH) weight(all:babi in 2009559) [DefaultSimilarity],
result of:
5.688705 = score(doc=2009559,freq=1.0 = termFreq=1.0
), product of:
0.54670167 = queryWeight, product of:
5.9460006 = idf(docFreq=912073, maxDocs=128248074)
0.091944434 = queryNorm
10.405501 = fieldWeight in 2009559, product of:
1.0 = tf(freq=1.0), with freq of:
1.0 = termFreq=1.0
5.9460006 = idf(docFreq=912073, maxDocs=128248074)
1.75 = fieldNorm(doc=2009559)
0.75 = coord(3/4)
</str>
<str name="1001419625706">
10.476225 = (MATCH) product of:
13.9683 = (MATCH) sum of:
7.988946 = (MATCH) weight(all:cook in 1482373) [DefaultSimilarity],
result of:
7.988946 = score(doc=1482373,freq=2.0 = termFreq=2.0
), product of:
0.5447923 = queryWeight, product of:
5.925234 = idf(docFreq=931212, maxDocs=128248074)
0.091944434 = queryNorm
14.664206 = fieldWeight in 1482373, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
5.925234 = idf(docFreq=931212, maxDocs=128248074)
1.75 = fieldNorm(doc=1482373)
0.29064935 = (MATCH) weight(all:book in 1482373) [DefaultSimilarity],
result of:
0.29064935 = score(doc=1482373,freq=1.0 = termFreq=1.0
), product of:
0.12357436 = queryWeight, product of:
1.3440113 = idf(docFreq=90917737, maxDocs=128248074)
0.091944434 = queryNorm
2.3520198 = fieldWeight in 1482373, product of:
1.0 = tf(freq=1.0), with freq of:
1.0 = termFreq=1.0
1.3440113 = idf(docFreq=90917737, maxDocs=128248074)
1.75 = fieldNorm(doc=1482373)
5.688705 = (MATCH) weight(all:babi in 1482373) [DefaultSimilarity],
result of:
5.688705 = score(doc=1482373,freq=1.0 = termFreq=1.0
), product of:
0.54670167 = queryWeight, product of:
5.9460006 = idf(docFreq=912073, maxDocs=128248074)
0.091944434 = queryNorm
10.405501 = fieldWeight in 1482373, product of:
1.0 = tf(freq=1.0), with freq of:
1.0 = termFreq=1.0
5.9460006 = idf(docFreq=912073, maxDocs=128248074)
1.75 = fieldNorm(doc=1482373)
0.75 = coord(3/4)
</str>
<str name="1004804435353">
10.476225 = (MATCH) product of:
13.9683 = (MATCH) sum of:
7.988946 = (MATCH) weight(all:cook in 3353258) [DefaultSimilarity],
result of:
7.988946 = score(doc=3353258,freq=2.0 = termFreq=2.0
), product of:
0.5447923 = queryWeight, product of:
5.925234 = idf(docFreq=931212, maxDocs=128248074)
0.091944434 = queryNorm
14.664206 = fieldWeight in 3353258, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
5.925234 = idf(docFreq=931212, maxDocs=128248074)
1.75 = fieldNorm(doc=3353258)
0.29064935 = (MATCH) weight(all:book in 3353258) [DefaultSimilarity],
result of:
0.29064935 = score(doc=3353258,freq=1.0 = termFreq=1.0
), product of:
0.12357436 = queryWeight, product of:
1.3440113 = idf(docFreq=90917737, maxDocs=128248074)
0.091944434 = queryNorm
2.3520198 = fieldWeight in 3353258, product of:
1.0 = tf(freq=1.0), with freq of:
1.0 = termFreq=1.0
1.3440113 = idf(docFreq=90917737, maxDocs=128248074)
1.75 = fieldNorm(doc=3353258)
5.688705 = (MATCH) weight(all:babi in 3353258) [DefaultSimilarity],
result of:
5.688705 = score(doc=3353258,freq=1.0 = termFreq=1.0
), product of:
0.54670167 = queryWeight, product of:
5.9460006 = idf(docFreq=912073, maxDocs=128248074)
0.091944434 = queryNorm
10.405501 = fieldWeight in 3353258, product of:
1.0 = tf(freq=1.0), with freq of:
1.0 = termFreq=1.0
5.9460006 = idf(docFreq=912073, maxDocs=128248074)
1.75 = fieldNorm(doc=3353258)
0.75 = coord(3/4)
</str>
<str name="1000514089336">
9.333149 = (MATCH) product of:
12.444199 = (MATCH) sum of:
7.2630486 = (MATCH) weight(all:cook in 2988182) [DefaultSimilarity],
result of:
7.2630486 = score(doc=2988182,freq=9.0 = termFreq=9.0
), product of:
0.5447923 = queryWeight, product of:
5.925234 = idf(docFreq=931212, maxDocs=128248074)
0.091944434 = queryNorm
13.331776 = fieldWeight in 2988182, product of:
3.0 = tf(freq=9.0), with freq of:
9.0 = termFreq=9.0
5.925234 = idf(docFreq=931212, maxDocs=128248074)
0.75 = fieldNorm(doc=2988182)
0.30511826 = (MATCH) weight(all:book in 2988182) [DefaultSimilarity],
result of:
0.30511826 = score(doc=2988182,freq=6.0 = termFreq=6.0
), product of:
0.12357436 = queryWeight, product of:
1.3440113 = idf(docFreq=90917737, maxDocs=128248074)
0.091944434 = queryNorm
2.4691064 = fieldWeight in 2988182, product of:
2.4494898 = tf(freq=6.0), with freq of:
6.0 = termFreq=6.0
1.3440113 = idf(docFreq=90917737, maxDocs=128248074)
0.75 = fieldNorm(doc=2988182)
4.8760324 = (MATCH) weight(all:babi in 2988182) [DefaultSimilarity],
result of:
4.8760324 = score(doc=2988182,freq=4.0 = termFreq=4.0
), product of:
0.54670167 = queryWeight, product of:
5.9460006 = idf(docFreq=912073, maxDocs=128248074)
0.091944434 = queryNorm
8.919001 = fieldWeight in 2988182, product of:
2.0 = tf(freq=4.0), with freq of:
4.0 = termFreq=4.0
5.9460006 = idf(docFreq=912073, maxDocs=128248074)
0.75 = fieldNorm(doc=2988182)
0.75 = coord(3/4)
</str>
<str name="1000329540261">
9.333149 = (MATCH) product of:
12.444199 = (MATCH) sum of:
7.2630486 = (MATCH) weight(all:cook in 3352532) [DefaultSimilarity],
result of:
7.2630486 = score(doc=3352532,freq=9.0 = termFreq=9.0
), product of:
0.5447923 = queryWeight, product of:
5.925234 = idf(docFreq=931212, maxDocs=128248074)
0.091944434 = queryNorm
13.331776 = fieldWeight in 3352532, product of:
3.0 = tf(freq=9.0), with freq of:
9.0 = termFreq=9.0
5.925234 = idf(docFreq=931212, maxDocs=128248074)
0.75 = fieldNorm(doc=3352532)
0.30511826 = (MATCH) weight(all:book in 3352532) [DefaultSimilarity],
result of:
0.30511826 = score(doc=3352532,freq=6.0 = termFreq=6.0
), product of:
0.12357436 = queryWeight, product of:
1.3440113 = idf(docFreq=90917737, maxDocs=128248074)
0.091944434 = queryNorm
2.4691064 = fieldWeight in 3352532, product of:
2.4494898 = tf(freq=6.0), with freq of:
6.0 = termFreq=6.0
1.3440113 = idf(docFreq=90917737, maxDocs=128248074)
0.75 = fieldNorm(doc=3352532)
4.8760324 = (MATCH) weight(all:babi in 3352532) [DefaultSimilarity],
result of:
4.8760324 = score(doc=3352532,freq=4.0 = termFreq=4.0
), product of:
0.54670167 = queryWeight, product of:
5.9460006 = idf(docFreq=912073, maxDocs=128248074)
0.091944434 = queryNorm
8.919001 = fieldWeight in 3352532, product of:
2.0 = tf(freq=4.0), with freq of:
4.0 = termFreq=4.0
5.9460006 = idf(docFreq=912073, maxDocs=128248074)
0.75 = fieldNorm(doc=3352532)
0.75 = coord(3/4)
</str>
<str name="1001607999738">
8.979621 = (MATCH) product of:
11.972828 = (MATCH) sum of:
6.847668 = (MATCH) weight(all:cook in 524047) [DefaultSimilarity],
result of:
6.847668 = score(doc=524047,freq=2.0 = termFreq=2.0
), product of:
0.5447923 = queryWeight, product of:
5.925234 = idf(docFreq=931212, maxDocs=128248074)
0.091944434 = queryNorm
12.569319 = fieldWeight in 524047, product of:
1.4142135 = tf(freq=2.0), with freq of:
2.0 = termFreq=2.0
5.925234 = idf(docFreq=931212, maxDocs=128248074)
1.5 = fieldNorm(doc=524047)
0.24912801 = (MATCH) weight(all:book in 524047) [DefaultSimilarity],
result of:
0.24912801 = score(doc=524047,freq=1.0 = termFreq=1.0
), product of:
0.12357436 = queryWeight, product of:
1.3440113 = idf(docFreq=90917737, maxDocs=128248074)
0.091944434 = queryNorm
2.016017 = fieldWeight in 524047, product of:
1.0 = tf(freq=1.0), with freq of:
1.0 = termFreq=1.0
1.3440113 = idf(docFreq=90917737, maxDocs=128248074)
1.5 = fieldNorm(doc=524047)
4.8760324 = (MATCH) weight(all:babi in 524047) [DefaultSimilarity],
result of:
4.8760324 = score(doc=524047,freq=1.0 = termFreq=1.0
), product of:
0.54670167 = queryWeight, product of:
5.9460006 = idf(docFreq=912073, maxDocs=128248074)
0.091944434 = queryNorm
8.919001 = fieldWeight in 524047, product of:
1.0 = tf(freq=1.0), with freq of:
1.0 = termFreq=1.0
5.9460006 = idf(docFreq=912073, maxDocs=128248074)
1.5 = fieldNorm(doc=524047)
0.75 = coord(3/4)
</str>
</lst>
<str name="QParser">LuceneQParser</str>
<lst name="timing">
<double name="time">15464.0</double>
<lst name="prepare">
<double name="time">1.0</double>
<lst name="query">
<double name="time">1.0</double>
</lst>
<lst name="facet">
<double name="time">0.0</double>
</lst>
<lst name="mlt">
<double name="time">0.0</double>
</lst>
<lst name="highlight">
<double name="time">0.0</double>
</lst>
<lst name="stats">
<double name="time">0.0</double>
</lst>
<lst name="debug">
<double name="time">0.0</double>
</lst>
</lst>
<lst name="process">
<double name="time">15463.0</double>
<lst name="query">
<double name="time">15411.0</double>
</lst>
<lst name="facet">
<double name="time">0.0</double>
</lst>
<lst name="mlt">
<double name="time">0.0</double>
</lst>
<lst name="highlight">
<double name="time">0.0</double>
</lst>
<lst name="stats">
<double name="time">0.0</double>
</lst>
<lst name="debug">
<double name="time">52.0</double>
</lst>
</lst>
</lst>
</lst>
</response>
