Re: Simple search low speed
In case s1 will face the same problem, the thing was that SOLR caches were turned off, and I underestimated the meaning of caches in desire to save as much RAM as possible. -- View this message in context: http://lucene.472066.n3.nabble.com/Simple-search-low-speed-tp4202135p4202975.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Simple search low speed
Perhaps belated and IANA Solr expert, but take a look at mergeFactor and ramBufferSizeMB. I've had a situation where after Solr upgrade old values were kept. Those are used for buffering during indexing AFAIR. If buffer size is exceeded, merge happens. If merge factor is exceeded, coalesced segments are (AFAIR) written to IO, thus you might see high CPU. pozdrawiam, LAFK 2015-04-24 14:39 GMT+02:00 Joel Bernstein joels...@gmail.com: Try breaking down the query to see which part of it is slow. If it turns out to be the range query you may want to look into using an frange postfilter. Joel Bernstein http://joelsolr.blogspot.com/ On Fri, Apr 24, 2015 at 6:50 AM, Norgorn lsunnyd...@mail.ru wrote: Thanks for your reply. Yes, 100% CPU is used by SOLR (100% - I mean 1 core, not all cores), I'm totally sure. I have more than 80 GB RAM on test machine and about 50 is cached as disk cache, SOLR uses about 8, Xmx=40G. I use GC1, but it can't be the problem, cause memory usage is much lower than GC start limit (45% of heap). I think, the problem can be in fully optimized index, and search over one big segment is much slower than parallel search over lot of segments, but it sounds weird, so I'm not sure. Setups with big indexes which I know are all with optimized indexes. Index scheme: field name=text type=text_general indexed=true stored=false termVectors=true termPositions=true termOffsets=true / field name=title type=text_general indexed=true stored=false termVectors=true termPositions=true termOffsets=true / field name=cat type=tint indexed=true stored=true multiValued=false required=true omitNorms=true omitTermFreqAndPositions=true/ field name=date type=date indexed=true stored=false omitNorms=true omitTermFreqAndPositions=true/ field name=field1 type=int indexed=true stored=true required=false omitNorms=true omitTermFreqAndPositions=true/ field name=field2 type=int indexed=true stored=true required=false omitNorms=true omitTermFreqAndPositions=true/ -- View this message in context: http://lucene.472066.n3.nabble.com/Simple-search-low-speed-tp4202135p4202157.html Sent from the Solr - User mailing list archive at Nabble.com.
Simple search low speed
We have simple search over 50 GB index.
And it's slow.
I can't even wonder why, whole index is in RAM (and a lot of free space is
available) and CPU is a bottleneck (100% load).
The query is simple (except tvrh):
q=(text:(word1+word2)++title:(word1+word2))tv=trueisShard=trueqt=/tvrhfq=cat:(10+11+12)fq=field1:(150)fq=field2:(0)fq=date:[2015-01-01T00:00:00Z+TO+2015-04-24T23:59:59Z]
text, title - text_general fields
cat, field1, field2 - tint fields
date - a date field (I know, it's deprecated, will be changed soon).
All fields are indexed, some of them are stored.
And search time is 15 seconds (for warmed searcher, it's not the first
query).
debug=true shows timings process={time=15382.0,query={time=15282.0}
What can I check?
--
View this message in context:
http://lucene.472066.n3.nabble.com/Simple-search-low-speed-tp4202135.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Simple search low speed
Thanks for your reply. Yes, 100% CPU is used by SOLR (100% - I mean 1 core, not all cores), I'm totally sure. I have more than 80 GB RAM on test machine and about 50 is cached as disk cache, SOLR uses about 8, Xmx=40G. I use GC1, but it can't be the problem, cause memory usage is much lower than GC start limit (45% of heap). I think, the problem can be in fully optimized index, and search over one big segment is much slower than parallel search over lot of segments, but it sounds weird, so I'm not sure. Setups with big indexes which I know are all with optimized indexes. Index scheme: field name=text type=text_general indexed=true stored=false termVectors=true termPositions=true termOffsets=true / field name=title type=text_general indexed=true stored=false termVectors=true termPositions=true termOffsets=true / field name=cat type=tint indexed=true stored=true multiValued=false required=true omitNorms=true omitTermFreqAndPositions=true/ field name=date type=date indexed=true stored=false omitNorms=true omitTermFreqAndPositions=true/ field name=field1 type=int indexed=true stored=true required=false omitNorms=true omitTermFreqAndPositions=true/ field name=field2 type=int indexed=true stored=true required=false omitNorms=true omitTermFreqAndPositions=true/ -- View this message in context: http://lucene.472066.n3.nabble.com/Simple-search-low-speed-tp4202135p4202157.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Simple search low speed
The number of documents in collection is about 100m. -- View this message in context: http://lucene.472066.n3.nabble.com/Simple-search-low-speed-tp4202135p4202152.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Simple search low speed
Java side: - launch jvisualvm - see how heap and CPU are occupied What are your JVM settings (heap) and how much RAM do you have? The CPU100% is used only by Solr? That is, are you 100% certain it's Solr that drives CPU to it's limit? pozdrawiam, LAFK 2015-04-24 12:14 GMT+02:00 Norgorn lsunnyd...@mail.ru: The number of documents in collection is about 100m. -- View this message in context: http://lucene.472066.n3.nabble.com/Simple-search-low-speed-tp4202135p4202152.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Simple search low speed
Try breaking down the query to see which part of it is slow. If it turns out to be the range query you may want to look into using an frange postfilter. Joel Bernstein http://joelsolr.blogspot.com/ On Fri, Apr 24, 2015 at 6:50 AM, Norgorn lsunnyd...@mail.ru wrote: Thanks for your reply. Yes, 100% CPU is used by SOLR (100% - I mean 1 core, not all cores), I'm totally sure. I have more than 80 GB RAM on test machine and about 50 is cached as disk cache, SOLR uses about 8, Xmx=40G. I use GC1, but it can't be the problem, cause memory usage is much lower than GC start limit (45% of heap). I think, the problem can be in fully optimized index, and search over one big segment is much slower than parallel search over lot of segments, but it sounds weird, so I'm not sure. Setups with big indexes which I know are all with optimized indexes. Index scheme: field name=text type=text_general indexed=true stored=false termVectors=true termPositions=true termOffsets=true / field name=title type=text_general indexed=true stored=false termVectors=true termPositions=true termOffsets=true / field name=cat type=tint indexed=true stored=true multiValued=false required=true omitNorms=true omitTermFreqAndPositions=true/ field name=date type=date indexed=true stored=false omitNorms=true omitTermFreqAndPositions=true/ field name=field1 type=int indexed=true stored=true required=false omitNorms=true omitTermFreqAndPositions=true/ field name=field2 type=int indexed=true stored=true required=false omitNorms=true omitTermFreqAndPositions=true/ -- View this message in context: http://lucene.472066.n3.nabble.com/Simple-search-low-speed-tp4202135p4202157.html Sent from the Solr - User mailing list archive at Nabble.com.
