Hi Eric,
Measuring running queries via JMeter. Values provided are rounded median
of multiple samples. Medians are just slightly better than 99th
percentile for all samples.
Filter cache is useless as you mentioned; they are effectively not used.
There is auto-warming through cache autoWarm but no auto-warming
queries.
Small experiment with passing &NOW=... seems not to make any difference
which would not be surprising given caches are barely involved.
Thanks for the suggestion on IO. After stopping indexing, the response
time barely changed on Solr 5. On Solr 4, with indexing running it is
still fast. So to effectively, Solr 4 under indexing load is faster than
idle Solr 5. Both set-ups have same heap size and available RAM on
machine (2x heap).
One other thing I am testing is issuing request to specific core, with
distrib=false. No significant improvements there.
Now what is interesting is that aforementioned query takes the same
amount of time to execute despite the number of documents found.
- Whether it is 0 or 10k, it takes couple seconds on Solr 5.5.0.
- Meanwhile, on Solr 4.10.3, the response time is dependent on results
size. For Solr 4 no results returns in few ms and few seconds for couple
thousands of results.
(query used {!cache=false}q=...)
Thanks,
Jarek
On Wed, 27 Apr 2016, at 04:39, Erick Erickson wrote:
> Well, the first question is always "how are you measuring this"?
> Measuring a few queries is almost completely uninformative,
> especially if the two systems have differing warmups. The only
> meaningful measurements are when throwing away the first bunch
> of queries then measuring a meaningful sample.
>
> The setup you describe will be very sensitive to disk access
> with the autowarm of 1 second, so if there's much at all in
> the way of differences in I/O that would be a red flag.
>
> From here on down doesn't really respond to the question, but
> I thought I'd mention it.
>
> And you don't have to worry about disabling your fitlerCache since
> any filter query of the form fq=field:[mention NOW in here without
> rounding]
> never be re-used. So you might as well use {!cache=false}. Here's the
> background:
>
> https://lucidworks.com/blog/2012/02/23/date-math-now-and-filter-queries/
>
> And your soft commit is probably throwing out all the filter caches
> anyway.
>
> I doubt you're doing any autowarming at all given the autocommit interval
> of 1 second and continuously updating documents and your reported
> query times. So you can pretty much forget what I said about throwing
> out your first N queries since you're (probably) not getting any benefit
> out of caches anyway.
>
> On Tue, Apr 26, 2016 at 10:34 AM, Jaroslaw Rozanski
> <s...@jarekrozanski.com> wrote:
> > Hi all,
> >
> > I am migrating a large Solr Cloud cluster from Solr 4.10 to Solr 5.5.0
> > and I observed big difference in query execution time.
> >
> > First a setup summary:
> > - multiple collections - 6
> > - each has multiple shards - 6
> > - same/similar hardware
> > - indexing tens of messages per second
> > - autoSoftCommit with 1s; hard commit few tens of seconds
> > - Java 8
> >
> > The query has following form: field1:[* TO NOW-14DAYS] OR (-field1:[* TO
> > *] AND field2:[* TO NOW-14DAYS])
> >
> > The fields field1 & field2 are of date type:
> > <fieldType name="date" class="solr.TrieDateField" precisionStep="0"
> > positionIncrementGap="0"/>
> >
> > As query (q={!cache=false}...)
> > Solr 4.10 -> 5s
> > Solr 5.5.0 -> 12s
> >
> > As filter query (q={!cache=false}*:*&fq=..,)
> > Solr 4.10 -> 9s
> > Solr 5.5.0 -> 11s
> >
> > The query itself is bad and its optimization aside, I am wondering if
> > there is anything in Lucene/Solr that would have such an impact on query
> > execution time between versions.
> >
> > Originally I though it might be related to
> > https://issues.apache.org/jira/browse/SOLR-8251 and testing on small
> > scale proved that there is a difference in performance. However upgraded
> > version is already 5.5.0.
> >
> >
> >
> > Thanks,
> > Jarek
> >
