Also, attempting to facet across a large number of docs is going to take some time. Perhaps you might gain some performance benefit by sharding your index?
Upayavira On Thu, Aug 6, 2015, at 04:48 PM, Mikhail Khludnev wrote: > On Thu, Aug 6, 2015 at 3:56 PM, Bernd Fehling < > bernd.fehl...@uni-bielefeld.de> wrote: > > > > > > > Am 06.08.2015 um 14:33 schrieb Upayavira: > > > Typically such performance issues with faceting are to do with the time > > > spend uninverting the index before calculating the facet counts. > > > > > > If you indexed the fields with docValues enabled, perhaps you could then > > > use them for faceting, which might improve performance. > > > > Well, this is against my observations. When I used uninverted fields > > without > > docValues I had a much better 99 percentile qtime but a very high heap > > consumption. > > Now with docValues the heap usage went down, but the 99 percentile > > qtime for MatchAllDocsQuery(*:*) went up to above 33 seconds. > > > > Note about performance optimization for DocValues faceting in > forthcoming > 5.3 > > > > > > > > > > If you are using a non-docValues field, and the second query is faster, > > > then you could add the query to your static warming, look for > > > newSearcher in your solrconfig.xml. That will execute your query, > > > warming the caches used by faceting, before a new searcher is made > > > available for searches. > > > > The q=*.* with sorting and facetting is always the first query I'm doing > > at static warming and it helped until switching to docValues :-( > > > > Bernd > > > > > > > > Upayavira > > > > > > On Thu, Aug 6, 2015, at 12:38 PM, Toke Eskildsen wrote: > > >> On Thu, 2015-08-06 at 13:00 +0200, Bernd Fehling wrote: > > >>> Single Index Solr 4.10.4, optimized Index, 76M docs, 235GB index size. > > >>> > > >>> I was analysing my solr logs and it turned out that I have some queries > > >>> which are above 30 seconds qtime while normally the qtime is below 1 > > second. > > >>> Looking closer about the queries it turned out that this is for > > MatchAllDocsQuery(*:*). > > >>> Next was to turn debugQuery on and see where the bottleneck is. > > >>> The result was that the facetting is consuming most of the qtime. > > >>> > > >>> So the question is, are facets or is facetting not cached? > > >> > > >> As far as I know it is not. 35 seconds for a match-all faceting sounds > > >> fairly on par with what we are seeing (250M docs, 900GB shard). > > >> > > >> Of course response time is very depending on the field itself. If you > > >> have very few unique values in your facet field(s), you might try > > >> facet.method=enum. If that is not the case, your best bet would probably > > >> be to cache the match-all outside of Solr. > > >> > > >>> My assumption is that the queryResultCache is catching such a > > MatchAllDocsQuery(*:*). > > >> > > >> It only stores the docIDs. > > >> > > >> I don't know why there is is no all_parameters -> complete_response > > >> cache in Solr. > > >> > > >> - Toke Eskildsen, State and University Library, Denmark > > >> > > >> > > > > > > -- > Sincerely yours > Mikhail Khludnev > Principal Engineer, > Grid Dynamics > > <http://www.griddynamics.com> > <mkhlud...@griddynamics.com>
