Thanks, Erick, The process time execution based on debugQuery between the query and facets is as follows
query 10ms facets 4900ms since max time is spent on facet processing (docValues enabled), query and filter cache do no apply to this, correct? - Autowarm count is at 32 for both and autowarm time is 25 for query-result cache and 1724 for filter cache - We have hard-commit every 5 mins with opensearcher=false and soft-commit every 2 secs. - facet are a mix of pivot facets,range facets and facet queries - when the same facets criteria bring a smaller result set, response is much faster On Mon, Mar 30, 2020 at 4:47 PM Erick Erickson <erickerick...@gmail.com> wrote: > OK, sounds like docValues is set. > > Sure, in solrconfig.xml, there are two sections “firstSearcher” and > “newSearcher”. > These are queries (or lists of queries) that are fired as part of > autowarming > when Solr is first started (firstSearcher) or when a commit happens that > opens > a new searcher (newSearcher). These are hand-crafted static queries. So > create one or more newSearcher sections in that block that exercise your > faceting and it’ll be fired as part of autowarming. That should smooth out > the delay your user’s experience when commits happen. > > Best, > Erick > > > On Mar 30, 2020, at 4:06 PM, Revas <revas2...@gmail.com> wrote: > > > > Thanks, Eric. > > > > 1) We are using dynamic string field for faceting where indexing =false > and > > stored=false . By default docValues are enabled for primitive fields > (solr > > 6.6.), so not explicitly defined in schema. Do you think its wrong > > assumption? Also I do not this field listed in feild cache, but dont see > > any dynamic fields listed. > > 2) Autowarm count is at 32 for both and autowarm time is 25 for > queryresult > > and 17 > > 3)Can you elaborate what you mean here > > > > > > > > On Mon, Mar 30, 2020 at 1:43 PM Erick Erickson <erickerick...@gmail.com> > > wrote: > > > >> Response spikes after commits are almost always something to do > >> with autowarming or docValues being set to false. So here’s what > >> I’d look at, in order. > >> > >> 1> are the fields used defined with docValues=true? They should be. > >> With this much variance it sounds like you don’t have that value set. > >> You’ll have to rebuild your entire index, first deleting all documents… > >> > >> You assert that they are all docValues, but the variance is so > >> high that I wonder whether they _all_ are. They may very well be, but > >> I’ve been tripped up by things I know are true that aren’t too often ;) > >> > >> You can insure this by setting 'uninvertible=“true” ‘ in your field > type, > >> see: https://issues.apache.org/jira/browse/SOLR-12962 if you’re on > >> 7.6 or later. > >> > >> 2>what are your autowarming settings for queryResultCache and/or > >> filterCache. Start with a relatively small number, say 16 and look at > >> your autowarm times to insure they aren’t excessive. > >> > >> 3> if autowarming doesn’t help, consider specifying a newSearcher > >> event in solrconfig.xml that exercises the facets. > >> > >> NOTE: <2> and <3> will mask any fields that are docValues=false that > >> slipped through the cracks, so I’d double check <1> first. > >> > >> Best, > >> Erick > >> > >>> On Mar 30, 2020, at 12:20 PM, sujatha arun <revas2...@gmail.com> > wrote: > >>> > >>> A facet heavy query which uses docValue fields for faceting returns > >> about > >>> 5k results executes between 10ms to 5 secs and the 5 secs time seems > to > >>> coincide with after a hard commit. > >>> > >>> Does that have any relation? Why the fluctuation in execution time? > >>> > >>> Thanks, > >>> Revas > >> > >> > >
