Re: Why does faceting add an entry to the filter cache for the q parameter?
> whether DocList can even be used for facets, No way. For sure. Overall, FacetComponent explicitly requires docset https://github.com/apache/solr/blob/main/solr/core/src/java/org/apache/solr/handler/component/FacetComponent.java#L82 and then https://github.com/apache/solr/blob/main/solr/core/src/java/org/apache/solr/search/SolrIndexSearcher.java#L1564 https://github.com/apache/solr/blob/main/solr/core/src/java/org/apache/solr/search/SolrIndexSearcher.java#L971 I don't recommend changing anything in this flow. On Thu, Nov 17, 2022 at 9:05 PM Shawn Heisey wrote: > On 11/17/22 08:45, Andy Lester wrote: > > Short of that, wouldn't it make sense for facets to put the q in the > queryResultsCache, not the filterCache? > > The queryResultCache is defined as ... very > different from the that filterCache uses. I have no idea > whether DocList can even be used for facets, but it is likely that > DocSet is faster and more directly applicable. Because filterCache was > already available, a choice was probably made to just use that rather > than introduce a whole new cache. > > That decision is a bad one for your use case. Having a separate > facetCache seems like a very good thing for your use case and probably > would be generally helpful for overall performance in a variety of use > cases. > > That part of Solr code is very unfamiliar to me, I wouldn't have any > idea where to begin or what to do for implementing facetCache. I would > tackle it if I knew how. > > Thanks, > Shawn > > > - > To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org > For additional commands, e-mail: dev-h...@solr.apache.org > > -- Sincerely yours Mikhail Khludnev
Re: Why does faceting add an entry to the filter cache for the q parameter?
On 11/17/22 08:45, Andy Lester wrote: Short of that, wouldn't it make sense for facets to put the q in the queryResultsCache, not the filterCache? The queryResultCache is defined as ... very different from the that filterCache uses. I have no idea whether DocList can even be used for facets, but it is likely that DocSet is faster and more directly applicable. Because filterCache was already available, a choice was probably made to just use that rather than introduce a whole new cache. That decision is a bad one for your use case. Having a separate facetCache seems like a very good thing for your use case and probably would be generally helpful for overall performance in a variety of use cases. That part of Solr code is very unfamiliar to me, I wouldn't have any idea where to begin or what to do for implementing facetCache. I would tackle it if I knew how. Thanks, Shawn - To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org For additional commands, e-mail: dev-h...@solr.apache.org
Re: Why does faceting add an entry to the filter cache for the q parameter?
> I think it likely means that the intended benefit of filterCache, which is > the caching of fqs, is not happening because those cache entries will be > evicted almost as soon as they are created. Yes, that's what I suspect is happening. Another problem with having q in the filterCache is that if your newSearcher is doing autowarming on some number of filterCache entries, it will be doing warming on q entries. This could be quite a bit of unnecessary work. > I wonder if maybe it would be a good idea to have a facetCache in addition to > filterCache. Same K,V as filterCache in the code, but entirely separate so > it does not interfere with caching of filters and can have a different > definition. That would make a lot of sense to me. Short of that, wouldn't it make sense for facets to put the q in the queryResultsCache, not the filterCache? Whatever comes of this, the docs could use some updates and clarifications, and especially discussion of tuning filterCache. I've started a ticket for that task, both as placeholder for the work, and as a place to aggregate notes and thoughts for when it gets done. Andy - To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org For additional commands, e-mail: dev-h...@solr.apache.org
Re: Why does faceting add an entry to the filter cache for the q parameter?
On 11/16/22 13:21, Mikhail Khludnev wrote: Why do you think that thousands of evictions is a bad thing? I think it likely means that the intended benefit of filterCache, which is the caching of fqs, is not happening because those cache entries will be evicted almost as soon as they are created. In my experience an uncached fq is significantly slower than a cached fq, so caching them can be very important. In the setup I used to manage, facets were not used by the application. I used them for manual data mining and some daily cronjobs that would report statistics from the index during hours of low usage. So I did not run into this. I wonder if maybe it would be a good idea to have a facetCache in addition to filterCache. Same K,V as filterCache in the code, but entirely separate so it does not interfere with caching of filters and can have a different definition. Thanks, Shawn - To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org For additional commands, e-mail: dev-h...@solr.apache.org
