[
https://issues.apache.org/jira/browse/SOLR-13890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17002424#comment-17002424
]
Joel Bernstein edited comment on SOLR-13890 at 12/23/19 8:23 PM:
-----------------------------------------------------------------
The other really big aspect of this is caching.
Even though the scorer based filter can be fast if it's applied with the main
query, in Solr that's not going to happen.
The reason is the filter cache. Solr will apply the filter against the *entire
index* and create a DocSet to cache. This will be slow compared to the
postfilter if the number of search results is small relative to the size of the
index. Which might be acceptable if the filter cache provided a big advantage
on subsequent requests. But ...
Solr's filter cache is top level so it gets dumped after a single document is
loaded. So in scenarios where there is lot's of indexing going on the filter
cache becomes problematic.
There are ways around this issue, like turning off caching using local params,
or not using filter queries. But these approaches are not what users typically
do with a filter.
So, the postfilters behavior (not cached in filter cache) provides the best
solution for certain situations where the filter cache is problematic.
was (Author: joel.bernstein):
The other really big aspect of this is caching.
Even though the scorer based filter can be fast if it's applied with the main
query, in Solr that's not going to happen.
The reason is the filter cache. Solr will apply the filter against the entire
index and create a DocSet to cache.
Our filter cache is top level so it gets dumped after a single document is
loaded. So in scenarios where there is lot's of indexing going on the filter
cache becomes problematic.
There are ways around this issue, like turning off caching using local params,
or not using filter queries. But these approaches are not what users typically
do with a filter.
So, the postfilters behavior (not cached in filter cache) provides the best
solution for certain situations where the filter cache is problematic.
> Add postfilter support to {!terms} queries
> ------------------------------------------
>
> Key: SOLR-13890
> URL: https://issues.apache.org/jira/browse/SOLR-13890
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: query parsers
> Affects Versions: master (9.0)
> Reporter: Jason Gerlowski
> Assignee: Jason Gerlowski
> Priority: Major
> Attachments: SOLR-13890.patch, SOLR-13890.patch, SOLR-13890.patch
>
>
> There are some use-cases where it'd be nice if the "terms" qparser created a
> query that could be run as a postfilter. Particularly, when users are
> checking for hundreds or thousands of terms, a postfilter implementation can
> be more performant than the standard processing.
> WIth this issue, I'd like to propose a post-filter implementation for the
> {{docValuesTermsFilter}} "method". Postfilter creation can use a
> SortedSetDocValues object to populate a DV bitset with the "terms" being
> checked for. Each document run through the post-filter can look at their
> doc-values for the field in question and check them efficiently against the
> constructed bitset.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
