[
https://issues.apache.org/jira/browse/SOLR-17615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17911641#comment-17911641
]
Yue Yu commented on SOLR-17615:
-------------------------------
[~abenedetti] [~hossman] Thanks for the respond.
Yes I'm aware the {{excludeTags}} support was added to the {{knn}} Qparser. And
currently it's the workaround to get facet exclusion to work with vector search
query.
However I'd say it's a tradeoff where we'll lose all the benefit of adding fqs
as knn query's prefilter, especially for the vector search recall when topk is
small and several fqs are added as post-filters.
Would it be fair to say that "as of now, there is a limitation that for facet
exclusion to work with knn query, ALL fqs to be excluded by the facet request
MUST ALSO be excluded from the knn query's pre-filters" ? I'm wondering could
we fix the facet exclusion logic in
[SimpleFacets.java|https://github.com/apache/solr/blob/main/solr/core/src/java/org/apache/solr/request/SimpleFacets.java#L223]
so that we can remove this limitation?
Looking at the computeDocSet function in
[SimpleFacets.java|https://github.com/apache/solr/blob/main/solr/core/src/java/org/apache/solr/request/SimpleFacets.java#L223]
, the logic was working perfectly fine until some special Query (knn query)
comes with pre-filters. It feels natural to update this function's logic to
accommodate Query with pre-filters.
The fix seems not trivial. We can use the similar idea in computeDocSet
function to reconstruct and update knn query's pre-filters, but the challenging
part is how to find all the knn queries in the base query {*}rb.getQuery(){*}.
Anyway if you guys think this is worth fixing, we can discuss further.
> facets exclusion does not exclude dense vector search prefilters
> ----------------------------------------------------------------
>
> Key: SOLR-17615
> URL: https://issues.apache.org/jira/browse/SOLR-17615
> Project: Solr
> Issue Type: Bug
> Components: faceting
> Reporter: Yue Yu
> Priority: Major
>
> Vector search adds all fqs as implicit pre-filters by default:
> [https://solr.apache.org/guide/solr/latest/query-guide/dense-vector-search.html#implicit-pre-filtering]
> for example:
> {code:java}
> ?q={!knn f=vector topK=10}[1.0, 2.0, 3.0, 4.0]
> &fq={!tag="category"}category:AAA
> &fq={!tag="instock"}inStock:true {code}
> where both "category" and "instock" fq are added as the main knn query's
> pre-filters.
>
> However, for the facet multi-select use case, we want to exclude these fqs in
> the facet request so that the facet shows other values
> {code:java}
> ?q={!knn f=vector topK=10}[1.0, 2.0, 3.0, 4.0]
> &fq={!tag="category"}category:AAA
> &fq={!tag="instock"}inStock:true
> &facet.field={!ex="category" key="category"}category
> &facet.field={!ex="instock" key="instock"}inStock{code}
> This is done by
> [SimpleFacets.java|https://github.com/apache/solr/blob/main/solr/core/src/java/org/apache/solr/request/SimpleFacets.java#L223]
> who recalculates the DocSet using the main query and all fqs except the
> excluded one:
> {code:java}
> for (String excludeTag : excludeTagList) {
> Object olst = tagMap.get(excludeTag);
> // tagMap has entries of List<String,List<QParser>>, but subject to
> change in the future
> if (!(olst instanceof Collection)) continue;
> for (Object o : (Collection<?>) olst) {
> if (!(o instanceof QParser qp)) continue;
> excludeSet.put(qp.getQuery(), Boolean.TRUE);
> }
> }
> if (excludeSet.size() == 0) return baseDocSet;
>
> List<Query> qlist = new ArrayList<>();
> // add the base query
> if (!excludeSet.containsKey(rb.getQuery())) {
> qlist.add(rb.getQuery());
> }
> // add the filters
> if (rb.getFilters() != null) {
> for (Query q : rb.getFilters()) {
> if (!excludeSet.containsKey(q)) {
> qlist.add(q);
> }
> }
> }
> // get the new base docset for this facet
> DocSet base = searcher.getDocSet(qlist); {code}
> This works fine for non-knn main queries as they don't have any pre-filters.
> For knn main queries, the base query *rb.getQuery()* added to the *qlist*
> contains all the fqs as pre-filters, so this facet exclusion logic has no
> effect.
>
> In the example above, the facet values for "category" will only have AAA and
> "instock" will only have inStock.
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
