[jira] [Updated] (SOLR-17615) facets exclusion does not exclude dense vector search prefilters
[
https://issues.apache.org/jira/browse/SOLR-17615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yue Yu updated SOLR-17615:
--
Affects Version/s: (was: 9.7)
> facets exclusion does not exclude dense vector search prefilters
>
>
> Key: SOLR-17615
> URL: https://issues.apache.org/jira/browse/SOLR-17615
> Project: Solr
> Issue Type: Bug
> Components: faceting
>Reporter: Yue Yu
>Priority: Major
>
> Vector search adds all fqs as implicit pre-filters by default:
> [https://solr.apache.org/guide/solr/latest/query-guide/dense-vector-search.html#implicit-pre-filtering]
> for example:
> {code:java}
> ?q={!knn f=vector topK=10}[1.0, 2.0, 3.0, 4.0]
> &fq={!tag="category"}category:AAA
> &fq={!tag="instock"}inStock:true {code}
> where both "category" and "instock" fq are added as the main knn query's
> pre-filters.
>
> However, for the facet multi-select use case, we want to exclude these fqs in
> the facet request so that the facet shows other values
> {code:java}
> ?q={!knn f=vector topK=10}[1.0, 2.0, 3.0, 4.0]
> &fq={!tag="category"}category:AAA
> &fq={!tag="instock"}inStock:true
> &facet.field={!ex="category" key="category"}category
> &facet.field={!ex="instock" key="instock"}inStock{code}
> This is done by
> [SimpleFacets.java|https://github.com/apache/solr/blob/main/solr/core/src/java/org/apache/solr/request/SimpleFacets.java#L223]
> who recalculates the DocSet using the main query and all fqs except the
> excluded one:
> {code:java}
>for (String excludeTag : excludeTagList) {
> Object olst = tagMap.get(excludeTag);
> // tagMap has entries of List>, but subject to
> change in the future
> if (!(olst instanceof Collection)) continue;
> for (Object o : (Collection) olst) {
> if (!(o instanceof QParser qp)) continue;
> excludeSet.put(qp.getQuery(), Boolean.TRUE);
> }
> }
> if (excludeSet.size() == 0) return baseDocSet;
>
> List qlist = new ArrayList<>();
> // add the base query
> if (!excludeSet.containsKey(rb.getQuery())) {
> qlist.add(rb.getQuery());
> }
> // add the filters
> if (rb.getFilters() != null) {
> for (Query q : rb.getFilters()) {
> if (!excludeSet.containsKey(q)) {
> qlist.add(q);
> }
> }
> }
> // get the new base docset for this facet
> DocSet base = searcher.getDocSet(qlist); {code}
> This works fine for non-knn main queries as they don't have any pre-filters.
> For knn main queries, the base query *rb.getQuery()* added to the *qlist*
> contains all the fqs as pre-filters, so this facet exclusion logic has no
> effect.
>
> In the example above, the facet values for "category" will only have AAA and
> "instock" will only have inStock.
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
[jira] [Updated] (SOLR-17615) facets exclusion does not exclude dense vector search prefilters
[
https://issues.apache.org/jira/browse/SOLR-17615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yue Yu updated SOLR-17615:
--
Component/s: faceting
> facets exclusion does not exclude dense vector search prefilters
>
>
> Key: SOLR-17615
> URL: https://issues.apache.org/jira/browse/SOLR-17615
> Project: Solr
> Issue Type: Bug
> Components: faceting
>Affects Versions: 9.7
>Reporter: Yue Yu
>Priority: Major
>
> Vector search adds all fqs as implicit pre-filters by default:
> [https://solr.apache.org/guide/solr/latest/query-guide/dense-vector-search.html#implicit-pre-filtering]
> for example:
> {code:java}
> ?q={!knn f=vector topK=10}[1.0, 2.0, 3.0, 4.0]
> &fq={!tag="category"}category:AAA
> &fq={!tag="instock"}inStock:true {code}
> where both "category" and "instock" fq are added as the main knn query's
> pre-filters.
>
> However, for the facet multi-select use case, we want to exclude these fqs in
> the facet request so that the facet shows other values
> {code:java}
> ?q={!knn f=vector topK=10}[1.0, 2.0, 3.0, 4.0]
> &fq={!tag="category"}category:AAA
> &fq={!tag="instock"}inStock:true
> &facet.field={!ex="category" key="category"}category
> &facet.field={!ex="instock" key="instock"}inStock{code}
> This is done by
> [SimpleFacets.java|https://github.com/apache/solr/blob/main/solr/core/src/java/org/apache/solr/request/SimpleFacets.java#L223]
> who recalculates the DocSet using the main query and all fqs except the
> excluded one:
> {code:java}
>for (String excludeTag : excludeTagList) {
> Object olst = tagMap.get(excludeTag);
> // tagMap has entries of List>, but subject to
> change in the future
> if (!(olst instanceof Collection)) continue;
> for (Object o : (Collection) olst) {
> if (!(o instanceof QParser qp)) continue;
> excludeSet.put(qp.getQuery(), Boolean.TRUE);
> }
> }
> if (excludeSet.size() == 0) return baseDocSet;
>
> List qlist = new ArrayList<>();
> // add the base query
> if (!excludeSet.containsKey(rb.getQuery())) {
> qlist.add(rb.getQuery());
> }
> // add the filters
> if (rb.getFilters() != null) {
> for (Query q : rb.getFilters()) {
> if (!excludeSet.containsKey(q)) {
> qlist.add(q);
> }
> }
> }
> // get the new base docset for this facet
> DocSet base = searcher.getDocSet(qlist); {code}
> This works fine for non-knn main queries as they don't have any pre-filters.
> For knn main queries, the base query *rb.getQuery()* added to the *qlist*
> contains all the fqs as pre-filters, so this facet exclusion logic has no
> effect.
>
> In the example above, the facet values for "category" will only have AAA and
> "instock" will only have inStock.
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
[jira] [Updated] (SOLR-17615) facets exclusion does not exclude dense vector search prefilters
[
https://issues.apache.org/jira/browse/SOLR-17615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yue Yu updated SOLR-17615:
--
Affects Version/s: 9.7
> facets exclusion does not exclude dense vector search prefilters
>
>
> Key: SOLR-17615
> URL: https://issues.apache.org/jira/browse/SOLR-17615
> Project: Solr
> Issue Type: Bug
>Affects Versions: 9.7
>Reporter: Yue Yu
>Priority: Major
>
> Vector search adds all fqs as implicit pre-filters by default:
> [https://solr.apache.org/guide/solr/latest/query-guide/dense-vector-search.html#implicit-pre-filtering]
> for example:
> {code:java}
> ?q={!knn f=vector topK=10}[1.0, 2.0, 3.0, 4.0]
> &fq={!tag="category"}category:AAA
> &fq={!tag="instock"}inStock:true {code}
> where both "category" and "instock" fq are added as the main knn query's
> pre-filters.
>
> However, for the facet multi-select use case, we want to exclude these fqs in
> the facet request so that the facet shows other values
> {code:java}
> ?q={!knn f=vector topK=10}[1.0, 2.0, 3.0, 4.0]
> &fq={!tag="category"}category:AAA
> &fq={!tag="instock"}inStock:true
> &facet.field={!ex="category" key="category"}category
> &facet.field={!ex="instock" key="instock"}inStock{code}
> This is done by
> [SimpleFacets.java|https://github.com/apache/solr/blob/main/solr/core/src/java/org/apache/solr/request/SimpleFacets.java#L223]
> who recalculates the DocSet using the main query and all fqs except the
> excluded one:
> {code:java}
>for (String excludeTag : excludeTagList) {
> Object olst = tagMap.get(excludeTag);
> // tagMap has entries of List>, but subject to
> change in the future
> if (!(olst instanceof Collection)) continue;
> for (Object o : (Collection) olst) {
> if (!(o instanceof QParser qp)) continue;
> excludeSet.put(qp.getQuery(), Boolean.TRUE);
> }
> }
> if (excludeSet.size() == 0) return baseDocSet;
>
> List qlist = new ArrayList<>();
> // add the base query
> if (!excludeSet.containsKey(rb.getQuery())) {
> qlist.add(rb.getQuery());
> }
> // add the filters
> if (rb.getFilters() != null) {
> for (Query q : rb.getFilters()) {
> if (!excludeSet.containsKey(q)) {
> qlist.add(q);
> }
> }
> }
> // get the new base docset for this facet
> DocSet base = searcher.getDocSet(qlist); {code}
> This works fine for non-knn main queries as they don't have any pre-filters.
> For knn main queries, the base query *rb.getQuery()* added to the *qlist*
> contains all the fqs as pre-filters, so this facet exclusion logic has no
> effect.
>
> In the example above, the facet values for "category" will only have AAA and
> "instock" will only have inStock.
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
[jira] [Updated] (SOLR-17615) facets exclusion does not exclude dense vector search prefilters
[
https://issues.apache.org/jira/browse/SOLR-17615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yue Yu updated SOLR-17615:
--
Description:
Vector search adds all fqs as implicit pre-filters by default:
[https://solr.apache.org/guide/solr/latest/query-guide/dense-vector-search.html#implicit-pre-filtering]
for example:
{code:java}
?q={!knn f=vector topK=10}[1.0, 2.0, 3.0, 4.0]
&fq={!tag="category"}category:AAA
&fq={!tag="instock"}inStock:true {code}
where both "category" and "instock" fq are added as the main knn query's
pre-filters.
However, for the facet multi-select use case, we want to exclude these fqs in
the facet request so that the facet shows other values
{code:java}
?q={!knn f=vector topK=10}[1.0, 2.0, 3.0, 4.0]
&fq={!tag="category"}category:AAA
&fq={!tag="instock"}inStock:true
&facet.field={!ex="category" key="category"}category
&facet.field={!ex="instock" key="instock"}inStock{code}
This is done by
[SimpleFacets.java|https://github.com/apache/solr/blob/main/solr/core/src/java/org/apache/solr/request/SimpleFacets.java#L223]
who recalculates the DocSet using the main query and all fqs except the
excluded one:
{code:java}
for (String excludeTag : excludeTagList) {
Object olst = tagMap.get(excludeTag);
// tagMap has entries of List>, but subject to
change in the future
if (!(olst instanceof Collection)) continue;
for (Object o : (Collection) olst) {
if (!(o instanceof QParser qp)) continue;
excludeSet.put(qp.getQuery(), Boolean.TRUE);
}
}
if (excludeSet.size() == 0) return baseDocSet;
List qlist = new ArrayList<>();
// add the base query
if (!excludeSet.containsKey(rb.getQuery())) {
qlist.add(rb.getQuery());
}
// add the filters
if (rb.getFilters() != null) {
for (Query q : rb.getFilters()) {
if (!excludeSet.containsKey(q)) {
qlist.add(q);
}
}
}
// get the new base docset for this facet
DocSet base = searcher.getDocSet(qlist); {code}
This works fine for non-knn main queries as they don't have any pre-filters.
For knn main queries, the base query *rb.getQuery()* added to the *qlist*
contains all the fqs as pre-filters, so this facet exclusion logic has no
effect.
In the example above, the facet values for "category" will only have AAA and
"instock" will only have inStock.
was:
Vector search adds all fqs as implicit pre-filters by default:
[https://solr.apache.org/guide/solr/latest/query-guide/dense-vector-search.html#implicit-pre-filtering]
for example:
{code:java}
?q={!knn f=vector topK=10}[1.0, 2.0, 3.0, 4.0]
&fq={!tag="category"}category:AAA
&fq={!tag="instock"}inStock:true {code}
where both "category" and "instock" fq are added as the main knn query's
pre-filters.
However, for the facet multi-select use case, we want to exclude these fqs in
the facet request so that the facet shows other values
{code:java}
?q={!knn f=vector topK=10}[1.0, 2.0, 3.0, 4.0]
&fq={!tag="category"}category:AAA
&fq={!tag="instock"}inStock:true
&facet.field={!ex="category" key="category"}category
&facet.field={!ex="instock" key="instock"}inStock{code}
This is done by
[SimpleFacets.java|https://github.com/apache/solr/blob/main/solr/core/src/java/org/apache/solr/request/SimpleFacets.java#L223]
who recalculates the DocSet using the main query and all fqs except the
excluded one:
{code:java}
for (String excludeTag : excludeTagList) {
Object olst = tagMap.get(excludeTag);
// tagMap has entries of List>, but subject to
change in the future
if (!(olst instanceof Collection)) continue;
for (Object o : (Collection) olst) {
if (!(o instanceof QParser qp)) continue;
excludeSet.put(qp.getQuery(), Boolean.TRUE);
}
}
if (excludeSet.size() == 0) return baseDocSet;
List qlist = new ArrayList<>();
// add the base query
if (!excludeSet.containsKey(rb.getQuery())) {
qlist.add(rb.getQuery());
}
// add the filters
if (rb.getFilters() != null) {
for (Query q : rb.getFilters()) {
if (!excludeSet.containsKey(q)) {
qlist.add(q);
}
}
}
// get the new base docset for this facet
DocSet base = searcher.getDocSet(qlist); {code}
This works fine for non-knn main queries as they don't have any pre-filters.
For knn main queries, the base query *rb.getQuery()* added to the *qlist*
contains all the fqs as pre-filters, so this facet exclusion logic has no
effect.
In the example above, the facet values for "category" will only have AAA and
"instock" will only have inStock.
> facets exclusion does not exclude dense vector search prefilters
>
>
> Key: SOLR-17615
> URL: https://issues.apache.org/jira/browse/SOLR-17615
>
[jira] [Updated] (SOLR-17615) facets exclusion does not exclude dense vector search prefilters
[
https://issues.apache.org/jira/browse/SOLR-17615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yue Yu updated SOLR-17615:
--
Description:
Vector search adds all fqs as implicit pre-filters by default:
[https://solr.apache.org/guide/solr/latest/query-guide/dense-vector-search.html#implicit-pre-filtering]
for example:
{code:java}
?q={!knn f=vector topK=10}[1.0, 2.0, 3.0, 4.0]
&fq={!tag="category"}category:AAA
&fq={!tag="instock"}inStock:true {code}
where both "category" and "instock" fq are added as the main knn query's
pre-filters.
However, for the facet multi-select use case, we want to exclude these fqs in
the facet request so that the facet shows other values
{code:java}
?q={!knn f=vector topK=10}[1.0, 2.0, 3.0, 4.0]
&fq={!tag="category"}category:AAA
&fq={!tag="instock"}inStock:true
&facet.field={!ex="category" key="category"}category
&facet.field={!ex="instock" key="instock"}inStock{code}
This is done by
[SimpleFacets.java|https://github.com/apache/solr/blob/main/solr/core/src/java/org/apache/solr/request/SimpleFacets.java#L223]
who recalculates the DocSet using the main query and all fqs except the
excluded one:
{code:java}
for (String excludeTag : excludeTagList) {
Object olst = tagMap.get(excludeTag);
// tagMap has entries of List>, but subject to
change in the future
if (!(olst instanceof Collection)) continue;
for (Object o : (Collection) olst) {
if (!(o instanceof QParser qp)) continue;
excludeSet.put(qp.getQuery(), Boolean.TRUE);
}
}
if (excludeSet.size() == 0) return baseDocSet;
List qlist = new ArrayList<>();
// add the base query
if (!excludeSet.containsKey(rb.getQuery())) {
qlist.add(rb.getQuery());
}
// add the filters
if (rb.getFilters() != null) {
for (Query q : rb.getFilters()) {
if (!excludeSet.containsKey(q)) {
qlist.add(q);
}
}
}
// get the new base docset for this facet
DocSet base = searcher.getDocSet(qlist); {code}
This works fine for non-knn main queries as they don't have any pre-filters.
For knn main queries, the base query *rb.getQuery()* added to the *qlist*
contains all the fqs as pre-filters, so this facet exclusion logic has no
effect.
In the example above, the facet values for "category" will only have AAA and
"instock" will only have inStock.
was:
Vector search adds all fqs as implicit pre-filters by default:
[https://solr.apache.org/guide/solr/latest/query-guide/dense-vector-search.html#implicit-pre-filtering]
for example:
{code:java}
?q={!knn f=vector topK=10}[1.0, 2.0, 3.0, 4.0]
&fq={!tag="category"}category:AAA
&fq={!tag="instock"}inStock:true {code}
where both "category" and "instock" fq are added as the main knn query's
pre-filters.
However, for the facet multi-select use case, we want to exclude these fqs in
the facet request so that the facet shows other values
{code:java}
?q={!knn f=vector topK=10}[1.0, 2.0, 3.0, 4.0]
&fq={!tag="category"}category:AAA
&fq={!tag="instock"}inStock:true
&facet.field={!ex="category" key="category"}category
&facet.field={!ex="instock" key="instock"}inStock{code}
This is done by
[SimpleFacets.java|https://github.com/apache/solr/blob/main/solr/core/src/java/org/apache/solr/request/SimpleFacets.java#L223]
who recalculates the DocSet using the main query and all fqs except the
excluded one:
{code:java}
for (String excludeTag : excludeTagList) {
Object olst = tagMap.get(excludeTag);
// tagMap has entries of List>, but subject to
change in the future
if (!(olst instanceof Collection)) continue;
for (Object o : (Collection) olst) {
if (!(o instanceof QParser qp)) continue;
excludeSet.put(qp.getQuery(), Boolean.TRUE);
}
}
if (excludeSet.size() == 0) return baseDocSet;
List qlist = new ArrayList<>(); // add the base query
if (!excludeSet.containsKey(rb.getQuery())) {
qlist.add(rb.getQuery());
}
// add the filters
if (rb.getFilters() != null) {
for (Query q : rb.getFilters()) {
if (!excludeSet.containsKey(q)) {
qlist.add(q);
}
}
}
// get the new base docset for this facet
DocSet base = searcher.getDocSet(qlist); {code}
This works fine for non-knn main queries as they don't have any pre-filters.
For knn main queries, the base query *rb.getQuery()* added to the *qlist*
contains all the fqs as pre-filters, so this facet exclusion logic has no
effect.
In the example above, the facet values for "category" will only have AAA and
"instock" will only have inStock.
> facets exclusion does not exclude dense vector search prefilters
>
>
> Key: SOLR-17615
> URL: https://issues.apache.org/jira/browse/SOLR-1761
