[
https://issues.apache.org/jira/browse/SOLR-18541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18127698#comment-18127698
]
Amrit Sarkar commented on SOLR-18541:
-------------------------------------
Same bug reported on dev@ against 10.1.0 RC2:
https://lists.apache.org/thread/pfkgtpno6h4drw7s081ymkn9pj6bsw6c
Reproduced on the RC2 tarball, 1 and 2 shards:
||case||union||numFound||numFoundExact||docs||
|{{/select}} cat OR dog|5|5|true|5|
|hybrid cat(3) + dog(2), no overlap|5|3|false|5|
|hybrid cat(3) + {{id:(3 4)}}, one overlap|4|3|false|4|
The response returns more docs than {{numFound}} says exist. The cause is
{{max()}} across sub-queries in {{CombinedQueryComponent.mergeIds}} and
{{QueryAndResponseCombiner.simpleCombine}}, on main, 10.x and 9.x.
On the open question: I think {{numFound}} should be the distinct docs matching
any sub-query, same as {{/select}}. That's the user-visible bug, so I'd make it
the priority here.
It also affects PR #5117. On the PR head, the repro above returns
{{numFound=3}}, {{numFoundExact=true}}. The exactness fix makes the wrong count
look exact. Either:
* fix the count: each shard reports its own union count and the coordinator
sums them (shards don't share docs, so this is exact), or
* at minimum, set {{numFoundExact=false}} when more than one sub-query
contributes, and drop the two {{largestSubQueryTotalWins*}} tests that lock in
{{max()}}.
Happy to help with the union count. I can take it as a follow-up JIRA if you'd
rather keep #5117 scoped.
> CombinedQueryComponent.mergeIds gives wrong numFound, approximateTotalHits
> and numFoundExact
> --------------------------------------------------------------------------------------------
>
> Key: SOLR-18541
> URL: https://issues.apache.org/jira/browse/SOLR-18541
> Project: Solr
> Issue Type: Bug
> Components: query
> Reporter: Nick Shanin
> Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> 🤖 _AI text below_ 🤖 _(posted on behalf of Nick Shanin)_
> Combined queries merge their per-query shard results in
> {{CombinedQueryComponent.mergeIds}}, and three defects sit in the totals that
> merge reports. First, a partial shard that sends a per-query entry is counted
> in that query's totals before the partial skip is tested, so its documents
> are dropped but its hits are not: in a constructed partial-response case run
> on unmodified main, the merged {{numFound}} was 7 instead of 3 and
> {{approximateTotalHits}} was 15 instead of 5. Second, when a shard sends no
> {{approximateTotalHits}} header, the fallback adds the running per-query
> total, which already includes the current shard, so two shards with 2 and 3
> documents merged to 7 instead of 5. Third, the {{hitCountIsExact}} fold is
> inverted: an all-exact merge reports {{numFoundExact=false}} and an
> all-inexact merge reports true, both confirmed by run on unmodified main.
> The fix I recommend is to test the skip before accumulating, with the other
> two corrections in the same change: the fallback should add the shard's own
> count instead of the running total, and the fold should keep the merged flag
> true only when every shard reports its count exact. Correcting the totals
> after the skip fires instead would mean reconstructing what the skipped shard
> added, and the exactness fold cannot be un-folded without per-shard tracking.
> One question for the maintainer: when several sub-queries contribute,
> {{numFound}} is currently the largest single sub-query count; should it
> instead count the distinct documents that match any sub-query, and should
> {{numFoundExact}} be false whenever more than one sub-query contributes?
> AI agents assisted with research, implementation, review, and drafting. Nick
> Shanin directed the work and takes responsibility for the above text.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
