[
https://issues.apache.org/jira/browse/LUCENE-8060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16556071#comment-16556071
]
Hoss Man commented on LUCENE-8060:
----------------------------------
I don't know a lot about how the current estimation code works, but would it be
worthwhile to capture info in the TopDocs about how accurate/confident the
totalHits is, and instead of an {{int maxTotalHits}} option make it an {{int
minExactTotalHits}} option?
So if a caller specifies {{minExactTotalHits=5000}} and the resulting
{{TopDocs.totalHits == 42}} then there would also be a
{{TopDocs.totalHitsAccuracy == 1.0D}} because we're 100% confident that that's
the number of hits ... but if {{TopDocs.totalHits == 42000000}} then maybe
{{TopDocs.totalHitsAccuracy == 0.1D}} or maybe {{TopDocs.totalHitsAccuracy ==
0.9D}} depending on how confident the estimation is.
would that even be possible?
----
bq. Regarding integration in IndexSearcher, I am thinking of 3 ideas:
I think as long as {{totalHits}} is renamed/replaced to force a compilation
error and draw attention to the need to use a Collector if you want to control
if/how-much the total number of hits is accurately recorded, it's fine to
hadcode a default in the IndexSearcher methods that return TopDocs directly ...
i would go so far as to suggest that in that in that situation, hardcoding
maxTotalHits/minExactTotalHits to "0" (ie: don't bother trying to track exactly
at all) would be fine.
> Require users to tell us whether they need total hit counts
> -----------------------------------------------------------
>
> Key: LUCENE-8060
> URL: https://issues.apache.org/jira/browse/LUCENE-8060
> Project: Lucene - Core
> Issue Type: Improvement
> Reporter: Adrien Grand
> Priority: Minor
> Fix For: master (8.0)
>
>
> We are getting optimizations when hit counts are not required (sorted
> indexes, MAXSCORE, short-circuiting of phrase queries) but our users won't
> benefit from them unless we disable exact hit counts by default or we require
> them to tell us whether hit counts are required.
> I think making hit counts approximate by default is going to be a bit trappy,
> so I'm rather leaning towards requiring users to tell us explicitly whether
> they need total hit counts. I can think of two ways to do that: either by
> passing a boolean to the IndexSearcher constructor or by adding a boolean to
> all methods that produce TopDocs instances. I like the latter better but I'm
> open to discussion or other ideas?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
