[
https://issues.apache.org/jira/browse/LUCENE-8010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16304098#comment-16304098
]
Robert Muir commented on LUCENE-8010:
-------------------------------------
And the TODOs about randomizing parameters are kinda serious for these. For the
other sims we randomize parameters within the range it will accept and then
fixed sim's checks to match. We should issue IAE if a parameter is out of range
rather than silently screw up later, this fixed a lot of possible bugs.
Its just important as some of these sims dont even have a default ctor (not
sure if this is intentional) and we fixed this everywhere else. We can make it
a followup issue though.
> fix or sandbox similarities in core with problems
> -------------------------------------------------
>
> Key: LUCENE-8010
> URL: https://issues.apache.org/jira/browse/LUCENE-8010
> Project: Lucene - Core
> Issue Type: Bug
> Reporter: Robert Muir
> Attachments: LUCENE-8010.patch
>
>
> We want to support scoring optimizations such as LUCENE-4100 and LUCENE-7993,
> which put very minimal requirements on the similarity impl. Today
> similarities of various quality are in core and tests.
> The ones with problems currently have warnings in the javadocs about their
> bugs, and if the problems are severe enough, then they are also disabled in
> randomized testing too.
> IMO lucene core should only have practical functions that won't return
> {{NaN}} scores at times or cause relevance to go backwards if the user's
> stopfilter isn't configured perfectly. Also it is important for unit tests to
> not deal with broken or semi-broken sims, and the ones in core should pass
> all unit tests.
> I propose we move the buggy ones to sandbox and deprecate them. If they can
> be fixed we can put them back in core, otherwise bye-bye.
> FWIW tests developed in LUCENE-7997 document the following requirements:
> * scores are non-negative and finite.
> * score matches the explanation exactly.
> * internal explanations calculations are sane (e.g. sum of: and so on
> actually compute sums)
> * scores don't decrease as term frequencies increase: e.g. score(freq=N +
> 1) >= score(freq=N)
> * scores don't decrease as documents get shorter, e.g. score(len=M) >=
> score(len=M+1)
> * scores don't decrease as terms get rarer, e.g. score(term=N) >=
> score(term=N+1)
> * scoring works for floating point frequencies (e.g. sloppy phrase and
> span queries will work)
> * scoring works for reasonably large 64-bit statistic values (e.g.
> distributed search will work)
> * scoring works for reasonably large boost values (0 .. Integer.MAX_VALUE,
> e.g. query boosts will work)
> * scoring works for parameters randomized within valid ranges
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
