[
https://issues.apache.org/jira/browse/LUCENE-6276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14963465#comment-14963465
]
David Smiley commented on LUCENE-6276:
--------------------------------------
RE BooleanQuery stable ordering: thanks for correcting me; I'm very glad it's
stable. At least this gives the user some control.
I think we may need to percolate the matchCost concept further into other APIs
-- namely ValueSource/FunctionValues. This way the 2-phase iterator can
enclose one of them to fetch the matchCost of it when composing it's aggregate
matchCost.
And I question if a numeric DV lookup or reading a posting is equivalent to one
mathematical operation since those things involve some code behind them that
aren't cheap one-liners that a math operation are. Nonetheless, I get the
concept you are suggesting.
I kind of like the time based approach you suggested better but what's needed
is some automation to aid in establishing a baseline such that someone on their
own machine can get an approximation slower/faster factor multiplier compared
to some baseline server. Like what if there was an ant target that ran some
test based on Wikipedia data that established that the matchCost for a some
PhraseQuery on the machine it's run on is ____ nanoseconds. Then the output
also displays some hard-coded value that we got when running this on some
baseline server. Dividing the two yields a relative difference between the
local machine and the baseline server. Then when working on a custom query, I
could locally temporarily either modify the timing test to test my new query
(perhaps plucking the geo data in Wikipedia out to a lat-lon spatial field) or
timing it in my own way. Then I apply the multiplier to determine which number
to hard-code into the query going into Lucene. Make sense?
There is another aspect of the cost beyond a per-postings iteration cost. The
cost of reading the N+1 position is generally going to be much cheaper than
reading the very first position since the first once possibly involves a seek.
If only postings based queries are being compared via matchCost, this is a wash
since all of them have this cost but it'd be different for a doc-values based
query. Although perhaps it's a wash there too -- assume one disk seek? Worst
case of course.
> Add matchCost() api to TwoPhaseDocIdSetIterator
> -----------------------------------------------
>
> Key: LUCENE-6276
> URL: https://issues.apache.org/jira/browse/LUCENE-6276
> Project: Lucene - Core
> Issue Type: Improvement
> Reporter: Robert Muir
> Attachments: LUCENE-6276-ExactPhraseOnly.patch,
> LUCENE-6276-NoSpans.patch, LUCENE-6276-NoSpans2.patch, LUCENE-6276.patch,
> LUCENE-6276.patch, LUCENE-6276.patch, LUCENE-6276.patch
>
>
> We could add a method like TwoPhaseDISI.matchCost() defined as something like
> estimate of nanoseconds or similar.
> ConjunctionScorer could use this method to sort its 'twoPhaseIterators' array
> so that cheaper ones are called first. Today it has no idea if one scorer is
> a simple phrase scorer on a short field vs another that might do some geo
> calculation or more expensive stuff.
> PhraseScorers could implement this based on index statistics (e.g.
> totalTermFreq/maxDoc)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
