[jira] [Updated] (LUCENE-5325) Move ValueSource and FunctionValues under core/
[
https://issues.apache.org/jira/browse/LUCENE-5325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Woodward updated LUCENE-5325:
--
Attachment: LUCENE-5325-6x-matchingbits.patch
bq. shouldn't it check docsWithField?
Hm, you're quite right. Here's a patch fixing that - will commit shortly.
> Move ValueSource and FunctionValues under core/
> ---
>
> Key: LUCENE-5325
> URL: https://issues.apache.org/jira/browse/LUCENE-5325
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/search
>Reporter: Shai Erera
> Attachments: LUCENE-5325-6x-matchingbits.patch, LUCENE-5325-6x.patch,
> LUCENE-5325.patch, LUCENE-5325.patch, LUCENE-5325.patch, LUCENE-5325.patch,
> LUCENE-5325.patch, LUCENE-5325.patch, LUCENE-5325.patch
>
>
> Spinoff from LUCENE-5298: ValueSource and FunctionValues are abstract APIs
> which exist under the queries/ module. That causes any module which wants to
> depend on these APIs (but not necessarily on any of their actual
> implementations!), to depend on the queries/ module. If we move these APIs
> under core/, we can eliminate these dependencies and add some mock impls for
> testing purposes.
> Quoting Robert from LUCENE-5298:
> {quote}
> we should eliminate the suggest/ dependencies on expressions and queries, the
> expressions/ on queries, the grouping/ dependency on queries, the spatial/
> dependency on queries, its a mess.
> {quote}
> To add to that list, facet/ should not depend on queries too.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5325) Move ValueSource and FunctionValues under core/
[
https://issues.apache.org/jira/browse/LUCENE-5325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Woodward updated LUCENE-5325:
--
Attachment: LUCENE-5325-6x.patch
The patch for 6x is slightly different, as the DocValues iterator API hasn't
been backported, so the interaction with the FieldComparators has to take into
account missing values
> Move ValueSource and FunctionValues under core/
> ---
>
> Key: LUCENE-5325
> URL: https://issues.apache.org/jira/browse/LUCENE-5325
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/search
>Reporter: Shai Erera
> Attachments: LUCENE-5325-6x.patch, LUCENE-5325.patch,
> LUCENE-5325.patch, LUCENE-5325.patch, LUCENE-5325.patch, LUCENE-5325.patch,
> LUCENE-5325.patch, LUCENE-5325.patch
>
>
> Spinoff from LUCENE-5298: ValueSource and FunctionValues are abstract APIs
> which exist under the queries/ module. That causes any module which wants to
> depend on these APIs (but not necessarily on any of their actual
> implementations!), to depend on the queries/ module. If we move these APIs
> under core/, we can eliminate these dependencies and add some mock impls for
> testing purposes.
> Quoting Robert from LUCENE-5298:
> {quote}
> we should eliminate the suggest/ dependencies on expressions and queries, the
> expressions/ on queries, the grouping/ dependency on queries, the spatial/
> dependency on queries, its a mess.
> {quote}
> To add to that list, facet/ should not depend on queries too.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5325) Move ValueSource and FunctionValues under core/
[
https://issues.apache.org/jira/browse/LUCENE-5325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Woodward updated LUCENE-5325:
--
Attachment: LUCENE-5325.patch
Final patch, to apply after LUCENE-7607.
> Move ValueSource and FunctionValues under core/
> ---
>
> Key: LUCENE-5325
> URL: https://issues.apache.org/jira/browse/LUCENE-5325
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/search
>Reporter: Shai Erera
> Attachments: LUCENE-5325.patch, LUCENE-5325.patch, LUCENE-5325.patch,
> LUCENE-5325.patch, LUCENE-5325.patch, LUCENE-5325.patch, LUCENE-5325.patch
>
>
> Spinoff from LUCENE-5298: ValueSource and FunctionValues are abstract APIs
> which exist under the queries/ module. That causes any module which wants to
> depend on these APIs (but not necessarily on any of their actual
> implementations!), to depend on the queries/ module. If we move these APIs
> under core/, we can eliminate these dependencies and add some mock impls for
> testing purposes.
> Quoting Robert from LUCENE-5298:
> {quote}
> we should eliminate the suggest/ dependencies on expressions and queries, the
> expressions/ on queries, the grouping/ dependency on queries, the spatial/
> dependency on queries, its a mess.
> {quote}
> To add to that list, facet/ should not depend on queries too.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5325) Move ValueSource and FunctionValues under core/
[
https://issues.apache.org/jira/browse/LUCENE-5325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Woodward updated LUCENE-5325:
--
Attachment: LUCENE-5325.patch
Thanks for the review, Adrien. Here's an updated patch:
* advanceTo renamed to advanceExact
* fromIntField delegates to fromLongField in both Long and DoubleValuesSource
* I ended up removing the automatic wrapping of doubles to longs in
LongValuesSource entirely, in favour of calling
DoubleValuesSource.toLongValuesSource(). The javadocs on the latter now
explicitly say that the conversion is done via casting.
* Javadocs generally are more explanatory, particularly around scores
* getValues() now takes a DoubleValues object rather than a Scorer - this ended
up being a very nice change, and will allow us to remove several of the
FakeScorer implementations that the current ValueSource API forces us to use in
things like expressions or facets.
I'll open a separate issue for adding the throws clause for
FieldComparator.setScorer().
> Move ValueSource and FunctionValues under core/
> ---
>
> Key: LUCENE-5325
> URL: https://issues.apache.org/jira/browse/LUCENE-5325
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/search
>Reporter: Shai Erera
> Attachments: LUCENE-5325.patch, LUCENE-5325.patch, LUCENE-5325.patch,
> LUCENE-5325.patch, LUCENE-5325.patch, LUCENE-5325.patch
>
>
> Spinoff from LUCENE-5298: ValueSource and FunctionValues are abstract APIs
> which exist under the queries/ module. That causes any module which wants to
> depend on these APIs (but not necessarily on any of their actual
> implementations!), to depend on the queries/ module. If we move these APIs
> under core/, we can eliminate these dependencies and add some mock impls for
> testing purposes.
> Quoting Robert from LUCENE-5298:
> {quote}
> we should eliminate the suggest/ dependencies on expressions and queries, the
> expressions/ on queries, the grouping/ dependency on queries, the spatial/
> dependency on queries, its a mess.
> {quote}
> To add to that list, facet/ should not depend on queries too.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5325) Move ValueSource and FunctionValues under core/
[
https://issues.apache.org/jira/browse/LUCENE-5325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Woodward updated LUCENE-5325:
--
Attachment: LUCENE-5325.patch
Waking this one up again. Here's a patch that takes into account the new doc
values iterator API. The new classes look like this:
DoubleValuesSource
- getDoubleValues(LeafReaderContext, Scorer) which returns a DoubleValues
DoubleValues
- advanceTo(doc) : returns true if doc has a value
- doubleValue()
And similar for LongValuesSource/LongValues. There is already a LongValues
class in lucene.util, so this probably needs a better name. The XValuesSource
classes have a bunch of static methods for generating sources from doc values
fields or from a Scorer.
I also have patches that convert to and from the old ValueSource API, and that
migrate the facet, expression and suggest modules, but I think they should be
kept for follow-up issues.
> Move ValueSource and FunctionValues under core/
> ---
>
> Key: LUCENE-5325
> URL: https://issues.apache.org/jira/browse/LUCENE-5325
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/search
>Reporter: Shai Erera
> Attachments: LUCENE-5325.patch, LUCENE-5325.patch, LUCENE-5325.patch,
> LUCENE-5325.patch, LUCENE-5325.patch
>
>
> Spinoff from LUCENE-5298: ValueSource and FunctionValues are abstract APIs
> which exist under the queries/ module. That causes any module which wants to
> depend on these APIs (but not necessarily on any of their actual
> implementations!), to depend on the queries/ module. If we move these APIs
> under core/, we can eliminate these dependencies and add some mock impls for
> testing purposes.
> Quoting Robert from LUCENE-5298:
> {quote}
> we should eliminate the suggest/ dependencies on expressions and queries, the
> expressions/ on queries, the grouping/ dependency on queries, the spatial/
> dependency on queries, its a mess.
> {quote}
> To add to that list, facet/ should not depend on queries too.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5325) Move ValueSource and FunctionValues under core/
[
https://issues.apache.org/jira/browse/LUCENE-5325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Woodward updated LUCENE-5325:
--
Attachment: LUCENE-5325.patch
You're right, it is unrelated, and there's a way around it, I just needed to
get away from thinking in the mode of the previous API. So thanks for the
push-back :-)
Here's a new patch which cuts over expressions, facets and suggest. There's
one new method on DoubleValuesSource and LongValuesSource, for retrieving a
bitset marking which docs actually have a value here, which is required by the
facet counting code. Field wrappers just forward to
DocValues.getDocsWithField(), and everything else returns a MatchAllBits for
now. Ideally, expressions would AND together the bitsets from all child
variables, but that can be done in a followup once I've found the methods to
actually do that efficiently...
I think this is good enough to start with? Grouping, join and queries modules
are going to be more complicated I think, so should be done in a separate JIRA.
And this patch is already pretty big, and can probably be broken up into
separate commits for each module.
> Move ValueSource and FunctionValues under core/
> ---
>
> Key: LUCENE-5325
> URL: https://issues.apache.org/jira/browse/LUCENE-5325
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/search
>Reporter: Shai Erera
> Attachments: LUCENE-5325.patch, LUCENE-5325.patch, LUCENE-5325.patch,
> LUCENE-5325.patch
>
>
> Spinoff from LUCENE-5298: ValueSource and FunctionValues are abstract APIs
> which exist under the queries/ module. That causes any module which wants to
> depend on these APIs (but not necessarily on any of their actual
> implementations!), to depend on the queries/ module. If we move these APIs
> under core/, we can eliminate these dependencies and add some mock impls for
> testing purposes.
> Quoting Robert from LUCENE-5298:
> {quote}
> we should eliminate the suggest/ dependencies on expressions and queries, the
> expressions/ on queries, the grouping/ dependency on queries, the spatial/
> dependency on queries, its a mess.
> {quote}
> To add to that list, facet/ should not depend on queries too.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5325) Move ValueSource and FunctionValues under core/
[
https://issues.apache.org/jira/browse/LUCENE-5325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Woodward updated LUCENE-5325:
--
Attachment: LUCENE-5325.patch
Another iteration, this time pushing access to Scores entirely back into the
expressions code. The API is pretty minimal now:
- DoubleValues just has ```double get(int)```
- DoubleValuesSource just has ```DoubleValues getValues(LeafReaderContext)```,
plus some factory methods for access to DV fields and a SortField implementation
> Move ValueSource and FunctionValues under core/
> ---
>
> Key: LUCENE-5325
> URL: https://issues.apache.org/jira/browse/LUCENE-5325
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/search
>Reporter: Shai Erera
> Attachments: LUCENE-5325.patch, LUCENE-5325.patch, LUCENE-5325.patch
>
>
> Spinoff from LUCENE-5298: ValueSource and FunctionValues are abstract APIs
> which exist under the queries/ module. That causes any module which wants to
> depend on these APIs (but not necessarily on any of their actual
> implementations!), to depend on the queries/ module. If we move these APIs
> under core/, we can eliminate these dependencies and add some mock impls for
> testing purposes.
> Quoting Robert from LUCENE-5298:
> {quote}
> we should eliminate the suggest/ dependencies on expressions and queries, the
> expressions/ on queries, the grouping/ dependency on queries, the spatial/
> dependency on queries, its a mess.
> {quote}
> To add to that list, facet/ should not depend on queries too.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5325) Move ValueSource and FunctionValues under core/
[
https://issues.apache.org/jira/browse/LUCENE-5325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Woodward updated LUCENE-5325:
--
Attachment: LUCENE-5325.patch
Updated patch, nuking setScorer() and exists().
There's a bit of hinky logic around creating sorts now, because FieldComparator
separates out setNextReader() and setScorer(), but it is at least contained.
We still need to pass in a Scorer somewhere though, so that scores are
available to the leaf values.
Another option would be to pull the Scorer logic back into the expressions
module itself, but I think having a generic DoubleValues.SCORES instance is
going to be useful in lots of situations; plus it means that all of the sorting
logic is in DoubleValuesProducer itself.
Also adds a test for converting a DoubleValuesSource into a generic ValueSource
(not the other way round, which I had originally put in and isn't needed yet!)
> Move ValueSource and FunctionValues under core/
> ---
>
> Key: LUCENE-5325
> URL: https://issues.apache.org/jira/browse/LUCENE-5325
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/search
>Reporter: Shai Erera
> Attachments: LUCENE-5325.patch, LUCENE-5325.patch
>
>
> Spinoff from LUCENE-5298: ValueSource and FunctionValues are abstract APIs
> which exist under the queries/ module. That causes any module which wants to
> depend on these APIs (but not necessarily on any of their actual
> implementations!), to depend on the queries/ module. If we move these APIs
> under core/, we can eliminate these dependencies and add some mock impls for
> testing purposes.
> Quoting Robert from LUCENE-5298:
> {quote}
> we should eliminate the suggest/ dependencies on expressions and queries, the
> expressions/ on queries, the grouping/ dependency on queries, the spatial/
> dependency on queries, its a mess.
> {quote}
> To add to that list, facet/ should not depend on queries too.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5325) Move ValueSource and FunctionValues under core/
[
https://issues.apache.org/jira/browse/LUCENE-5325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Woodward updated LUCENE-5325:
--
Attachment: LUCENE-5325.patch
Patch.
To keep it simple, I've just added DoubleValues / DoubleValuesSource, and cut
over the expressions module to use it. Still needs some work:
- the ValueSource.toDoubleValuesSource() method needs a test
- I'm not quite sure how to deal with exceptions throw by Scorer.score() in
DoubleValuesSource.SCORES
- I've copied the exists() method from FunctionValues to DoubleValues, but I'm
not convinced we need it. The expressions module doesn't actually use it, at
any rate, but in some cases I can see it's useful to distinguish between a
zero-value and a non-existent value. Maybe I should remove it here, and it can
be added in later if it's needed for a specific function?
- DoubleValues is an abstract class rather than an interface, simply because I
couldn't get the ASM stuff in JavascriptCompiler to work with interfaces.
> Move ValueSource and FunctionValues under core/
> ---
>
> Key: LUCENE-5325
> URL: https://issues.apache.org/jira/browse/LUCENE-5325
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/search
>Reporter: Shai Erera
> Attachments: LUCENE-5325.patch
>
>
> Spinoff from LUCENE-5298: ValueSource and FunctionValues are abstract APIs
> which exist under the queries/ module. That causes any module which wants to
> depend on these APIs (but not necessarily on any of their actual
> implementations!), to depend on the queries/ module. If we move these APIs
> under core/, we can eliminate these dependencies and add some mock impls for
> testing purposes.
> Quoting Robert from LUCENE-5298:
> {quote}
> we should eliminate the suggest/ dependencies on expressions and queries, the
> expressions/ on queries, the grouping/ dependency on queries, the spatial/
> dependency on queries, its a mess.
> {quote}
> To add to that list, facet/ should not depend on queries too.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
