Ok, so after I followed Rob's hints and clues, I found LongFieldSource, which uses FieldCache API under the hood, which access the NDV field. I feel sorry for the poor user who will try to figure it out himself though, because there's no evident anywhere that this is what you should do!
LongFieldSource jdocs are completely erroneous, seems a copy-paste bug from FloatFieldSource. Then, if you get passed that and read FieldCache.getLongs jdocs, they say: * Checks the internal cache for an appropriate entry, and if none is * found, *reads the terms in <code>field</code> as longs* and returns an array * of size <code>reader.maxDoc()</code> of the value each document * has in the given field. Nothing about NumericDocValues. I actually looked at FieldCache before sending the first email, but all I could conclude from the jdocs is that it parses terms. I didn't bother looking at FieldCacheImpl implementation, and users shouldn't be expected to do that. I'll open an issue to clean/clarify javadocs. Shai On Fri, Sep 20, 2013 at 6:53 PM, Robert Muir <[email protected]> wrote: > You asked about how to access a NumericDocValues field from a > valuesource in lucene. > > Yet you showed an example where you did just this with expressions, so > I'm just recommending you look at expressions/ source code (they use > valuesource under the hood) to see how its done! > > On Fri, Sep 20, 2013 at 11:49 AM, Shai Erera <[email protected]> wrote: > > What do Expressions have to do here? Do they replace CustomScoreQuery? > Maybe > > they should, I don't know. But today, if you want to use CSQ, to boost a > > document by an NDV field, you need to write a ValueSource which reads > from > > the field. And that's the object that I don't see. > > > > Maybe you want to say that Expressions will eventually replace CSQ, and > so > > it's moot to add a NumericDVFieldSource to Lucene? Or we want to > document on > > CSQ that you should really consider using Expressions? > > > > Shai > > > > > > On Fri, Sep 20, 2013 at 6:41 PM, Robert Muir <[email protected]> wrote: > >> > >> why dont you look and see how expressions is doing it? > >> > >> On Fri, Sep 20, 2013 at 11:39 AM, Shai Erera <[email protected]> wrote: > >> > Thanks Rob. So is there a NumericDVFieldSource-like in Lucene? I think > >> > it's > >> > important that we have one. > >> > > >> > Shai > >> > > >> > > >> > On Fri, Sep 20, 2013 at 6:10 PM, Robert Muir <[email protected]> > wrote: > >> >> > >> >> thats what it does. its more like a computed field. and you can sort > >> >> by more than one of them. > >> >> > >> >> please see the JIRA issue for a description of the differences > between > >> >> function queries. > >> >> > >> >> On Fri, Sep 20, 2013 at 10:49 AM, Shai Erera <[email protected]> > wrote: > >> >> > Yes, you're right, but that's unrelated to this thread. I passed > >> >> > doScore=true and the scores come out the same, meaning Expression > >> >> > didn't > >> >> > affect the actual score, only the sort-by value (which is ok). > >> >> > > >> >> > search Expression > >> >> > doc=1, score=0.37158427, field=0.7431685328483582 > >> >> > doc=0, score=0.37158427, field=0.3715842664241791 > >> >> > > >> >> > Shai > >> >> > > >> >> > > >> >> > On Fri, Sep 20, 2013 at 5:10 PM, Robert Muir <[email protected]> > >> >> > wrote: > >> >> >> > >> >> >> On Fri, Sep 20, 2013 at 8:01 AM, Shai Erera <[email protected]> > >> >> >> wrote: > >> >> >> > > >> >> >> > Expression > >> >> >> > I tried the new module, following TestDemoExpression and > compiled > >> >> >> > the > >> >> >> > expression using this code: > >> >> >> > > >> >> >> > Expression expr = JavascriptCompiler.compile("_score * > >> >> >> > boost"); > >> >> >> > SimpleBindings bindings = new SimpleBindings(); > >> >> >> > bindings.add(new SortField("_score", SortField.Type.SCORE)); > >> >> >> > bindings.add(new SortField("boost", SortField.Type.LONG)); > >> >> >> > > >> >> >> > The result scores are: > >> >> >> > > >> >> >> > search Expression > >> >> >> > doc=1, score=NaN, field=0.7431685328483582 > >> >> >> > doc=0, score=NaN, field=0.3715842664241791 > >> >> >> > > >> >> >> > As you can see, both CustomScoreProvider and Expression methods > >> >> >> > return > >> >> >> > same > >> >> >> > scores for the docs, while the FunctionQuery method returns > >> >> >> > different > >> >> >> > scores. The reason is that when using FunctionQuery, the scores > of > >> >> >> > the > >> >> >> > ValueSources are multiplied by queryWeight, which seems correct > to > >> >> >> > me. > >> >> >> > > >> >> >> > Expression is more about sorting than scoring as far as I > >> >> >> > understand > >> >> >> > (for > >> >> >> > instance, the result FieldDocs.score is NaN) > >> >> >> > >> >> >> Why does that come as a surprise to you? Pass true to > indexsearcher > >> >> >> to get the documents score back here. > >> >> >> > >> >> >> ======================= Release 2.9.0 2009-09-23 > >> >> >> ======================= > >> >> >> > >> >> >> Changes in backwards compatibility policy > >> >> >> > >> >> >> LUCENE-1575: Searchable.search(Weight, Filter, int, Sort) no > longer > >> >> >> computes a document score for each hit by default. > >> >> >> ... (Shai Erera via Mike McCandless) > >> >> >> > >> >> >> > >> >> >> > --------------------------------------------------------------------- > >> >> >> To unsubscribe, e-mail: [email protected] > >> >> >> For additional commands, e-mail: [email protected] > >> >> >> > >> >> > > >> >> > >> >> --------------------------------------------------------------------- > >> >> To unsubscribe, e-mail: [email protected] > >> >> For additional commands, e-mail: [email protected] > >> >> > >> > > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: [email protected] > >> For additional commands, e-mail: [email protected] > >> > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
