Re: Function query matching

2014-01-06 Thread Peter Keegan
: The bottom line for Peter is still the same: using scale() wrapped arround : a function/query does involve a computing hte results for every document, : and that is going to scale linearly as the size of hte index grows -- but : it it is *only* because of the scale function. Another problem with

Re: Function query matching

2013-12-07 Thread Chris Hostetter
(This is why i shouldn't send emails just before going to bed.) I woke up this morning realizing that of course I was completley wrong when i said this... : I want to be clear for 99% of the people reading this, if you find : yourself writting a query structure like this... : : q={!func}..f

Re: Function query matching

2013-12-07 Thread Peter Keegan
>But for your specific goal Peter: Yes, if the whole point of a function >you have is to wrap generated a "scaled" score of your base $qq, ... Thanks for the confirmation, Chris. So, to do this efficiently, I think I need to implement a custom Collector that performs the scaling (and other mat

Re: Function query matching

2013-12-06 Thread Chris Hostetter
I had to do a double take when i read this sentence... : Even with any improvements to 'scale', all function queries will add a : linear increase to the Qtime as index size increases, since they match all : docs. ...because that smelled like either a bug in your methodology, or a bug in Solr.

Re: Function query matching

2013-12-06 Thread Peter Keegan
In my previous posting, I said: "Subsequent calls to ScaleFloatFuntion.getValues bypassed 'createScaleInfo and added ~0 time." These subsequent calls are for the remaining segments in the index reader (21 segments). Peter On Fri, Dec 6, 2013 at 2:10 PM, Peter Keegan wrote: > I added some

Re: Function query matching

2013-12-06 Thread Peter Keegan
I added some timing logging to IndexSearcher and ScaleFloatFunction and compared a simple DisMax query with a DisMax query wrapped in the scale function. The index size was 500K docs, 61K docs match the DisMax query. The simple DisMax query took 33 ms, the function query took 89 ms. What I found wa

Re: Function query matching

2013-12-02 Thread Trey Grainger
We're working on the same problem with the combination of the scale(query(...)) combination, so I'd like to share a bit more information that may be useful. *On the scale function:* Even thought the scale query has to calculate the scores for all documents, it is actually doing this work twice for

Re: Function query matching

2013-12-02 Thread Peter Keegan
I'm persuing this possible PostFilter solution, I can see how to collect all the hits and recompute the scores in a PostFilter, after all the hits have been collected (for scaling). Now, I can't see how to get the custom doc/score values back into the main query's HitQueue. Any advice? Thanks, Pet

Re: Function query matching

2013-11-29 Thread Peter Keegan
Instead of using a function query, could I use the edismax query (plus some low cost filters not shown in the example) and implement the scale/sum/product computation in a PostFilter? Is the query's maxScore available there? Thanks, Peter On Wed, Nov 27, 2013 at 1:58 PM, Peter Keegan wrote: > A

Re: Function query matching

2013-11-27 Thread Peter Keegan
Although the 'scale' is a big part of it, here's a closer breakdown. Here are 4 queries with increasing functions, and theei response times (caching turned off in solrconfig): 100 msec: select?q={!edismax v='news' qf='title^2 body'} 135 msec: select?qq={!edismax v='news' qf='title^2 body'}q={!fun

Re: Function query matching

2013-11-27 Thread Chris Hostetter
: So, this query does just what I want, but it's typically 3 times slower : than the edismax query without the functions: that's because the scale() function is inhernetly slow (it has to compute the min & max value for every document in order to know how to scale them) what you are seeing is

Re: Function query matching

2013-11-27 Thread Peter Keegan
Hi, So, this query does just what I want, but it's typically 3 times slower than the edismax query without the functions: select?qq={!edismax v='news' qf='title^2 body'}&scaledQ=scale(product( query($qq),1),0,1)&q={!func}sum(product(0.75,$scaledQ), product(0.25,field(myfield)))&fq={!query v=$qq}

Re: Function query matching

2013-11-11 Thread Peter Keegan
Thanks On Mon, Nov 11, 2013 at 11:46 AM, Yonik Seeley wrote: > On Mon, Nov 11, 2013 at 11:39 AM, Peter Keegan > wrote: > > fq=$qq > > > > What is the proper syntax? > > fq={!query v=$qq} > > -Yonik > http://heliosearch.com -- making solr shine >

Re: Function query matching

2013-11-11 Thread Yonik Seeley
On Mon, Nov 11, 2013 at 11:39 AM, Peter Keegan wrote: > fq=$qq > > What is the proper syntax? fq={!query v=$qq} -Yonik http://heliosearch.com -- making solr shine

Re: Function query matching

2013-11-11 Thread Peter Keegan
I replaced the frange filter with the following filter and got the correct no. of results and it was 3X faster: select?qq={!edismax v='news' qf='title^2 body'}&scaledQ=scale(product(query($qq),1),0,1)&q={!func}sum(product(0.75,$scaledQ),product(0.25,field(myfield)))&fq={!edismax v='news' qf='title

Re: Function query matching

2013-11-07 Thread Peter Keegan
I'm trying to used a normalized score in a query as I described in a recent thread titled "Re: How to get similarity score between 0 and 1 not relative score" I'm using this query: select?qq={!edismax v='news' qf='title^2 body'}&scaledQ=scale(product(query($qq),1),0,1)&q={!func}sum(product(0.75,$s

Re: Function query matching

2013-11-07 Thread Jason Hellman
You can, of course, us a function range query: select?q=text:news&fq={!frange l=0 u=100}sum(x,y) http://lucene.apache.org/solr/4_5_1/solr-core/org/apache/solr/search/FunctionRangeQParserPlugin.html This will give you a bit more flexibility to meet your goal. On Nov 7, 2013, at 7:26 AM, Erik Hat

Re: Function query matching

2013-11-07 Thread Erik Hatcher
Function queries score (all) documents, but don't filter them. All documents effectively match a function query. Erik On Nov 7, 2013, at 1:48 PM, Peter Keegan wrote: > Why does this function query return docs that don't match the embedded > query? > select?qq=text:news&q={!func}sum

Function query matching

2013-11-07 Thread Peter Keegan
Why does this function query return docs that don't match the embedded query? select?qq=text:news&q={!func}sum(query($qq),0)