Re: Does sorting skip everything having to do with relevancy?
Arhgh... It seems like Functions Queries (obviously) never throw an exception. I have to scratch my own which throws always. Here is the proof that boost is lazy https://gist.github.com/m-khl/10010541 On Sun, Apr 6, 2014 at 12:54 AM, Shawn Heisey s...@elyograg.org wrote: On 4/5/2014 1:21 PM, Mikhail Khludnev wrote: I suppose e yields syntax error. Therefore, this case doesn't prove anything yet. Haven't you tried sqrt(-1) or log(-1) ? Using boost=sqrt(-1) is error-free whether I include the sort parameter or not. That seems like a bug. Thanks, Shawn -- Sincerely yours Mikhail Khludnev Principal Engineer, Grid Dynamics http://www.griddynamics.com mkhlud...@griddynamics.com
Re: Does sorting skip everything having to do with relevancy?
Shawn, I suppose e yields syntax error. Therefore, this case doesn't prove anything yet. Haven't you tried sqrt(-1) or log(-1) ? On Sat, Apr 5, 2014 at 1:47 AM, Shawn Heisey s...@elyograg.org wrote: On 4/4/2014 3:13 PM, Mikhail Khludnev wrote: I suppose SolrIndexSearcher.buildTopDocsCollector() doesn't create a Collector which calls score() in this case. Hence, it shouldn't waste CPU. Just my impression. Haven't you tried to check it supplying some weird formula, which throws exception? I didn't think of that. That's a good idea -- as long as there's not independent code that checks the function in addition to the code that actually runs it. With the following parameters added to an edismax query that otherwise works, I get an exception. It works if I change the e to 5. sort=registered_date ascboost=sum(5,e) I will take Alvaro's suggestion and add boost=1 to queries that use a sort parameter. It's probably a good idea to file that Jira. Thanks, Shawn -- Sincerely yours Mikhail Khludnev Principal Engineer, Grid Dynamics http://www.griddynamics.com mkhlud...@griddynamics.com
Re: Does sorting skip everything having to do with relevancy?
On 4/5/2014 1:21 PM, Mikhail Khludnev wrote: I suppose e yields syntax error. Therefore, this case doesn't prove anything yet. Haven't you tried sqrt(-1) or log(-1) ? Using boost=sqrt(-1) is error-free whether I include the sort parameter or not. That seems like a bug. Thanks, Shawn
Re: Does sorting skip everything having to do with relevancy?
Hi, By default solr is using the sort parameter over the score field. So if you overwrite it using other sort field, yes solr will use the parameter you've provided. Remember, you can use multiple fields for sortinghttp://wiki.apache.org/solr/CommonQueryParameters#sort so you can make something like: sort score desc, your_field1 asc, your_field2 desc The score of documents is calculated on every query (it does not depend on the sort parameter or the debugQueryParameter) and the debubQuery is only a mechanism for showing (or hidding) how score was calculated. If you want to see a document score for a particular query (apart from the debugQuery) you can ask for it in the solr response adding the parameter *fl=*,score* to your request. Regards. On Fri, Apr 4, 2014 at 4:42 AM, Shawn Heisey s...@elyograg.org wrote: If I provide a sort parameter, will Solr (4.6.1) skip score/boost processing? In particular I would like to know what happens if I have a boost parameter (with a complex function) for edismax search, but I include a sort parameter on one of my fields. I am using distributed search. I do know that if debugQuery is used, the score IS calculated, but I'm talking about when debugQuery is not used. Thanks, Shawn
Re: Does sorting skip everything having to do with relevancy?
On 4/4/2014 12:48 AM, Alvaro Cabrerizo wrote: By default solr is using the sort parameter over the score field. So if you overwrite it using other sort field, yes solr will use the parameter you've provided. Remember, you can use multiple fields for sortinghttp://wiki.apache.org/solr/CommonQueryParameters#sort so you can make something like: sort score desc, your_field1 asc, your_field2 desc The score of documents is calculated on every query (it does not depend on the sort parameter or the debugQueryParameter) and the debubQuery is only a mechanism for showing (or hidding) how score was calculated. If you want to see a document score for a particular query (apart from the debugQuery) you can ask for it in the solr response adding the parameter *fl=*,score* to your request. These are things that I already know. What I want to know is whether Solr has code in place that will avoid wasting CPU cycles calculating the score that will never be displayed or used, *especially* the complex boost parameter that's in the request handler definition (solrconfig.xml). str name=boostmin(recip(abs(ms(NOW/HOUR,registered_date)),1.92901e-10,1.5,1.5),0.85)/str Do I need to send 'boost=' as a parameter (along with my sort) to get it to avoid that calculation? Thanks, Shawn
Re: Does sorting skip everything having to do with relevancy?
Hi, If you dont want to waste your cpu time, then comment the boost parameter in the query parser defined in your solrconfig.xml. If you cant do that, then you can overwrite it sending the boost parameter for example using the constant function (e.g. http:///...boost=1sort=your_sort). The parameter boost will be overwritten if it is not defined as an invariant. Regards. On Fri, Apr 4, 2014 at 4:12 PM, Shawn Heisey s...@elyograg.org wrote: On 4/4/2014 12:48 AM, Alvaro Cabrerizo wrote: By default solr is using the sort parameter over the score field. So if you overwrite it using other sort field, yes solr will use the parameter you've provided. Remember, you can use multiple fields for sortinghttp://wiki.apache.org/solr/CommonQueryParameters#sort so you can make something like: sort score desc, your_field1 asc, your_field2 desc The score of documents is calculated on every query (it does not depend on the sort parameter or the debugQueryParameter) and the debubQuery is only a mechanism for showing (or hidding) how score was calculated. If you want to see a document score for a particular query (apart from the debugQuery) you can ask for it in the solr response adding the parameter *fl=*,score* to your request. These are things that I already know. What I want to know is whether Solr has code in place that will avoid wasting CPU cycles calculating the score that will never be displayed or used, *especially* the complex boost parameter that's in the request handler definition (solrconfig.xml). str name=boostmin(recip(abs(ms(NOW/HOUR,registered_date)),1.92901e-10,1.5,1.5),0.85)/str Do I need to send 'boost=' as a parameter (along with my sort) to get it to avoid that calculation? Thanks, Shawn
Re: Does sorting skip everything having to do with relevancy?
On 4/4/2014 1:48 PM, Alvaro Cabrerizo wrote: If you dont want to waste your cpu time, then comment the boost parameter in the query parser defined in your solrconfig.xml. If you cant do that, then you can overwrite it sending the boost parameter for example using the constant function (e.g. http:///...boost=1sort=your_sort). The parameter boost will be overwritten if it is not defined as an invariant. Thank you for responding. I know how I can override the behavior, what I want to find out is whether or not it's necessary to do so -- if it's not necessary because Solr skips it, then everything is good. If it is necessary, I can open an issue in Jira asking for Solr to get smarter. That way everyone benefits and they don't have to do anything except upgrade Solr. Thanks, Shawn
Re: Does sorting skip everything having to do with relevancy?
Hello Shawn, I suppose SolrIndexSearcher.buildTopDocsCollector() doesn't create a Collector which calls score() in this case. Hence, it shouldn't waste CPU. Just my impression. Haven't you tried to check it supplying some weird formula, which throws exception? On Sat, Apr 5, 2014 at 12:02 AM, Shawn Heisey s...@elyograg.org wrote: On 4/4/2014 1:48 PM, Alvaro Cabrerizo wrote: If you dont want to waste your cpu time, then comment the boost parameter in the query parser defined in your solrconfig.xml. If you cant do that, then you can overwrite it sending the boost parameter for example using the constant function (e.g. http:///...boost=1sort=your_sort). The parameter boost will be overwritten if it is not defined as an invariant. Thank you for responding. I know how I can override the behavior, what I want to find out is whether or not it's necessary to do so -- if it's not necessary because Solr skips it, then everything is good. If it is necessary, I can open an issue in Jira asking for Solr to get smarter. That way everyone benefits and they don't have to do anything except upgrade Solr. Thanks, Shawn -- Sincerely yours Mikhail Khludnev Principal Engineer, Grid Dynamics http://www.griddynamics.com mkhlud...@griddynamics.com
Re: Does sorting skip everything having to do with relevancy?
On 4/4/2014 3:13 PM, Mikhail Khludnev wrote: I suppose SolrIndexSearcher.buildTopDocsCollector() doesn't create a Collector which calls score() in this case. Hence, it shouldn't waste CPU. Just my impression. Haven't you tried to check it supplying some weird formula, which throws exception? I didn't think of that. That's a good idea -- as long as there's not independent code that checks the function in addition to the code that actually runs it. With the following parameters added to an edismax query that otherwise works, I get an exception. It works if I change the e to 5. sort=registered_date ascboost=sum(5,e) I will take Alvaro's suggestion and add boost=1 to queries that use a sort parameter. It's probably a good idea to file that Jira. Thanks, Shawn
Does sorting skip everything having to do with relevancy?
If I provide a sort parameter, will Solr (4.6.1) skip score/boost processing? In particular I would like to know what happens if I have a boost parameter (with a complex function) for edismax search, but I include a sort parameter on one of my fields. I am using distributed search. I do know that if debugQuery is used, the score IS calculated, but I'm talking about when debugQuery is not used. Thanks, Shawn
