Re: Solr Sorting, merging/weighting sort fields
I have trouble balancing between popularity and search relevance. The trouble is combining boost factors and a mm(minimum match) of less than 100%. The mm leads the search to return also less relevant items. Two conflicting main scenarios exist: - generic category search (say something like tft) mainly i just want to list the most popular ones - product specific search eos 400d for instance If i set a low boost on popularity the category search doesn't attach enough significance to popularity If i set a high boost on popularity the search for eos 400d ends you with the more popular eos 450d Anything higher than ^5 messes up the second scenario and anything below ^500 messes up the first scenario. There are two problems in solving this: - seemingly equally matching items return strongly different relevancy scores, based on string length - the effect of popularity and relevancy seems to be addition instead of a multiplying effect. Any tips on how to better understand these effects? Two examples: name^1.0 category^1.1 description^0.2 color^0.2 popularity^5.0 1 and popularity^1500 mm set to default Walter Underwood wrote: > > The boost is a way to adjust the weight of that field, just like you > adjust the weight of any other field. If the boost is dominating the > score, reduce the weight and vice versa. > > wunder > > On 5/10/07 9:22 PM, "Chris Hostetter" <[EMAIL PROTECTED]> wrote: > >> >> : Is this correct? bf is a boosting function, so a function is needed >> there, >> no? >> >> : If I'm not missing someting, the ^0.5 is just a boost, and "popularity" >> : is just a (numeric) field. So boosting a numeric field wouldn't make >> : sense, but appying it to a function would. Am I missing something? >> >> the function parser "does the right thing" when you give it a bare field >> name, from the javadocs... >> >> http://lucene.apache.org/solr/api/org/apache/solr/search/QueryParsing.html#par >> seFunction(java.lang.String,%20org.apache.solr.schema.IndexSchema) >> // Numeric fields default to correct type >> // (ie: IntFieldSource or FloatFieldSource) >> // Others use implicit ord(...) to generate numeric field value >> myfield >> >> you are correct about 0.5 being the boost, using either the _val_ hack on >> the SolrQueryParser or using he bf param of dismax the ^0.5 will be used >> as a boost on the resulting function query... >> >>qt=standard&q=%2Bfoo%20_val_:popularity^0.5 >>qt=dismax&q=foo&bf=popularity^0.5 >> >> >> -Hoss >> > > > -- View this message in context: http://www.nabble.com/Solr-Sorting%2C-merging-weighting-sort-fields-tp10405022p20525893.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Sorting, merging/weighting sort fields
The boost is a way to adjust the weight of that field, just like you adjust the weight of any other field. If the boost is dominating the score, reduce the weight and vice versa. wunder On 5/10/07 9:22 PM, "Chris Hostetter" <[EMAIL PROTECTED]> wrote: > > : Is this correct? bf is a boosting function, so a function is needed there, > no? > > : If I'm not missing someting, the ^0.5 is just a boost, and "popularity" > : is just a (numeric) field. So boosting a numeric field wouldn't make > : sense, but appying it to a function would. Am I missing something? > > the function parser "does the right thing" when you give it a bare field > name, from the javadocs... > > http://lucene.apache.org/solr/api/org/apache/solr/search/QueryParsing.html#par > seFunction(java.lang.String,%20org.apache.solr.schema.IndexSchema) > // Numeric fields default to correct type > // (ie: IntFieldSource or FloatFieldSource) > // Others use implicit ord(...) to generate numeric field value > myfield > > you are correct about 0.5 being the boost, using either the _val_ hack on > the SolrQueryParser or using he bf param of dismax the ^0.5 will be used > as a boost on the resulting function query... > >qt=standard&q=%2Bfoo%20_val_:popularity^0.5 >qt=dismax&q=foo&bf=popularity^0.5 > > > -Hoss >
Re: Solr Sorting, merging/weighting sort fields
: I actually used the _val_ hack, so query foo:bar : _val_:"ord(popularity)^0.5", still playing with it though its still : skewing results a bit to much. if your query string is... foo:bar _val_:ord(popularity)^0.5 ..then part of your problem may be that (unless you've changed the default operator) that will match all docs with popularity even if they don't match foo:bar besides that, debugQuery=1 and the explain output is your friend. -Hoss
Re: Solr Sorting, merging/weighting sort fields
: Is this correct? bf is a boosting function, so a function is needed there, no? : If I'm not missing someting, the ^0.5 is just a boost, and "popularity" : is just a (numeric) field. So boosting a numeric field wouldn't make : sense, but appying it to a function would. Am I missing something? the function parser "does the right thing" when you give it a bare field name, from the javadocs... http://lucene.apache.org/solr/api/org/apache/solr/search/QueryParsing.html#parseFunction(java.lang.String,%20org.apache.solr.schema.IndexSchema) // Numeric fields default to correct type // (ie: IntFieldSource or FloatFieldSource) // Others use implicit ord(...) to generate numeric field value myfield you are correct about 0.5 being the boost, using either the _val_ hack on the SolrQueryParser or using he bf param of dismax the ^0.5 will be used as a boost on the resulting function query... qt=standard&q=%2Bfoo%20_val_:popularity^0.5 qt=dismax&q=foo&bf=popularity^0.5 -Hoss
Re: Solr Sorting, merging/weighting sort fields
Hi Otis I used a boosting function, bf pointed me in the right direction :) I actually used the _val_ hack, so query foo:bar _val_:"ord(popularity)^0.5", still playing with it though its still skewing results a bit to much. -Nick On 5/11/07, Otis Gospodnetic <[EMAIL PROTECTED]> wrote: Is this correct? bf is a boosting function, so a function is needed there, no? e.g. ord(popularity)^0.5 recip(rord(price),1,1000,1000)^0.3 If I'm not missing someting, the ^0.5 is just a boost, and "popularity" is just a (numeric) field. So boosting a numeric field wouldn't make sense, but appying it to a function would. Am I missing something? Thanks, Otis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Simpy -- http://www.simpy.com/ - Tag - Search - Share - Original Message From: Walter Underwood <[EMAIL PROTECTED]> To: solr-user@lucene.apache.org Sent: Wednesday, May 9, 2007 8:05:53 PM Subject: Re: Solr Sorting, merging/weighting sort fields No problem. Use a boost function. In a DisMaxRequestHandler spec in solrconfig.xml, specify this: popularity^0.5 This value will be added to the score before ranking. You will probably need to fuss with the multiplier to get the popularity to the right proportion of the total score. I find it handy to return the score and the popularity value and look over a few test queries to adjust that. wunder On 5/9/07 4:58 PM, "Nick Jenkin" <[EMAIL PROTECTED]> wrote: > Hi all, > > I have a popularity field in my solr index, this field is a popularity > rating of a particular product (based on the number of product views > etc). > > I want to be able to integrate this number into the search result > sorting such that a product with a higher popularity rating is ranking > higher in the search results. > > I can always do: > title:(harry potter); popularity desc, score desc; > > In this example I will say be searching for harry potter books, > obviously the latest book has a very high popularity rating, but lets > say a completely unrelated product with a high popularity also matches > for harry potter and gets into the search results (but has a very low > score value), this will bring it to the top of the results. > > I want to be able to weight popularity in such a way that it boosts > the score, but will not greatly affect the search results. > > Is this possible? > > Thanks
Re: Solr Sorting, merging/weighting sort fields
Is this correct? bf is a boosting function, so a function is needed there, no? e.g. ord(popularity)^0.5 recip(rord(price),1,1000,1000)^0.3 If I'm not missing someting, the ^0.5 is just a boost, and "popularity" is just a (numeric) field. So boosting a numeric field wouldn't make sense, but appying it to a function would. Am I missing something? Thanks, Otis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Simpy -- http://www.simpy.com/ - Tag - Search - Share - Original Message From: Walter Underwood <[EMAIL PROTECTED]> To: solr-user@lucene.apache.org Sent: Wednesday, May 9, 2007 8:05:53 PM Subject: Re: Solr Sorting, merging/weighting sort fields No problem. Use a boost function. In a DisMaxRequestHandler spec in solrconfig.xml, specify this: popularity^0.5 This value will be added to the score before ranking. You will probably need to fuss with the multiplier to get the popularity to the right proportion of the total score. I find it handy to return the score and the popularity value and look over a few test queries to adjust that. wunder On 5/9/07 4:58 PM, "Nick Jenkin" <[EMAIL PROTECTED]> wrote: > Hi all, > > I have a popularity field in my solr index, this field is a popularity > rating of a particular product (based on the number of product views > etc). > > I want to be able to integrate this number into the search result > sorting such that a product with a higher popularity rating is ranking > higher in the search results. > > I can always do: > title:(harry potter); popularity desc, score desc; > > In this example I will say be searching for harry potter books, > obviously the latest book has a very high popularity rating, but lets > say a completely unrelated product with a high popularity also matches > for harry potter and gets into the search results (but has a very low > score value), this will bring it to the top of the results. > > I want to be able to weight popularity in such a way that it boosts > the score, but will not greatly affect the search results. > > Is this possible? > > Thanks
Re: Solr Sorting, merging/weighting sort fields
Thanks, worked perfectly! -Nick On 5/10/07, Walter Underwood <[EMAIL PROTECTED]> wrote: No problem. Use a boost function. In a DisMaxRequestHandler spec in solrconfig.xml, specify this: popularity^0.5 This value will be added to the score before ranking. You will probably need to fuss with the multiplier to get the popularity to the right proportion of the total score. I find it handy to return the score and the popularity value and look over a few test queries to adjust that. wunder On 5/9/07 4:58 PM, "Nick Jenkin" <[EMAIL PROTECTED]> wrote: > Hi all, > > I have a popularity field in my solr index, this field is a popularity > rating of a particular product (based on the number of product views > etc). > > I want to be able to integrate this number into the search result > sorting such that a product with a higher popularity rating is ranking > higher in the search results. > > I can always do: > title:(harry potter); popularity desc, score desc; > > In this example I will say be searching for harry potter books, > obviously the latest book has a very high popularity rating, but lets > say a completely unrelated product with a high popularity also matches > for harry potter and gets into the search results (but has a very low > score value), this will bring it to the top of the results. > > I want to be able to weight popularity in such a way that it boosts > the score, but will not greatly affect the search results. > > Is this possible? > > Thanks -- - Nick
Re: Solr Sorting, merging/weighting sort fields
No problem. Use a boost function. In a DisMaxRequestHandler spec in solrconfig.xml, specify this: popularity^0.5 This value will be added to the score before ranking. You will probably need to fuss with the multiplier to get the popularity to the right proportion of the total score. I find it handy to return the score and the popularity value and look over a few test queries to adjust that. wunder On 5/9/07 4:58 PM, "Nick Jenkin" <[EMAIL PROTECTED]> wrote: > Hi all, > > I have a popularity field in my solr index, this field is a popularity > rating of a particular product (based on the number of product views > etc). > > I want to be able to integrate this number into the search result > sorting such that a product with a higher popularity rating is ranking > higher in the search results. > > I can always do: > title:(harry potter); popularity desc, score desc; > > In this example I will say be searching for harry potter books, > obviously the latest book has a very high popularity rating, but lets > say a completely unrelated product with a high popularity also matches > for harry potter and gets into the search results (but has a very low > score value), this will bring it to the top of the results. > > I want to be able to weight popularity in such a way that it boosts > the score, but will not greatly affect the search results. > > Is this possible? > > Thanks
Solr Sorting, merging/weighting sort fields
Hi all, I have a popularity field in my solr index, this field is a popularity rating of a particular product (based on the number of product views etc). I want to be able to integrate this number into the search result sorting such that a product with a higher popularity rating is ranking higher in the search results. I can always do: title:(harry potter); popularity desc, score desc; In this example I will say be searching for harry potter books, obviously the latest book has a very high popularity rating, but lets say a completely unrelated product with a high popularity also matches for harry potter and gets into the search results (but has a very low score value), this will bring it to the top of the results. I want to be able to weight popularity in such a way that it boosts the score, but will not greatly affect the search results. Is this possible? Thanks -- - Nick
