We originally thought rescoring would work as well! I actually implemented it, but low relevancy documents continued to show up at the top of our results. I didn't understand this at first, but on re-reading the documentation I saw what the problem was: the rescore is executed on the shard *before* the results are returned to the node handling the overall request. So: if an individual shard only had low relevancy, high trending documents for a query, then we'd run into the exact same problem as before. And, in fact, that does seem to happen with enough frequency to be an issue for us.
At the moment, the only way I see to solve the problem is to do a post-processing step on the returned documents. On Monday, December 22, 2014 11:50:24 PM UTC-5, vineeth mohan wrote: > > Hi , > > Maybe you are looking for this - > http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-rescore.html > > Thanks > Vineeth > > On Tue, Dec 23, 2014 at 2:46 AM, hespoddi <ch...@publishthis.com > <javascript:>> wrote: > >> Hi all, >> >> The problem isn't really the query. The problem is we'd like the limit >> the results of the query to just "high" scores before we apply the >> function_score. There is a min_score parameter we could use: >> >> >> http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-min-score.html >> >> But what the min_score should be will, obviously, vary significantly >> depending on the query. Ideally, we'd set the min_score to some percentile >> of the max score for query, but I don't think that's possible: >> >> https://github.com/elasticsearch/elasticsearch/issues/719 >> >> I was curious if anyone had any other ideas about how to do this (or >> something close)? >> >> -Chris >> >> On Monday, December 22, 2014 1:34:08 PM UTC-5, vineeth mohan wrote: >>> >>> Hi , >>> >>> I dont see why it should run into performance issue. >>> In anyway you do it , the _score and score due to a field have to be >>> computed/loaded. >>> If you precompile your script by placing it in config directory , that >>> should be good enough. >>> Also feel free to write the same in Java code and attach that as script. >>> >>> Thanks >>> Vineeth >>> >>> On Mon, Dec 22, 2014 at 10:30 PM, Scott Decker <sc...@publishthis.com> >>> wrote: >>> >>>> If you did the script score path, don't you run into performance >>>> issues? I would think, running say thousands of queries like that would >>>> probably not be performant. >>>> unless I am missing something. >>>> >>>> On Sunday, December 21, 2014 8:01:06 PM UTC-8, vineeth mohan wrote: >>>>> >>>>> Hello , >>>>> >>>>> My advice would be to use the script function type. Inside it you can >>>>> access _score which is the score given by the query and the value of the >>>>> field. Mix them together in whatever logic you want. >>>>> >>>>> Thanks >>>>> Vineeth >>>>> >>>>> On Thu, Dec 11, 2014 at 7:28 PM, hespoddi <ch...@publishthis.com> >>>>> wrote: >>>>> >>>>>> Hi all, >>>>>> >>>>>> We'd like to combine the query score with our own custom trending >>>>>> score for a given document. Currently, our query looks like: >>>>>> >>>>>> { >>>>>> "query": { >>>>>> "filtered": { >>>>>> "filter": { >>>>>> "and": [ >>>>>> { >>>>>> "range": { >>>>>> "trendingScore": { >>>>>> "gt": 0.0 >>>>>> } >>>>>> } >>>>>> } >>>>>> ] >>>>>> }, >>>>>> "query": { >>>>>> "function_score": { >>>>>> "functions": [ >>>>>> { >>>>>> "field_value_factor": { >>>>>> "field": "trendingScore" >>>>>> } >>>>>> } >>>>>> ], >>>>>> "query": { >>>>>> "bool": { >>>>>> "minimum_should_match": 1, >>>>>> "should": [ >>>>>> { >>>>>> "match_phrase": { >>>>>> "title": { >>>>>> "boost": 1.5, >>>>>> "query": "<phrase>" >>>>>> } >>>>>> } >>>>>> }, >>>>>> { >>>>>> "match_phrase": { >>>>>> "content": { >>>>>> "boost": 1.0, >>>>>> "query": "<phrase>" >>>>>> } >>>>>> } >>>>>> } >>>>>> ] >>>>>> } >>>>>> } >>>>>> } >>>>>> } >>>>>> } >>>>>> } >>>>>> } >>>>>> >>>>>> So, this query seems to do want we want: it multiples the query score >>>>>> for document with our custom trending score (stored on the >>>>>> "trendingScore" >>>>>> field). The problem is that the trending score, in many cases, >>>>>> overwhelms >>>>>> the query score. Thus, documents with very low relevancy, but very high >>>>>> trending, are at the top of our results. Ideally, we'd filter the query >>>>>> to >>>>>> only return the top N percentage of documents that matched, but I don't >>>>>> think that's possible. We've looked at the min_score parameter for >>>>>> queries >>>>>> as well, but I don't know what a "good" value would be for this. >>>>>> >>>>>> Does anyone have any ideas on the best way to solve this problem? >>>>>> Thank you ahead of time! >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "elasticsearch" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to elasticsearc...@googlegroups.com. >>>>>> To view this discussion on the web visit https://groups.google.com/d/ >>>>>> msgid/elasticsearch/7bc5f7ed-4021-4044-bde0-a9e0f538e213%40goo >>>>>> glegroups.com >>>>>> <https://groups.google.com/d/msgid/elasticsearch/7bc5f7ed-4021-4044-bde0-a9e0f538e213%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>> >>>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "elasticsearch" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to elasticsearc...@googlegroups.com. >>>> To view this discussion on the web visit https://groups.google.com/d/ >>>> msgid/elasticsearch/72359b37-a130-4984-a329-f1dff43848d4% >>>> 40googlegroups.com >>>> <https://groups.google.com/d/msgid/elasticsearch/72359b37-a130-4984-a329-f1dff43848d4%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "elasticsearch" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to elasticsearc...@googlegroups.com <javascript:>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/elasticsearch/304f009b-ddc3-4826-8686-cc653c81d946%40googlegroups.com >> >> <https://groups.google.com/d/msgid/elasticsearch/304f009b-ddc3-4826-8686-cc653c81d946%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> >> For more options, visit https://groups.google.com/d/optout. >> > > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/16ec7395-4d06-4cdf-a1aa-02c6559f6051%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
