Re: How to manages diversity in search results in Solr

[Toke Eskildsen]

Daisy  wrote:
> The main issue is: Our page needs 80 products to display from different
> suppliers per keyword search. We have some of the search keywords
> which only have less than 80 suppliers.

Okay. The number of products per supplier gets averaged? So if 1 supplier has 
80 products and 79 suppliers has 1 product, you only show 1 product from 
supplier 1? 


You can group by supplier with group size 80, then prune the result in the 
client end. But if you have 80 or more suppliers, this means resolving up to 
80*80 documents per query, which can be somewhat heavy.

Alternatively you can facet on supplier to get the distribution of products, 
then perform a group query with limits tuned from the facet result.

Either way you need to implement the logic yourself.

- Toke Eskildsen

RE: How to manages diversity in search results in Solr

[Daisy]

The main issue is: Our page needs 80 products to display from different 
suppliers per keyword search. We have some of the search keywords which only 
have less than 80 suppliers. If we use groupby(supplier), the 80 products per 
page is not possible anymore.

Regards,
Daisy

-Original Message-
From: Toke Eskildsen [mailto:t...@statsbiblioteket.dk] 
Sent: Thursday, December 22, 2016 6:24 PM
To: solr_user lucene_apache
Subject: Re: How to manages diversity in search results in Solr

On Thu, 2016-12-22 at 17:35 +0800, Daisy wrote:
> How to restrict the product search in a marketplace where no more than 
> 3 results per retailer are permitted in search results?
> 
> I understand the groupby/collapse could solve the issue but is there 
> any other way to do it?

Grouping is the obvious solution. Since that does not work for you, you need to 
describe what the problem is with that solution, in order for us to suggest 
alternatives.

- Toke Eskildsen, State and University Library, Denmark


--
CONFIDENTIALITY NOTICE 

This e-mail (including any attachments) may contain confidential and/or 
privileged information. If you are not the intended recipient or have received 
this e-mail in error, please inform the sender immediately and delete this 
e-mail (including any attachments) from your computer, and you must not use, 
disclose to anyone else or copy this e-mail (including any attachments), 
whether in whole or in part. 

This e-mail and any reply to it may be monitored for security, legal, 
regulatory compliance and/or other appropriate reasons.

Re: How to manages diversity in search results in Solr

[Toke Eskildsen]

On Thu, 2016-12-22 at 17:35 +0800, Daisy wrote:
> How to restrict the product search in a marketplace where no more
> than 3 results per retailer are permitted in search results?
> 
> I understand the groupby/collapse could solve the issue but is there
> any other way to do it?

Grouping is the obvious solution. Since that does not work for you, you
need to describe what the problem is with that solution, in order for
us to suggest alternatives.

- Toke Eskildsen, State and University Library, Denmark

How to manages diversity in search results in Solr

[Daisy]

How to restrict the product search in a marketplace where no more than 3 
results per retailer are permitted in search results?

I understand the groupby/collapse could solve the issue but is there any other 
way to do it? Thank you.

 

Regards,

Daisy

 


--
CONFIDENTIALITY NOTICE 

This e-mail (including any attachments) may contain confidential and/or 
privileged information. If you are not the intended recipient or have received 
this e-mail in error, please inform the sender immediately and delete this 
e-mail (including any attachments) from your computer, and you must not use, 
disclose to anyone else or copy this e-mail (including any attachments), 
whether in whole or in part. 

This e-mail and any reply to it may be monitored for security, legal, 
regulatory compliance and/or other appropriate reasons.

diversity of search results?

[Paul Libbrecht]

Hello SOLR expert,

yesterday in our group we realized that a danger we may need to face is that a 
search result includes very similar results.
Of course, one would expect skimming so that duplicates that show almost the 
same results in a search result would be avoided but we fear that this is not 
possible.

I was wondering if some technology, plugin, or even research was existing that 
would enable a search result to be partially reordered so that diversity is 
ensured for a first page of results at least.

I suppose that might be doable by processing the result page and the next (and 
the five next?) and pushing down some results if they are too similar to 
previous ones.

Hope I am being clear.

Paul

Re: diversity of search results?

[dirk]

Hi Paul,

yes that`s a typical problem in configuring a search engine. A solution
depends on your data. Sometimes you can overcome this problem by fine tuning
your search engine on boosting level. Thats not easy and always based on
trail and error tests.

Another thing you can do is to try to realize a data pre-processing which
compensate the reasons of similar content in certain fields, e.g. in a title
field. 
For example if you have products with very similar titles and you boost such
a field. The result is, that you always will found all documents in the
result list. But if you go on and add some informations (perhaps out of
other search fields) in this title field you perhaps can reduce the
similarity. (typical example in my branch: Book titles in different volumes,
then I add the volumn  number and der year to the title field.) 

Perhaps it is also necessary to cape with a pre-processed deduplication.
Here you can find an entry point:
http://wiki.apache.org/solr/Deduplication

Dirk

   



-
my developer logs 
--
View this message in context: 
http://lucene.472066.n3.nabble.com/diversity-of-search-results-tp4014692p4014696.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: diversity of search results?

[Otis Gospodnetic]

Hi Paul,

We've done this for a client in the past via a custom SearchComponent
and it worked well.  Yes, it involved some post-processing, but on the
server, not client.  I *think* we saw 10% performance degradation.

Otis
--
Search Analytics - http://sematext.com/search-analytics/index.html
Performance Monitoring - http://sematext.com/spm/index.html


On Fri, Oct 19, 2012 at 3:26 AM, Paul Libbrecht p...@hoplahup.net wrote:
 Hello SOLR expert,

 yesterday in our group we realized that a danger we may need to face is that 
 a search result includes very similar results.
 Of course, one would expect skimming so that duplicates that show almost the 
 same results in a search result would be avoided but we fear that this is not 
 possible.

 I was wondering if some technology, plugin, or even research was existing 
 that would enable a search result to be partially reordered so that 
 diversity is ensured for a first page of results at least.

 I suppose that might be doable by processing the result page and the next 
 (and the five next?) and pushing down some results if they are too similar 
 to previous ones.

 Hope I am being clear.

 Paul