Re: Related Search

[Trey Grainger]

Yeah, the approach listed by Grant and Markus is a common approach. I've
worked on systems that mined query logs like this, and it's a good approach
if you have sufficient query logs to pull it off.

There are a lot of linguistic nuances you'll encounter along the way,
including how you disambiguate homonyms and their related terms, identify
synonyms/acronyms as having the same underlying meaning, how you parse and
handle unknown phrases, removing noise present in the query logs, and even
how you weight the strength or relationship between related queries. I gave
a presentation on this topic at Lucene/Solr Revolution in 2015 if you're
interested in learning more about how to build such a system (
http://www.treygrainger.com/posts/presentations/leveraging-lucene-solr-as-a-knowledge-graph-and-intent-engine/
).

Another approach (also referenced in the above presentation), for those
with more of a cold-start problem with query logs, is to mine related terms
and phrases out of the underlying content in the search engine (inverted
index) itself. The Semantic Knowledge Graph that was recently open sourced
by CareerBuilder and contributed back to Solr (disclaimer: I worked on it,
and it's available both a Solr plugin and patch, but it's not ready to be
committed into Solr yet.) enables such a capability. See
https://issues.apache.org/jira/browse/SOLR-9480 for the most current patch.

It is a request handler that can take in any query and discover the most
related other terms to that entire query from the inverted index, sorted by
strength of relationship to that query (it can also traverse from those
terms across fields/relationships to other terms, but that's probably
overkill for the basic related searches use case). Think of it as a way to
run a query and find the most relevant other keywords, as opposed to
finding the most relevant documents.

Using this, you can then either return the related keywords as your related
searches, or you can modify your query to include them and power a
conceptual/semantic search instead of the pure text-based search you
started with. It's effectively a (better) way to implement More Like This,
where instead of taking a document and using tf-idf to extract out the
globally-interesting terms from the document (like MLT), you can instead
use a query to find contextually-relevant keywords across many documents,
score them based upon their similarity to the original query, and then turn
around and use the top most semantically-relevant terms as your related
search(es).

I don't have near-term plans to expose the semantic knowledge graph as a
search component (it's a request handler right now), but once it's finished
that could certainly be done. Just wanted to mention it as another approach
to solve this specific problem.

-Trey Grainger
SVP of Engineering @ Lucidworks
Co-author, Solr in Action



On Wed, Oct 26, 2016 at 1:59 PM, Markus Jelsma <markus.jel...@openindex.io>
wrote:

> Indeed, we have similar processes running of which one generates a
> 'related query collection' which just contains a (normalized) query and its
> related queries. I would not know how this is even possible without
> continuously processing query and click logs.
>
> M.
>
>
> -Original message-
> > From:Grant Ingersoll <gsing...@apache.org>
> > Sent: Tuesday 25th October 2016 23:51
> > To: solr-user@lucene.apache.org
> > Subject: Re: Related Search
> >
> > Hi Rick,
> >
> > I typically do this stuff just by searching a different collection that I
> > create offline by analyzing query logs and then indexing them and
> searching.
> >
> > On Mon, Oct 24, 2016 at 8:32 PM Rick Leir <rl...@leirtech.com> wrote:
> >
> > > Hi all,
> > >
> > > There is an issue 'Create a Related Search Component' which has been
> > > open for some years now.
> > >
> > > It has a priority: major.
> > >
> > > https://issues.apache.org/jira/browse/SOLR-2080
> > >
> > >
> > > I discovered it linked from Lucidwork's very useful blog on ecommerce:
> > >
> > >
> > > https://lucidworks.com/blog/2011/01/25/implementing-the-
> ecommerce-checklist-with-apache-solr-and-lucidworks/
> > >
> > >
> > > Did people find a better way to accomplish Related Search? Perhaps MLT
> > > http://wiki.apache.org/solr/MoreLikeThis ?
> > >
> > > cheers -- Rick
> > >
> > >
> > >
> >
>

RE: Related Search

[Markus Jelsma]

Indeed, we have similar processes running of which one generates a 'related 
query collection' which just contains a (normalized) query and its related 
queries. I would not know how this is even possible without continuously 
processing query and click logs.

M.
 
 
-Original message-
> From:Grant Ingersoll <gsing...@apache.org>
> Sent: Tuesday 25th October 2016 23:51
> To: solr-user@lucene.apache.org
> Subject: Re: Related Search
> 
> Hi Rick,
> 
> I typically do this stuff just by searching a different collection that I
> create offline by analyzing query logs and then indexing them and searching.
> 
> On Mon, Oct 24, 2016 at 8:32 PM Rick Leir <rl...@leirtech.com> wrote:
> 
> > Hi all,
> >
> > There is an issue 'Create a Related Search Component' which has been
> > open for some years now.
> >
> > It has a priority: major.
> >
> > https://issues.apache.org/jira/browse/SOLR-2080
> >
> >
> > I discovered it linked from Lucidwork's very useful blog on ecommerce:
> >
> >
> > https://lucidworks.com/blog/2011/01/25/implementing-the-ecommerce-checklist-with-apache-solr-and-lucidworks/
> >
> >
> > Did people find a better way to accomplish Related Search? Perhaps MLT
> > http://wiki.apache.org/solr/MoreLikeThis ?
> >
> > cheers -- Rick
> >
> >
> >
> 

Re: Related Search

[Grant Ingersoll]

Hi Rick,

I typically do this stuff just by searching a different collection that I
create offline by analyzing query logs and then indexing them and searching.

On Mon, Oct 24, 2016 at 8:32 PM Rick Leir  wrote:

> Hi all,
>
> There is an issue 'Create a Related Search Component' which has been
> open for some years now.
>
> It has a priority: major.
>
> https://issues.apache.org/jira/browse/SOLR-2080
>
>
> I discovered it linked from Lucidwork's very useful blog on ecommerce:
>
>
> https://lucidworks.com/blog/2011/01/25/implementing-the-ecommerce-checklist-with-apache-solr-and-lucidworks/
>
>
> Did people find a better way to accomplish Related Search? Perhaps MLT
> http://wiki.apache.org/solr/MoreLikeThis ?
>
> cheers -- Rick
>
>
>

Re: Related Search

[Erick Erickson]

Rick:

The priority isn't particularly helpful for two reasons:

1> it's the default so often gets set without intent.
2> what the originator thinks of as major may or may not translate
into someone actually doing work on it.

In this case there's a lot of work that'd need to be done. "some
model" just begs for clarification. In this case it might be a major
feature, but nobody's felt the need or had the time to put into making
it a reality. This is really just a topic for conversation at this
point

Best,
Erick

On Mon, Oct 24, 2016 at 5:32 PM, Rick Leir  wrote:
> Hi all,
>
> There is an issue 'Create a Related Search Component' which has been open
> for some years now.
>
> It has a priority: major.
>
> https://issues.apache.org/jira/browse/SOLR-2080
>
>
> I discovered it linked from Lucidwork's very useful blog on ecommerce:
>
> https://lucidworks.com/blog/2011/01/25/implementing-the-ecommerce-checklist-with-apache-solr-and-lucidworks/
>
>
> Did people find a better way to accomplish Related Search? Perhaps MLT
> http://wiki.apache.org/solr/MoreLikeThis ?
>
> cheers -- Rick
>
>

Re: related search

[Suram]

Thanx lot Ahmet Arslan

How can make query to get synonym value ,any suggestion

-- 
View this message in context: 
http://old.nabble.com/related-search-tp27933778p27974649.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: related search

[Andrew Greenburg]

On Sun, Mar 21, 2010 at 4:30 AM, Suram reactive...@yahoo.com wrote:

 Thanx lot Ahmet Arslan

 How can make query to get synonym value ,any suggestion

You need to apply the SynonymFilterFactory to your queries as well.

Re: related search

[Suram]

Suram wrote:
 
 How can i make related search in solr.if i search ipod i need to get
 answer like ipodsuffle,ipodnano,ipone with out using morelikethis option
 


any solution please kindly tell me

-- 
View this message in context: 
http://old.nabble.com/related-search-tp27933778p27951060.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: related search

[Ahmet Arslan]

  How can i make related search in solr.if i search ipod
 i need to get
  answer like ipodsuffle,ipodnano,ipone with out using
 morelikethis option

First of all you need to construct these list. Either automatic or manually. 
You can use SynonymFilterFactory [1] at index time with this settings.

filter class=solr.SynonymFilterFactory synonyms=index_synonyms.txt 
ignoreCase=true expand=true/

SolrHome/conf/index_synonyms.txt file will contain comma separated entries per 
line:
ipodsuffle, ipodnano, ipone

[1]http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.SynonymFilterFactory


  

Re: Related Search

[Cool Coder]

Take a look at this thread 
http://www.gossamer-threads.com/lists/lucene/java-user/54996
   
  There was a need to get all related topics for any selected topic. I have 
taken help of lucene-sand-box wordnet project to get all synoms of user 
selected topics. I am not sure whether wordnet project would help you as you 
look for products synonyms. In your case, you might need to maintain a vector 
of product synonyms E.g. If User searches TV, internally you would search for 
lcd tv, lcd, hdtv etc...
   
   
  Take a look at www.ajaxtrend.com, how all related topics are displayed and I 
am also keep on refining the related query search as this site is evolving. 
This is just prototype. 
   
  - BR
William Silva [EMAIL PROTECTED] wrote:
  Hi,
What is the best way to implement a related search like CNET with SOLR ?
Ex.: Searching for tv the related searches are: lcd tv, lcd, hdtv,
vizio, plasma tv, panasonic, gps, plasma
Thanks,
William.


   
-
Be a better sports nut! Let your teams follow you with Yahoo Mobile. Try it now.