Re: AnalyzingInfixSuggester performance

[Michael McCandless]

It also indexes edge ngrams for short sequences (e.g. a*, b*, etc.) and
switches to ordinary PrefixQuery for longer sequences, and does some work
to at search time to do the "infixing".

But yeah otherwise that's it.

If your ranking at lookup isn't exactly matching the weight, but "roughly"
has some correlation to it, you could still use the fast early termination,
except collect deeper than just the top N to ensure you likely found the
best hits according to your ranking function.

Mike McCandless

http://blog.mikemccandless.com

On Tue, Apr 18, 2017 at 4:35 PM, OTH  wrote:

> I see.  I had actually overlooked the fact that Suggester provides a
> 'weightField', and I could possibly use that in my case instead of the
> regular Solr index with bq.
>
> So if I understand then - the main advantage of using the
> AnalyzingInfixSuggester instead of a regular Solr index (since both are
> using standard Lucene?) is that the AInfixSuggester does sorting at
> index-time using the weightField?  So it's only ever advantageous to use
> this Suggester if you need sorting based on a field?
>
> Thanks
>
> On Tue, Apr 18, 2017 at 2:20 PM, Michael McCandless <
> luc...@mikemccandless.com> wrote:
>
> > AnalyzingInfixSuggester uses index-time sort, to sort all postings by the
> > suggest weight, so that lookup, as long as your sort by the suggest
> weight
> > is extremely fast.
> >
> > But if you need to rank at lookup time by something not "congruent" with
> > the index-time sort then you lose that benefit.
> >
> > Mike McCandless
> >
> > http://blog.mikemccandless.com
> >
> > On Sun, Apr 16, 2017 at 11:46 AM, OTH  wrote:
> >
> > > Hello,
> > >
> > > From what I understand, the AnalyzingInfixSuggester is using a simple
> > > Lucene query; so I was wondering, how then would this suggester have
> > better
> > > performance than using a simple Solr 'select' query on a regular Solr
> > index
> > > (with an asterisk placed at the start and end of the query string).  I
> > > could understand why say an FST based suggester would be faster, but I
> > > wanted to confirm if that indeed is the case with
> > AnalyzingInfixSuggester.
> > >
> > > One reason I ask is:
> > > I needed the results to be boosted based on the value of another field;
> > > e.g., if a user in the UK is searching for cities, then I'd need the
> > cities
> > > which are in the UK to be boosted.  I was able to do this with a
> regular
> > > Solr index by adding something like these parameters:
> > > defType=edismax=country:UK^2.0
> > >
> > > However, I'm not sure if this is possible with the Suggester.
> Moreover -
> > > other than the 'country' field above, there are other fields as well
> > which
> > > I need to be returned with the results.  Since the Suggester seems to
> > only
> > > allow one additional field, called 'payload', I'm able to do this by
> > > putting the values of all the other fields into a JSON and then placing
> > > that into the 'payload' field - however, I don't know if it would be
> > > possible then to incorporate the boosting mechanism I showed above.
> > >
> > > So I was thinking of just using a regular Solr index instead of the
> > > Suggester; I wanted to confirm, what if any is the performance
> > improvement
> > > in using the AnalyzingInfixSuggester over using a regular index?
> > >
> > > Much thanks
> > >
> >
>

Re: AnalyzingInfixSuggester performance

[OTH]

I see.  I had actually overlooked the fact that Suggester provides a
'weightField', and I could possibly use that in my case instead of the
regular Solr index with bq.

So if I understand then - the main advantage of using the
AnalyzingInfixSuggester instead of a regular Solr index (since both are
using standard Lucene?) is that the AInfixSuggester does sorting at
index-time using the weightField?  So it's only ever advantageous to use
this Suggester if you need sorting based on a field?

Thanks

On Tue, Apr 18, 2017 at 2:20 PM, Michael McCandless <
luc...@mikemccandless.com> wrote:

> AnalyzingInfixSuggester uses index-time sort, to sort all postings by the
> suggest weight, so that lookup, as long as your sort by the suggest weight
> is extremely fast.
>
> But if you need to rank at lookup time by something not "congruent" with
> the index-time sort then you lose that benefit.
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
> On Sun, Apr 16, 2017 at 11:46 AM, OTH  wrote:
>
> > Hello,
> >
> > From what I understand, the AnalyzingInfixSuggester is using a simple
> > Lucene query; so I was wondering, how then would this suggester have
> better
> > performance than using a simple Solr 'select' query on a regular Solr
> index
> > (with an asterisk placed at the start and end of the query string).  I
> > could understand why say an FST based suggester would be faster, but I
> > wanted to confirm if that indeed is the case with
> AnalyzingInfixSuggester.
> >
> > One reason I ask is:
> > I needed the results to be boosted based on the value of another field;
> > e.g., if a user in the UK is searching for cities, then I'd need the
> cities
> > which are in the UK to be boosted.  I was able to do this with a regular
> > Solr index by adding something like these parameters:
> > defType=edismax=country:UK^2.0
> >
> > However, I'm not sure if this is possible with the Suggester.  Moreover -
> > other than the 'country' field above, there are other fields as well
> which
> > I need to be returned with the results.  Since the Suggester seems to
> only
> > allow one additional field, called 'payload', I'm able to do this by
> > putting the values of all the other fields into a JSON and then placing
> > that into the 'payload' field - however, I don't know if it would be
> > possible then to incorporate the boosting mechanism I showed above.
> >
> > So I was thinking of just using a regular Solr index instead of the
> > Suggester; I wanted to confirm, what if any is the performance
> improvement
> > in using the AnalyzingInfixSuggester over using a regular index?
> >
> > Much thanks
> >
>

Re: AnalyzingInfixSuggester performance

[Michael McCandless]

AnalyzingInfixSuggester uses index-time sort, to sort all postings by the
suggest weight, so that lookup, as long as your sort by the suggest weight
is extremely fast.

But if you need to rank at lookup time by something not "congruent" with
the index-time sort then you lose that benefit.

Mike McCandless

http://blog.mikemccandless.com

On Sun, Apr 16, 2017 at 11:46 AM, OTH  wrote:

> Hello,
>
> From what I understand, the AnalyzingInfixSuggester is using a simple
> Lucene query; so I was wondering, how then would this suggester have better
> performance than using a simple Solr 'select' query on a regular Solr index
> (with an asterisk placed at the start and end of the query string).  I
> could understand why say an FST based suggester would be faster, but I
> wanted to confirm if that indeed is the case with AnalyzingInfixSuggester.
>
> One reason I ask is:
> I needed the results to be boosted based on the value of another field;
> e.g., if a user in the UK is searching for cities, then I'd need the cities
> which are in the UK to be boosted.  I was able to do this with a regular
> Solr index by adding something like these parameters:
> defType=edismax=country:UK^2.0
>
> However, I'm not sure if this is possible with the Suggester.  Moreover -
> other than the 'country' field above, there are other fields as well which
> I need to be returned with the results.  Since the Suggester seems to only
> allow one additional field, called 'payload', I'm able to do this by
> putting the values of all the other fields into a JSON and then placing
> that into the 'payload' field - however, I don't know if it would be
> possible then to incorporate the boosting mechanism I showed above.
>
> So I was thinking of just using a regular Solr index instead of the
> Suggester; I wanted to confirm, what if any is the performance improvement
> in using the AnalyzingInfixSuggester over using a regular index?
>
> Much thanks
>