[
https://issues.apache.org/jira/browse/LUCENE-5214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13783998#comment-13783998
]
Michael McCandless commented on LUCENE-5214:
--------------------------------------------
bq. I was curious to know why you did not implement the load and store methods
for your AnalyzingInfixSuggester rather build the index at the ctor?
Well ... once you .build() the AnalyzingInfixSuggester, it's already "stored"
since it's backed by an on-disk index. So this suggester is somewhat different
from others (it's not RAM resident ... hmm unless you provide a RAMDir in
getDirectory).
In the ctor, if there's already a previously built suggester, I just open the
searcher there. I suppose we could move that code into load() instead?
bq. was it because of the fact that they take a Input/output stream?
That is sort of weird; I think we have an issue open to change that to
Directory or maybe IndexInput/Output or something ...
bq. What are your thoughts on generalizing the interface so that the index can
be loaded up and stored as it is done by all the other suggesters?
+1 to somehow improve the suggester APIs (I think there's yet another issue
opened for that).
Do you mean loaded into a RAMDir?
bq. this may not be the most relevant place to ask but will do anyways.
That's fine :) Just send an email to dev@ next time ...
bq. your AnalyzingInfixSuggester
It's not "mine" :) Anyone can and should go fix it!
> Add new FreeTextSuggester, to handle "long tail" suggestions
> ------------------------------------------------------------
>
> Key: LUCENE-5214
> URL: https://issues.apache.org/jira/browse/LUCENE-5214
> Project: Lucene - Core
> Issue Type: Improvement
> Components: modules/spellchecker
> Reporter: Michael McCandless
> Assignee: Michael McCandless
> Fix For: 5.0, 4.6
>
> Attachments: LUCENE-5214.patch, LUCENE-5214.patch
>
>
> The current suggesters are all based on a finite space of possible
> suggestions, i.e. the ones they were built on, so they can only
> suggest a full suggestion from that space.
> This means if the current query goes outside of that space then no
> suggestions will be found.
> The goal of FreeTextSuggester is to address this, by giving
> predictions based on an ngram language model, i.e. using the last few
> tokens from the user's query to predict likely following token.
> I got the idea from this blog post about Google's suggest:
> http://googleblog.blogspot.com/2011/04/more-predictions-in-autocomplete.html
> This is very much still a work in progress, but it seems to be
> working. I've tested it on the AOL query logs, using an interactive
> tool from luceneutil to show the suggestions, and it seems to work well.
> It's fun to use that tool to explore the word associations...
> I don't think this suggester would be used standalone; rather, I think
> it'd be a fallback for times when the primary suggester fails to find
> anything. You can see this behavior on google.com, if you type "the
> fast and the ", you see entire queries being suggested, but then if
> the next word you type is "burning" then suddenly you see the
> suggestions are only based on the last word, not the entire query.
> It uses ShingleFilter under-the-hood to generate the token ngrams;
> once LUCENE-5180 is in it will be able to properly handle a user query
> that ends with stop-words (e.g. "wizard of "), and then stores the
> ngrams in an FST.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
