Re: Solr with Auto-suggest
Hi Ryan, I gone through your post https://issues.apache.org/jira/browse/SOLR-357 where you mention about prefix filter,can you tell me how to use that patch,and you mentioned to use the code as bellow, fieldType name=prefix_full class=solr.TextField positionIncrementGap=1 analyzer type=index tokenizer class=solr.KeywordTokenizerFactory/ filter class=solr.LowerCaseFilterFactory / filter class=solr.EdgeNGramFilterFactory minGramSize=1 maxGramSize=20/ /analyzer analyzer type=query tokenizer class=solr.KeywordTokenizerFactory/ filter class=solr.LowerCaseFilterFactory / /analyzer /fieldType fieldType name=prefix_token class=solr.TextField positionIncrementGap=1 analyzer type=index tokenizer class=solr.WhitespaceTokenizerFactory/ filter class=solr.LowerCaseFilterFactory / filter class=solr.EdgeNGramFilterFactory minGramSize=1 maxGramSize=20/ /analyzer analyzer type=query tokenizer class=solr.WhitespaceTokenizerFactory/ filter class=solr.LowerCaseFilterFactory / /analyzer /fieldType ... field name=prefix1 type=prefix_full indexed=true stored=false/ field name=prefix2 type=prefix_token indexed=true stored=false/ ... copyField source=name dest=prefix1/ copyField source=name dest=prefix2/ For using the above code is that you are using EdgeNGramFilterFactory or PrefixingFilterFactory. or the above code works for EdgeNGramFilterFactory,i am not clear about it,with out using the PrefixingFilterFactory patch, is that i can write the above code. And the next is name in copyFiled is text type or string type waiting for your reply, Regards, Rekha -- View this message in context: http://www.nabble.com/Solr-with-Auto-suggest-tp16880894p25530993.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr with Auto-suggest
On Wed, Sep 23, 2009 at 11:30 AM, dharhsana rekha.dharsh...@gmail.comwrote: Hi Ryan, I gone through your post https://issues.apache.org/jira/browse/SOLR-357 where you mention about prefix filter,can you tell me how to use that patch,and you mentioned to use the code as bellow, fieldType name=prefix_full class=solr.TextField positionIncrementGap=1 analyzer type=index tokenizer class=solr.KeywordTokenizerFactory/ filter class=solr.LowerCaseFilterFactory / filter class=solr.EdgeNGramFilterFactory minGramSize=1 maxGramSize=20/ /analyzer analyzer type=query tokenizer class=solr.KeywordTokenizerFactory/ filter class=solr.LowerCaseFilterFactory / /analyzer /fieldType fieldType name=prefix_token class=solr.TextField positionIncrementGap=1 analyzer type=index tokenizer class=solr.WhitespaceTokenizerFactory/ filter class=solr.LowerCaseFilterFactory / filter class=solr.EdgeNGramFilterFactory minGramSize=1 maxGramSize=20/ /analyzer analyzer type=query tokenizer class=solr.WhitespaceTokenizerFactory/ filter class=solr.LowerCaseFilterFactory / /analyzer /fieldType ... field name=prefix1 type=prefix_full indexed=true stored=false/ field name=prefix2 type=prefix_token indexed=true stored=false/ ... copyField source=name dest=prefix1/ copyField source=name dest=prefix2/ For using the above code is that you are using EdgeNGramFilterFactory or PrefixingFilterFactory. or the above code works for EdgeNGramFilterFactory,i am not clear about it,with out using the PrefixingFilterFactory patch, is that i can write the above code. There is no such thing in Solr as a PrefixingFilterFactory. Use EdgeNGramFilterFactory. And the next is name in copyFiled is text type or string type Name was a field in his schema. Whatever fields' values you want for auto-suggest, copy them over to the field. -- Regards, Shalin Shekhar Mangar.
Re: Solr with Auto-suggest
Thanks a lot for your advice/suggestion. I have made good progress and could able to extract all facets based on facet.prefix query. The auto-suggest works fine for single word suggestion. I was wondering to extract all nearest token for any token selected by user in auto-suggest mode. Example: Ex-series products are in market - Tokens - exseries products market exseries switches are expensive - Tokens - exseries swicthes expensive In the above example, how can I keep track of relative distance between tokens. I.e. if user selects exseries then next should be available are products, swicthes in the order of appearance. Your input/suggestion would be highly appreciated - RB On 4/25/08, Lance Norskog [EMAIL PROTECTED] wrote: This what the spellchecker does. It makes a separate Lucene index of n-gram letters and searches those. Works pretty well and it is outside the main index. I did an experimental variation indexing word pairs as phrases, and it worked well too. Lance Norskog -Original Message- From: Ryan McKinley [mailto:[EMAIL PROTECTED] Sent: Thursday, April 24, 2008 2:18 PM To: solr-user@lucene.apache.org Subject: Re: Solr with Auto-suggest On Apr 24, 2008, at 12:25 PM, Rantjil Bould wrote: Hi Group, I was asked in my project to implement google suggest kind of functionality for searching help system. I have seen one thread http://www.mail-archive.com/solr-user@lucene.apache.org/ msg06739.html which deals with the way to index if large index. But I am not able to get much information to start with. I am using JQuery's plugin for auto- suggest and query field is a large text(appx 2000 char long). I am just wondering how can I extract all tokens for any character typed by user? Somebody might have already implemented the same functionality and I would appreciate your help on this, even a hint might be a great help. I don't think there is a magic one-size-fits-all solution to this, only a set of approaches you will need to modify for your specific index. You will need to modify the jquery plugin to grab results from a solr query. For starters that can be just a standard query whatever. Unless your index is small, you will likely need to configure your index with special fields to use for the auto-complete search. This is the approach pointed to in SOLR-357. Eseentially you index: Bould as b bo bou boul bould. ryan . Checked by AVG. Version: 8.0.100 / Virus Database: 269.23.4/1397 - Release Date: 25.04.2008 7:42
Re: Solr with Auto-suggest
Nice. Great help. I have added following fields to hold tokens. fieldType name=prefix_full class=solr.TextField positionIncrementGap=1 analyzer type=index tokenizer class=solr.KeywordTokenizerFactory/ filter class=solr.LowerCaseFilterFactory / filter class=solr.EdgeNGramFilterFactory minGramSize=1 maxGramSize=20/ /analyzer analyzer type=query tokenizer class=solr.KeywordTokenizerFactory/ filter class=solr.LowerCaseFilterFactory / /analyzer /fieldType fieldType name=prefix_token class=solr.TextField positionIncrementGap=1 analyzer type=index tokenizer class=solr.WhitespaceTokenizerFactory/ filter class=solr.LowerCaseFilterFactory / filter class=solr.EdgeNGramFilterFactory minGramSize=1 maxGramSize=20/ /analyzer analyzer type=query tokenizer class=solr.WhitespaceTokenizerFactory/ filter class=solr.LowerCaseFilterFactory / /analyzer /fieldType field name=prefix1 type=prefix_full indexed=true stored=true/ field name=prefix2 type=prefix_token indexed=true stored=true/ copyField source=content dest=prefix1/ copyField source=content dest=prefix2/ I am wondering how can I extract tokens? I can see all tokens http://localhost:8080/solr/admin/schema.jsp page for fields prefix1 and prefix2 but when I query http://localhost:8080/solr/select?fl=prefix1,idq=prefix2:jun%20prefix2:jun to get the content for prefix2, it does not display any content for prefix2. Am I doing anything wrong??? - RB On 4/24/08, Ryan McKinley [EMAIL PROTECTED] wrote: On Apr 24, 2008, at 12:25 PM, Rantjil Bould wrote: Hi Group, I was asked in my project to implement google suggest kind of functionality for searching help system. I have seen one thread http://www.mail-archive.com/solr-user@lucene.apache.org/msg06739.html which deals with the way to index if large index. But I am not able to get much information to start with. I am using JQuery's plugin for auto-suggest and query field is a large text(appx 2000 char long). I am just wondering how can I extract all tokens for any character typed by user? Somebody might have already implemented the same functionality and I would appreciate your help on this, even a hint might be a great help. I don't think there is a magic one-size-fits-all solution to this, only a set of approaches you will need to modify for your specific index. You will need to modify the jquery plugin to grab results from a solr query. For starters that can be just a standard query whatever. Unless your index is small, you will likely need to configure your index with special fields to use for the auto-complete search. This is the approach pointed to in SOLR-357. Eseentially you index: Bould as b bo bou boul bould. ryan
Re: Solr with Auto-suggest
On Apr 25, 2008, at 3:02 AM, Rantjil Bould wrote: Nice. Great help. I have added following fields to hold tokens. fieldType name=prefix_full class=solr.TextField positionIncrementGap=1 analyzer type=index tokenizer class=solr.KeywordTokenizerFactory/ filter class=solr.LowerCaseFilterFactory / filter class=solr.EdgeNGramFilterFactory minGramSize=1 maxGramSize=20/ /analyzer analyzer type=query tokenizer class=solr.KeywordTokenizerFactory/ filter class=solr.LowerCaseFilterFactory / /analyzer /fieldType fieldType name=prefix_token class=solr.TextField positionIncrementGap=1 analyzer type=index tokenizer class=solr.WhitespaceTokenizerFactory/ filter class=solr.LowerCaseFilterFactory / filter class=solr.EdgeNGramFilterFactory minGramSize=1 maxGramSize=20/ /analyzer analyzer type=query tokenizer class=solr.WhitespaceTokenizerFactory/ filter class=solr.LowerCaseFilterFactory / /analyzer /fieldType field name=prefix1 type=prefix_full indexed=true stored=true/ field name=prefix2 type=prefix_token indexed=true stored=true/ copyField source=content dest=prefix1/ copyField source=content dest=prefix2/ I am wondering how can I extract tokens? I can see all tokens http://localhost:8080/solr/admin/schema.jsp page for fields prefix1 and prefix2 but when I query http://localhost:8080/solr/select?fl=prefix1,idq=prefix2:jun%20prefix2:jun to get the content for prefix2, it does not display any content for prefix2. Am I doing anything wrong??? what do you mean extract tokens? The documents returned from /select? are the stored field values not the tokens -- you don't get to see the analyzed tokens (nor do you need to). If you want to interact with tokens consider using faceting. ryan - RB On 4/24/08, Ryan McKinley [EMAIL PROTECTED] wrote: On Apr 24, 2008, at 12:25 PM, Rantjil Bould wrote: Hi Group, I was asked in my project to implement google suggest kind of functionality for searching help system. I have seen one thread http://www.mail-archive.com/solr-user@lucene.apache.org/ msg06739.html which deals with the way to index if large index. But I am not able to get much information to start with. I am using JQuery's plugin for auto- suggest and query field is a large text(appx 2000 char long). I am just wondering how can I extract all tokens for any character typed by user? Somebody might have already implemented the same functionality and I would appreciate your help on this, even a hint might be a great help. I don't think there is a magic one-size-fits-all solution to this, only a set of approaches you will need to modify for your specific index. You will need to modify the jquery plugin to grab results from a solr query. For starters that can be just a standard query whatever. Unless your index is small, you will likely need to configure your index with special fields to use for the auto-complete search. This is the approach pointed to in SOLR-357. Eseentially you index: Bould as b bo bou boul bould. ryan
RE: Solr with Auto-suggest
This what the spellchecker does. It makes a separate Lucene index of n-gram letters and searches those. Works pretty well and it is outside the main index. I did an experimental variation indexing word pairs as phrases, and it worked well too. Lance Norskog -Original Message- From: Ryan McKinley [mailto:[EMAIL PROTECTED] Sent: Thursday, April 24, 2008 2:18 PM To: solr-user@lucene.apache.org Subject: Re: Solr with Auto-suggest On Apr 24, 2008, at 12:25 PM, Rantjil Bould wrote: Hi Group, I was asked in my project to implement google suggest kind of functionality for searching help system. I have seen one thread http://www.mail-archive.com/solr-user@lucene.apache.org/ msg06739.html which deals with the way to index if large index. But I am not able to get much information to start with. I am using JQuery's plugin for auto- suggest and query field is a large text(appx 2000 char long). I am just wondering how can I extract all tokens for any character typed by user? Somebody might have already implemented the same functionality and I would appreciate your help on this, even a hint might be a great help. I don't think there is a magic one-size-fits-all solution to this, only a set of approaches you will need to modify for your specific index. You will need to modify the jquery plugin to grab results from a solr query. For starters that can be just a standard query whatever. Unless your index is small, you will likely need to configure your index with special fields to use for the auto-complete search. This is the approach pointed to in SOLR-357. Eseentially you index: Bould as b bo bou boul bould. ryan . Checked by AVG. Version: 8.0.100 / Virus Database: 269.23.4/1397 - Release Date: 25.04.2008 7:42
Re: Solr with Auto-suggest
On Apr 24, 2008, at 12:25 PM, Rantjil Bould wrote: Hi Group, I was asked in my project to implement google suggest kind of functionality for searching help system. I have seen one thread http://www.mail-archive.com/solr-user@lucene.apache.org/ msg06739.html which deals with the way to index if large index. But I am not able to get much information to start with. I am using JQuery's plugin for auto- suggest and query field is a large text(appx 2000 char long). I am just wondering how can I extract all tokens for any character typed by user? Somebody might have already implemented the same functionality and I would appreciate your help on this, even a hint might be a great help. I don't think there is a magic one-size-fits-all solution to this, only a set of approaches you will need to modify for your specific index. You will need to modify the jquery plugin to grab results from a solr query. For starters that can be just a standard query whatever. Unless your index is small, you will likely need to configure your index with special fields to use for the auto-complete search. This is the approach pointed to in SOLR-357. Eseentially you index: Bould as b bo bou boul bould. ryan
