Re: Implementing phrase autopop up
On Tue, Nov 24, 2009 at 11:58 PM, darniz wrote: > > > i created a filed as same as the lucid blog says. > > omitNorms="true" omitTermFreqAndPositions="true"/> > > with the following field configurtion > > positionIncrementGap="100"> > − > > > > maxGramSize="25"/> > > − > > > > > > > Now when i query i get the correct phrases for example if search for > autocomp:"how to" i get all the correct phrases like > > How to find a car > How to find a mechanic > How to choose the right insurance company > > etc... which is good. > > Now I have two question. > 1) Is it necessary to give the query in quote. My gut feeling is yes, since > if you dont give quote i get phrases beginning with How followed by some > other words like How can etc... > Yes since we want to do phrase searches on n-grams > 2)if i search for word for example choose, it gives me nothing > I was expecting to see a result considering there is a word "choose" in the > phrase > How to choose the right insurance company > > i might look more at documentation but do you have anything to advice. > > EdgeNgram creates n-grams from the starting or the ending edge therefore you can't match words in the middle of a phrase. Try using NGramFilterFactory instead. -- Regards, Shalin Shekhar Mangar.
Re: Implementing phrase autopop up
can anybody update me if its possible that a word within a phrase is match, that phrase can be displayed. darniz darniz wrote: > > Thanks for your input > You made a valid point, if we are using field type as text to get > autocomplete it wont work because it goes through tokenizer. > Hence looks like for my use case i need to have a field which uses ngram > and copy. Here is what i did > > i created a filed as same as the lucid blog says. > > omitNorms="true" omitTermFreqAndPositions="true"/> > > with the following field configurtion > > positionIncrementGap="100"> > − > > > > maxGramSize="25"/> > > − > > > > > > > Now when i query i get the correct phrases for example if search for > autocomp:"how to" i get all the correct phrases like > > How to find a car > How to find a mechanic > How to choose the right insurance company > > etc... which is good. > > Now I have two question. > 1) Is it necessary to give the query in quote. My gut feeling is yes, > since if you dont give quote i get phrases beginning with How followed by > some other words like How can etc... > > 2)if i search for word for example choose, it gives me nothing > I was expecting to see a result considering there is a word "choose" in > the phrase > How to choose the right insurance company > > i might look more at documentation but do you have anything to advice. > > darniz > > > > > > > > > > Shalin Shekhar Mangar wrote: >> >> On Tue, Nov 24, 2009 at 10:12 AM, darniz wrote: >> >>> >>> hello all >>> Let me first explain the task i am trying to do. >>> i have article with title for example >>> >>> >Car Insurance for Teenage Drivers >>> >>> − >>> >>> A Total Loss? >>> >>> If a user begins to type car insu i want the autopop to show up with the >>> entire phrase. >>> There are two ways to implement this. >>> First is to use the termcomponent and the other is to use a field with >>> field >>> type which uses solr.EdgeNGramFilterFactor filter. >>> >>> I started with using with Term component and i declared a term request >>> handler and gave the following query >>> >>> http://localhost:8080/solr/terms?terms.fl=title&terms.prefix=car >>> The issue is that its not giving the entire pharse, it gives me back >>> results >>> like car, caravan, carbon. Now i know using terms.prefix will only give >>> me >>> results where the sentence start with car. On top of this i also want if >>> there is word like car somewhere in between the title that should also >>> show >>> up in autopop very much similar like google where a word is not >>> necessarily >>> start at the beginning but it could be present anywhere in the middle of >>> the >>> title. >>> The question is does TermComponent is a good candidate or using a >>> custom >>> field lets the name is autoPopupText with field type configured with all >>> filter and EdgeNGramFilterFactor defined and copying the title to the >>> autoPopupText field and using it to power autopopup. >>> >>> The other thing is that using EdgeNGramFilterFactor is more from index >>> point of view when you index document you need to know which fields you >>> want >>> to copy to autoPopupText field where as using Term component is more >>> like >>> you can define at query time what fields you want to use to fetch >>> autocomplete from. >>> >>> Any idea whats the best and why the Term component is not giving me an >>> entire phrase which i mentioned earlier. >>> FYI >>> my title field is of type text. >>> >> >> >> You are using a tokenized field type with TermsComponent therefore each >> word >> in your phrase gets indexed as a separate token. You should use a >> non-tokenized type (such as a string type) with TermsComponent. However, >> this will only let you search by prefix and not by words in between the >> phrase. >> >> Your best bet here would be to use EdgeNGramFilterFactory. If your index >> is >> very large, you can consider doing a prefix search on shingles too. >> >> -- >> Regards, >> Shalin Shekhar Mangar. >> >> > > -- View this message in context: http://old.nabble.com/Implementing-phrase-autopop-up-tp26490419p26506470.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Implementing phrase autopop up
Thanks for your input You made a valid point, if we are using field type as text to get autocomplete it wont work because it goes through tokenizer. Hence looks like for my use case i need to have a field which uses ngram and copy. Here is what i did i created a filed as same as the lucid blog says. with the following field configurtion − − Now when i query i get the correct phrases for example if search for autocomp:"how to" i get all the correct phrases like How to find a car How to find a mechanic How to choose the right insurance company etc... which is good. Now I have two question. 1) Is it necessary to give the query in quote. My gut feeling is yes, since if you dont give quote i get phrases beginning with How followed by some other words like How can etc... 2)if i search for word for example choose, it gives me nothing I was expecting to see a result considering there is a word "choose" in the phrase How to choose the right insurance company i might look more at documentation but do you have anything to advice. darniz Shalin Shekhar Mangar wrote: > > On Tue, Nov 24, 2009 at 10:12 AM, darniz wrote: > >> >> hello all >> Let me first explain the task i am trying to do. >> i have article with title for example >> >> >Car Insurance for Teenage Drivers >> >> − >> >> A Total Loss? >> >> If a user begins to type car insu i want the autopop to show up with the >> entire phrase. >> There are two ways to implement this. >> First is to use the termcomponent and the other is to use a field with >> field >> type which uses solr.EdgeNGramFilterFactor filter. >> >> I started with using with Term component and i declared a term request >> handler and gave the following query >> >> http://localhost:8080/solr/terms?terms.fl=title&terms.prefix=car >> The issue is that its not giving the entire pharse, it gives me back >> results >> like car, caravan, carbon. Now i know using terms.prefix will only give >> me >> results where the sentence start with car. On top of this i also want if >> there is word like car somewhere in between the title that should also >> show >> up in autopop very much similar like google where a word is not >> necessarily >> start at the beginning but it could be present anywhere in the middle of >> the >> title. >> The question is does TermComponent is a good candidate or using a custom >> field lets the name is autoPopupText with field type configured with all >> filter and EdgeNGramFilterFactor defined and copying the title to the >> autoPopupText field and using it to power autopopup. >> >> The other thing is that using EdgeNGramFilterFactor is more from index >> point of view when you index document you need to know which fields you >> want >> to copy to autoPopupText field where as using Term component is more like >> you can define at query time what fields you want to use to fetch >> autocomplete from. >> >> Any idea whats the best and why the Term component is not giving me an >> entire phrase which i mentioned earlier. >> FYI >> my title field is of type text. >> > > > You are using a tokenized field type with TermsComponent therefore each > word > in your phrase gets indexed as a separate token. You should use a > non-tokenized type (such as a string type) with TermsComponent. However, > this will only let you search by prefix and not by words in between the > phrase. > > Your best bet here would be to use EdgeNGramFilterFactory. If your index > is > very large, you can consider doing a prefix search on shingles too. > > -- > Regards, > Shalin Shekhar Mangar. > > -- View this message in context: http://old.nabble.com/Implementing-phrase-autopop-up-tp26490419p26499912.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Implementing phrase autopop up
On Tue, Nov 24, 2009 at 10:12 AM, darniz wrote: > > hello all > Let me first explain the task i am trying to do. > i have article with title for example > > >Car Insurance for Teenage Drivers > > − > > A Total Loss? > > If a user begins to type car insu i want the autopop to show up with the > entire phrase. > There are two ways to implement this. > First is to use the termcomponent and the other is to use a field with > field > type which uses solr.EdgeNGramFilterFactor filter. > > I started with using with Term component and i declared a term request > handler and gave the following query > > http://localhost:8080/solr/terms?terms.fl=title&terms.prefix=car > The issue is that its not giving the entire pharse, it gives me back > results > like car, caravan, carbon. Now i know using terms.prefix will only give me > results where the sentence start with car. On top of this i also want if > there is word like car somewhere in between the title that should also show > up in autopop very much similar like google where a word is not necessarily > start at the beginning but it could be present anywhere in the middle of > the > title. > The question is does TermComponent is a good candidate or using a custom > field lets the name is autoPopupText with field type configured with all > filter and EdgeNGramFilterFactor defined and copying the title to the > autoPopupText field and using it to power autopopup. > > The other thing is that using EdgeNGramFilterFactor is more from index > point of view when you index document you need to know which fields you > want > to copy to autoPopupText field where as using Term component is more like > you can define at query time what fields you want to use to fetch > autocomplete from. > > Any idea whats the best and why the Term component is not giving me an > entire phrase which i mentioned earlier. > FYI > my title field is of type text. > You are using a tokenized field type with TermsComponent therefore each word in your phrase gets indexed as a separate token. You should use a non-tokenized type (such as a string type) with TermsComponent. However, this will only let you search by prefix and not by words in between the phrase. Your best bet here would be to use EdgeNGramFilterFactory. If your index is very large, you can consider doing a prefix search on shingles too. -- Regards, Shalin Shekhar Mangar.
Implementing phrase autopop up
hello all Let me first explain the task i am trying to do. i have article with title for example >Car Insurance for Teenage Drivers − A Total Loss? If a user begins to type car insu i want the autopop to show up with the entire phrase. There are two ways to implement this. First is to use the termcomponent and the other is to use a field with field type which uses solr.EdgeNGramFilterFactor filter. I started with using with Term component and i declared a term request handler and gave the following query http://localhost:8080/solr/terms?terms.fl=title&terms.prefix=car The issue is that its not giving the entire pharse, it gives me back results like car, caravan, carbon. Now i know using terms.prefix will only give me results where the sentence start with car. On top of this i also want if there is word like car somewhere in between the title that should also show up in autopop very much similar like google where a word is not necessarily start at the beginning but it could be present anywhere in the middle of the title. The question is does TermComponent is a good candidate or using a custom field lets the name is autoPopupText with field type configured with all filter and EdgeNGramFilterFactor defined and copying the title to the autoPopupText field and using it to power autopopup. The other thing is that using EdgeNGramFilterFactor is more from index point of view when you index document you need to know which fields you want to copy to autoPopupText field where as using Term component is more like you can define at query time what fields you want to use to fetch autocomplete from. Any idea whats the best and why the Term component is not giving me an entire phrase which i mentioned earlier. FYI my title field is of type text. Thanks darniz -- View this message in context: http://old.nabble.com/Implementing-phrase-autopop-up-tp26490419p26490419.html Sent from the Solr - User mailing list archive at Nabble.com.