Highlighting on Prefix-Search Bug/Workaround (Re: query with stemming, prefix and fuzzy?)

2009-02-04 Thread Gert Brinkmann
Mark Miller wrote: >> Currently I think about dropping the stemming and only use >> prefix-search. But as highlighting does not work with a prefix "house*" >> this is a problem for me. The hint to use "house?*" instead does not >> work here. >> > Thats because wildcard queries are also not high

Re: query with stemming, prefix and fuzzy?

2009-01-30 Thread Shalin Shekhar Mangar
On Fri, Jan 30, 2009 at 11:37 PM, Mark Miller wrote: > >> >>> you can try indexing the full term at the same position as the stemmed >>> term, >>> >>> >> >> what does this mean "at the same position" and how could I do this? >> >> > Write a custom filter. Normally, for every term, its position is

Re: query with stemming, prefix and fuzzy?

2009-01-30 Thread Mark Miller
Gert Brinkmann wrote: 57971 Its a lot for a small index. The fuzzy query will enumerate all of those terms and calculate an edit distance. Its not an insane amount of work, but it jives with the slowness you see. Doing that 60,000 times for a query is not that fast. Unfortunately, withou

Re: query with stemming, prefix and fuzzy?

2009-01-30 Thread Gert Brinkmann
Mark Miller wrote: > Try hitting /solr/admin/luke and see what it says. Oh, interesting. I think I have to check the stopword list. Is there a way to filter single characters like the "h"? text_de_de ITS-- ITS-- 2340 57971 1454 1016 1008 980 927 924 895 843 730 730

Re: query with stemming, prefix and fuzzy?

2009-01-30 Thread Mark Miller
Gert Brinkmann wrote: Mark Miller wrote: Yeah, sounds small. Its odd you would see such slow performance. It depends though. You may still have a *lot* of unique terms in there. Is there a way to retrieve the list of terms in the index? Gert Try hitting /solr/admin/luke and see wh

Re: query with stemming, prefix and fuzzy?

2009-01-30 Thread Gert Brinkmann
Mark Miller wrote: > Yeah, sounds small. Its odd you would see such slow performance. It > depends though. You may still have a *lot* of unique terms in there. Is there a way to retrieve the list of terms in the index? Gert

Re: query with stemming, prefix and fuzzy?

2009-01-30 Thread Mark Miller
Gert Brinkmann wrote: Thanks, Mark, for your answer, Mark Miller wrote: Truncation queries and stemming are difficult partners. You likely have to accept compromise. You can try using multiple fields like you are, I already have multiple fields, one per language, to be able to use dif

Re: query with stemming, prefix and fuzzy?

2009-01-30 Thread Gert Brinkmann
Thanks, Mark, for your answer, Mark Miller wrote: > Truncation queries and stemming are difficult partners. You likely have > to accept compromise. You can try using multiple fields like you are, I already have multiple fields, one per language, to be able to use different stemmers. Wouldn't bec

Re: query with stemming, prefix and fuzzy?

2009-01-29 Thread Mark Miller
Truncation queries and stemming are difficult partners. You likely have to accept compromise. You can try using multiple fields like you are, you can try indexing the full term at the same position as the stemmed term, or you can accept the weirdness that comes from matching on a stemmed form (

Re: query with stemming, prefix and fuzzy?

2009-01-29 Thread Gert Brinkmann
Gert Brinkmann wrote: >> A) fuzzy search >> >> What can I do to speed up the fuzzy query? Setting ramBufferSizeMB to a higher value seems to speed up the query slightly. I have to continue with tuning though. >> B) combine stemming, prefix and fuzzy search >> >> Is there a way to combine all th

Re: query with stemming, prefix and fuzzy?

2009-01-29 Thread Gert Brinkmann
Shalin Shekhar Mangar wrote: Quite the opposite, you are actually working with some advanced stuff :) Thank you for the response. Please have some patience, someone is Ok, I will have (what else could I do? ;) ). Meanwhile I while try some things and continue to search the web. Greetings

Re: query with stemming, prefix and fuzzy?

2009-01-28 Thread Shalin Shekhar Mangar
On Thu, Jan 29, 2009 at 12:39 AM, Gert Brinkmann wrote: > > Hello again, > > is there nobody who could help me with this? Or is it an FAQ and my > questions are dumb somehow? Maybe I should try to shorten the questions: ;) > Quite the opposite, you are actually working with some advanced stuff :

Re: query with stemming, prefix and fuzzy?

2009-01-28 Thread Gert Brinkmann
Hello again, is there nobody who could help me with this? Or is it an FAQ and my questions are dumb somehow? Maybe I should try to shorten the questions: ;) > A) fuzzy search > > What can I do to speed up the fuzzy query? > B) combine stemming, prefix and fuzzy search > > Is there a way to

query with stemming, prefix and fuzzy?

2009-01-27 Thread Gert Brinkmann
Hello, I am trying to get Solr to properly work. I have set up a Solr test server (using jetty as mentioned in the tutorial). Also I had to modify the schema.xml so that I have different fields for different languages (with their own stemmers) that occur in the content management system that I am