Re: Slow query time on stemmed fields
with my parameter overload or does highlighting on the spell field actually take place ? I Noticed a 13MB file poping up only if the search results are extended via the spell field. but highlighting this doc on a query that brings only this doc up does not take any amount of time like this. Thanks for your comments and time. Best, Jens 2014-02-24 17:32 GMT+01:00 Jack Krupansky j...@basetechnology.com: Maybe some heap/GC issue from using more of this 20 GB index. Maybe it was running at the edge and just one more field was too much for the heap. The timing section of the debug query response should shed a little light. -- Jack Krupansky -Original Message- From: Erick Erickson Sent: Monday, February 24, 2014 11:25 AM To: solr-user@lucene.apache.org Subject: Re: Slow query time on stemmed fields This is really strange. You should have _fewer_ tokens in your stemmed field. Plus, the up-front processing to stem the field in the query shouldn't be noticeable. Let's see the query and results from debug=all being added to the URL because something is completely strange here. Best, Erick On Mon, Feb 24, 2014 at 7:18 AM, Jens Meiners snej.sren...@gmail.com wrote: Hi, we've built an index (Solr 4.3), which contains approx. 1 Million docs and its size is around 20 GB (optimized). In our index we have one field which contains the tokenized words of indexed documents and a second field with the stemmed contents (SnowballFilter, German2). During our tests we've found out that some keywords are just taking too long to process. When we exclude the stemmed field from our edismax configuration (qf) the query time was surprisingly quick (10 000x faster). Had one of you the same experience ? We are using the stemmed field only to increase the returned documents and not for highlighting. We know that by applying highlighting on stemmed values is not good for query speed. Best Regards, Jens Meiners
Slow query time on stemmed fields
Hi, we've built an index (Solr 4.3), which contains approx. 1 Million docs and its size is around 20 GB (optimized). In our index we have one field which contains the tokenized words of indexed documents and a second field with the stemmed contents (SnowballFilter, German2). During our tests we've found out that some keywords are just taking too long to process. When we exclude the stemmed field from our edismax configuration (qf) the query time was surprisingly quick (10 000x faster). Had one of you the same experience ? We are using the stemmed field only to increase the returned documents and not for highlighting. We know that by applying highlighting on stemmed values is not good for query speed. Best Regards, Jens Meiners
Re: Slow query time on stemmed fields
This is really strange. You should have _fewer_ tokens in your stemmed field. Plus, the up-front processing to stem the field in the query shouldn't be noticeable. Let's see the query and results from debug=all being added to the URL because something is completely strange here. Best, Erick On Mon, Feb 24, 2014 at 7:18 AM, Jens Meiners snej.sren...@gmail.comwrote: Hi, we've built an index (Solr 4.3), which contains approx. 1 Million docs and its size is around 20 GB (optimized). In our index we have one field which contains the tokenized words of indexed documents and a second field with the stemmed contents (SnowballFilter, German2). During our tests we've found out that some keywords are just taking too long to process. When we exclude the stemmed field from our edismax configuration (qf) the query time was surprisingly quick (10 000x faster). Had one of you the same experience ? We are using the stemmed field only to increase the returned documents and not for highlighting. We know that by applying highlighting on stemmed values is not good for query speed. Best Regards, Jens Meiners
Re: Slow query time on stemmed fields
Maybe some heap/GC issue from using more of this 20 GB index. Maybe it was running at the edge and just one more field was too much for the heap. The timing section of the debug query response should shed a little light. -- Jack Krupansky -Original Message- From: Erick Erickson Sent: Monday, February 24, 2014 11:25 AM To: solr-user@lucene.apache.org Subject: Re: Slow query time on stemmed fields This is really strange. You should have _fewer_ tokens in your stemmed field. Plus, the up-front processing to stem the field in the query shouldn't be noticeable. Let's see the query and results from debug=all being added to the URL because something is completely strange here. Best, Erick On Mon, Feb 24, 2014 at 7:18 AM, Jens Meiners snej.sren...@gmail.comwrote: Hi, we've built an index (Solr 4.3), which contains approx. 1 Million docs and its size is around 20 GB (optimized). In our index we have one field which contains the tokenized words of indexed documents and a second field with the stemmed contents (SnowballFilter, German2). During our tests we've found out that some keywords are just taking too long to process. When we exclude the stemmed field from our edismax configuration (qf) the query time was surprisingly quick (10 000x faster). Had one of you the same experience ? We are using the stemmed field only to increase the returned documents and not for highlighting. We know that by applying highlighting on stemmed values is not good for query speed. Best Regards, Jens Meiners
