Re: Anyway to not bother scoring less good matches ?

2011-05-05 Thread Paul Taylor
On 05/05/2011 11:59, Ian Lea wrote: See http://www.hathitrust.org/blogs/large-scale-search/slow-queries-and-common-words-part-1 for an excellent article and solution to the problem with common words. Would this work when the user doesnt actualy use a phrase query You could also consider using

Re: Anyway to not bother scoring less good matches ?

2011-05-05 Thread Ian Lea
See http://www.hathitrust.org/blogs/large-scale-search/slow-queries-and-common-words-part-1 for an excellent article and solution to the problem with common words. You could also consider using, and caching and reusing, filters for the tnum and tracks fields. -- Ian. On Thu, May 5, 2011 at 11

Re: Anyway to not bother scoring less good matches ?

2011-05-05 Thread Paul Taylor
On 05/05/2011 11:13, Ahmet Arslan wrote: Yes correct, but I have looked and the list of optimizations before. What was clear from profiling was that it wasnt the searching part that was slow (a query run on the same index with only a few matching docs ran super fast) the slowness only occurs when

Re: Anyway to not bother scoring less good matches ?

2011-05-05 Thread Ahmet Arslan
> Yes correct, but I have looked and the list of > optimizations before. What was clear from profiling was that > it wasnt the searching part that was slow (a query run on > the same index with only a few matching docs ran super fast) > the slowness only occurs when there are loads of matching > do

Re: Anyway to not bother scoring less good matches ?

2011-05-05 Thread Paul Taylor
On 05/05/2011 00:24, Ahmet Arslan wrote: Thanks again, now done that but still not having much effect on total ime, So your main concern is enhancing the running time? , not to decrease the number of returned results. Additionally http://wiki.apache.org/lucene-java/ImproveSearchingSpeed Yes c

Re: Anyway to not bother scoring less good matches ?

2011-05-05 Thread Paul Taylor
On 05/05/2011 00:24, Chris Hostetter wrote: : Well I did extend QuerParser, and the method is being called but rather : disappointingly it had no noticeablke effect on how long queries took. I : really thought by reducing the number of matches the corresponding scoring : phase would be quicker.

Re: Anyway to not bother scoring less good matches ?

2011-05-04 Thread Chris Hostetter
: Well I did extend QuerParser, and the method is being called but rather : disappointingly it had no noticeablke effect on how long queries took. I : really thought by reducing the number of matches the corresponding scoring : phase would be quicker. "matching" and "scoring" go hand in hand ...

Re: Anyway to not bother scoring less good matches ?

2011-05-04 Thread Ahmet Arslan
> Thanks again, now done that but still not having much > effect on total > ime, So your main concern is enhancing the running time? , not to decrease the number of returned results. Additionally http://wiki.apache.org/lucene-java/ImproveSearchingSpeed ---

Re: Anyway to not bother scoring less good matches ?

2011-05-04 Thread Paul Taylor
On 04/05/2011 15:02, Ahmet Arslan wrote: Thanks for the hint, so this could be done by overriding getBooleanQuery() in QueryParser ? I think something like this should do the trick. Without overriding anything. Query query= QueryParser.parse("User Entered String"); if (query instanceof B

Re: Anyway to not bother scoring less good matches ?

2011-05-04 Thread Ahmet Arslan
> Thanks for the hint, so this could be done by overriding getBooleanQuery() in > QueryParser ? > I think something like this should do the trick. Without overriding anything. Query query= QueryParser.parse("User Entered String"); if (query instanceof BooleanQuery) ((BooleanQuery)query).se

Re: Anyway to not bother scoring less good matches ?

2011-05-04 Thread Paul Taylor
On 04/05/2011 12:51, Paul Taylor wrote: On 04/05/2011 12:39, Ahmet Arslan wrote: Im receiving a number of searches with many ORs so that the total number of matches is huge (> 1 million) although only the first 20 results are required. Analysis shows most time is spent scoring the results.

Re: Anyway to not bother scoring less good matches ?

2011-05-04 Thread Paul Taylor
On 04/05/2011 12:39, Ahmet Arslan wrote: Im receiving a number of searches with many ORs so that the total number of matches is huge (> 1 million) although only the first 20 results are required. Analysis shows most time is spent scoring the results. Now it seems to me if you sending a query

Re: Anyway to not bother scoring less good matches ?

2011-05-04 Thread Ahmet Arslan
Im receiving a number of searches with many ORs so that the total number of matches is huge ( > 1 million) although only the first 20 results are required. Analysis shows most time is spent scoring the results. Now it seems to me if you sending a query with 10 OR components, documents that mat

Anyway to not bother scoring less good matches ?

2011-05-03 Thread Paul Taylor
Im receiving a number of searches with many ORs so that the total number of matches is huge ( > 1 million) although only the first 20 results are required. Analysis shows most time is spent scoring the results. Now it seems to me if you sending a query with 10 OR components, documents that matc