Re: lucene (search) performance tuning

2012-05-28 Thread Lance Norskog
And, no RamDirectory does not help. On Mon, May 28, 2012 at 5:54 PM, Lance Norskog wrote: > Can you use filter queries? Filters short-circuit a lot of search > processing. "City:San Francisco" is a classic filter - it is a small > part of the documents and it is reused a lot. > > On Sat, May 26,

Re: lucene (search) performance tuning

2012-05-28 Thread Lance Norskog
Can you use filter queries? Filters short-circuit a lot of search processing. "City:San Francisco" is a classic filter - it is a small part of the documents and it is reused a lot. On Sat, May 26, 2012 at 7:32 AM, Yang wrote: > I'm using disjunction (OR) query. unfortunately all of the clauses ar

Re: lucene (search) performance tuning

2012-05-26 Thread Yang
I'm using disjunction (OR) query. unfortunately all of the clauses are optional On Sat, May 26, 2012 at 4:38 AM, Simon Willnauer < simon.willna...@googlemail.com> wrote: > On Sat, May 26, 2012 at 2:59 AM, Yang wrote: > > I tested with more threads / processes. indeed this is completely > > cpu-b

Re: lucene (search) performance tuning

2012-05-26 Thread Li Li
if you don't score but sort by id, it may be a little bit faster. but for 3.x, you can hardly speed up by simpler scoring function. for your situation, the bottleneck is cpu. you can speed up by paralleling. so the best one is to split index and searching concurrently. so the cpus can be fully used

Re: lucene (search) performance tuning

2012-05-26 Thread Simon Willnauer
On Sat, May 26, 2012 at 2:59 AM, Yang wrote: > I tested with more threads / processes. indeed this is completely > cpu-bound, since running 1 thread gives the same latency as 4 threads (my > box has 4 cores) > > > given this, is there any way to simplify the scoring computation (i'm only > using l

Re: lucene (search) performance tuning

2012-05-25 Thread Yang
I tested with more threads / processes. indeed this is completely cpu-bound, since running 1 thread gives the same latency as 4 threads (my box has 4 cores) given this, is there any way to simplify the scoring computation (i'm only using lucene as a first level "rough" search, so the search quali

Re: lucene (search) performance tuning

2012-05-25 Thread Yang
thanks a lot guys On Tue, May 22, 2012 at 1:34 AM, Ian Lea wrote: > Lots of good tips in > http://wiki.apache.org/lucene-java/ImproveSearchingSpeed, linked from > the FAQ. > > > -- > Ian. > > > On Tue, May 22, 2012 at 2:08 AM, Li Li wrote: > > something wrong when writing in my android client.

Re: lucene (search) performance tuning

2012-05-22 Thread Ian Lea
Lots of good tips in http://wiki.apache.org/lucene-java/ImproveSearchingSpeed, linked from the FAQ. -- Ian. On Tue, May 22, 2012 at 2:08 AM, Li Li wrote: > something wrong when writing in my android client. > if RAMDirectory do not help, i think the bottleneck is cpu. you may try to > tune jvm

Re: lucene (search) performance tuning

2012-05-21 Thread Li Li
something wrong when writing in my android client. if RAMDirectory do not help, i think the bottleneck is cpu. you may try to tune jvm but i do not expect much improvement. the best one is splitting your index into 2 or more smaller ones. you can then use solr s distributed searching. if the cpu is

Re: lucene (search) performance tuning

2012-05-21 Thread Li Li
在 2012-5-22 凌晨4:59,"Yang" 写道: > > I'm trying to make my search faster. right now a query like > > name:Joe Moe Pizza address:77 main street city:San Francisco >is this a conjunction query or a disjunction query? > in a index with 20mil such short business descriptions (total size about 3GB) take