Re: [Fwd: Re: Lucene 3.0 Search Performance Stats]

2010-03-22 Thread Jamie
Hi Suman Here are some of the things we did: - cache searcher/s - cache indexreader/s - all users use the same searchers - perform a background search when apps starts to warm up search engine - use numerics where necessary - use shorter dates (i.e. do you really need a granularity of up to the

[Fwd: Re: Lucene 3.0 Search Performance Stats]

2010-03-22 Thread suman . holani
Hi , I am also using range based searches for dates .I am converting time to utc based seconds format and storing them to indexes. and then running range queries Is there something needed to make it more efficient. Thanks, Suman > Very nice! Thanks for sharing :) > > Mike > > On Fri, Ma

Re: Lucene 3.0 Search Performance Stats

2010-03-22 Thread Michael McCandless
Looks like the bulk of your RAM usage is from the 370K index terms in your terms dict... The flex branch (once it lands) should substantially reduce that... Mike On Mon, Mar 22, 2010 at 8:35 AM, Jamie wrote: > Hi Everyone > > The stats I sent through earlier were erroneous due to fact the date

Re: Lucene 3.0 Search Performance Stats

2010-03-22 Thread Jamie
Hi Everyone The stats I sent through earlier were erroneous due to fact the date range query selected fewer records than stated. The correct stats are: Lucene 3.0 Stats: Search conducted using Lucene's Realtime search feature (writer.getReader() for each search) Analyzer: Russian Analyzer

RE: Lucene 3.0 Search Performance Stats

2010-03-20 Thread Uwe Schindler
Hi Jamie, thanks for reporting back the numbers about your usage of NumericField and NumericRangeQuery! I am glad to hear about it. > Sure. As soon as I get access to the server again, I'll get the mem > stats for you. I will say that Lucene was consuming a large amount of > memory before we mov

Re: Lucene 3.0 Search Performance Stats

2010-03-20 Thread Jamie
Hi Monique Sure. As soon as I get access to the server again, I'll get the mem stats for you. I will say that Lucene was consuming a large amount of memory before we moved over to using Numerics. The reason for this is that we were encoding dates as strings. Our date time strings were unique,

Re: Lucene 3.0 Search Performance Stats

2010-03-19 Thread Monique Monteiro
Hi Jamie, could you please tell us how much memory does your application consume with Lucene? I'm asking it because we are having memory consumption problems with a 32GB index and 1.5GB od RAM allocated to our web application. At the momento, we use textual search. Thanks in advance, Monique On

Re: Lucene 3.0 Search Performance Stats

2010-03-19 Thread Michael McCandless
Very nice! Thanks for sharing :) Mike On Fri, Mar 19, 2010 at 6:53 AM, Jamie wrote: > I forgot to point out, this is a search using the Lucene realtime search > feature. We get the reader from indexwriter.getReader() for each search. > > On 2010/03/19 01:49 PM, Jamie wrote: >> >> Hi Guys >> >>

Re: Lucene 3.0 Search Performance Stats

2010-03-19 Thread Jamie
I forgot to point out, this is a search using the Lucene realtime search feature. We get the reader from indexwriter.getReader() for each search. On 2010/03/19 01:49 PM, Jamie wrote: Hi Guys I just wanted to congratulate the Lucene guys for a fine job on 3.0!! Since we switched our indexes to