RE: Search performance with one index vs. many indexes

2005-02-28 Thread Runde, Kevin
Hi All, Sorry about that please disregard that last email. I must not be fully awake yet. Sorry, Kevin Runde -Original Message- From: Runde, Kevin [mailto:[EMAIL PROTECTED] Sent: Monday, February 28, 2005 7:34 AM To: Lucene Users List Subject: RE: Search performance with one index vs

RE: Search performance with one index vs. many indexes

2005-02-28 Thread Runde, Kevin
Follow Up to the article from Friday -Original Message- From: Morus Walter [mailto:[EMAIL PROTECTED] Sent: Monday, February 28, 2005 1:30 AM To: Lucene Users List Subject: Re: Search performance with one index vs. many indexes Jochen Franke writes: > Topic: Search performance w

Re: Search performance with one index vs. many indexes

2005-02-27 Thread Morus Walter
Jochen Franke writes: > Topic: Search performance with large numbers of indexes vs. one large index > > > My questions are: > > - Is the size of the "wordlist" the problem? > - Would we be a lot faster, when we have a smaller number > of files per index? sure. Look: Index lookup of a word is O

Re: Search Performance

2005-02-19 Thread sergiu gordea
tests. Best, Sergiu Michael -Original Message- From: David Townsend [mailto:[EMAIL PROTECTED] Sent: Friday, February 18, 2005 11:50 AM To: Lucene Users List Subject: RE: Search Performance IndexSearchers are thread safe, so you can use the same object on multiple requests. If the ind

Re: Search Performance

2005-02-18 Thread Otis Gospodnetic
Yes, until it's cleaned up, and as soon as the last client is done with Hits, the originating IndexSearcher is ready for cleanup if nobody else is holding references to it. You can close it explicityly, as you are doing, too, no harm. Otis --- Chris Lamprecht <[EMAIL PROTECTED]> wrote: > Wouldn

Re: Search Performance

2005-02-18 Thread Chris Lamprecht
Wouldn't this leave open file handles? I had a problem where there were lots of open file handles for deleted index files, because the old searchers were not being closed. On Fri, 18 Feb 2005 13:41:37 -0800 (PST), Otis Gospodnetic <[EMAIL PROTECTED]> wrote: > Or you could just open a new IndexSe

Re: Search Performance

2005-02-18 Thread David Spencer
Users List; Chris Lamprecht Subject: Re: Search Performance Or you could just open a new IndexSearcher, forget the old one, and have GC collect it when everyone is done with it. Otis --- Chris Lamprecht <[EMAIL PROTECTED]> wrote: I should have mentioned, the reason for not doing this the o

RE: Search Performance

2005-02-18 Thread Michael Celona
Just tried that... works like a charm... thanks... Michael -Original Message- From: Otis Gospodnetic [mailto:[EMAIL PROTECTED] Sent: Friday, February 18, 2005 4:42 PM To: Lucene Users List; Chris Lamprecht Subject: Re: Search Performance Or you could just open a new IndexSearcher

Re: Search Performance

2005-02-18 Thread Otis Gospodnetic
Or you could just open a new IndexSearcher, forget the old one, and have GC collect it when everyone is done with it. Otis --- Chris Lamprecht <[EMAIL PROTECTED]> wrote: > I should have mentioned, the reason for not doing this the obvious, > simple way (just close the Searcher and reopen it if a

RE: Search Performance

2005-02-18 Thread Michael Celona
Thanks... I am seeing this problem right now Has anyone implemented a better solution...? Michael -Original Message- From: Chris Lamprecht [mailto:[EMAIL PROTECTED] Sent: Friday, February 18, 2005 4:14 PM To: Lucene Users List Subject: Re: Search Performance I should have mentioned

Re: Search Performance

2005-02-18 Thread Chris Lamprecht
I should have mentioned, the reason for not doing this the obvious, simple way (just close the Searcher and reopen it if a new version is available) is because some threads could be in the middle of iterating through the search Hits. If you close the Searcher they get a Bad file descriptor IOExcep

Re: Search Performance

2005-02-18 Thread Chris Lamprecht
nal Message- > From: David Townsend [mailto:[EMAIL PROTECTED] > Sent: Friday, February 18, 2005 11:50 AM > To: Lucene Users List > Subject: RE: Search Performance > > IndexSearchers are thread safe, so you can use the same object on multiple > requests. If the index is stat

RE: Search Performance

2005-02-18 Thread Michael Celona
I am using the highlighter... does this matter -Original Message- From: David Spencer [mailto:[EMAIL PROTECTED] Sent: Friday, February 18, 2005 2:05 PM To: Lucene Users List Subject: Re: Search Performance Are you using the highlighter or doing anything non-trivial in displaying the

Re: Search Performance

2005-02-18 Thread David Spencer
Are you using the highlighter or doing anything non-trivial in displaying the results? Are the pages being compressed (mod_gzip or some servlet equivalent)? This definitely helps, though to see the effect you may have to make sure your simulated users are "remote". Also consider caching search

Re: Search Performance

2005-02-18 Thread David Spencer
Noone has mentioned JVM options yet. [a] -server [b] -XX:CompileThreshold=1000 [c] Raise the -Xms value if you haven't done so (-Xms...) I think by default the VM runs with "-client" but -server makes more sense for web containers (Tomcat etc). [b] tells the hotspot compiler to compile methods soo

RE: Search Performance

2005-02-18 Thread Michael Celona
My index is changing in real time constantly... in this case I guess this will not work for me any suggestions... Michael -Original Message- From: David Townsend [mailto:[EMAIL PROTECTED] Sent: Friday, February 18, 2005 11:50 AM To: Lucene Users List Subject: RE: Search Performance

RE: Search Performance

2005-02-18 Thread David Townsend
t: 18 February 2005 16:15 To: Lucene Users List Subject: Re: Search Performance Try a singleton pattern or an static field. Stefan Michael Celona wrote: >I am creating new IndexSearchers... how do I cache my IndexSearcher... > >Michael > >-Original Message- >From: David

Re: Search Performance

2005-02-18 Thread Stefan Groschupf
: RE: Search Performance Are you creating new IndexSearchers or IndexReaders on each search? Caching your IndexSearchers has a dramatic effect on speed. David Townsend -Original Message- From: Michael Celona [mailto:[EMAIL PROTECTED] Sent: 18 February 2005 15:55 To: Lucene Users List

RE: Search Performance

2005-02-18 Thread Michael Celona
I am creating new IndexSearchers... how do I cache my IndexSearcher... Michael -Original Message- From: David Townsend [mailto:[EMAIL PROTECTED] Sent: Friday, February 18, 2005 11:00 AM To: Lucene Users List Subject: RE: Search Performance Are you creating new IndexSearchers or

RE: Search Performance

2005-02-18 Thread David Townsend
Are you creating new IndexSearchers or IndexReaders on each search? Caching your IndexSearchers has a dramatic effect on speed. David Townsend -Original Message- From: Michael Celona [mailto:[EMAIL PROTECTED] Sent: 18 February 2005 15:55 To: Lucene Users List Subject: Search Performance