Re: Closing IndexWriter can be very slow on large indexes

2011-08-01 Thread Michael McCandless
On Mon, Aug 1, 2011 at 8:04 AM, Simon Willnauer wrote: > On Mon, Aug 1, 2011 at 12:57 AM, kiwi clive wrote: >> Hi Mike, >> >> The problem was due to close().  A shutdown was calling close() which seems >> to cause lucene to perform a merge. For a busy very large index (with lots >> of deletes a

Re: Closing IndexWriter can be very slow on large indexes

2011-08-01 Thread Simon Willnauer
flushed segment(s). simon > > Clive > > > > - Original Message - > From: Michael McCandless > To: java-user@lucene.apache.org > Cc: > Sent: Tuesday, July 26, 2011 5:30 PM > Subject: Re: Closing IndexWriter can be very slow on large indexes > > Which me

Re: Closing IndexWriter can be very slow on large indexes

2011-07-31 Thread kiwi clive
this appears to close the index without performing the merge. At least that is my understanding of things ! Clive - Original Message - From: Michael McCandless To: java-user@lucene.apache.org Cc: Sent: Tuesday, July 26, 2011 5:30 PM Subject: Re: Closing IndexWriter can be very slow

Re: Closing IndexWriter can be very slow on large indexes

2011-07-26 Thread Michael McCandless
Which method (abort or close) do you see taking so much time? It's odd, because IW.abort should quickly stop any running BG merges. Can you get a dump of the thread stacks during this long abort/close and post that back? Can't answer if Lucene 3.x will improve this situation until we find the so

Closing IndexWriter can be very slow on large indexes

2011-07-26 Thread Chris Bamford
Hi I think I must be doing something wrong, but not sure what. I have some long running indexing code which sometimes needs to be shutdown in a hurry. To achieve this, I set a shutdown flag which causes it to break from the loop and call first abort() and then close(). The problem is that w