Oops, this fell off lucene-dev... re-adding it. -Yonik On 5/16/05, Yonik Seeley <[EMAIL PROTECTED]> wrote: > In general, people shouldn't be relying on GC to clean up non-memory > resources. If you don't explicitly call close() on an IndexReader > when you are done with it, you are likely to run out of file > descriptors at some point. Finalizers are a bad solution to clean up > descriptors. > > With non-shared readers, things would continue to work like normal. > The constructor could set all the counts to 1, and close() could > decrement and really do a close if it hits 0. > > IndexReader.reopen() could then just increment the count of any reader > it wished to share. > > -Yonik > > On 5/16/05, Robert Engels <[EMAIL PROTECTED]> wrote: > > Agreed, but ref counting can be a tricky business in a GC environment. It > > may be better to use weak references, and let Java collect them when needed. > > ? > > > > -----Original Message----- > > From: Yonik Seeley [mailto:[EMAIL PROTECTED] > > Sent: Monday, May 16, 2005 1:11 PM > > To: [EMAIL PROTECTED] > > Subject: Re: optimized reopen? > > > > If the SegmentReaders can be shared, it looks like the only thing that > > would need changing is a reference count on all IndexReaders to avoid > > closing of a shared reader. > > > > -Yonik > > > > On 5/14/05, Robert Engels <[EMAIL PROTECTED]> wrote: > > > It appears so, as the SegmentReaders are read-only. > > > > > > -----Original Message----- > > > From: Yonik Seeley [mailto:[EMAIL PROTECTED] > > > Sent: Saturday, May 14, 2005 2:43 PM > > > To: [EMAIL PROTECTED] > > > Subject: Re: optimized reopen? > > > > > > Our uses involve overlapping IndexReaders... one searcher is "warming > > > up" while the other one is finishing queries. Even after the new > > > searcher switches to be the default one, the old one must be kept > > > around until all queries using it finish. > > > > > > So can the segment readers be shared by multiple active index readers? > > > > > > -Yonik > > > > > > On 5/11/05, Robert Engels <[EMAIL PROTECTED]> wrote: > > > > I did an implementation of IndexReader.reopen() in my modified Lucene > > lib. > > > > > > > > It is over 200% faster than closing and opening the index. > > > > > > > > I just made the semantics of IndexReader.reopen() be that the original > > > > IndexReader is now closed, and can no longer be used (since I cannot > > > close() > > > > the original, otherwise the SegmentReaders will close, so I close the > > > unused > > > > SegmentReaders directly). In order to make this work I also needed to > > make > > > > commit() public, since the commit() needs to be performed prior to the > > > > segment merging, and I needed to add a package method to MultiReader to > > > get > > > > the underlying subReaders. > > > > > > > > Although this may not be useful to many, in our case since we use a > > single > > > > shared reader and a single exclusive writer in a server process (with > > > > external locking, so we can guarantee than when the IndexReader is > > > "rolled", > > > > there are no references to the previous reader. > > > > > > > > Robert > > > > > > > > -----Original Message----- > > > > From: Yonik Seeley [mailto:[EMAIL PROTECTED] > > > > Sent: Wednesday, May 11, 2005 4:50 PM > > > > To: [email protected]; [EMAIL PROTECTED] > > > > Subject: Re: optimized reopen? > > > > > > > > Things are cached using an IndexReader as the key, so you would have > > > > to be careful not to break the current behaviour (that an > > > > IndexReader's view of an index doesn't change - deletes from that > > > > specific reader aside). > > > > > > > > Maybe you could invoke reopen() on an existing IndexReader and it > > > > would return a new IndexReader that shares sub-readers that haven't > > > > changed? But I don't think that sub-readers can be shared like this > > > > right now... the difficulty may lie in deleted-docs. > > > > > > > > -Yonik > > > > > > > > On 5/11/05, Robert Engels <[EMAIL PROTECTED]> wrote: > > > > > Is there any way to optimize the closing & reopening of an Index? > > > > > > > > > > Since the IndexReader.open() opens a MultiReader is there are multiple > > > > > segments, it seems a reopen() method could be implemented, which > > detects > > > > > which segments are the same as the current open index, and then passes > > > > those > > > > > SegementReaders to a new Multireader rather than creating new > > > > SegmentReaders > > > > > for the unmodified segments. > > > > > > > > > > Does this sounds feasible? Would it improve the performance? > > > > > > > > > > Thanks > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
