On 1/14/07, Michael McCandless <[EMAIL PROTECTED]> wrote:
* The "support deleteDocuments in IndexWriter" (LUCENE-565) feature
could have a more efficient implementation (just like Solr) when
autoCommit is false, because deletes don't need to be flushed
until commit() is called. Whereas, now, they must be aggressively
flushed on each checkpoint.
The idea of adding "explicit commits" is good! And in time - I was
just about to submit a latest patch for LUCENE-565. With this feature,
the frequency of reader open/close on large old segments could be
reduced when autoCommit is false.
Based on your proposal, however, an application wouldn't be able to
delete any documents that have not been committed since a reader
always opens a snapshot (segments_N), but not a checkpoint
(segmentsx_N). This functionality will be supported by LUCENE-565, but
I wonder if it should be supported in general. So maybe a reader can
open the latest checkpoint for modification, but only snapshots for
search...
If a reader can only open snapshots both for search and for
modification, I think another change is needed besides the ones
listed: assume the latest snapshot is segments_5 and the latest
checkpoint is segmentsx_7 with 2 new segments, then a reader opens
snapshot segments_5, performs a few deletes and writes a new
checkpoint segmentsx_8. The summary file segmentsx_8 should include
the 2 new segments which are in segmentsx_7 but not in segments_5.
Such segments to include are easily identifiable only if they are not
merged with segments in the latest snapshot... All these won't be
necessary if a reader always opens the latest checkpoint for
modification, which will also support deletion of non-committed
documents.
Lastly, hopefully the term "transaction" won't cause any confusion
since this "explicit commit" is much simpler than database transaction
where a database can guarantee the ACID properties for each of
multiple concurrent transactions.
Cheers,
Ning
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
