[
https://issues.apache.org/jira/browse/LUCENE-1703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12722057#action_12722057
]
Shai Erera commented on LUCENE-1703:
------------------------------------
{quote}
one thread may call addDocument() (or maybeMerge() to be more to the point)
this thread could result in the SerialMergeScheduler to start merging
(addDocument() won't return until this merge completes)
{quote}
Again, if autoCommit=false, how can this happen? I thought that if
autoCommit=false, no commit happens and therefore no segment merging?
bq. I think it makes sense to expose waitForMerges in IW (instead of
duplicating the code in every merge scheduler). We may be able to then
deprecate CMS.sync?
I guess I'm still not convinced what simplicity would that bring. For Tim's use
case, just two threads, using SMS, that might work. But for the general use
case, or one which uses multiple indexing threads, one of which may call
commit() at some point, another daemon may run optimize(), I dunno, this would
still require syncing all threads around that waitForMerges call, if the intent
is to prevent document additions while merges occur. Therefore this method is
not expected to make my life any easier, except that if I need to report "no
merges are running" or make a decision based on "no merges are running" I
should have one thread call this waitForMerges()
Of course, still playing the devil's advocate, you could call waitForMerges()
which will return immediately b/c there are no merges to do or that are
running, and soon as that happens, a context switch also happens, and an
innocent addDocument will trigger a 50-segments merge, at which point whatever
you thought to do b/c there are no merges, will hit the exact scenario you were
trying to avoid all that time :).
I'll admit though that it's late here (1 AM), and perhaps I'm not seeing this
clearly. And .. I still don't understand how if autocommit=false, and
addDocument/deleteDocument can trigger a merge.
> Add a waitForMerges() method to IndexWriter
> -------------------------------------------
>
> Key: LUCENE-1703
> URL: https://issues.apache.org/jira/browse/LUCENE-1703
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Affects Versions: 2.4
> Reporter: Tim Smith
> Attachments: IndexWriter.java.diff, IndexWriter.java.diff
>
>
> It would be very useful to have a waitForMerges() method on the IndexWriter.
> Right now, the only way i can see to achieve this is to call
> IndexWriter.close()
> ideally, there would be a method on the IndexWriter to wait for merges
> without actually closing the index.
> This would make it so that background merges (or optimize) can be waited for
> without closing the IndexWriter, and then reopening a new IndexWriter
> the close() reopen IndexWriter method can be problematic if the close() fails
> as the write lock won't be released
> this could then result in the following sequence:
> * close() - fails
> * force unlock the write lock (per close() documentation)
> * new IndexWriter() (acquires write lock)
> * finalize() on old IndexWriter releases the write lock
> * Index is now not locked, and another IndexWriter pointing to the same
> directory could be opened
> If you don't force unlock the write lock, opening a new IndexWriter will fail
> until garbage collection calls finalize() the old IndexWriter
> If the waitForMerges() method is available, i would likely never need to
> close() the IndexWriter until right before the process being shutdown, so
> this issue would not occur (worst case scenario, the waitForMerges() fails)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-dev-h...@lucene.apache.org
