[
https://issues.apache.org/jira/browse/LUCENE-1703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12722019#action_12722019
]
Shai Erera commented on LUCENE-1703:
------------------------------------
{quote}
MergeScheduler does not provide a sync() method in its abstract class.
Therefore, an implementation could be used that would not derive from or
provide a sync() method
If it did, this would be a more than acceptable solution instead of putting the
method on the IndexWriter. (i potentially want to allow configuration to
specify what MergeScheduler to use, and if sync() isn't specified on the base
class, i can't reliably use it (will have to do instanceof checks, and a new
concurrent based scheduler not derived from ConcurrentMergeScheduler breaks
this)
{quote}
We can add a sync() method to MergeScheduler or you do it on your side. There
are only two MS today: CMS and SMS. If you're going to write your own CMS
variant, then you can have it extend CMS, or define your own interface with a
sync() method, which will delegate that call to whatever MS it wraps. Again,
the way I see it, you know which MS you instantiate, and therefore you should
be able to declare its type and use whatever methods it has. If you want
something configurable, I suggest we move sync() to MS (w/ a default impl of
doing nothing, for back-compat).
What do you think?
bq. ... another thread calling addDocument triggering a merge
This will only happen if you use autoCommit=true, right? You can use
autoCommit=true, and when you call commit(), use the one that waits for merges
(the one that does not exist yet :) ), and sync all your addDoc threads with
that current commit thread (make sure you use a ReadWriteLock, or otherwise all
your addDocs are going to sync on each other).
bq. this will allow blocking adding more documents until background merges are
complete periodically
What's "periodically"? If you're using autoCommit=false, those merges cannot
happen suddenly. There are only 4 methods that can trigger them (I think?) and
you can sync on calling them w/ the rest of the addDoc threads.
Don't get me wrong - I just play the devil's advocate here. Recently, we've had
this thread http://www.nabble.com/ReadOnly-IndexReaders-td23787255.html - a
naive question on adding an ever more naive method to IndexReader - isReadOnly.
If you read it, you'll see that it wasn't accepted at the end, because we've
understood it will just try to cover a design flaw of your system.
Therefore I'm just trying to verify whether this method is absolutely
necessary, and there aren't any "design decisions" we should make in our apps
to better handle that case.
Having said that, I'm not a committer and if one decides it's important enough,
one would take this issue and commit it. I just enjoy the discussion for now
(hope you too :) ).
> Add a waitForMerges() method to IndexWriter
> -------------------------------------------
>
> Key: LUCENE-1703
> URL: https://issues.apache.org/jira/browse/LUCENE-1703
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Affects Versions: 2.4
> Reporter: Tim Smith
> Attachments: IndexWriter.java.diff, IndexWriter.java.diff
>
>
> It would be very useful to have a waitForMerges() method on the IndexWriter.
> Right now, the only way i can see to achieve this is to call
> IndexWriter.close()
> ideally, there would be a method on the IndexWriter to wait for merges
> without actually closing the index.
> This would make it so that background merges (or optimize) can be waited for
> without closing the IndexWriter, and then reopening a new IndexWriter
> the close() reopen IndexWriter method can be problematic if the close() fails
> as the write lock won't be released
> this could then result in the following sequence:
> * close() - fails
> * force unlock the write lock (per close() documentation)
> * new IndexWriter() (acquires write lock)
> * finalize() on old IndexWriter releases the write lock
> * Index is now not locked, and another IndexWriter pointing to the same
> directory could be opened
> If you don't force unlock the write lock, opening a new IndexWriter will fail
> until garbage collection calls finalize() the old IndexWriter
> If the waitForMerges() method is available, i would likely never need to
> close() the IndexWriter until right before the process being shutdown, so
> this issue would not occur (worst case scenario, the waitForMerges() fails)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-dev-h...@lucene.apache.org
