[
https://issues.apache.org/jira/browse/LUCENE-2095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782715#action_12782715
]
Michael McCandless commented on LUCENE-2095:
--------------------------------------------
I think the main cost is often fsync'ing the new files.
But I agree we should simply lock so only one thread can be in commit
at once. Concurrency inside commit (when fscyning) seems silly (it
used to be slightly more interesting when we had autoCommit=true).
We shouldn't use IW's lock. First, it's overkill (doing so would
unnecessarily block other sync'd ops from running). Second, it leads
to deadlock, at least in CMS (if it waits because too many merges are
running). I'll use a separate lock.
I'm not adding locking around the separate calls to prepareCommit then
commit. An app must ensure these two calls are always balanced.
Making flush unsynchronized would be great but I think wouldn't be
that big a gain in practice, unless we could make it truly unsync'd
even with adding new docs. Ie, if while we are flushing the last
segment, we can accept adding/deleting docs into a new ram buffer in
DW, that'd be quite interesting. But that's a big change!
> Document not guaranteed to be found after write and commit
> ----------------------------------------------------------
>
> Key: LUCENE-2095
> URL: https://issues.apache.org/jira/browse/LUCENE-2095
> Project: Lucene - Java
> Issue Type: Bug
> Components: Index
> Affects Versions: 2.4.1, 2.9.1
> Environment: Linux 64bit
> Reporter: Sanne Grinovero
> Assignee: Michael McCandless
> Fix For: 2.9.2, 3.0.1, 3.1
>
> Attachments: lucene-stresstest.patch
>
>
> after same email on developer list:
> "I developed a stress test to assert that a new document containing a
> specific term "X" is always found after a commit on the IndexWriter.
> This works most of the time, but it fails under load in rare occasions.
> I'm testing with 40 Threads, both with a SerialMergeScheduler and a
> ConcurrentMergeScheduler, all sharing a common IndexWriter.
> Attached testcase is using a RAMDirectory only, but I verified a
> FSDirectory behaves in the same way so I don't believe it's the
> Directory implementation or the MergeScheduler.
> This test is slow, so I don't consider it a functional or unit test.
> It might give false positives: it doesn't always fail, sorry I
> couldn't find out how to make it more likely to happen, besides
> scheduling it to run for a longer time."
> I tested this to affect versions 2.4.1 and 2.9.1;
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-dev-h...@lucene.apache.org
