[
https://issues.apache.org/jira/browse/LUCENE-3348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13073678#comment-13073678
]
Simon Willnauer commented on LUCENE-3348:
-----------------------------------------
mike I can not reproduce this failure.. what exactly is failing there? maybe
you can put the output in a text file and attache it?
Regarding the latest patch, I think we can call
DWFlushControl#addFlushableState() from DWFlushControl#markForFullFlush() and
use a global list to collect the DWPT for the full flush.
I think we should move the getAndLock call into DWFlushControl something like
DWFlushControl#obtainAndLock(), this would allow us to make the check and the
DWFlushControl#addFlushableState() method private to DWFC. Further we can also
simplify the deleteQueue check a little since we already obtained a ThreadState
we don't need to unlock the state again after calling addFlushableState(),
something like this:
{code}
ThreadState obtainAndLock() {
final ThreadState perThread = perThreadPool.getAndLock(Thread
.currentThread(), documentsWriter);
if (perThread.isActive()
&& perThread.perThread.deleteQueue != documentsWriter.deleteQueue) {
// There is a flush-all in process and this DWPT is
// now stale -- enroll it for flush and try for
// another DWPT:
addFlushableState(perThread);
}
return perThread;
}
{code}
Eventually we are spending too much time in full flush since we lock all
ThreadStates at least once while some indexing threads might have already
helped out with swapping out DWPT instances. I think we can collect already
swapped out ThreadStates during a full flush and only check the ones that have
not been processed?
> IndexWriter applies wrong deletes during concurrent flush-all
> -------------------------------------------------------------
>
> Key: LUCENE-3348
> URL: https://issues.apache.org/jira/browse/LUCENE-3348
> Project: Lucene - Java
> Issue Type: Bug
> Components: core/index
> Reporter: Michael McCandless
> Assignee: Michael McCandless
> Fix For: 3.4, 4.0
>
> Attachments: LUCENE-3348.patch, LUCENE-3348.patch, LUCENE-3348.patch
>
>
> Yonik uncovered this with the TestRealTimeGet test: if a flush-all is
> underway, it is possible for an incoming update to pick a DWPT that is
> stale, ie, not yet pulled/marked for flushing, yet the DW has cutover
> to a new deletes queue. If this happens, and the deleted term was
> also updated in one of the non-stale DWPTs, then the wrong document is
> deleted and the test fails by detecting the wrong value.
> There's a 2nd failure mode that I haven't figured out yet, whereby 2
> docs are returned when searching by id (there should only ever be 1
> doc since the test uses updateDocument which is atomic wrt
> commit/reopen).
> Yonik verified the test passes pre-DWPT, so my guess is (but I
> have yet to verify) this test also passes on 3.x. I'll backport
> the test to 3.x to be sure.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
