[
https://issues.apache.org/jira/browse/LUCENE-870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steven Parkes updated LUCENE-870:
---------------------------------
Attachment: CMP.patch.txt
Mike expressed interest in pursuing this with an alternative strargey, so I
thought I'd give a "work in progress" snapshot of the way I'd be going.
This code doesn't work, but it has some ideas, so it's only of interest to
people who really want to make suggestions on how to do the parallelization.
Overall, the idea of generating new threads for non-conflicting primitive
merges seems okay. Need to make sure you don't overload the i/o system and that
throttling code isn't in there.
A couple of things things off the top that I haven't worked through yet:
My current thinking is that when you are not going to do a merge serially, you
need to copy the segmentInfo objects that you will be using. It may be possible
to do this with a lock, but that gets harry. Ther'es also state in the
SegmentInfo objescts themselves, like docStoreIsCompoundFile that can get
changed on the fly.
flushDocStore is challenging to parallelize. It's synchronized now, but you
probably would rather it not be? It's complicated by the fact that doc stores
are shared by multiple segments and so non-conflicting merges may stll share
doc stores.
> add concurrent merge policy
> ---------------------------
>
> Key: LUCENE-870
> URL: https://issues.apache.org/jira/browse/LUCENE-870
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Index
> Reporter: Steven Parkes
> Assignee: Steven Parkes
> Attachments: CMP.patch.txt, concurrentMerge.patch
>
>
> Provide the ability to handle merges in one or more concurrent threads, i.e.,
> concurrent with other IndexWriter operations.
> I'm factoring the code from LUCENE-847 for this.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
