[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12526029
]
Ning Li commented on LUCENE-847:
--------------------------------
Comments on optimize():
- In the while loop of optimize(), LogMergePolicy.findMergesForOptimize
returns a merge spec with one merge. If ConcurrentMergeScheduler is used, the
one merge will be started in MergeScheduler.merge() and findMergesForOptimize
will be called again. Before the one merge finishes, findMergesForOptimize will
return the same spec but the one merge is already started. So only one
concurrent merge is possible and the main thread will spin on calling
findMergesForOptimize and attempting to merge.
- One possible solution is to make LogMergePolicy.findMergesForOptimize
return multiple merge candidates. It allows higher level of concurrency. It
also alleviates a bit the problem of main thread spinning. To solve this
problem, maybe we can check if a merge is actually started, then sleep briefly
if not (which means all merges candidates are in conflict)?
A comment on concurrent merge threads:
- One difference between the current approach on concurrent merge and the
patch I posted a while back is that, in the current approach, a MergeThread
object is created and started for every concurrent merge. In my old patch,
maxThreadCount of threads are created and started at the beginning and are used
throughout. Both have pros and cons.
> Factor merge policy out of IndexWriter
> --------------------------------------
>
> Key: LUCENE-847
> URL: https://issues.apache.org/jira/browse/LUCENE-847
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Reporter: Steven Parkes
> Assignee: Steven Parkes
> Fix For: 2.3
>
> Attachments: concurrentMerge.patch, LUCENE-847.patch.txt,
> LUCENE-847.patch.txt, LUCENE-847.take3.patch, LUCENE-847.take4.patch,
> LUCENE-847.take5.patch, LUCENE-847.txt
>
>
> If we factor the merge policy out of IndexWriter, we can make it pluggable,
> making it possible for apps to choose a custom merge policy and for easier
> experimenting with merge policy variants.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
