[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12526628
]
Ning Li commented on LUCENE-847:
--------------------------------
> OK, another rev of the patch (take6). I think it's close!
Yes, it's close! :)
> I made one simplification to the approach: IndexWriter now keeps track
> of "pendingMerges" (merges that mergePolicy has declared are necessary
> but have not yet been started), and "runningMerges" (merges currently
> in flight). Then MergeScheduler just asks IndexWriter for the next
> pending merge when it's ready to run it. This also cleaned up how
> cascading works.
I like this simplification.
> * Optimize: optimize is now fully concurrent (it can run multiple
> merges at once, new segments can be flushed during an optimize,
> etc). Optimize will optimize only those segments present when it
> started (newly flushed segments may remain separate).
This semantics does add a bit complexity - segmentsToOptimize,
OneMerge.optimize.
> Good idea! I took exactly this approach in patch I just attached. I
> made a simple change: LogMergePolicy.findMergesForOptimize first
> checks if "normal merging" would want to do merges and returns them if
> so. Since "normal merging" exposes concurrent merges, this gains us
> concurrency for optimize in cases where the index has too many
> segments. I wasn't sure how otherwise to expose concurrency...
Another option is to schedule merges for the newest N segments and the next
newest N segments and the next next... N is the merge factor.
A couple of other things:
- It seems you intended sync() to be part of the MergeScheduler interface?
- IndexWriter.close([true]), abort(): The behaviour should be the same
whether the calling thread is the one that actually gets to do the closing.
Right now, only the thread that actually does the closing waits for the
closing. The others do not wait for the closing.
> Factor merge policy out of IndexWriter
> --------------------------------------
>
> Key: LUCENE-847
> URL: https://issues.apache.org/jira/browse/LUCENE-847
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Reporter: Steven Parkes
> Assignee: Steven Parkes
> Fix For: 2.3
>
> Attachments: concurrentMerge.patch, LUCENE-847.patch.txt,
> LUCENE-847.patch.txt, LUCENE-847.take3.patch, LUCENE-847.take4.patch,
> LUCENE-847.take5.patch, LUCENE-847.take6.patch, LUCENE-847.txt
>
>
> If we factor the merge policy out of IndexWriter, we can make it pluggable,
> making it possible for apps to choose a custom merge policy and for easier
> experimenting with merge policy variants.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
