[ https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless updated LUCENE-847: -------------------------------------- Attachment: LUCENE-847.take3.patch OK I started from the original patch and made the changes described below. This is still a work in progress, but I think I think the new stateless approach works very well. All unit tests pass (one assert had to be changed in TestAddIndexesNoOptimize). I created a ConcurrentMergePolicyWrapper along with this (I'll post patch to LUCENE-870). I've also included the two merge policies from LUCENE-845 (still defaulting to LogDocMergePolicy). Here are the changes: - Renamed merge -> maybeMerge - Changed the API to be "stateless" meaning the merge policy is no longer responsible for running the merges itself. Instead, it quickly returns the specification, which describes which merges are needed, back to the writer and the writer then runs them. I also changed MergeSpecification to contain a list of OneMerge instances. - Removed IndexMerger interface (just use IndexWriter instead) - Put isOptimized() logic into LogMergePolicy: on thinking about this more (and seeing response to a thread on java-dev), I now agree with Steve that this logically belongs in LogMergePolicy because each MergePolicy is free to define just what it considers "optimized" to mean. Then I removed the MergePolicyBase. - Un-deprecated {get/set}{UseCompoundFile,MergeFactor,MaxMergeDocs}. But I did leave the static constants deprecated. - IndexWriter keeps track of which segments are involved in running merges and throws a MergeException if it's asked to initiate a merge that involves a segment that's already being merged. - Fixed LogMergePolicy to return all possible merges (exposes concurrency). - Implemented the "merge deletes when commiting the merge" algorithm that Ning suggested (this is in commitMerge). - Assert that the merge request is in fact contiguous (at start & finish of merge) & throw MergeException if not. - Fixed a number of sneaky concurrency issues so that CMPW would work. Broke "merge" into mergeInit, mergeMiddle and mergeFinish. The first & last are carefully sychronized. - I put copyDirFiles in IW and call this in addIndexesNoOptimize before committing new segments file: we can't let mergePolicy leave the index inconsistent. - I reverted the changes to addIndexes(IndexReader[]): I think the change here wasn't valid: you can't assume that you can re-create any IndexReader instance by loading from its directory; I put the original back for this method. - the changes to addIndexes I'm not sure are good. - Fixed LogMergePolicy to return more than 1 merge - Made CMPW - Renamed replace -> commitMerge; made it private. > Factor merge policy out of IndexWriter > -------------------------------------- > > Key: LUCENE-847 > URL: https://issues.apache.org/jira/browse/LUCENE-847 > Project: Lucene - Java > Issue Type: Improvement > Components: Index > Reporter: Steven Parkes > Assignee: Steven Parkes > Attachments: concurrentMerge.patch, LUCENE-847.patch.txt, > LUCENE-847.patch.txt, LUCENE-847.take3.patch, LUCENE-847.txt > > > If we factor the merge policy out of IndexWriter, we can make it pluggable, > making it possible for apps to choose a custom merge policy and for easier > experimenting with merge policy variants. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]