[
https://issues.apache.org/jira/browse/LUCENE-847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael McCandless updated LUCENE-847:
--------------------------------------
Attachment: LUCENE-847.take3.patch
OK I started from the original patch and made the changes described
below.
This is still a work in progress, but I think I think the new
stateless approach works very well.
All unit tests pass (one assert had to be changed in
TestAddIndexesNoOptimize).
I created a ConcurrentMergePolicyWrapper along with this (I'll post
patch to LUCENE-870).
I've also included the two merge policies from LUCENE-845 (still
defaulting to LogDocMergePolicy).
Here are the changes:
- Renamed merge -> maybeMerge
- Changed the API to be "stateless" meaning the merge policy is no
longer responsible for running the merges itself. Instead, it
quickly returns the specification, which describes which merges
are needed, back to the writer and the writer then runs them. I
also changed MergeSpecification to contain a list of OneMerge
instances.
- Removed IndexMerger interface (just use IndexWriter instead)
- Put isOptimized() logic into LogMergePolicy: on thinking about
this more (and seeing response to a thread on java-dev), I now
agree with Steve that this logically belongs in LogMergePolicy
because each MergePolicy is free to define just what it considers
"optimized" to mean. Then I removed the MergePolicyBase.
- Un-deprecated {get/set}{UseCompoundFile,MergeFactor,MaxMergeDocs}.
But I did leave the static constants deprecated.
- IndexWriter keeps track of which segments are involved in running
merges and throws a MergeException if it's asked to initiate a
merge that involves a segment that's already being merged.
- Fixed LogMergePolicy to return all possible merges (exposes
concurrency).
- Implemented the "merge deletes when commiting the merge" algorithm
that Ning suggested (this is in commitMerge).
- Assert that the merge request is in fact contiguous (at start &
finish of merge) & throw MergeException if not.
- Fixed a number of sneaky concurrency issues so that CMPW would
work. Broke "merge" into mergeInit, mergeMiddle and mergeFinish.
The first & last are carefully sychronized.
- I put copyDirFiles in IW and call this in addIndexesNoOptimize
before committing new segments file: we can't let mergePolicy
leave the index inconsistent.
- I reverted the changes to addIndexes(IndexReader[]): I think the
change here wasn't valid: you can't assume that you can re-create
any IndexReader instance by loading from its directory; I put the
original back for this method.
- the changes to addIndexes I'm not sure are good.
- Fixed LogMergePolicy to return more than 1 merge
- Made CMPW
- Renamed replace -> commitMerge; made it private.
> Factor merge policy out of IndexWriter
> --------------------------------------
>
> Key: LUCENE-847
> URL: https://issues.apache.org/jira/browse/LUCENE-847
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Reporter: Steven Parkes
> Assignee: Steven Parkes
> Attachments: concurrentMerge.patch, LUCENE-847.patch.txt,
> LUCENE-847.patch.txt, LUCENE-847.take3.patch, LUCENE-847.txt
>
>
> If we factor the merge policy out of IndexWriter, we can make it pluggable,
> making it possible for apps to choose a custom merge policy and for easier
> experimenting with merge policy variants.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
