[ http://issues.apache.org/jira/browse/LUCENE-528?page=comments#action_12443940 ] Yonik Seeley commented on LUCENE-528: -------------------------------------
> Algorithm looks good? Yep, looks good. A possible optimization *far* down the road for segments that need copying could be to "steal" the segments if rename/mv is supported (if they are both of the same Directory type, on the same partition for FSDirectories, etc), and if the caller indicated it was OK. Rather than try to detect when it's safe, there could be an attachSegments(Directory[]) call. > Optimization for IndexWriter.addIndexes() > ----------------------------------------- > > Key: LUCENE-528 > URL: http://issues.apache.org/jira/browse/LUCENE-528 > Project: Lucene - Java > Issue Type: Improvement > Components: Index > Reporter: Steven Tamm > Assigned To: Otis Gospodnetic > Priority: Minor > Attachments: AddIndexes.patch > > > One big performance problem with IndexWriter.addIndexes() is that it has to > optimize the index both before and after adding the segments. When you have > a very large index, to which you are adding batches of small updates, these > calls to optimize make using addIndexes() impossible. It makes parallel > updates very frustrating. > Here is an optimized function that helps out by calling mergeSegments only on > the newly added documents. It will try to avoid calling mergeSegments until > the end, unless you're adding a lot of documents at once. > I also have an extensive unit test that verifies that this function works > correctly if people are interested. I gave it a different name because it > has very different performance characteristics which can make querying take > longer. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
