Well, we have our first Java 1.5 patch... Now that we have had a week
or two to digest the comments, do we want to reopen the discussion?
Chuck Williams (JIRA) wrote:
[ http://issues.apache.org/jira/browse/LUCENE-600?page=all ]
Chuck Williams updated LUCENE-600:
----------------------------------
Attachment: ParallelWriter.patch
Patch to create and integrate ParallelWriter, Writable and TestParallelWriter
-- also modifies build to use java 1.5.
ParallelWriter companion to ParallelReader
------------------------------------------
Key: LUCENE-600
URL: http://issues.apache.org/jira/browse/LUCENE-600
Project: Lucene - Java
Type: Improvement
Components: Index
Versions: 2.1
Reporter: Chuck Williams
Attachments: ParallelWriter.patch
A new class ParallelWriter is provided that serves as a companion to
ParallelReader. ParallelWriter meets all of the doc-id synchronization
requirements of ParallelReader, subject to:
1. ParallelWriter.addDocument() is synchronized, which might have an
adverse effect on performance. The writes to the sub-indexes are, however,
done in parallel.
2. The application must ensure that the ParallelReader is never reopened
inside ParallelWriter.addDocument(), else it might find the sub-indexes out of
sync.
3. The application must deal with recovery from
ParallelWriter.addDocument() exceptions. Recovery must restore the
synchronization of doc-ids, e.g. by deleting any trailing document(s) in one
sub-index that were not successfully added to all sub-indexes, and then
optimizing all sub-indexes.
A new interface, Writable, is provided to abstract IndexWriter and
ParallelWriter. This is in the same spirit as the existing Searchable and
Fieldable classes.
This implementation uses java 1.5. The patch applies against today's svn head.
All tests pass, including the new TestParallelWriter.
--
Grant Ingersoll
Sr. Software Engineer
Center for Natural Language Processing
Syracuse University
School of Information Studies
335 Hinds Hall
Syracuse, NY 13244
http://www.cnlp.org
Voice: 315-443-5484
Fax: 315-443-6886
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
