Dear Doug,
thanks for your message.
On Fri, Jun 03, 2005 at 09:37:01AM -0700, Doug Cutting wrote:
> Sebastian Marius Kirsch wrote:
> >I took up your suggestion to use a ParallelReader for adding more
> >fields to existing documents. I now have two indexes with the same
> >number of documents, but different fields.
> Does search work using the ParalleReader?
I have to admit that I didn't test *that* yet, sorry. I thought that
merging the index into another one would be a good test anyway.
> >One field is duplicated (the id field.)
> Why is this duplicated? Just curious. That shouldn't cause a problem.
Just as a precaution, so that I can tell afterwards whether two
indexes are in sync or not. (Iterate over the documents in both
indexes and check whether the id fields match.)
> Why are you merging? Why not just search using the ParallelReader?
> Again, just curious. This should work.
Interoperability. That way, I can later hand the index over to another
application that knows nothing about parallel indexes, and don't have
to make sure that this other application combines the indexes the
right way.
(Oh, and I can use Luke to check the index for plausibility. That's an
important point for me.)
> This could be a bug. I have not tested merging with a ParallelReader.
> Can you please try to adding a test case to TestParallelReader that
> demonstrates this?
I have attached the diff for the test case, and the output of the test
run.
I have played around with the code for a couple of hours, but cannot
find a fix for this. If I change ParallelTermPosition.seek(TermEnum
termEnum) to check for termEnum.term() being null, and then hand this
null over to the correct IndexReader, instead of calling
.seek(termEnum.term()), then I get a different
error. (NullPointerException in
org.apache.lucene.index.TermInfosReader.get(TermInfosReader.java:140).)
Apparently, TermPositions are not made for seeking to a null.
On the other hand, I don't know where the null is coming from in the
first case. It comes from the termEnum of one of the underlying
IndexReaders, and if that's a problem, it should be a problem outside
of ParallelReader too.
I'm confused (in case you couldn't tell that yet.) I'll try to find
out more tomorrow.
Regards, Sebastian
PS: I'm rather new to this whole Java thing; I tried to import lucene
into Eclipse for easier debugging, but failed. If any of the
developers use Eclipse, I'd be grateful for some hints regarding
this. Thanks for bearing with me.
$ svn diff
Index: src/test/org/apache/lucene/index/TestParallelReader.java
===================================================================
--- src/test/org/apache/lucene/index/TestParallelReader.java (revision
179785)
+++ src/test/org/apache/lucene/index/TestParallelReader.java (working copy)
@@ -57,6 +57,13 @@
}
+ public void testMerge() throws Exception {
+ Directory dir = new RAMDirectory();
+ IndexWriter w = new IndexWriter(dir, new StandardAnalyzer(), true);
+ w.addIndexes(new IndexReader[] { ((IndexSearcher)
parallel).getIndexReader() });
+ w.close();
+ }
+
private void queryTest(Query query) throws IOException {
Hits parallelHits = parallel.search(query);
Hits singleHits = single.search(query);
$ ant -Dtestcase=TestParallelReader test
Buildfile: build.xml
[...]
test:
[mkdir] Created dir:
/Users/skirsch/text/lectures/da/thirdparty/lucene-trunk/build/test
[junit] Testsuite: org.apache.lucene.index.TestParallelReader
[junit] Tests run: 2, Failures: 0, Errors: 1, Time elapsed: 1.993 sec
[junit] Testcase: testMerge(org.apache.lucene.index.TestParallelReader):
Caused an ERROR
[junit] null
[junit] java.lang.NullPointerException
[junit] at
org.apache.lucene.index.ParallelReader$ParallelTermPositions.seek(ParallelReader.java:318)
[junit] at
org.apache.lucene.index.ParallelReader$ParallelTermDocs.seek(ParallelReader.java:294)
[junit] at
org.apache.lucene.index.SegmentMerger.appendPostings(SegmentMerger.java:325)
[junit] at
org.apache.lucene.index.SegmentMerger.mergeTermInfo(SegmentMerger.java:296)
[junit] at
org.apache.lucene.index.SegmentMerger.mergeTermInfos(SegmentMerger.java:270)
[junit] at
org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:234)
[junit] at
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:96)
[junit] at
org.apache.lucene.index.IndexWriter.addIndexes(IndexWriter.java:596)
[junit] at
org.apache.lucene.index.TestParallelReader.testMerge(TestParallelReader.java:63)
[junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit] at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
[junit] at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit] Test org.apache.lucene.index.TestParallelReader FAILED
BUILD FAILED
/Users/skirsch/text/lectures/da/thirdparty/lucene-trunk/common-build.xml:188:
Tests failed!
Total time: 16 seconds
$
--
Sebastian Kirsch <[EMAIL PROTECTED]> [http://www.sebastian-kirsch.org/]
NOTE: New email address! Please update your address book.
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
