I am using a custom analyzer upgrading from Lucene 2.x.
Need to get more familiar with Lucene 3.0 behavior.
I think this is one of the upgrade pitfalls.
Thanks for the help!
Chris
Uwe Schindler wrote:
This problem occurs, if you have a Tokenizer or TokenFilter that produces new
tokens but does not call clearAttributes(). What TokenStreams do you use in
your analyzer?
If you not call clearAttributes() (see javadocs of Tokenizer!) whenever you
produce new tokens (in any type of TokenStream), the positionIncrement is never
reset back to 1 (and when in adiition the analyzer reuses the tokenstream, it
is really never reset even across multiple documents). So whenever a stopword
occurs it get larger...
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: [email protected]
-----Original Message-----
From: Chris Lu [mailto:[email protected]]
Sent: Thursday, January 14, 2010 10:41 PM
To: [email protected]
Subject: Re: IllegalArgumentException when IndexWriter.addDocument
Notes: I am using Lucene 3.0
Seems a integer overflow problem?
java.lang.IllegalArgumentException: Increment must be zero or
greater:
-472893952
at
org.apache.lucene.analysis.tokenattributes.PositionIncrementAttributeIm
pl.setPositionIncrement(PositionIncrementAttributeImpl.java:58)
at
org.apache.lucene.analysis.StopFilter.incrementToken(StopFilter.java:25
6)
at
org.apache.lucene.index.DocInverterPerField.processFields(DocInverterPe
rField.java:188)
at
org.apache.lucene.index.DocFieldProcessorPerThread.processDocument(DocF
ieldProcessorPerThread.java:246)
at
org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.
java:774)
at
org.apache.lucene.index.DocumentsWriter.addDocument(DocumentsWriter.jav
a:752)
at
org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1931)
at
org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1905)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
