Hi,
I have tryed to get all the tokens from a TokenStream in the same way as I was
doing in the 3.x version of Lucene, but now (at least with WhitespaceTokenizer)
I get an exception:
Exception in thread main java.lang.ArrayIndexOutOfBoundsException: -1
at
I didn't read your code, but do you have the reset that is now mandatory
and throws AIOOBE if not present?
-- Jack Krupansky
-Original Message-
From: andi rexha
Sent: Monday, April 15, 2013 10:21 AM
To: java-user@lucene.apache.org
Subject: WhitespaceTokenizer, incrementToke
://www.thetaphi.de
eMail: u...@thetaphi.de
-Original Message-
From: Jack Krupansky [mailto:j...@basetechnology.com]
Sent: Monday, April 15, 2013 4:25 PM
To: java-user@lucene.apache.org
Subject: Re: WhitespaceTokenizer, incrementToke()
ArrayOutOfBoundException
I didn't read your code
@lucene.apache.org
Subject: RE: WhitespaceTokenizer, incrementToke() ArrayOutOfBoundException
Hi,
It was always mandatory! In Lucene 2.x/3.x some Tokenizers just returned
bogus, undefined stuff if not correctly reset before usage, especially when
Tokenizers are reused by the Analyzer, which is now
