See Tokenizer.java for the state machine logic. In general you should not have to do anything if the tokenizer is well-behaved (e.g. close calls super.close() and so on).
On Tue, Jan 7, 2014 at 2:50 PM, Benson Margulies <[email protected]> wrote: > In 4.6.0, > org.apache.lucene.analysis.BaseTokenStreamTestCase#checkResetException > > fails if incrementToken fails to throw if there's a missing reset. > > How am I supposed to organize this in a Tokenizer? A quick look at > CharTokenizer did not reveal any code for the purpose. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
