Hello,
Noticed the following issue during our recent code migration to LUCENE_40. The test below will fail with an ArrayIndexOutOfBoundsException -1. It will pass only if tokenizer.reset() is called before incrementing the tokens. @Test public void whitespaceTokTest() throws IOException { String text = "a b c d"; Tokenizer tokenizer = new WhitespaceTokenizer(Version.LUCENE_40, new StringReader(text)); List<String> tokens = new ArrayList<String>(); while (tokenizer.incrementToken()) { tokens.add(tokenizer.getAttribute(CharTermAttribute.class).toString()); } assertEquals(tokens, Arrays.asList(new String[]{"a","b","c","d"})); } This used to work, at least until LUCENE_33. Is this a bug, or am I missing something? Thank you, Spyros