[ https://issues.apache.org/jira/browse/LUCENE-5202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13762011#comment-13762011 ]
Michael McCandless commented on LUCENE-5202: -------------------------------------------- bq. I suspect that there's something that LTF does that I don't need that explains why it is so complex. I think it's trying to support arbitrary lookahead, and insertion of new tokens. Sort of what a SynonymFilter would need. But it's obviously not easy to use yet :) > LookaheadTokenFilter consumes an extra token in nextToken > --------------------------------------------------------- > > Key: LUCENE-5202 > URL: https://issues.apache.org/jira/browse/LUCENE-5202 > Project: Lucene - Core > Issue Type: Bug > Affects Versions: 4.3.1 > Reporter: Benson Margulies > Attachments: LUCENE-5202.patch, LUCENE-5202.patch > > > This is a bit hard to explain except by looking at the test case. I've coded > a filter that uses LookaheadTokenFilter. The incrementToken method peeks some > tokens. Then, it seems, nextToken in the Lookahead class calls peekToken > itself, which seems to me to consume a token so that it's not seen when the > derived class sets out to process the next set of tokens. > In passing, this test case can be used to demonstrate that it does not work > to try to use the afterPosition method to set up attributes of the token that > we're 'after'. Probably that was never intended. However, I'm hoping for some > feedback as to whether the rest of the structure here is as intended for > subclasses of LookaheadTokenFilter. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org