[ https://issues.apache.org/jira/browse/LUCENE-6121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14252471#comment-14252471 ]
Robert Muir commented on LUCENE-6121: ------------------------------------- {quote} If you still call reset on the input (but not in CachingTokenFilter) before then calling incrementToken, it'll all work out fine. {quote} How is that? MockTokenizer should be throwing an exception here. its not ok to reset twice. Please change the text to "you must". {quote} In fact there's one query builder ('flexible' AnalyzerQueryNodeProcessor) that I didn't change to move what it calls reset() on and it still works. {quote} Then it should be fixed, ideally to use MockAnalyzer so the double-reset causes a test failure first. We dont need an API that has "flexibility", consumers must follow the normal contract and it will work: reset() .. incrementToken().... end(), close() I am also ok if we only forward close() a single time for now. I do not trust that all TokenStreams obey Closeable yet and will ignore the second invocation. But technically, CachingTokenFilter shouldn't need to do anything special there. > Fix CachingTokenFilter to propagate reset() the first time > ---------------------------------------------------------- > > Key: LUCENE-6121 > URL: https://issues.apache.org/jira/browse/LUCENE-6121 > Project: Lucene - Core > Issue Type: Improvement > Reporter: David Smiley > Assignee: David Smiley > Fix For: 5.0, Trunk > > Attachments: > LUCENE-6121_CachingTokenFilter_reset_propagates_reset_if_not_cached.patch > > > CachingTokenFilter should have been propagating reset() _but only the first > time_ and thus you would then use CachingTokenFilter in a more normal way – > wrap it and call reset() then increment in a loop, etc., instead of knowing > you need to reset() on what it wraps but not this token filter itself. That's > weird. It's ab-normal for a TokenFilter to never propagate reset, so every > user of CachingTokenFilter to date has worked around this by calling reset() > on the underlying input instead of the final wrapping token filter > (CachingTokenFilter in this case). -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org