[
https://issues.apache.org/jira/browse/LUCENE-6121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14252471#comment-14252471
]
Robert Muir commented on LUCENE-6121:
-------------------------------------
{quote}
If you still call reset on the input (but not in CachingTokenFilter) before
then calling incrementToken, it'll all work out fine.
{quote}
How is that? MockTokenizer should be throwing an exception here. its not ok to
reset twice. Please change the text to "you must".
{quote}
In fact there's one query builder ('flexible' AnalyzerQueryNodeProcessor) that
I didn't change to move what it calls reset() on and it still works.
{quote}
Then it should be fixed, ideally to use MockAnalyzer so the double-reset causes
a test failure first.
We dont need an API that has "flexibility", consumers must follow the normal
contract and it will work:
reset() .. incrementToken().... end(), close()
I am also ok if we only forward close() a single time for now. I do not trust
that all TokenStreams obey Closeable yet and will ignore the second invocation.
But technically, CachingTokenFilter shouldn't need to do anything special there.
> Fix CachingTokenFilter to propagate reset() the first time
> ----------------------------------------------------------
>
> Key: LUCENE-6121
> URL: https://issues.apache.org/jira/browse/LUCENE-6121
> Project: Lucene - Core
> Issue Type: Improvement
> Reporter: David Smiley
> Assignee: David Smiley
> Fix For: 5.0, Trunk
>
> Attachments:
> LUCENE-6121_CachingTokenFilter_reset_propagates_reset_if_not_cached.patch
>
>
> CachingTokenFilter should have been propagating reset() _but only the first
> time_ and thus you would then use CachingTokenFilter in a more normal way –
> wrap it and call reset() then increment in a loop, etc., instead of knowing
> you need to reset() on what it wraps but not this token filter itself. That's
> weird. It's ab-normal for a TokenFilter to never propagate reset, so every
> user of CachingTokenFilter to date has worked around this by calling reset()
> on the underlying input instead of the final wrapping token filter
> (CachingTokenFilter in this case).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
