[ https://issues.apache.org/jira/browse/LUCENE-8273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16452676#comment-16452676 ]
Alan Woodward commented on LUCENE-8273: --------------------------------------- bq. You mean any filter that uses captureState? capture/restoreState works fine, the problem comes when you get a filter that needs to look ahead in the tokenstream, so for example if SynonymGraphFilter has a multiword synonym "a b c -> d", and you hit token "a", then the filter pulls in two more tokens to see if it matches the whole synonym; but ConditionalTokenFilter only allows you to pull in one token at a time, because it needs to distinguish between incrementToken() as called by the next filter down the line, and incrementToken() as called by its delegate. > Add a ConditionalTokenFilter > ---------------------------- > > Key: LUCENE-8273 > URL: https://issues.apache.org/jira/browse/LUCENE-8273 > Project: Lucene - Core > Issue Type: New Feature > Reporter: Alan Woodward > Priority: Major > Attachments: LUCENE-8273.patch, LUCENE-8273.patch > > > Spinoff of LUCENE-8265. It would be useful to be able to wrap a TokenFilter > in such a way that it could optionally be bypassed based on the current state > of the TokenStream. This could be used to, for example, only apply > WordDelimiterFilter to terms that contain hyphens. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org