[
https://issues.apache.org/jira/browse/LUCENE-6667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14618309#comment-14618309
]
Uwe Schindler commented on LUCENE-6667:
---------------------------------------
In filters the approach should be the following:
- Of the original token capture the state
- when you insert new tokens, restore the state instead of clearAttributes()
- set the "changed" attributes
This approach is used by stemmers that insert stemmed tokens (preserve
original), so the original attributes keep alive.
clearAttributes should only be called in Tokenizers or root TokenStreams.
> Custom attributes get cleared by filters
> ----------------------------------------
>
> Key: LUCENE-6667
> URL: https://issues.apache.org/jira/browse/LUCENE-6667
> Project: Lucene - Core
> Issue Type: Bug
> Affects Versions: 4.10.4
> Reporter: Oliver Becker
>
> I believe the Lucene API enables users to define their custom attributes (by
> extending {{AttributeImpl}}) which may be added by custom Tokenizers.
> It seems, the {{clear}} and {{copyTo}} methods must be implemented to clear
> and restore the state of this custom attribute.
> However, some filters (in our case the SynonymFilter) simply call
> {{AttributeSource.clearAttributes}} without invoking {{copyTo}}. Instead the
> filter just resets some known attributes, simply ignoring all other custom
> attributes. In the end our custom attribute value is lost.
> Is this a bug in {{SynonymFilter}} (and others) or are we using the API in
> the wrong way?
> A solution might be of course to provide empty implementations of {{clear}}
> and {{copyTo}}, but I'm not sure if this has other unwanted effects.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
