[
https://issues.apache.org/jira/browse/LUCENE-1775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12738026#action_12738026
]
Uwe Schindler commented on LUCENE-1775:
---------------------------------------
This ShingleMatrixFilter is really a pain!
I think the ShingeMatrix is very "special" and only produce tokens with few
correlation to the original input stream, so it is not so bad, if the extra
attributes get lost.
You could use a simple AttributeSource instead of EmptyTokenStream and create
it with the same AttributeFactory as the filter isself. Because of this, you
could copyTo the extra Tokens (currently implemented by the Token instance).
This reuseableToken could also be an AttributeSource? For me it is not really
clear what all this copying between the attributes and the Token instance does,
but it seems that it could be converted to Attributes, too. If you do it that
way, would it be not work also with custom attributes? One possibility would be
to copyTo the Tokens around (or use States) and then modify the shingle
speicfic things.
> Change org.apache.lucene.analysis.shingle to use new TokenStream API
> --------------------------------------------------------------------
>
> Key: LUCENE-1775
> URL: https://issues.apache.org/jira/browse/LUCENE-1775
> Project: Lucene - Java
> Issue Type: Task
> Components: contrib/analyzers
> Reporter: Michael Busch
> Assignee: Michael Busch
> Priority: Minor
> Fix For: 2.9
>
> Attachments: lucene-1775.patch, lucene-1775.patch, lucene-1775.patch
>
>
> All other contrib streams/filters have already been converted with
> LUCENE-1460.
> The two shingle filters are the last ones we need to convert.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
