[
https://issues.apache.org/jira/browse/LUCENE-1137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Grant Ingersoll resolved LUCENE-1137.
-------------------------------------
Resolution: Fixed
Lucene Fields: (was: [New])
Committed on 614891
> Token type as BitSet: typeBits()
> --------------------------------
>
> Key: LUCENE-1137
> URL: https://issues.apache.org/jira/browse/LUCENE-1137
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Analysis
> Reporter: Grant Ingersoll
> Assignee: Grant Ingersoll
> Priority: Minor
> Fix For: 2.4
>
> Attachments: LUCENE-1137.patch, LUCENE-1137.patch, LUCENE-1137.patch
>
>
> It is sometimes useful to have a more compact, easy to parse, type
> representation for Token than the current type() String. This patch adds a
> BitSet onto Token, defaulting to null, with accessors for setting bit flags
> on a Token. This is useful for communicating information about a token to
> TokenFilters further down the chain.
> For example, in the WikipediaTokenizer, the possibility exists that a token
> could be both a category and bold (or many other variations), yet it is
> difficult to communicate this without adding in a lot of different Strings
> for type. Unlike using the payload information (which could serve this
> purpose), the BitSet does not get added to the index (although one could
> easily convert it to a payload.)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
