Does a TokenStream have to return always the same number of attributes
with the same underlying classes for all the tokens it generates?
I mean, during the tokenization phase, can the first "token" have a Term
and Offset Attribute and the second "token" only a Type Attribute or
does this mean that the first token has to have an empty Type attribute
as well?
I'm just not sure,
Daniel Shane
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
