Hey there,
in lucene 3.0 / 2.9 the Token class has been remove / replaced with an
Attribute based API. A TokenStream operates on Attibutes it has
declared which are eventually accessed by the IndexWriter to create
the inverted index. There are Attributes like TermAttribute,
PositionIncrementAttribute and OffsetAttribute that roughly reflect
the fields of the Token class. If you want to return an extra token /
Term from the Attribute based API you just set the TermAttribute value
and all other attributes you wanna use and return true from the
TokenStream#incrementToken()
something like this
public boolean incrementToken() {
if(input.incrementToken()){
// process the tokens in the stream
return true;
} else if (!extraTokens.isEmtpy()){
// return extra token
termAttribute.setTerm(extraTokens.pop())
return true;
}
return false;
}
does that make sense?
simon
On Mon, Jun 7, 2010 at 4:43 PM, Aad Nales <[email protected]> wrote:
> Hi All,
>
> Years ago we implemented a Lucene solution which we are updating
> today, and i am a bit lost on the following.
>
> In Lucene 1.x and 2.x it was possible to add a token in a Filter
> simply by returning an extra Token when next was being called. What i
> can not find is an equivalent possiblity for adding a Token in a
> filter. The incrementToken() seem to be the logical place to implement
> similar functionality (at least it could trigger it) but i am a bit
> lost how to access the stream and insert an extra token. Any
> suggestions?
>
> cheers,
> Aad
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [email protected]
> For additional commands, e-mail: [email protected]
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
