Since you are writing a custom token filter, it's up to you to return successive tokens by setting the appropriate attributes when nextToken is called. Have you read the tokenstream javadocs?
On Mar 15, 2018 10:35 AM, "deepu srinivasan" <sdeep...@gmail.com> wrote: > Hi . > How do i split a single token and index them both. For eg : if i recieve a > token "&&11&" in my custom token filter , i would like to index as "&" and > 11. >
