Re: getting the token position

2013-01-10 Thread Igal @ getRailo.org
hi Denis, thanks for your reply. OffsetAttribute gives the character position whereas I was looking for the Token Position. I ended up adding the attached PositionAttribute/PositionAttributeImpl/PositionFilter. as it turned out though I didn't need that attribute as there was an easier way

Re: getting the token position

2013-01-10 Thread Denis Bazhenov
What you are looking for is OffsetAttribute. Also consider the possibility of using ShingleFilter with position increment > 1 and then filtering tokens containing "_" (underscore). This will be easier, I guess. On Jan 11, 2013, at 7:14 AM, Igal @ getRailo.org wrote: > hi all, > > how can I ge

getting the token position

2013-01-10 Thread Igal @ getRailo.org
hi all, how can I get the Token's Position from the TokenStream / Tokenizer / Analyzer ? I know that there's a TokenPositionIncrement Attribute and a TokenPositionLength Attribute, but is there an easy way to get the token position or do I need to implement my own attribute by adding one of t