position increments should be implemented by TokenStream.end()
--------------------------------------------------------------
Key: LUCENE-3849
URL: https://issues.apache.org/jira/browse/LUCENE-3849
Project: Lucene - Java
Issue Type: Bug
Affects Versions: 3.6, 4.0
Reporter: Robert Muir
if you have pages of a book as multivalued fields, with the default position
increment gap
of analyzer.java (0), phrase queries won't work across pages if one ends with
stopword(s).
This is because the 'trailing holes' are not taken into account in end(). So I
think in
TokenStream.end(), subclasses of FilteringTokenFilter (e.g. stopfilter) should
do:
{code}
super.end();
posIncAtt += skippedPositions;
{code}
One problem is that these filters need to 'add' to the posinc, but currently
nothing clears
the attributes for end() [they are dirty, except offset which is set by the
tokenizer].
Also the indexer should be changed to pull posIncAtt from end().
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
