[
https://issues.apache.org/jira/browse/LUCENE-1253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Uwe Schindler reopened LUCENE-1253:
-----------------------------------
Assignee: Uwe Schindler
After some discussion with Robert, we realized, that all TokenFilters that
remove tokens from the stream must preserve the position increment like
StopFilter. Else it could also happen that synonyms of a removed token appear
as synonyms of the token before the removed one. If the removed one has
posIncr=1, this would produce wrong synonyms.
Two filters that remove tokanes need to be fixed in the same way like
StopFilter:
- LengthFilter
- KeepWordFilter
- ... (find more and add here)
> LengthFilter ignoring relative positionIncrement of tokens skipped
> ------------------------------------------------------------------
>
> Key: LUCENE-1253
> URL: https://issues.apache.org/jira/browse/LUCENE-1253
> Project: Lucene - Java
> Issue Type: Bug
> Components: contrib/analyzers
> Affects Versions: 2.3.1
> Reporter: Walter Ferrara
> Assignee: Uwe Schindler
> Priority: Minor
> Fix For: 3.1, 4.0
>
>
> See for reference:
> http://www.nabble.com/WordDelimiterFilter%2BLenghtFilter-results-in-termPosition%3D%3D-1-td16306788.html
> and http://www.nabble.com/Lucene---Java-f24284.html
> It seems that LengthFilter (at least) could produce a stream in which the
> first Token has a positionIncrement of 0, which make CheckIndex and Luke
> function "Reconstruct&Edit" to generate exception.
> Should something be done to avoid this situation, or could the error be
> ignored (by allowing Term with a position of -1, and relaxing CheckIndex
> checks?)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
