[
https://issues.apache.org/jira/browse/LUCENE-8192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16385233#comment-16385233
]
Robert Muir commented on LUCENE-8192:
-------------------------------------
I wrote a patch to do this, but it causes many tests to fail.
I think this boolean currently mixes up two concerns:
* "correct offsets" as far as what IndexWriter will check. This is the useless
boolean, its mandatory that the tokenstream behave correctly here or its
basically broke.
* "graph offsets checks". This seems to be a higher bar, and even tests for
filters that claim to support graphs (SynonymGraphFilter) screw this up?
Just at a glance, it seems like we want to separate these concerns. The first
one should not be optional.
> Remove offsetsAreCorrect from BaseTokenStreamTestCase
> -----------------------------------------------------
>
> Key: LUCENE-8192
> URL: https://issues.apache.org/jira/browse/LUCENE-8192
> Project: Lucene - Core
> Issue Type: Bug
> Reporter: Robert Muir
> Priority: Major
> Attachments: LUCENE-8192_prototype.patch
>
>
> Similar to LUCENE-8191, now that indexwriter checks the offsets, this boolean
> is useless: if offsets are broken it will still fail.
> We should just remove the boolean.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
