[jira] [Updated] (LUCENE-3113) fix analyzer bugs found by MockTokenizer
[ https://issues.apache.org/jira/browse/LUCENE-3113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Muir updated LUCENE-3113: Component/s: modules/analysis Fix Version/s: 4.0 3.2 fix analyzer bugs found by MockTokenizer Key: LUCENE-3113 URL: https://issues.apache.org/jira/browse/LUCENE-3113 Project: Lucene - Java Issue Type: Bug Components: modules/analysis Reporter: Robert Muir Fix For: 3.2, 4.0 Attachments: LUCENE-3113.patch In LUCENE-3064, we beefed up MockTokenizer with assertions, and I've switched over the analysis tests to use MockTokenizer for better coverage. However, this found a few bugs (one of which is LUCENE-3106): * incrementToken() after it returns false in CommonGramsQueryFilter, HyphenatedWordsFilter, ShingleFilter, SynonymFilter * missing end() implementation for PrefixAwareTokenFilter * double reset() in QueryAutoStopWordAnalyzer and ReusableAnalyzerBase * missing correctOffset()s in MockTokenizer itself. I think it would be nice to just fix all the bugs on one issue... I've fixed everything except Shingle and Synonym -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-3113) fix analyzer bugs found by MockTokenizer
[ https://issues.apache.org/jira/browse/LUCENE-3113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Muir updated LUCENE-3113: Attachment: LUCENE-3113.patch attached is a patch, the synonyms and shingles tests still fail. fix analyzer bugs found by MockTokenizer Key: LUCENE-3113 URL: https://issues.apache.org/jira/browse/LUCENE-3113 Project: Lucene - Java Issue Type: Bug Components: modules/analysis Reporter: Robert Muir Fix For: 3.2, 4.0 Attachments: LUCENE-3113.patch In LUCENE-3064, we beefed up MockTokenizer with assertions, and I've switched over the analysis tests to use MockTokenizer for better coverage. However, this found a few bugs (one of which is LUCENE-3106): * incrementToken() after it returns false in CommonGramsQueryFilter, HyphenatedWordsFilter, ShingleFilter, SynonymFilter * missing end() implementation for PrefixAwareTokenFilter * double reset() in QueryAutoStopWordAnalyzer and ReusableAnalyzerBase * missing correctOffset()s in MockTokenizer itself. I think it would be nice to just fix all the bugs on one issue... I've fixed everything except Shingle and Synonym -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-3113) fix analyzer bugs found by MockTokenizer
[ https://issues.apache.org/jira/browse/LUCENE-3113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Muir updated LUCENE-3113: Attachment: LUCENE-3113.patch updated patch, fixing the bugs in Synonyms and ShingleFilter. also, i found two more bugs: the ShingleAnalyzerWrapper was double-resetting, and the PrefixAndSuffixAwareTokenFilter was missing end() also fix analyzer bugs found by MockTokenizer Key: LUCENE-3113 URL: https://issues.apache.org/jira/browse/LUCENE-3113 Project: Lucene - Java Issue Type: Bug Components: modules/analysis Reporter: Robert Muir Fix For: 3.2, 4.0 Attachments: LUCENE-3113.patch, LUCENE-3113.patch In LUCENE-3064, we beefed up MockTokenizer with assertions, and I've switched over the analysis tests to use MockTokenizer for better coverage. However, this found a few bugs (one of which is LUCENE-3106): * incrementToken() after it returns false in CommonGramsQueryFilter, HyphenatedWordsFilter, ShingleFilter, SynonymFilter * missing end() implementation for PrefixAwareTokenFilter * double reset() in QueryAutoStopWordAnalyzer and ReusableAnalyzerBase * missing correctOffset()s in MockTokenizer itself. I think it would be nice to just fix all the bugs on one issue... I've fixed everything except Shingle and Synonym -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
