[jira] [Commented] (LUCENE-3455) All Analysis Consumers should use reusableTokenStream
[ https://issues.apache.org/jira/browse/LUCENE-3455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13114458#comment-13114458 ] Chris Male commented on LUCENE-3455: Committed revision 1175670. All Analysis Consumers should use reusableTokenStream - Key: LUCENE-3455 URL: https://issues.apache.org/jira/browse/LUCENE-3455 Project: Lucene - Java Issue Type: Sub-task Components: modules/analysis Reporter: Chris Male Assignee: Chris Male Attachments: LUCENE-3455-test-consumers.patch, LUCENE-3455-test-consumers.patch With Analyzer now using TokenStreamComponents, theres no reason for Analysis consumers to use tokenStream() (it just gives bad performance). Consequently all consumers will be moved over to using reusableTokenStream(). The only challenge here is that reusableTokenStream throws an IOException which many consumers are not rigged to deal with. Once all consumers have been moved, we can rename reusableTokenStream() back to tokenStream(). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3455) All Analysis Consumers should use reusableTokenStream
[ https://issues.apache.org/jira/browse/LUCENE-3455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13114153#comment-13114153 ] Robert Muir commented on LUCENE-3455: - +1, there is a lot of crazy code around this area, consumers catching the exception from reusableTokenStream() and falling back to tokenStream() and other silly things. All Analysis Consumers should use reusableTokenStream - Key: LUCENE-3455 URL: https://issues.apache.org/jira/browse/LUCENE-3455 Project: Lucene - Java Issue Type: Sub-task Components: modules/analysis Reporter: Chris Male With Analyzer now using TokenStreamComponents, theres no reason for Analysis consumers to use tokenStream() (it just gives bad performance). Consequently all consumers will be moved over to using reusableTokenStream(). The only challenge here is that reusableTokenStream throws an IOException which many consumers are not rigged to deal with. Once all consumers have been moved, we can rename reusableTokenStream() back to tokenStream(). -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
