[
https://issues.apache.org/jira/browse/LUCENE-4640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13536911#comment-13536911
]
Uwe Schindler commented on LUCENE-4640:
---------------------------------------
By the way: the code you have posted contains a bug and is inefficient: reading
= new String(termAttr.buffer()); - this will create a string with garbage at
the end, use termAttr.toString().
> JapaneseCharFilter can't be reusable
> ------------------------------------
>
> Key: LUCENE-4640
> URL: https://issues.apache.org/jira/browse/LUCENE-4640
> Project: Lucene - Core
> Issue Type: Bug
> Components: modules/analysis
> Affects Versions: 4.0
> Reporter: Wang Han
> Assignee: Uwe Schindler
>
> com.hulu.lucene.analysis.JapaneseCharFilter is used in JPRomajiAnalyzer
> but incrementToken() can't be reusable.
> -------------------------------------------------
> if (!isEnd) { //can't be false when
> analyzer.tokenStream(...) is called
> if (input.incrementToken()) {
> String reading = readingAttr.getReading();
> if (reading == null) {
> reading = new String(termAttr.buffer());
> }
> if(reading != null) {
> reading =
> JapaneseStringTools.getRomanization(reading.trim());
> reading =
> StringTools.removeAccentCharacter(reading);
> for (int i = 0; i < reading.length();
> i++) {
> chars.add(reading.charAt(i));
> }
> }
> isEnd = false;
> } else {
> isEnd = true;
> }
> }
> if (chars.size() > 0 || isEnd == false) {
> if (chars.size() > 0) {
> char ch = chars.poll();
> termAttr.setEmpty().append(ch);
> }
> return true;
> }
> return false;
> --------------------------------------------------
> So i can't the call analyzer.tokenStream for the same field twice.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
