Token of  "" returns in CJK
---------------------------

                 Key: LUCENE-973
                 URL: https://issues.apache.org/jira/browse/LUCENE-973
             Project: Lucene - Java
          Issue Type: Bug
          Components: Analysis
    Affects Versions: 2.3
            Reporter: Toru Matsuzawa


The "" string returns as Token in the boundary of two byte character and one 
byte character. 

There is no problem in CJKAnalyzer. 
When CJKTokenizer is used with the unit, it becomes a problem. (Use it with 
Solr etc.)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Reply via email to