CharTokenizer can not detect the end of stream
----------------------------------------------
Key: LUCENENET-119
URL: https://issues.apache.org/jira/browse/LUCENENET-119
Project: Lucene.Net
Issue Type: Bug
Environment: Lucene.Net 2.3.1
Reporter: Digy
Attachments: CharTokenizer.patch
CharTokenizer can not detect the end of stream since input stream does not
return -1 on EOS, instead it returns 0.
{code}
dataLen = input.Read(....)
if (dataLen == -1 )
{code}
Analyzers that use that class (like WhitespaceAnalyzer, SimpleAnalyzer etc.)
return tokens with length 255 at the end (only last token).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
