Update StandardTokenizer and UAX29Tokenizer to Unicode 6.0.0
------------------------------------------------------------
Key: LUCENE-2699
URL: https://issues.apache.org/jira/browse/LUCENE-2699
Project: Lucene - Java
Issue Type: Improvement
Components: contrib/analyzers
Affects Versions: 3.1, 4.0
Reporter: Steven Rowe
Assignee: Steven Rowe
Priority: Minor
Fix For: 3.1, 4.0
Newly released Unicode 6.0.0 contains some character property changes from the
previous release (5.2.0) that affect word segmentation (UAX#29), and JFlex
1.5.0-SNAPSHOT now supports Unicode 6.0.0, so Lucene's UAX#29-based tokenizers
should be updated accordingly.
Note that the UAX#29 word break rules themselves did not change between Unicode
versions 5.2.0 and 6.0.0.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
