Richard Zowalla created OPENNLP-1964:
----------------------------------------
Summary: Remove the Japanese add-on
Key: OPENNLP-1964
URL: https://issues.apache.org/jira/browse/OPENNLP-1964
Project: OpenNLP
Issue Type: Task
Components: Name Finder
Reporter: Richard Zowalla
Assignee: Richard Zowalla
Remove the {{japanese}} module and its leftover Ant {{build.xml}} from
opennlp-addons. Most of its feature generators are copies of the ones in
{{opennlp.tools.util.featuregen}}.
*Replacement:* core now handles Japanese itself:
* {{StringPattern}} detects Han, Hiragana and Katakana, and
{{FeatureGeneratorUtil}} emits the {{jah}}/{{jak}} token classes.
* {{LatticeTokenizer}} with a MeCab dictionary (OPENNLP-1894) segments the text
and returns each morpheme with its features, which covers the {{word/POS}}
tokens {{AuxiliaryInfo*}} expected.
The module also put classes in {{opennlp.tools.namefind}} and
{{opennlp.tools.util.featuregen}}, packages that {{opennlp-runtime}} already
owns.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
