Martin Wiesner created OPENNLP-1539:
---------------------------------------
Summary: Introduce parameter for POSTaggerME to configure tag
format
Key: OPENNLP-1539
URL: https://issues.apache.org/jira/browse/OPENNLP-1539
Project: OpenNLP
Issue Type: Improvement
Components: POS Tagger
Affects Versions: 2.3.0, 2.2.0, 2.1.0, 2.0.0
Reporter: Martin Wiesner
Classic (legacy) POS models provide tags in the [PENN Treebank POS tag
format|https://www.ling.upenn.edu/courses/Fall_2003/ling001/penn_treebank_pos.html].
The modern UD-based models, however, differ in the format and are longer, e.g.
"VB" (Penn) vs. "VERB" (UD).
To provide compatibility for existing applications and/or use-cases, we need to
provide a way to retrieve both POS formats.
Aims:
- Introduce a constructor parameter for POSTaggerME to configure tag format /
style: Penn or UD style
- Implement a mapping between both POS tag formats: UD <==> Penn
- Update the OpenNLP Manual to explain differences of POS tag format and
configuration parameter
Conceptual idea:
- {{new POSTaggerME("en")}} => by _default_: UD format "as is"
- {{new POSTaggerME("en", POSTagFormat.PENN)}} => by _intention_, here: Penn
style
Benefit:
1. It should be explicit so devs / user see what they will get via
{{POSTagFormat}}. Enum values: POSTagFormat.UD, POSTagFormat.PENN,
POSTagFormat.DEFAULT
2. IT tests can now be formulated to work on both modern and legacy models.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
