[jira] [Closed] (OPENNLP-1555) TokenizerME should detect multi-dot abbreviations
[ https://issues.apache.org/jira/browse/OPENNLP-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Wiesner closed OPENNLP-1555. --- Resolution: Fixed > TokenizerME should detect multi-dot abbreviations > - > > Key: OPENNLP-1555 > URL: https://issues.apache.org/jira/browse/OPENNLP-1555 > Project: OpenNLP > Issue Type: Improvement > Components: Tokenizer >Affects Versions: 2.1.0, 2.2.0, 2.3.0, 2.3.1, 2.3.2, 2.3.3 >Reporter: Martin Wiesner >Assignee: Martin Wiesner >Priority: Minor > Fix For: 2.3.4 > > > TokenizerME should detect and handle multi-dot abbreviations correctly. > Currently, this is not handled correctly. For instance, > German: "z.B." = "zum Beispiel" (for example) or, > Dutch: "e.v." = "en volgende" (and following) > are not tokenized correctly and extra tokens are returned. NOTE: no > whitespaces in between the dots in the above examples. > Aims: > * Fix the detection / handling of abbreviations for multi-dot abbreviations > * Provide test cases that cover these cases -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (OPENNLP-1555) TokenizerME should detect multi-dot abbreviations
[ https://issues.apache.org/jira/browse/OPENNLP-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842864#comment-17842864 ] ASF GitHub Bot commented on OPENNLP-1555: - mawiesne merged PR #599: URL: https://github.com/apache/opennlp/pull/599 > TokenizerME should detect multi-dot abbreviations > - > > Key: OPENNLP-1555 > URL: https://issues.apache.org/jira/browse/OPENNLP-1555 > Project: OpenNLP > Issue Type: Improvement > Components: Tokenizer >Affects Versions: 2.1.0, 2.2.0, 2.3.0, 2.3.1, 2.3.2, 2.3.3 >Reporter: Martin Wiesner >Assignee: Martin Wiesner >Priority: Minor > Fix For: 2.3.4 > > > TokenizerME should detect and handle multi-dot abbreviations correctly. > Currently, this is not handled correctly. For instance, > German: "z.B." = "zum Beispiel" (for example) or, > Dutch: "e.v." = "en volgende" (and following) > are not tokenized correctly and extra tokens are returned. NOTE: no > whitespaces in between the dots in the above examples. > Aims: > * Fix the detection / handling of abbreviations for multi-dot abbreviations > * Provide test cases that cover these cases -- This message was sent by Atlassian Jira (v8.20.10#820010)
