[
https://issues.apache.org/jira/browse/OPENNLP-1933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18120444#comment-18120444
]
ASF GitHub Bot commented on OPENNLP-1933:
-----------------------------------------
krickert commented on PR #1320:
URL: https://github.com/apache/opennlp/pull/1320#issuecomment-5882375303
I replaced regex in Leipzig file name checks, Morfologik suffix handling and
frequency dictionary parsing with direct character checks.
It matches only a trailing .info suffix. Split frequency columns on tabs or
spaces, preserve Unicode terms and digits, and report line numbers for invalid
counts.
I added tests for file names, separators, Unicode input and invalid counts.
Documented the dictionary formats in the manual.
LMK if I need to do anything else.
> RegEx removal part 5a: Remove regex from Leipzig filenames and dictionary
> loading
> ----------------------------------------------------------------------------------
>
> Key: OPENNLP-1933
> URL: https://issues.apache.org/jira/browse/OPENNLP-1933
> Project: OpenNLP
> Issue Type: Improvement
> Reporter: Kristian Rickert
> Assignee: Kristian Rickert
> Priority: Major
> Fix For: 2.5.13, 3.0.0
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Replace regex operations in Leipzig filename checks, Morfologik suffix
> handling, and frequency-dictionary column parsing with character scans.
> Preserve Unicode dictionary terms and supported integer-count
> representations, removing the proposed ASCII-only count restriction. Keep
> space/tab column separators and existing negative-count and overflow checks.
> Add focused regression tests and document the supported format, referencing
> SymSpell's dictionary documentation:
> https://github.com/wolfgarbe/SymSpell#dictionary-file-format
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
