Kristian Rickert created OPENNLP-1933:
-----------------------------------------
Summary: RegEx removal part 5: String.split, matches, replaceAll,
and replaceFirst calls that compile a pattern per call
Key: OPENNLP-1933
URL: https://issues.apache.org/jira/browse/OPENNLP-1933
Project: OpenNLP
Issue Type: Improvement
Reporter: Kristian Rickert
Assignee: Kristian Rickert
||Class||Call||Replacement||
|{{SimpleEventStreamBuilder}}, {{RealValueFileEventStream}},
{{RealBasicEventStream}}|{{split("\\s+")}}|{{StringUtil.splitOnAsciiWhitespace}}
from part 1|
|{{LeipzigLanguageSampleStream}}|{{matches("[a-z]+")}} on the file name
prefix|ASCII lowercase scan; the length 3 becomes a shared constant|
|{{MascNamedEntityParser}}, {{MascPennTagParser}}, {{MascWordParser}}|six
{{replaceFirst}} calls removing id prefixes|package-private
{{MascIdentifiers.removeFirst}}, first occurrence only|
|{{MorfologikDictionaryBuilder}}|anchored {{replaceAll}} of the metadata
suffix|{{endsWith}} and {{substring}}|
|{{FrequencyDictionaryLoader}}|{{[\t ]+}} column split|tab-and-space scan with
{{String.split}} semantics|
Each new helper is tested against the old call as an oracle, and each parser
has a test through its public surface. Single-character literal splits
elsewhere (comma, tab, space, semicolon) take the JDK fast path and compile
nothing, so they are not touched.
Not changed here: {{SpellCheckingCharSequenceNormalizer.URL_LIKE}} is a
multi-scheme URL and email heuristic and goes on the exempt list;
{{BasicContextGenerator}} is handled in its own bug ticket.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
