[
https://issues.apache.org/jira/browse/OPENNLP-1931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18119511#comment-18119511
]
ASF GitHub Bot commented on OPENNLP-1931:
-----------------------------------------
rzo1 commented on PR #1277:
URL: https://github.com/apache/opennlp/pull/1277#issuecomment-5838121615
Thanks for the update. Maintainer edits don't work on the `ai-pipestream`
fork, so here is a patch with the remaining mechanical fixes:
- `JsonScan`: class Javadoc reduced to what it reads and the three
deviations from RFC 8259. Dropped "in one pass", the call-stack sentence, and
"the only API of this class".
- `DocumentCategorizerDL.readCategories`: a non-integer `id2label` key now
fails with "id2label key must be an integer: <key>" instead of the bare "For
input string: ..." from `NumberFormatException`.
- `StringUtilByteOrderMarkTest`: `` / `�` escapes instead of invisible
characters in the source.
- `doccat.xml`: condensed both paragraphs, dropped the Python
`NaN`/`Infinity` sentence, and made OPENNLP-1988 a `<ulink>`.
Apply it with `git am 1277.patch`, and please don't force-push over it
without applying it first.
<details>
<summary>1277.patch</summary>
```diff
From fb41615b1b87be53ed9398367978f17be16f26c0 Mon Sep 17 00:00:00 2001
From: Richard Zowalla <[email protected]>
Date: Fri, 25 Sep 2026 21:10:17 +0200
Subject: [PATCH] OPENNLP-1931: Trim commentary and name the bad id2label key
- JsonScan: reduce the class Javadoc to what is read and the three
deviations from RFC 8259.
- DocumentCategorizerDL: report a non-integer id2label key by name
instead of the bare NumberFormatException message.
- StringUtilByteOrderMarkTest: use escapes instead of invisible
characters.
- doccat.xml: condense the vocabulary and configuration paragraphs and
link OPENNLP-1988.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
---
.../util/StringUtilByteOrderMarkTest.java | 4 +--
.../src/main/java/opennlp/dl/JsonScan.java | 26 +++++++++----------
.../dl/doccat/DocumentCategorizerDL.java | 17 +++++++++++-
opennlp-docs/src/docbkx/doccat.xml | 23 ++++++----------
4 files changed, 38 insertions(+), 32 deletions(-)
diff --git
a/opennlp-api/src/test/java/opennlp/tools/util/StringUtilByteOrderMarkTest.java
b/opennlp-api/src/test/java/opennlp/tools/util/StringUtilByteOrderMarkTest.java
index 0914c7673..8f34a0a90 100644
-
> RegEx removal part 3: JSON vocabulary and id2label scrape in opennlp-dl
> -----------------------------------------------------------------------
>
> Key: OPENNLP-1931
> URL: https://issues.apache.org/jira/browse/OPENNLP-1931
> Project: OpenNLP
> Issue Type: Improvement
> Components: Deep Learning
> Reporter: Kristian Rickert
> Assignee: Kristian Rickert
> Priority: Major
> Fix For: 2.5.13, 3.0.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> {{AbstractDL.loadJsonVocab}} and {{DocumentCategorizerConfig.fromJson}}
> located string keys and values with regular expressions. Both now use a small
> cursor helper, {{opennlp.dl.JsonScan}} (public, marked internal), that finds
> string literals honoring backslash escapes, skips ASCII whitespace, and reads
> digit runs.
> Behavior is unchanged, including the quirks of the old patterns, which the
> tests pin: a value with a decimal fraction yields its integer prefix, a
> negative id is skipped, the id2label object is cut at the first closing brace
> even inside a nested value, and a key spanning a newline is accepted while a
> value with a newline is skipped.
> Verification: the old patterns and the new scan were compared over the real
> vocabulary and configuration resources, 57 hand-written edge cases, and
> 400,000 generated inputs with zero differences. {{JsonScanTest}} adds 71
> cases, {{LoadVocabTest}} 27, {{DocumentCategorizerConfigTest}} 24.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
