Kristian Rickert created OPENNLP-1918:
-----------------------------------------
Summary: Add ONNX-backed constituency parsing with reproducible
English model training
Key: OPENNLP-1918
URL: https://issues.apache.org/jira/browse/OPENNLP-1918
Project: OpenNLP
Issue Type: Improvement
Components: language model
Reporter: Kristian Rickert
Assignee: Kristian Rickert
h3. Background
OpenNLP provides a constituency parser API and classic parser implementation,
but the current Apache model distribution does not include a recently trained
English constituency parser model.
The remaining downloadable model is a legacy SourceForge-era artifact and
should not be the default for a modern OpenNLP demonstration.
The {{opennlp-dl}} module already provides shared ONNX Runtime, tokenization,
CPU/GPU selection, validation, and lifecycle foundations. OpenNLP should
provide a modern parser implementation while preserving its existing parser API
and tree representation.
h3. Proposed work
Add an ONNX-backed constituency parser with a documented OpenNLP model contract.
* Implement {{ParserDL}} behind the existing OpenNLP parser API.
* Define and document one supported ONNX parser architecture and its input,
output, vocabulary, label, and model-manifest contract.
* Preserve caller-supplied token boundaries and original sentence spans.
* Decode model output into the existing OpenNLP {{Parse}} tree representation.
* Require one complete rooted tree with valid nested spans, ordered children,
terminal coverage, labels, and finite scores.
* Reject malformed tensor names, shapes, label indexes, spans, scores, and
incomplete trees.
* Enforce configured sentence-length, tensor-size, memory, and execution
limits before inference.
* Support thread-safe inference and deterministic native-resource cleanup.
* Add a reproducible English constituency-parser training and ONNX export
pipeline.
* Pin corpus revisions, preprocessing configuration, tag conventions, tool
versions, and artifact checksums.
* Produce held-out labeled and unlabeled bracketing evaluation reports.
* Document model provenance and licensing before publishing an artifact
through {{apache/opennlp-models}}.
* Add usage and training documentation with executable or mirror-tested
examples.
The runtime implementation and model artifact may be reviewed as separate
pull requests under this JIRA issue.
h3. Acceptance criteria
* {{ParserDL}} passes the existing OpenNLP parser contract tests.
* Every successful parse contains one root, ordered and properly nested
children, complete terminal coverage, valid labels, and document-consistent
spans.
* Tests cover Unicode text, punctuation, unknown tokens, empty input, long
sentences, malformed outputs, invalid trees, model-loading failures,
concurrency, and cleanup.
* Training and evaluation are reproducible from documented, checksum-pinned
inputs.
* Evaluation publishes labeled precision, labeled recall, labeled F1,
unlabeled F1, sentence-complete accuracy, and throughput on a clearly
identified held-out corpus.
* The model manifest records the label inventory, vocabulary identity,
preprocessing rules, corpus provenance, artifact hash, supported sentence
length, and OpenNLP compatibility.
* No test, build, or demonstration requires a SourceForge-era model.
* The resulting model can be loaded by the OpenNLP gRPC service without
changing the public constituency-parse annotation contract.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
