[
https://issues.apache.org/jira/browse/OPENNLP-1888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Richard Zowalla updated OPENNLP-1888:
-------------------------------------
Fix Version/s: 3.0.0
> Creating a Document annotation container: typed, offset-anchored layer s so
> every future analysis result is an additive 3.x feature
> -----------------------------------------------------------------------------------------------------------------------------------
>
> Key: OPENNLP-1888
> URL: https://issues.apache.org/jira/browse/OPENNLP-1888
> Project: OpenNLP
> Issue Type: Improvement
> Affects Versions: 3.0.0-M4
> Reporter: Kristian Rickert
> Assignee: Kristian Rickert
> Priority: Major
> Fix For: 3.0.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Designing the document generic type in Java:
> To be explicit, because it is the first question this proposal should answer:
> *this ticket deprecates nothing and replaces nothing.* It is something I've
> brought up in the chats - the importance of now being the time to land API
> changes.
> h2. Motivation
> OpenNLP's Java API is tool-by-tool: a tokenizer returns spans, a tagger
> returns strings, a name finder returns spans, and the caller reassembles the
> pieces by hand, differently in every application. Each new capability adds
> another entry point and result type, so the cost compounds. I ran into this
> head-on writing the gRPC layer, which is mostly mapping tool outputs into a
> common document structure that the Java API itself never defines. Since 3.0
> has not landed yet, this feels like the right time to discuss adding that
> layer.
> We already have two data points. {{Term}} validated the layered-container
> idea, but it is token-scoped, string-valued, and keyed by a closed enum, so
> it cannot hold anything structural. The emoji annotation layer (OPENNLP-1870)
> is the first structured result that did not fit, and it correctly became a
> separate accessor beside {{{}Term{}}}. Every future structured result
> (dependency arcs, coreference, relations, money, PII) would repeat that
> pattern, each growing its own parallel surface, while the combined-document
> view still would not exist.
> The fix is one small container in {{{}opennlp-api{}}}: typed annotation
> layers over the original text, identified by open-ended keys rather than an
> enum. Any future capability then becomes three additions that touch no
> existing type: a payload record, a layer key, and an annotator. Since
> anything published in 3.x is frozen until 4.0, shipping this in 3.0 is what
> keeps the parallel surfaces from ever accumulating.
> h2. Design
> Four public types, all in a new package in {{{}opennlp-api{}}}:
> {code:java}
> public interface Document {
> CharSequence text(); // the original text, never
> a normalized form
> <T> List<Annotation<T>> get(LayerKey<T> layer); // empty list when the
> layer is absent
> Set<LayerKey<?>> layers();
> }
> public record Annotation<T>(Span span, T value) { }
> public final class LayerKey<T> {
> public static <T> LayerKey<T> of(String id, Class<T> type) { ... }
> }
> public interface DocumentAnnotator {
> Document annotate(Document document); // returns a new document
> with layers added
> Set<LayerKey<?>> requires();
> Set<LayerKey<?>> provides();
> }
> {code}
> Standard keys for today's results (sentences, tokens, tags, chunks, entities)
> ship with thin annotators wrapping the existing task interfaces, which stay
> exactly as they are. A {{DocumentAnalyzer}} builder assembles annotators into
> a pipeline ordered by {{{}requires(){}}}/{{{}provides(){}}}, mirroring
> {{{}TermAnalyzer{}}}.
> Rules that protect the additive property:
> * Spans always point into the original text; normalization maps back through
> {{{}Alignment{}}}.
> * Layer identity is an open key, never an enum; payload types live in the
> producing feature's package, and the container never imports or enumerates
> them.
> * Annotations reference other annotations by layer and index (a dependency
> arc names token indices), keeping documents immutable and mappable to any
> encoding.
> * No new dependencies, and no serialization framework in
> {{{}opennlp-api{}}}; encodings live where their dependencies already are.
> * Evolution by default methods and new types only; nothing sealed, no
> abstract classes at the seam.
> h2. Coexistence with the task-level API
> The task interfaces are not a legacy layer. They remain the primary API for
> single-task use, with no deprecation and no planned removal; annotators
> delegate to them, so there is one behavior reachable two ways (the same
> relationship {{TermAnalyzer}} has to the normalizer rungs). Deprecation is
> reserved for the rare surface a layer supersedes outright, and even then
> removal waits for a major, so existing call sites compile and run for the
> life of 3.x. Models are unaffected: nothing about training or decoding
> changes.
> h2. Validation before freeze
> # The existing analyze pipeline (sentences, tokens, tags, entities)
> expressed as annotators, proving the adapters and ordering model.
> # Dependency parsing as the first graph-shaped layer: its values reference
> other annotations, so it exercises the cross-layer rules that coreference and
> relation extraction will inherit. A transition-based parser over the existing
> maxent machinery, trained from CoNLL-U treebanks, suffices and is a wanted
> feature in its own right.
> h2. Out of scope
> Downstream features themselves (each is its own ticket), changes to any
> existing task interface or result type, and document serialization.
> h2. Acceptance criteria
> * Container types in {{{}opennlp-api{}}}, no new dependencies, full javadoc;
> existing tools populate a document through adapters unchanged.
> * Both validation consumers pass tests covering cross-layer references and
> span fidelity, including a normalization-shifted case.
> * A new layer can be added in a test without touching the container's
> package.
> * Binary compatibility from the previous release is verified mechanically in
> CI.
> * No existing type is modified or deprecated; the task-level API remains
> fully supported.
> h4. Guardrail Note:
> The container's package must never grow knowledge of specific layers. The
> moment it imports a payload type or enumerates layer identities, the additive
> property is lost and every future feature becomes an API change again.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
