Thanks for the summary. I did some homework on how other systems solve this before forming an opinion. Short version: the proposed design is not novel architecture. That is its strength. The same pattern has emerged independently in at least four mature systems.
INCEpTION defines no container of its own. It sits on the UIMA CAS [1][2]. The CAS holds the original text and typed feature structures with begin and end offsets into it. That is classic standoff annotation, the same rule the proposal states for spans. What INCEpTION calls a layer is a UIMA type plus configurable behaviors. It distinguishes span layers, relation layers, chain layers for coreference, and metadata layers without offsets. The difference is weight. UIMA declares its type system in XML descriptors and carries reflection and serialization machinery with it. The proposal is deliberately the lightweight end of that spectrum. For a library, as opposed to an annotation platform, that is the right decision, IMHO. Stanford CoreNLP is the closest match (to us), almost line for line. Its Annotation is a type safe heterogeneous map in the sense of Bloch [3][4][5]. Keys are class objects parameterized by the value type. The container never enumerates keys. Any module adds a new key without touching the core. The proposed LayerKey is the same pattern with string identity instead of class identity. That design has held up in Standford's CoreNLP for over fifteen years. spaCy has one owning Doc object, spans as views into it, and doc.spans as a dict of named span groups under open string keys [6]. Pipeline components declare assigns and requires metadata, and the pipeline analyzer validates ordering. That is a direct precedent for the proposed requires and provides methods. GATE uses named annotation sets with untyped feature maps [7]. It is open ended like the proposal but stringly typed in its values, which the Class parameter in LayerKey correctly avoids. So the field has converged four times on the same shape: standoff spans over immutable original text, open keyed typed layers, a container ignorant of payloads, and pipelines ordered by declared dependencies. The guardrail note in the ticket, that the doc container must never learn about specific layers, is the single most important sentence in it. Four points where the precedents diverge and where I would tighten the spec before the freeze: 1. Key identity. CoreNLP gets collision free identity from class objects. String ids do not. Two extensions can both mint "entities". The spec should define key equality and adopt namespaced ids for the standard keys, IMHO. 2. Annotations without spans. Language id, document categories, provenance. Every precedent grew a home for these. UIMA has types without offsets, spaCy has doc level attributes, INCEpTION has metadata layers. Retrofitting this later creates exactly the parallel surface the proposal wants to prevent, so we should think about that upfront. 3. Index based references need stated invariants. UIMA and CoreNLP reference annotations by object identity. Indices are better for immutability and serialization, but only if layer order is defined and existing layers are never mutated or reordered. The spec should - therefore- also say what happens when an annotator provides a layer that already exists. This needs to be defined. 4. Gold versus predicted layers: GATE annotation sets and UIMA views exist because corpora might carry competing versions of the same layer. Open string keys already handle this. The convention should simply be documented. At a first glance, the validatio plan from the Jira looks right. Dependency parsing is precisely the consumer that stresses cross layer references, and coreference will inherit the same rules. I would also make the acceptance criterion, that a new layer can be added without touching the container package. Gruß Richard [1] INCEpTION Developer Guide. https://inception-project.github.io/releases/26.8/docs/developer-guide.html [2] Klie JC, Bugert M, Boullosa B, Eckart de Castilho R, Gurevych I. The INCEpTION Platform: Machine Assisted and Knowledge Oriented Interactive Annotation. COLING 2018. https://aclanthology.org/C18-2002.pdf [3] TypesafeMap. Stanford CoreNLP API. https://nlp.stanford.edu/nlp/javadoc/javanlp-3.5.0/edu/stanford/nlp/util/TypesafeMap.html [4] Annotation. Stanford CoreNLP API. https://nlp.stanford.edu/nlp/javadoc/javanlp-3.5.0/edu/stanford/nlp/pipeline/Annotation.html [5] CoreAnnotation. Stanford CoreNLP API. https://nlp.stanford.edu/nlp/javadoc/javanlp-3.5.0/edu/stanford/nlp/ling/CoreAnnotation.html [6] spaCy Doc API. https://spacy.io/api/doc [7] GATE Documentation, Language Resources: Corpora, Documents and Annotations. https://gate.ac.uk/sale/tao/splitch5.html > Am 15.07.2026 um 21:33 schrieb Kristian Rickert <[email protected]>: > > I accidentally sent a draft. To clear things up, I’ve opened OPENNLP-1888 > to spark a design discussion around introducing a generic Document type > into the 3.0 API. > > Currently, our Java API is tool-by-tool; as we look toward adding more > complex, structured results (like dependency arcs or relations), continuing > this pattern means we'll end up building parallel surface areas for every > new capability. > > The proposal in the JIRA ticket aims to solve this by introducing a small, > unified container in opennlp-api that uses typed annotation layers over the > original text. I want to emphasize right away that this deprecates and > replaces absolutely nothing in the existing task-level API; it simply > provides a clean, additive way to handle combined-document views moving > forward. I'd really love it if we could take a look at the full > architecture, constraints, and motivations in the ticket and share your > thoughts. > > I think this design covers what you'd see in other APIs as well as some new > frontier ideas. > > Thoughts? > > On Wed, Jul 15, 2026 at 3:17 PM Richard Zowalla <[email protected]> wrote: > >> Did this mail get mangled? >> >>> Am 15.07.2026 um 00:17 schrieb Kristian Rickert <[email protected]>: >>> >>> Two main points: >>> >>> - OPENNLP-1888 has not been started; it was created as a design >>> discussion regarding a generic document shape I'm recommending. Details >> in >>> the ticket >>> - Thank you everyone for the detailed feedback. I'll ensure every PR has >>> a section in the documentation. >>> >>> OPENNLP-1888 was opened to discuss a generic Document type for the new >> API >>> interface. Details are in the ticket, but here is the initial shape and I >>> would like anyone's thoughts on it: >>> >>> public interface Document { >>> CharSequence text(); // the original text, >>> never a normalized form >>> <T> List<Annotation<T>> get(LayerKey<T> layer); // empty list when the >>> layer is absent >>> Set<LayerKey<?>> layers(); >>> } >>> >>> public record Annotation<T>(Span span, T value) { } >>> >>> public final class LayerKey<T> { >>> public static <T> LayerKey<T> of(String id, Class<T> type) { ... } >>> } >>> >>> public interface DocumentAnnotator { >>> Document annotate(Document document); // returns a new >> document >>> with layers added >>> Set<LayerKey<?>> requires(); >>> Set<LayerKey<?>> provides(); >>> } >>
