Kristian Rickert created OPENNLP-1937:
-----------------------------------------
Summary: Expose text embeddings as a generic interface
Key: OPENNLP-1937
URL: https://issues.apache.org/jira/browse/OPENNLP-1937
Project: OpenNLP
Issue Type: Improvement
Reporter: Kristian Rickert
Assignee: Kristian Rickert
Core has no contract for "text in, vector out". \{{WordVectorTable}} looks up a
stored vector for a single word, and \{{SentenceVectorsDL}} produces sentence
vectors but only through its own class, so nothing can accept an embedder
without knowing exactly which implementation it is holding.
This adds \{{opennlp.tools.embeddings.TextEmbedder}} to opennlp-api:
\{{embed(CharSequence)}}, a defaulted \{{embedAll(List)}} for implementations
that batch better than they loop, and \{{dimension()}}. \{{SentenceVectorsDL}}
implements it, so the interface ships with a working implementation in core
rather than as a shape nobody uses yet.
The reason to do it now rather than later: a static embedding add-on is coming,
and an add-on can only implement a contract that exists in a released core
version. Without this, every add-on invents its own embedder type, none of them
interoperate, and moving to a shared one afterwards breaks their users.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
