[
https://issues.apache.org/jira/browse/OPENNLP-1942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Richard Zowalla updated OPENNLP-1942:
-------------------------------------
Fix Version/s: (was: 3.0.0-M6)
> Test the shared deep-learning encoder and fix the token type javadoc
> --------------------------------------------------------------------
>
> Key: OPENNLP-1942
> URL: https://issues.apache.org/jira/browse/OPENNLP-1942
> Project: OpenNLP
> Issue Type: New Feature
> Components: Sentence Detector
> Reporter: Kristian Rickert
> Assignee: Kristian Rickert
> Priority: Major
> Fix For: 3.0.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The shared deep-learning encoder in {{AbstractDL}}, used by every opennlp-dl
> component, had no tests that run without an ONNX model. Its javadoc also
> misstated the token type array.
> * Adds {{SharedDlEncodingTest}}: verifies the complete
> {{tokens}}/{{ids}}/{{mask}}/{{types}} arrays over a plain vocabulary through
> a session-free {{AbstractDL}} stub, rejects null input, and checks that
> repeated and concurrent calls return independent arrays.
> * Corrects the {{Tokens#types}} javadoc: the values are segment ids, all 0
> for a single input segment, not "typically all 1".
> * Links the Unicode SpecialCasing.txt data file from the {{WordpieceEncoder}}
> case-mapping javadoc.
> No runtime behavior changes; the production diff is javadoc only.
> PR: https://github.com/apache/opennlp/pull/1288
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
