Kristian Rickert created OPENNLP-1987:
-----------------------------------------
Summary: Concurrent MorfologikTagDictionary lookups corrupt
results when an instance is shared
Key: OPENNLP-1987
URL: https://issues.apache.org/jira/browse/OPENNLP-1987
Project: OpenNLP
Issue Type: Bug
Reporter: Kristian Rickert
Assignee: Kristian Rickert
Fix For: 2.5.13, 3.0.0
h2. Problem
{\{MorfologikTagDictionary}} retains one mutable \{{DictionaryLookup}} instance.
Concurrent calls to \{{getTags}} on that shared wrapper can interfere with the
lookup state, producing incorrect tags or exceptions.
This report concerns sharing one dictionary wrapper. It does not claim that all
POS taggers or all TagDictionary implementations already guarantee thread
safety.
h2. Reproduction
Build a tagged dictionary with distinct results for two words, for example
{\{casa -> HOUSE}} and \{{carro -> CAR}}. Start eight workers together and have
each repeatedly query both words through the same MorfologikTagDictionary,
checking each result against its single-threaded reference.
An observed run against exact 2.x source reported 30 incorrect results and
85 exceptions. The counts depend on scheduling and are not acceptance criteria.
The same shared-lookup implementation is present on both branches checked on
2026-09-19:
* \{{opennlp-2.x}} at \{{26f394a6ab01bf757334ba88f03bc9e2a42f6c27}}.
* \{{main}} at \{{f2ab1ccd435c20e910008511a37467131683e9c7}}.
The 2.x reproduction used Morfologik 2.2.0. These branch checks do not establish
the earliest affected released version.
h2. Proposed fix and validation
Isolate mutable lookup state or synchronize the complete lookup and copying of
its results. Document the resulting sharing contract on MorfologikTagDictionary.
Assess the allocation or serialization cost of the chosen fix.
Add a regression comparing concurrent results with distinguishable
single-threaded references, with bounded execution and reliable worker cleanup.
Prefer a controlled interleaving where feasible rather than relying only on a
stress test to expose the race. Preserve sequential lookup behavior.
Keep tagless-entry handling and case-folding changes outside this issue. Prepare
focused fixes for main and opennlp-2.x under this issue.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
