[
https://issues.apache.org/jira/browse/OPENNLP-1884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martin Wiesner updated OPENNLP-1884:
------------------------------------
Fix Version/s: 3.0.0-M6
> Extend stemmer coverage: regenerate from Snowball 3.1.1 and add light stemmers
> ------------------------------------------------------------------------------
>
> Key: OPENNLP-1884
> URL: https://issues.apache.org/jira/browse/OPENNLP-1884
> Project: OpenNLP
> Issue Type: Improvement
> Reporter: Kristian Rickert
> Assignee: Kristian Rickert
> Priority: Major
> Fix For: 3.0.0-M6, 3.0.0
>
>
> OpenNLP currently ships 21 Snowball algorithms (20 languages, last
> regenerated for OPENNLP-1701). Upstream Snowball 3.1.1 (June 2026) provides
> ~35 algorithms across ~33 languages. Regenerating adds Armenian, Basque,
> Czech, Esperanto, Estonian, Hindi, Latin, Lithuanian, Nepali, Persian,
> Polish, Serbian, Sesotho, Tamil and Yiddish, plus the Early Modern English
> variant - generated code only, no algorithm work. Stemmer tests would now run
> in CI (OPENNLP-1882), so regeneration regressions surface immediately.
> Additionally, port the hand-written light/minimal stemmers from Apache Lucene
> (Arabic light10, Hindi light, Portuguese RSLP, Latvian, Bulgarian, and
> others) - ASL 2.0 within the ASF, with attribution in NOTICE. Light stemmers
> under-stem relative to Snowball, which is often preferable for high-precision
> matching. Expose everything through the {{StemmerFactory}} seam and register
> new languages in {{{}NormalizationProfiles{}}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
