[ https://issues.apache.org/jira/browse/LUCENE-9930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17334710#comment-17334710 ]
ASF subversion and git services commented on LUCENE-9930: --------------------------------------------------------- Commit 90d363ece7116954c530a74a014487fcbdee7610 in lucene's branch refs/heads/main from Alan Woodward [ https://gitbox.apache.org/repos/asf?p=lucene.git;h=90d363e ] LUCENE-9930: Only load Ukrainian morfologik dictionary once per JVM (#109) The UkrainianMorfologikAnalyzer was reloading its dictionary every time it created a new TokenStreamComponents, which meant that while the analyzer was open it would hold onto one copy of the dictionary per thread. This commit loads the dictionary in a lazy static initializer, alongside its stopword set. It also makes the normalizer charmap a singleton so that we do not rebuild the same immutable object on every call to initReader. > UkrainianMorfologikAnalyzer reloads its Dictionary for every new > TokenStreamComponents instance > ----------------------------------------------------------------------------------------------- > > Key: LUCENE-9930 > URL: https://issues.apache.org/jira/browse/LUCENE-9930 > Project: Lucene - Core > Issue Type: Bug > Reporter: Alan Woodward > Assignee: Alan Woodward > Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Large static data structures should be loaded in Analyzer constructors and > shared between threads, but the UkrainianMorfologikAnalyzer is loading its > dictionary in `createComponents`, which means it is reloaded and stored on > every new analysis thread. If you have a large dictionary and highly > concurrent indexing then this can lead to you running out of memory as > multiple copies of the dictionary are held in thread locals. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org