[
https://issues.apache.org/jira/browse/SOLR-8349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15032007#comment-15032007
]
Gus Heck commented on SOLR-8349:
--------------------------------
Further background for this feature... a precursor version of this patch (which
does not have the interface and thus can't fix SOLR-3443) is in use at a client
where we had a ~900MB hash map for looking up lat/lon from custom query
parameters. This map was needed for all our cores. This is anticipated to save
>35GB of ram since 40+ cores will live on a machine. The original
implementation of this lat/lon lookup feature for the client attempted to use a
static field, but the independent class loaders (each core gets it's own class
loader) loaded fresh copies of the class each with it's own static map.
It's worth noting that the analyzers such as the hunspell one in SOLR-3443 are
not loaded by the core's class loaders and the excess memory there is held in a
member field per instance, so a static variable based solution would be
possible there. I thought it was better to provide a uniform solution.
Another possible follow on feature (or perhaps enhancement to this one) would
be a means of reference counting the shared resources and removing them. In the
present (initial) patch, a long running solr instance where lots of cores are
added and removed would potentially have unused container resources hanging
around (though they would become used again with no loading time if a core were
re-installed that required them). I didn't go into the complexity of removal
because I wasn't sure if it would be deemed necessary.
> Allow sharing of large in memory data structures across cores
> -------------------------------------------------------------
>
> Key: SOLR-8349
> URL: https://issues.apache.org/jira/browse/SOLR-8349
> Project: Solr
> Issue Type: Improvement
> Components: Server
> Affects Versions: 5.3
> Reporter: Gus Heck
> Attachments: SOLR-8349.patch
>
>
> In some cases search components or analysis classes may utilize a large
> dictionary or other in-memory structure. When multiple cores are loaded with
> identical configurations utilizing this large in memory structure, each core
> holds it's own copy in memory. This has been noted in the past and a specific
> case reported in SOLR-3443. This patch provides a generalized capability, and
> if accepted, this capability will then be used to fix SOLR-3443.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
