[
https://issues.apache.org/jira/browse/SOLR-8349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15149286#comment-15149286
]
Gus Heck commented on SOLR-8349:
--------------------------------
I did see .build(CacheLoader). :) My writeup was an explanation of why that
(and other tactics) don't work (for the defined goals).
What would you pass into build(CacheLoader)? How will it know how to load the
data for a custom component that hasn't even been written yet. Furthermore,
that pattern is meant to load on the first get(String) for a key, which
(without further code to cause a warming-get before core init completes)
violates Goal 5, and could block a query or update for the time it takes to
load (~15-20 sec if I employ all 16 CPU with parallel streams in my case).
If you do use build(CacheLoader) you will have to make your implementation of
CacheLoader a wrapper object than can accept arbitrary loaders from yet to be
written components, and then manage the acceptance and invocation of those
individual loaders on it's own. One of the reasons I had the goal of not
blocking is the case that motivated this for a client of mine was one with 40
cores in a node all using this one resource, but I can accept that this might
be an unusual case.
I'll submit a patch backing off on the "don't block loading other cores" goal
soon.
> Allow sharing of large in memory data structures across cores
> -------------------------------------------------------------
>
> Key: SOLR-8349
> URL: https://issues.apache.org/jira/browse/SOLR-8349
> Project: Solr
> Issue Type: Improvement
> Components: Server
> Affects Versions: 5.3
> Reporter: Gus Heck
> Attachments: SOLR-8349.patch
>
>
> In some cases search components or analysis classes may utilize a large
> dictionary or other in-memory structure. When multiple cores are loaded with
> identical configurations utilizing this large in memory structure, each core
> holds it's own copy in memory. This has been noted in the past and a specific
> case reported in SOLR-3443. This patch provides a generalized capability, and
> if accepted, this capability will then be used to fix SOLR-3443.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
