[ https://issues.apache.org/jira/browse/SOLR-8349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15149286#comment-15149286 ]
Gus Heck commented on SOLR-8349: -------------------------------- I did see .build(CacheLoader). :) My writeup was an explanation of why that (and other tactics) don't work (for the defined goals). What would you pass into build(CacheLoader)? How will it know how to load the data for a custom component that hasn't even been written yet. Furthermore, that pattern is meant to load on the first get(String) for a key, which (without further code to cause a warming-get before core init completes) violates Goal 5, and could block a query or update for the time it takes to load (~15-20 sec if I employ all 16 CPU with parallel streams in my case). If you do use build(CacheLoader) you will have to make your implementation of CacheLoader a wrapper object than can accept arbitrary loaders from yet to be written components, and then manage the acceptance and invocation of those individual loaders on it's own. One of the reasons I had the goal of not blocking is the case that motivated this for a client of mine was one with 40 cores in a node all using this one resource, but I can accept that this might be an unusual case. I'll submit a patch backing off on the "don't block loading other cores" goal soon. > Allow sharing of large in memory data structures across cores > ------------------------------------------------------------- > > Key: SOLR-8349 > URL: https://issues.apache.org/jira/browse/SOLR-8349 > Project: Solr > Issue Type: Improvement > Components: Server > Affects Versions: 5.3 > Reporter: Gus Heck > Attachments: SOLR-8349.patch > > > In some cases search components or analysis classes may utilize a large > dictionary or other in-memory structure. When multiple cores are loaded with > identical configurations utilizing this large in memory structure, each core > holds it's own copy in memory. This has been noted in the past and a specific > case reported in SOLR-3443. This patch provides a generalized capability, and > if accepted, this capability will then be used to fix SOLR-3443. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org