Torsten Bøgh Köster created SOLR-10550: ------------------------------------------
Summary: Improve FileFloatSource eviction // reduce FileFloatSource memory footprint Key: SOLR-10550 URL: https://issues.apache.org/jira/browse/SOLR-10550 Project: Solr Issue Type: Improvement Security Level: Public (Default Security Level. Issues are Public) Components: Server Affects Versions: 6.5 Reporter: Torsten Bøgh Köster Attachments: solr_filefloatsource.patch As a follow up from {{SOLR-10506}} we found another possible memory leak in Solr. The values generated from an {{ExternalFileField}} are cached in a static cache inside the {{FileFloatSource}}. That cache caches both a {{IndexReader}} and {{FileFloatSource}}s loaded using that {{IndexReader}}. Cache eviction is left to the internally used {{WeakHashMap}} or a full eviction can be triggered via url. We are dealing with large synonym files and word lists stored in managed resources. Those are tied to the {{SolrCore}} as described in {{SOLR-10506}}. We're also using {{ExternalFileField}}s whose {{FileFloatSource}} are cached in said static cache. The {{FileFloatSource}} hold strong (transitive) references to the {{SolrCore}} they have been created for. After a couple of collection reloads, the cache eviction mechanism of the {{WeakHashMap}} gets activated pretty close to heap exhaustion. The patch attached adds a mechanism to evict cache entries created in the context of a {{SolrCore}} upon it's close using a close hook in the {{ExternalFileFieldReloader}}. It furthermore adds a static cache reset method for all entries bound to a given {{IndexReader}}. I'm not sure, if the added cache resets are too aggressive or executed too often, I'd like to leave that to the experts ;-) -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org