[
https://issues.apache.org/jira/browse/SOLR-13579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16878603#comment-16878603
]
Andrzej Bialecki commented on SOLR-13579:
------------------------------------------
This patch contains a draft of the API, with some details of the implementation
fleshed out, up for discussion - there are no tests and no integration with any
existing component yet. :)
A high-level design overview:
* A {{ResourceManager}} manages multiple named pools of resources (flat
hierarchy for now). A default instance of {{ResourceManager}} would be created
at a {{CoreContainer}} level so that it can manage global limits for a Solr
node.
* Each pool knows how to perform a single specific type of management. This
handling is actually performed by a {{ResourceManagerPlugin}}, which knows what
monitored values to retrieve from resources, and knows how to adjust the
controlled parameters of managed resources.
* There can be multiple pools of the same type (under different names) - they
will likely differ in their parameter. Eg. document cache size may be checked
every 1 min and have one limit, but the query / filter cache size may use
different parameters, even though the set of monitored parameters and
controlled parameters are the same (hence the same type).
* {{ResourceManager}} is responsible for periodically executing the
{{ResourceManagerPlugin}} of each of the pools, so that it can verify and
adjust the resources it manages in the pool.
* Each pool has its own parameters - currently the only global parameter is
scheduleDelaySeconds, which determines how often the pool will run the
management plugin to verify and adjust the resource usage.
* Each pool also has plugin-specific parameters, most notably the limits - eg.
max total cache size, which the CacheManagerPlugin knows how to use in order to
adjust cache sizes.
* Each managed resource can be managed by multiple types of plugins and it may
appear in multiple pools (of different types). This reflects the fact that a
single component may have multiple aspects of resource management - eg. cache
mgmt, cpu, threads, etc.
The patch also contains an example implementation of a management plugin -
{{CacheManagerPlugin}}. This plugin uses the API to enforce global limits on
the cache size. It knows how to retrieve and calculate the current resource
usage, as reported by the monitored values, and then it adjusts the controlled
limits of each resource to bring the usage back to the total values that fit
within the limits defined by the pool. In this case the pool can define global
limits on the cache {{size}} and {{maxRamMB}} (and these are also the
parameters to control for each cache), and the plugin uses {{size}} and
{{ramBytesUsed}} for monitoring the actual resource consumption.
Obviously {{SolrCache}} doesn't implement this API yet, but it's relatively
easy to add.
I'd appreciate review, comments and suggestions.
> Create resource management API
> ------------------------------
>
> Key: SOLR-13579
> URL: https://issues.apache.org/jira/browse/SOLR-13579
> Project: Solr
> Issue Type: Sub-task
> Security Level: Public(Default Security Level. Issues are Public)
> Reporter: Andrzej Bialecki
> Assignee: Andrzej Bialecki
> Priority: Major
> Attachments: SOLR-13579.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
