[
https://issues.apache.org/jira/browse/SOLR-5200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13753781#comment-13753781
]
Michael Della Bitta commented on SOLR-5200:
-------------------------------------------
We've wanted the ability to tune commit properties for bulk indexing, and then
switch to more incremental indexing-friendly setup on the fly, for a while. +1.
> Add REST support for reading and modifying Solr configuration
> -------------------------------------------------------------
>
> Key: SOLR-5200
> URL: https://issues.apache.org/jira/browse/SOLR-5200
> Project: Solr
> Issue Type: New Feature
> Reporter: Steve Rowe
> Assignee: Steve Rowe
>
> There should be a REST API to allow full read access to, and write access to
> some elements of, Solr's per-core and per-node configuration not already
> covered by the Schema REST API:
> {{solrconfig.xml}}/{{core.properties}}/{{solrcore.properties}} and
> {{solr.xml}}/{{solr.properties}} (SOLR-4718 discusses addition of
> {{solr.properties}}).
> Use cases for runtime configuration modification include scripted setup,
> troubleshooting, and tuning.
> Tentative rules-of-thumb about configuration items that should not be
> modifiable at runtime:
> # Startup-only items, e.g. where to start core discovery
> # Items that are deprecated in 4.X and will be removed in 5.0
> # Items that if modified should be followed by a full re-index
> Some issues to consider:
> Persistence: How (and even whether) to handle persistence for configuration
> modifications via REST API is not clear - e.g. persisting the entire config
> file or having one or more sidecar config files that get persisted. The
> extent of what should be modifiable will likely affect how persistence is
> implemented. For example, if the only {{solrconfig.xml}} modifiable items
> turn out to be plugin configurations, an alternative to
> full-{{solrconfig.xml}} persistence could be individual plugin registration
> of runtime config modifiable items, along with per-plugin sidecar config
> persistence.
> "Live" reload: Most (if not all) per-core configuration modifications will
> require core reload, though it will be a "live" reload, so some things won't
> be modifiable, e.g. {{<dataDir>}} and {{IndexWriter}} related settings in
> {{<indexConfig>}} - see SOLR-3592. (Should a full reload be supported to
> handle changes in these places?)
> Interpolation aka property substitution: I think it would be useful on read
> access to optionally return raw values in addition to the interpolated
> values, e.g. {{solr.xml}} {{hostPort}} raw value {{$\{jetty.port:8983}}} vs.
> interpolated value {{8983}}. Modification requests will accept raw values -
> property interpolation will be applied. At present interpolation is done
> once, at parsing time, but if property value modification is supported via
> the REST API, an alternative could be to delay interpolation until values are
> requested; in this way, property value modification would not trigger
> re-parsing the affected configuration source.
> Response format: Similarly to the schema REST API, results could be returned
> in XML, JSON, or any other response writer's output format.
> Transient cores: How should non-loaded transient cores be handled? Simplest
> thing would be to load the transient core before handling the request, just
> like other requests.
> Below I provide an exhaustive list of configuration items in the files in
> question and indicate which ones I think could be modifiable at runtime. I
> don't mean to imply that these must all be made modifiable, or for those that
> are made modifiable, that they must be made so at once - a piecemeal approach
> will very likely be more appropriate.
> h2. {{solrconfig.xml}}
> Note that XIncludes and includes via Document Entities won't survive a
> modification request (assuming persistence is via overwriting the original
> file).
> ||XPath under {{/config/}}||Should be modifiable via REST
> API?||Rationale||Description||
> |{{luceneMatchVersion}}|No|Modifying this should be followed by a full
> re-index|Controls what version of Lucene various components of Solr adhere to|
> |{{lib}}|Yes|Required for adding plugins at runtime|Contained jars available
> via classloader for {{solrconfig.xml}} and {{schema.xml}}|
> |{{dataDir}}|No|Not supported by "live" RELOAD|Holds all index data|
> |{{directoryFactory}}|No|Not supported by "live" RELOAD|index directory
> factory|
> |{{codecFactory}}|No|Modifying this should be followed by a full
> re-index|index codec factory, per-field SchemaCodecFactory by default|
> |{{schemaFactory}}|Partial|Although the class shouldn't be modifiable, it
> should be possible to modify an already Managed schema's mutability|Managed
> or Classic (non-mutable) schema factory|
> |{{indexConfig}}|No|{{IndexWriter}}-related settings not supported by "live"
> RELOAD|low-level indexing behavior|
> |{{jmx}}|Yes| |Enables JMX if an MBeanServer is found|
> |{{updateHandler@class}}|No| |Defaults to DirectUpdateHandler2|
> |{{updateHandler/updateLog}}|No| |Enables a transaction log, configures its
> directory and synchronization|
> |{{updateHandler/autoCommit}}|Yes| |Durability: enables hard autocommit,
> configures max interval and whether to open a searcher afterward|
> |{{updateHandler/autoSoftCommit}}|Yes| |Visibility: enables soft autocommit,
> configures max interval|
> |{{updateHandler/commitWithin/softCommit}}|Yes| |Whether commitWithin update
> request param should trigger a soft commit instead of hard commit|
> |{{updateHandler/listener}}|Yes| |Update-related event listeners, e.g.
> snapshooter|
> |{{indexReaderFactory}}|No| |Specify custom index reader factory (default
> StandardIndexReaderFactory)|
> |{{query/maxBooleanClauses}}|Yes| |Maximum boolean clauses allowed in a query|
> |{{query/filterCache}}|Yes| |Enables the filter cache - unordered docsets,
> configures class, initial size, max size, and entries to pull from an old
> cache|
> |{{query/queryResultCache}}|Yes| |Enables the query result cache - ordered
> docid lists,configures class, initial size, max size, and entries to pull
> from an old cache|
> |{{query/documentCache}}|Yes| |Enables the document cache - document stored
> fields, configures class, initial size, and max size|
> |{{query/fieldValueCache}}|Yes| |Enables the field value cache - field values
> by docid, created by default, configures class, size, # entries to report
> stats for (showItems)|
> |{{query/cache}}|Yes| |Enables a custom cache, configures name, class,
> initial size, max size, regenerator class, and entries to pull from an old
> cache|
> |{{query/enableLazyFieldLoading}}|Yes| |Whether to enable lazy field loading|
> |{{query/useFilterForSortedQuery}}|Yes| |Whether to use a filter for a sorted
> non-scoring search|
> |{{query/queryResultWindowSize}}|Yes| |Cached result window size|
> |{{query/queryResultMaxDocsCached}}|Yes| |Maximum number of documents to
> cache for any entry in the queryResultCache|
> |{{query/listener}}|Yes| |Query-related event listener, configures event
> type, class, and queries, e.g. newSearcher and firstSearcher events with
> solr.QuerySenderListener|
> |{{query/useColdSearcher}}|Yes| |Whether to interrupt searcher warming to
> service a query request if there are no registered searchers|
> |{{query/maxWarmingSearchers}}|Yes| |Max searchers to warm|
> |{{requestDispatcher}}|Yes| |Configures SolrDispatchFilter behavior,
> Including {{requestParsers}} and {{httpCaching}}|
> |{{requestHandler}}|Yes| |Configures request handlers, including
> SearchHandler, RealTimeGetHandler, UpdateRequestHandler, ReplicationHandler,
> etc., and their URL path mapping (name)|
> |{{searchComponent}}|Yes| |Configures search components available to
> SearchHandlers|
> |{{updateRequestProcessorChain}}|Yes| |Configures named update request
> processor chains usable by UpdateRequestHandler|
> |{{queryResponseWriter}}|Yes| |Configures named response writers|
> |{{queryParser}}|Yes| |Configures query parser plugins|
> |{{valueSourceParser}}|Yes| |Configures named function parsers, usable by the
> "func" QParser|
> |{{transformer}}|Yes| |Configures named document transformers, which
> transform documents returned to the user, e.g. adding fields - defaults are
> explain, value, shard, docid|
> |{{admin/defaultQuery}}|No| |Legacy config for the admin UI|
> h2. {{core.properties}}
> {{core.properties}} marks a core directory. Each core will parse its
> {{solrconfig.xml}} using these properties.
> I don't think any of the Solr-internal properties in this file should be
> modifiable at runtime: "name", "config", "instanceDir", "absoluteInstDir",
> "dataDir", "ulogDir", "schema", "shard", "collection", "roles", "properties",
> "loadOnStartup", "transient", "coreNodeName". But it would be useful to
> allow for addition/modification of user-defined properties here.
> Read/write access will be provided, both for individual properties and in
> bulk. {{solrconfig.xml}} will need to be re-parsed using new property
> values; alternatively, interpolation could be delayed until values are
> accessed. Problem: changing properties that aren't valid in a "live" RELOAD
> - see SOLR-3592.
> h2. {{solrcore.properties}}
> {{solrcore.properties}} is a per-config-set properties map used to
> interpolate property values when parsing {{solrconfig.xml}}.
> Read/write access will be provided, both for individual properties and in
> bulk. {{solrconfig.xml}} will need to be re-parsed using new property
> values; alternatively, interpolation could be delayed until values are
> accessed. Problem: changing properties that aren't valid in a "live" RELOAD
> - see SOLR-3592.
> h2. {{solr.xml}}
> {{solr.xml}} is used to configure multi-core and SolrCloud features.
> Most of the configuration items in this file are related to startup-only
> operations, and so shouldn't be changed at runtime.
> ||XPath under {{/solr/}} (4.X old-style)||||XPath under {{/solr/}} (5.0 and
> 4.4+ core discovery style)||Should be modifiable via REST
> API?||Description/rationale||
> |{{@persistent}}|N/A|No|Deprecated in 4.X old-style, removed in 5.0 and 4.4+
> core discovery style|
> |{{cores/@defaultCoreName}}|N/A|No|Deprecated in 4.X old-style, removed in
> 5.0 and 4.4+ core discovery style|
> |{{cores/@adminPath}}|N/A|No|Removed in 5.0, where it's always
> {{/admin/cores}} |
> |N/A|{{str[@name='coreRootDirectory']}}|No|The root of the core discovery
> tree, defaults to the solrhome|
> |{{@coreLoadThreads}}|{{int[@name='coreLoadThreads']}}|Yes|Core loading fixed
> thread pool size|
> |{{@sharedLib}}|{{str[@name='sharedLib']}}|No|Lib directory used by all cores
> on the same node|
> |{{cores/@adminHandler}}|{{str[@name='adminHandler']}}|No|Admin handler
> class, CoreAdminHandler by default|
> |{{cores/@managementPath}}|{{str[@name='managementPath']}}|No|Request URL
> path prefix that gets stripped by SolrDispatchFilter|
> |{{cores/@shareSchema}}|{{str[@name='shareSchema']}}|No|Whether to cache and
> share schema object among cores on the same node|
> |{{cores/@transientCacheSize}}|{{int[@name='transientCacheSize']}}|Yes|Max
> active transient cores; reducing this would trigger immediate unloading|
> |{{cores/shardHandlerFactory}}|{{shardHandlerFactory}}|No|Shard handler
> factory class and configuration|
> |{{logging/@class}}|{{logging/str[@name='class']}}|No|Logging class|
> |{{logging/@enabled}}|{{logging/str[@name='enabled']}}|Yes|Whether to enable
> logging|
> |{{logging/watcher/@size}}|{{logging/watcher/int[@name='size']}}|Yes|Max log
> history entries|
> |{{logging/watcher/@threshold}}|{{logging/watcher/int[@name='threshold']}}|No|Root
> logger level; per-logger level setting already available through
> LoggingHandler via the {{/admin/logging}} endpoint|
> |{{@zkHost}}|{{solrcloud/str[@name='zkHost']}}|No|SolrCloud: ZooKeeper host
> holding cluster state|
> |{{cores/@distribUpdateConnTimeout}}|{{solrcloud/int[@name='distribUpdateConnTimeout']}}|No|SolrCloud:
> initial distributed update connection timeout|
> |{{cores/@distribUpdateSoTimeout}}|{{solrcloud/int[@name='distribUpdateSoTimeout']}}|No|SolrCloud:
> distributed update socket read timeout|
> |{{cores/@host}}|{{solrcloud/str[@name='host']}}|No|SolrCloud: Local Solr
> host name|
> |{{cores/@hostContext}}|{{solrcloud/str[@name='hostContext']}}|No|SolrCloud:
> Local Solr servlet context path|
> |{{cores/@hostPort}}|{{solrcloud/int[@name='hostPort']}}|No|SolrCloud: Local
> Solr host port|
> |{{cores/@leaderVoteWait}}|{{solrcloud/int[@name='leaderVoteWait']}}|No|SolrCloud:
> Leader vote wait time (ms)|
> |{{cores/@genericCoreNodeNames}}|{{solrcloud/bool[@name='genericCoreNodeNames']"}}|No|SolrCloud:
> If true, don't base core node names on the node address|
> |{{cores/@zkClientTimeout}}|{{solrcloud/int[@name='zkClientTimeout']}}|No|SolrCloud:
> ZooKeeper connection timeout|
> h2. {{solr.properties}}
> Contains local per-node (not in ZooKeeper) properties used to parse
> {{solr.xml}}.
> Read/write access will be provided, both for individual properties and in
> bulk. {{solr.xml}} will need to be re-parsed using new property values.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
