[ https://issues.apache.org/jira/browse/SOLR-646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12624935#action_12624935 ]
Hoss Man commented on SOLR-646: ------------------------------- I've attempted to catch up on the comments on this issue, and somewhere i got pretty lost (i haven't even tried looking at the patches yet) but FWIW: there is at least the huge use cases i know of in favor of supporting include/import related to master/slave differences... # only masters should trigger snapshot creation in postCommit/postOptimize hooks, only slaves should use QuerySenderListener to do static warming of things in firstSearcher/newSearcher # different cache configs for master/slave -- want smaller caches on maters, with little/no autowarm, and in some cases don't want certain user caches to exist at all # don't register certain intensive request handlers on masters at all, so people don't inadvertantly hose the master by querying it directly. ...some of these things might be achieved by having a *lot* of properties, but ideally you want one property with a value of either "master" or "slave" and use that as part of hte name of fragment files to include. At CNET, we've been using m4 macros to generate master/slave configs from the same template file since forever, but it would certainly be nice to have Solr take care of this natively -- and it would make promoting a slave to a master in failover situations trivial (just bounce the port with one JNDI variable changed) *BUT* Solr users have lived without this feature for along time, not need to rush it if hte parties involved think it's better to hold off until after 1.3 > Configuration properties in multicore.xml > ----------------------------------------- > > Key: SOLR-646 > URL: https://issues.apache.org/jira/browse/SOLR-646 > Project: Solr > Issue Type: New Feature > Affects Versions: 1.3 > Reporter: Henri Biestro > Assignee: Shalin Shekhar Mangar > Fix For: 1.4 > > Attachments: solr-646.patch, solr-646.patch, SOLR-646.patch, > solr-646.patch, solr-646.patch, solr-646.patch, solr-646.patch, solr-646.patch > > > This patch refers to 'generalized configuration properties' as specified by > [HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834] > This means configuration & schema files can use expression based on > properties defined in *solr.xml*. > h3. Use cases: > Describe core data directories from solr.xml as properties. > Share the same schema and/or config file between multiple cores. > Share reusable fragments of schema & configuration between multiple cores. > h3. Usage: > h4. solr.xml > This *solr.xml* will be used to illustrates using properties for different > purpose. > {code:xml} > <solr persistent="true"> > <property name="version" value="1.3"/> > <property name="lang" value="english, french"/> > <property name="en-cores" value="en,core0"/> > <property name="fr-cores" value="fr,core1"/> > <!-- This experimental feature flag enables schema & solrconfig to include > other files --> > <property name="solr.experimental.enableConfigInclude" value="true"/> > <cores adminPath="/admin/cores"> > <core name="${en-cores}" instanceDir="./"> > <property name="version" value="3.5"/> > <property name="l10n" value="EN"/> > <property name="ctlField" value="core0"/> > <property name="comment" value="This is a sample"/> > </core> > <core name="${fr-cores}" instanceDir="./"> > <property name="version" value="2.4"/> > <property name="l10n" value="FR"/> > <property name="ctlField" value="core1"/> > <property name="comment" value="Ceci est un exemple"/> > </core> > </cores> > </solr> > {code} > {{version}} : if you update your solr.xml or your cores for various motives, > it can be useful to track of a version. In this example, this will be used to > define the {{dataDir}} for each core. > {{en-cores}},{{fr-cores}}: with aliases, if the list is long or repetitive, > it might be convenient to use a property that can then be used to describe > the Solr core name. > {{instanceDir}}: note that both cores will use the same instance directory, > sharing their configuration and schema. The {{dataDir}} will be set for each > of them from the *solrconfig.xml*. > h4. solrconfig.xml > This is where our *solr.xml* property are used to define the data directory > as a composition of, in our example, the language code {{l10n}} and the core > version stored in {{version}}. > {code:xml} > <config> > <dataDir>${solr.solr.home}/data/${l10n}-${version}</dataDir> > .... > </config> > {code} > h5. schema.xml > The {{include}} allows to import a file within the schema (or a solrconfig); > this can help de-clutter long schemas or reuse parts. > {color:red}This is an experimental feature that may not be kept in the > future.{color} > The {{ctlField}} is just illustrating that a field & its type can be set > through properties as well; in our example, we will want the 'english' core > to refer to an 'english-configured' field and the 'french' core to a > 'french-configured' one. The type for the field is defined as {{text-EN}} or > {{text-FR}} after expansion. > {code:xml} > <schema name="example core ${l10n}" version="1.1"> > <types> > ... > <include resource="text-l10n.xml"/> > </types> > <fields> > ... > <field name="${ctlField}" type="text-${l10n}" indexed="true" > stored="true" multiValued="true" /> > </fields> > {code} > This schema is importing this *text-l10n.xml* file which is a *fragment*; the > fragment tag must be present & indicates the file is to be included. Our > example only defines different stopwords for each language but you could of > course extend this to stemmers, synonyms, etc. > {code:xml} > <fragment> > <fieldType name="text-FR" class="solr.TextField" > positionIncrementGap="100"> > ... > <filter class="solr.StopFilterFactory" ignoreCase="true" > words="stopwords-fr.txt"/> > ... > </fieldType> > <fieldType name="text-EN" class="solr.TextField" > positionIncrementGap="100"> > ... > <filter class="solr.StopFilterFactory" ignoreCase="true" > words="stopwords-en.txt"/> > ... > </fieldType> > </fragment> > {code} > h4. Technical specifications > solr.xml can define properties at the multicore & each core level. > Properties defined in the multicore scope can override system properties. > Properties defined in a core scope can override multicore & system properties. > Property definitions can use expressions to define their name & value; these > expressions are evaluated in their outer scope context . > CoreContainer serialization keeps properties as defined; persistence is > idem-potent. (ie property expressions are written, not their evaluation). > The core descriptor properties are automatically defined in each core > context, namely: > solr.core.instanceDir > solr.core.name > solr.core.configName > solr.core.schemaName > h3. Coding notes: > - DOMUtil.java: > refactored substituteSystemProperties to use an Evaluator; > an Evaluator is a DOM visitor that expands property expressions "in place" > using a property map as an evaluation context > added an asString(node) method for logging purpose > - CoreDescriptor.java: > added an expression member to keep property expressions as defined in > solr.xml for persistence - allowing to write file as defined (not as expanded) > - CoreContainer.java: > add an expression member to keep property expression as defined in solr.xml > for persistence - allowing to write file as defined (not as expanded); > solrx.xml peristence is idem-potent > added a local DOMUtil.Evaluator that tracks property expressions to evaluate > & store them > *issues outlined through solr-646:* > fix in load: > CoreDescriptor p = new CoreDescriptor(this, names, ....); > was: CoreDescriptor p = new CoreDescriptor(this, name, ...); > fix in load; > register(aliases.get(a), core, false); > was of register(aliases.get(i), core, false); > - CoreAdminHandler.java > added an optional fileName to persist so it is possible to write the solr.xml > to a different file (for comparison purpose) > - CoreAdminRequest.java > added PersistRequest to allow passing optional fileName > - Config.java: > subsituteProperties has been moved out of constructor & doc member made > protected to allow override > added an IncludesEvaluator that deals with include/fragment > - SolrConfig.java & IndexSchema.ava > added explicit calls to substituteProperties to perform property/include > expansion > - SolrResourceLoader.java > added properties member to store CoreContainer & per-SolrCore properties > added constructor properties parameter & getter for properties > - SolrProperties.java: > test inspired by MulticoreExampleTestBase.java > loads 2 cores sharing a schema & config; > config define dataDir using a property > schema uses a localization (l10n) property to define an attribute > persists the file to check it keeps the expression properties -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.