[ https://issues.apache.org/jira/browse/SOLR-646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12619330#action_12619330 ]
Henri Biestro commented on SOLR-646: ------------------------------------ About documentation: I'll try to come with better (ie more meaningful) examples - although I kinda feel "impaired" to describe the 3 tags (property/import/fragment) if what I've already written does not convey 90% of how things would/should work already. I welcome any help/hint on what a useful documentation should look like (any template I could/should follow)? On the documentation editing/releasing process, I'd tend to go for refining this issue description till the code gets committed at which time I'd copy/paste it in a Wiki page if that's OK. About "automatic" properties: Those reflect the Core descriptor values or in more general terms, any Solr install/multicore/core configuration/schema value used in the system but defined through another API; the core name, instance directory, configuration file name (defaults to solrconfig.xml), schema file name (defaults to schema.xml) all come form the core descriptor . solr.core.instanceDir is strictly what the core descriptor instance directory is; solr.core.instancePath is the non-relative version of it. When solr.core.instanceDir is *not* an absolute path, is is relative to the solr.home instance directory; in this case, solr.core.instancePath is the non relative version. Otherwise, they are just the same; overall, this just reflect what the code does but removes any ambiguous implicits (which is what the PropertyMap.imports() also resolves*). I've also added the solr.core.schemaDir & solr.core.configDir- the directories where the config & schema files reside. We have no way yet to specify those last two but they are inferred from the other values - which would be their defaults if un-specified.There is a functional choice to make about how we want to complete the "core" properties; the "missing" parts are the config & schema directories. My preference would be to extend the core descriptor and reflect those as properties but we might want&try to go the other way, put everything as properties & retain the API as shortcuts. Regarding naming conventions/protection, I would tend to go for the natural attribute-like convention & protect those; the automatic/reserved namespaces would be any property starting with 'solr.' (including 'solr.core' and 'solr.multicore') and disallow/warn about any attempt to override. Would that be acceptable/enough ? About files/substituted properties & admin JSPs: As is, nothing has changed; getting the schema/config return the file as it exists in the file-system. The first natural extension would be that for config & schema, when allowed, the imported files should be downloadable too. It might be nice to have the download expanded version of each file for debugging purpose as well if this is actually useful and/or the expanded version of config & schema; any functional suggestions is welcome. Lance, what do you think would be good to have? About PropertyMap.imports(): The rationale behind this -besides debugging/logging purpose- would be "dynamic" replication of a core (a slave Slr 'install' could automatically replicate new "master' cores. The goal of this API is to have the canonical (local) path of each file - so we can read them- and their logical/symbolic counterparts so their destination (remote) path can be determined, each core's solrconfig.xml itself being "replicable". About the issue itself: How much of these should be specified/implemented before the patch can be committed and/or is there any chance to put this in the 1.3 scope ? > Configuration properties in multicore.xml > ----------------------------------------- > > Key: SOLR-646 > URL: https://issues.apache.org/jira/browse/SOLR-646 > Project: Solr > Issue Type: New Feature > Affects Versions: 1.3 > Reporter: Henri Biestro > Fix For: 1.3 > > Attachments: solr-646.patch, solr-646.patch > > > This patch refers to 'generalized configuration properties' as specified by > [HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834] > This means configuration & schema files can use expression based on > properties defined in multicore.xml. > h3. Use cases: > Share the same schema and/or config file between multiple cores but point to > different dataDirs using a <dataDir>${core.datadir}</dataDir> > Change the general layout between data, config & schema directories. > Define 'installation' wide properties (for replication for instance) in > multicore.properties (different base/install/data directories on different > hosts). > h3. Syntax: > h4. Defining properties: > Either in the multicore.properties file (usual property format): > {code:xml} > env.value=an installation value > env.other_value=another installation value > {code} > In the multicore.xml: > {code:xml} > <multicore'> > <property name='mp0'>a value</property> <!-- a basic property --> > <property name='mp1'>${env.value}</property> }<!-- a property whose value > is an expression --> > <core name='core_name'> > <property name='p0'>some value</property> > <property name='p1'>${mp0}-and some data</property> > </core> > </multicore> > {code} > h4. Using properties: > Besides used defined properties, the following core descriptor properties are > automatically defined in each core context, namely: > {code} > solr.core.instanceDir > solr.core.instancePath > solr.core.name > solr.core.configName > solr.core.schemaName > {code} > All properties can be used in any attribute or text node of a solrconfig.xml > or schema.xml as in: > {code:xml} > <dataDir>${core.dataDir}</dataDir> > {code} > h4. Technical specifications > Multicore.xml can define properties at the multicore & each core level. > Properties defined in the multicore scope can override system properties. > Properties defined in a core scope can override multicore & system properties. > Property definitions can use expressions to define their name & value; these > expressions are evaluated in their outer scope context . > Multicore serialization keeps properties as written (ie as expressions if > they were defined so). > The core descriptor properties are automatically defined in each core > context, namely: > solr.core.instanceDir > solr.core.instancePath > solr.core.name > solr.core.configName > solr.core.schemaName > h4. Test: > The following (contrived) multicore.xml: > {code:xml} > <multicore adminPath='/admin/multicore' persistent='true'> > <property name='revision'>33</property> <!-- a basic property --> > <property name='zero'>0</property> <!-- used to expand the core0 name --> > <property name='one'>1</property> <!-- used to expand the core1 name --> > <property name='id_type'>bogus</property> <!-- a bogus type that will be > overriden --> > <property name='updateHandler'>bogus</property> <!-- a bogus updateHandler > that will be overriden --> > <core name='core${zero}' instanceDir='core0/'> <!-- the name is expanded > --> > <property name='id_type'>core${zero}_id</property> <!-- so is a text node > --> > <property name='updateHandler'>solr.DirectUpdateHandler2</property> <!-- > a property can be overriden --> > <property name='revision'>11</property> > </core> > <core name='core${one}' instanceDir='core1/'> > <property name='id_type'>core${one}_id</property> > <property name='updateHandler'>solr.DirectUpdateHandler2</property> > <property name='revision'>22</property> > </core> > </multicore> > {code} > allows this config.xml: > {code:xml} > <config> > <!-- use the defined update handler property --> > <updateHandler class="${updateHandler}" /> > <requestDispatcher handleSelect="true" > > <requestParsers enableRemoteStreaming="false" > multipartUploadLimitInKB="2048" /> > </requestDispatcher> > > <requestHandler name="standard" class="solr.StandardRequestHandler" > default="true" /> > <requestHandler name="/update" class="solr.XmlUpdateRequestHandler" /> > <requestHandler name="/admin/luke" > class="org.apache.solr.handler.admin.LukeRequestHandler" /> > > <!-- config for the admin interface --> > <admin> > <defaultQuery>solr</defaultQuery> > <gettableFiles>solrconfig.xml schema.xml admin-extra.html</gettableFiles> > <pingQuery> > qt=standard&q=solrpingquery > </pingQuery> > </admin> > </config> > {code} > and this schema.xml: > {code:xml} > <schema name="example core zero" version="1.1"> > <types> > <!-- define a type name dynamically --> > <fieldtype name="${id_type:id_t}" class="solr.StrField" > sortMissingLast="true" omitNorms="true"/> > <fieldtype name="string" class="solr.StrField" sortMissingLast="true" > omitNorms="true"/> > </types> > <fields> > <!-- the type of unique key defined above --> > <field name="id" type="${id_type:id_t}" indexed="true" > stored="true" multiValued="false" required="true"/> > <field name="type" type="string" indexed="true" stored="true" > multiValued="false" /> > <field name="name" type="string" indexed="true" stored="true" > multiValued="false" /> > <field name="${solr.core.name:core}" type="string" indexed="true" > stored="true" multiValued="false" /> > </fields> > <uniqueKey>id</uniqueKey> > <defaultSearchField>name</defaultSearchField> > <solrQueryParser defaultOperator="OR"/> > </schema> > {code} > Allow the trunk test to work. > h3. Coding notes: > The code itself refactored some of DOMUtil (the ant based property > substitution) into one added class (PropertyMap & PropertyMap.Evaluator). > The PropertyMap are chained (one link chain between core to multicore map); > those maps are owned by each core's ResourceLoader. > Config is modified a little to accommodate delaying & specializing property > expansions. > Multicore is modified so it properly parses & serializes. > Tested against the example above. > Reviews & comments more than welcome. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.