[
https://issues.apache.org/jira/browse/SOLR-646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12619330#action_12619330
]
Henri Biestro commented on SOLR-646:
------------------------------------
About documentation:
I'll try to come with better (ie more meaningful) examples - although I kinda
feel "impaired" to describe the 3 tags (property/import/fragment) if what I've
already written does not convey 90% of how things would/should work already. I
welcome any help/hint on what a useful documentation should look like (any
template I could/should follow)?
On the documentation editing/releasing process, I'd tend to go for refining
this issue description till the code gets committed at which time I'd
copy/paste it in a Wiki page if that's OK.
About "automatic" properties:
Those reflect the Core descriptor values or in more general terms, any Solr
install/multicore/core configuration/schema value used in the system but
defined through another API; the core name, instance directory, configuration
file name (defaults to solrconfig.xml), schema file name (defaults to
schema.xml) all come form the core descriptor .
solr.core.instanceDir is strictly what the core descriptor instance directory
is; solr.core.instancePath is the non-relative version of it. When
solr.core.instanceDir is *not* an absolute path, is is relative to the
solr.home instance directory; in this case, solr.core.instancePath is the non
relative version. Otherwise, they are just the same; overall, this just reflect
what the code does but removes any ambiguous implicits (which is what the
PropertyMap.imports() also resolves*).
I've also added the solr.core.schemaDir & solr.core.configDir- the directories
where the config & schema files reside. We have no way yet to specify those
last two but they are inferred from the other values - which would be their
defaults if un-specified.There is a functional choice to make about how we want
to complete the "core" properties; the "missing" parts are the config & schema
directories. My preference would be to extend the core descriptor and reflect
those as properties but we might want&try to go the other way, put everything
as properties & retain the API as shortcuts.
Regarding naming conventions/protection, I would tend to go for the natural
attribute-like convention & protect those; the automatic/reserved namespaces
would be any property starting with 'solr.' (including 'solr.core' and
'solr.multicore') and disallow/warn about any attempt to override. Would that
be acceptable/enough ?
About files/substituted properties & admin JSPs:
As is, nothing has changed; getting the schema/config return the file as it
exists in the file-system. The first natural extension would be that for config
& schema, when allowed, the imported files should be downloadable too. It might
be nice to have the download expanded version of each file for debugging
purpose as well if this is actually useful and/or the expanded version of
config & schema; any functional suggestions is welcome. Lance, what do you
think would be good to have?
About PropertyMap.imports():
The rationale behind this -besides debugging/logging purpose- would be
"dynamic" replication of a core (a slave Slr 'install' could automatically
replicate new "master' cores. The goal of this API is to have the canonical
(local) path of each file - so we can read them- and their logical/symbolic
counterparts so their destination (remote) path can be determined, each core's
solrconfig.xml itself being "replicable".
About the issue itself:
How much of these should be specified/implemented before the patch can be
committed and/or is there any chance to put this in the 1.3 scope ?
> Configuration properties in multicore.xml
> -----------------------------------------
>
> Key: SOLR-646
> URL: https://issues.apache.org/jira/browse/SOLR-646
> Project: Solr
> Issue Type: New Feature
> Affects Versions: 1.3
> Reporter: Henri Biestro
> Fix For: 1.3
>
> Attachments: solr-646.patch, solr-646.patch
>
>
> This patch refers to 'generalized configuration properties' as specified by
> [HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
> This means configuration & schema files can use expression based on
> properties defined in multicore.xml.
> h3. Use cases:
> Share the same schema and/or config file between multiple cores but point to
> different dataDirs using a <dataDir>${core.datadir}</dataDir>
> Change the general layout between data, config & schema directories.
> Define 'installation' wide properties (for replication for instance) in
> multicore.properties (different base/install/data directories on different
> hosts).
> h3. Syntax:
> h4. Defining properties:
> Either in the multicore.properties file (usual property format):
> {code:xml}
> env.value=an installation value
> env.other_value=another installation value
> {code}
> In the multicore.xml:
> {code:xml}
> <multicore'>
> <property name='mp0'>a value</property> <!-- a basic property -->
> <property name='mp1'>${env.value}</property> }<!-- a property whose value
> is an expression -->
> <core name='core_name'>
> <property name='p0'>some value</property>
> <property name='p1'>${mp0}-and some data</property>
> </core>
> </multicore>
> {code}
> h4. Using properties:
> Besides used defined properties, the following core descriptor properties are
> automatically defined in each core context, namely:
> {code}
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> {code}
> All properties can be used in any attribute or text node of a solrconfig.xml
> or schema.xml as in:
> {code:xml}
> <dataDir>${core.dataDir}</dataDir>
> {code}
> h4. Technical specifications
> Multicore.xml can define properties at the multicore & each core level.
> Properties defined in the multicore scope can override system properties.
> Properties defined in a core scope can override multicore & system properties.
> Property definitions can use expressions to define their name & value; these
> expressions are evaluated in their outer scope context .
> Multicore serialization keeps properties as written (ie as expressions if
> they were defined so).
> The core descriptor properties are automatically defined in each core
> context, namely:
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> h4. Test:
> The following (contrived) multicore.xml:
> {code:xml}
> <multicore adminPath='/admin/multicore' persistent='true'>
> <property name='revision'>33</property> <!-- a basic property -->
> <property name='zero'>0</property> <!-- used to expand the core0 name -->
> <property name='one'>1</property> <!-- used to expand the core1 name -->
> <property name='id_type'>bogus</property> <!-- a bogus type that will be
> overriden -->
> <property name='updateHandler'>bogus</property> <!-- a bogus updateHandler
> that will be overriden -->
> <core name='core${zero}' instanceDir='core0/'> <!-- the name is expanded
> -->
> <property name='id_type'>core${zero}_id</property> <!-- so is a text node
> -->
> <property name='updateHandler'>solr.DirectUpdateHandler2</property> <!--
> a property can be overriden -->
> <property name='revision'>11</property>
> </core>
> <core name='core${one}' instanceDir='core1/'>
> <property name='id_type'>core${one}_id</property>
> <property name='updateHandler'>solr.DirectUpdateHandler2</property>
> <property name='revision'>22</property>
> </core>
> </multicore>
> {code}
> allows this config.xml:
> {code:xml}
> <config>
> <!-- use the defined update handler property -->
> <updateHandler class="${updateHandler}" />
> <requestDispatcher handleSelect="true" >
> <requestParsers enableRemoteStreaming="false"
> multipartUploadLimitInKB="2048" />
> </requestDispatcher>
>
> <requestHandler name="standard" class="solr.StandardRequestHandler"
> default="true" />
> <requestHandler name="/update" class="solr.XmlUpdateRequestHandler" />
> <requestHandler name="/admin/luke"
> class="org.apache.solr.handler.admin.LukeRequestHandler" />
>
> <!-- config for the admin interface -->
> <admin>
> <defaultQuery>solr</defaultQuery>
> <gettableFiles>solrconfig.xml schema.xml admin-extra.html</gettableFiles>
> <pingQuery>
> qt=standard&q=solrpingquery
> </pingQuery>
> </admin>
> </config>
> {code}
> and this schema.xml:
> {code:xml}
> <schema name="example core zero" version="1.1">
> <types>
> <!-- define a type name dynamically -->
> <fieldtype name="${id_type:id_t}" class="solr.StrField"
> sortMissingLast="true" omitNorms="true"/>
> <fieldtype name="string" class="solr.StrField" sortMissingLast="true"
> omitNorms="true"/>
> </types>
> <fields>
> <!-- the type of unique key defined above -->
> <field name="id" type="${id_type:id_t}" indexed="true"
> stored="true" multiValued="false" required="true"/>
> <field name="type" type="string" indexed="true" stored="true"
> multiValued="false" />
> <field name="name" type="string" indexed="true" stored="true"
> multiValued="false" />
> <field name="${solr.core.name:core}" type="string" indexed="true"
> stored="true" multiValued="false" />
> </fields>
> <uniqueKey>id</uniqueKey>
> <defaultSearchField>name</defaultSearchField>
> <solrQueryParser defaultOperator="OR"/>
> </schema>
> {code}
> Allow the trunk test to work.
> h3. Coding notes:
> The code itself refactored some of DOMUtil (the ant based property
> substitution) into one added class (PropertyMap & PropertyMap.Evaluator).
> The PropertyMap are chained (one link chain between core to multicore map);
> those maps are owned by each core's ResourceLoader.
> Config is modified a little to accommodate delaying & specializing property
> expansions.
> Multicore is modified so it properly parses & serializes.
> Tested against the example above.
> Reviews & comments more than welcome.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
