[
https://issues.apache.org/jira/browse/SOLR-646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Henri Biestro updated SOLR-646:
-------------------------------
Description:
This patch refers to 'generalized configuration properties' as specified by
[HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
This means configuration & schema files can use expression based on properties
defined in multicore.xml.
h3. Use cases:
Share the same schema and/or config file between multiple cores but point to
different dataDirs using a <dataDir>${core.datadir}</dataDir>
Change the general layout between data, config & schema directories.
Define 'installation' wide properties (for replication for instance) in
multicore.properties (different base/install/data directories on different
hosts).
h3. Syntax:
h4. Defining properties:
Either in the multicore.properties file (usual property format):
{code:xml}
env.value=an installation value
env.other_value=another installation value
{code}
In the multicore.xml:
{code:xml}
<multicore'>
<property name='mp0'>a value</property> <!-- a basic property -->
<property name='mp1'>${env.value}</property> }<!-- a property whose value is
an expression -->
<core name='core_name'>
<property name='p0'>some value</property>
<property name='p1'>${mp0}-and some data</property>
</core>
</multicore>
{code}
h4. Using properties:
Besides used defined properties, the following core descriptor properties are
automatically defined in each core context, namely:
{code}
solr.core.instanceDir
solr.core.instancePath
solr.core.name
solr.core.configName
solr.core.schemaName
{code}
All properties can be used in any attribute or text node of a solrconfig.xml or
schema.xml as in:
{code:xml}
<dataDir>${core.dataDir}</dataDir>
{code}
h4. Technical specifications
Multicore.xml can define properties at the multicore & each core level.
Properties defined in the multicore scope can override system properties.
Properties defined in a core scope can override multicore & system properties.
Property definitions can use expressions to define their name & value; these
expressions are evaluated in their outer scope context .
Multicore serialization keeps properties as written (ie as expressions if they
were defined so).
The core descriptor properties are automatically defined in each core context,
namely:
solr.core.instanceDir
solr.core.instancePath
solr.core.name
solr.core.configName
solr.core.schemaName
h4. Test:
The following (contrived) multicore.xml:
{code:xml}
<multicore adminPath='/admin/multicore' persistent='true'>
<property name='revision'>33</property> <!-- a basic property -->
<property name='zero'>0</property> <!-- used to expand the core0 name -->
<property name='one'>1</property> <!-- used to expand the core1 name -->
<property name='id_type'>bogus</property> <!-- a bogus type that will be
overriden -->
<property name='updateHandler'>bogus</property> <!-- a bogus updateHandler
that will be overriden -->
<core name='core${zero}' instanceDir='core0/'> <!-- the name is expanded
-->
<property name='id_type'>core${zero}_id</property> <!-- so is a text node
-->
<property name='updateHandler'>solr.DirectUpdateHandler2</property> <!-- a
property can be overriden -->
<property name='revision'>11</property>
</core>
<core name='core${one}' instanceDir='core1/'>
<property name='id_type'>core${one}_id</property>
<property name='updateHandler'>solr.DirectUpdateHandler2</property>
<property name='revision'>22</property>
</core>
</multicore>
{code}
allows this config.xml:
{code:xml}
<config>
<!-- use the defined update handler property -->
<updateHandler class="${updateHandler}" />
<requestDispatcher handleSelect="true" >
<requestParsers enableRemoteStreaming="false"
multipartUploadLimitInKB="2048" />
</requestDispatcher>
<requestHandler name="standard" class="solr.StandardRequestHandler"
default="true" />
<requestHandler name="/update" class="solr.XmlUpdateRequestHandler" />
<requestHandler name="/admin/luke"
class="org.apache.solr.handler.admin.LukeRequestHandler" />
<!-- config for the admin interface -->
<admin>
<defaultQuery>solr</defaultQuery>
<gettableFiles>solrconfig.xml schema.xml admin-extra.html</gettableFiles>
<pingQuery>
qt=standard&q=solrpingquery
</pingQuery>
</admin>
</config>
{code}
and this schema.xml:
{code:xml}
<schema name="example core zero" version="1.1">
<types>
<!-- define a type name dynamically -->
<fieldtype name="${id_type:id_t}" class="solr.StrField"
sortMissingLast="true" omitNorms="true"/>
<fieldtype name="string" class="solr.StrField" sortMissingLast="true"
omitNorms="true"/>
</types>
<fields>
<!-- the type of unique key defined above -->
<field name="id" type="${id_type:id_t}" indexed="true" stored="true"
multiValued="false" required="true"/>
<field name="type" type="string" indexed="true" stored="true"
multiValued="false" />
<field name="name" type="string" indexed="true" stored="true"
multiValued="false" />
<field name="${solr.core.name:core}" type="string" indexed="true"
stored="true" multiValued="false" />
</fields>
<uniqueKey>id</uniqueKey>
<defaultSearchField>name</defaultSearchField>
<solrQueryParser defaultOperator="OR"/>
</schema>
{code}
Allow the trunk test to work.
h3. Coding notes:
The code itself refactored some of DOMUtil (the ant based property
substitution) into one added class (PropertyMap & PropertyMap.Evaluator).
The PropertyMap are chained (one link chain between core to multicore map);
those maps are owned by each core's ResourceLoader.
Config is modified a little to accommodate delaying & specializing property
expansions.
Multicore is modified so it properly parses & serializes.
Tested against the example above.
Reviews & comments more than welcome.
was:
This patch refers to 'generalized configuration properties' as specified by
[HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
This means configuration & schema files can use expression based on properties
defined in multicore.xml.
Use cases:
Share the same schema and/or config file between multiple cores but point to
different dataDirs using a <dataDir>${core.datadir}</dataDir>
Change the general layout between data, config & schema directories.
Define 'installation' wide properties (for replication for instance) in
multicore.properties (different base/install/data directories on different
hosts).
The following (contrived) multicore.xml:
{code:xml}
<multicore adminPath='/admin/multicore' persistent='true'>
<property name='revision'>33</property> <!-- a basic property -->
<property name='zero'>0</property> <!-- used to expand the core0 name -->
<property name='one'>1</property> <!-- used to expand the core1 name -->
<property name='id_type'>bogus</property> <!-- a bogus type that will be
overriden -->
<property name='updateHandler'>bogus</property> <!-- a bogus updateHandler
that will be overriden -->
<core name='core${zero}' instanceDir='core0/'> <!-- the name is expanded
-->
<property name='id_type'>core${zero}_id</property> <!-- so is a text node
-->
<property name='updateHandler'>solr.DirectUpdateHandler2</property> <!-- a
property can be overriden -->
<property name='revision'>11</property>
</core>
<core name='core${one}' instanceDir='core1/'>
<property name='id_type'>core${one}_id</property>
<property name='updateHandler'>solr.DirectUpdateHandler2</property>
<property name='revision'>22</property>
</core>
</multicore>
{code}
allows this config.xml:
{code:xml}
<config>
<!-- use the defined update handler property -->
<updateHandler class="${updateHandler}" />
<requestDispatcher handleSelect="true" >
<requestParsers enableRemoteStreaming="false"
multipartUploadLimitInKB="2048" />
</requestDispatcher>
<requestHandler name="standard" class="solr.StandardRequestHandler"
default="true" />
<requestHandler name="/update" class="solr.XmlUpdateRequestHandler" />
<requestHandler name="/admin/luke"
class="org.apache.solr.handler.admin.LukeRequestHandler" />
<!-- config for the admin interface -->
<admin>
<defaultQuery>solr</defaultQuery>
<gettableFiles>solrconfig.xml schema.xml admin-extra.html</gettableFiles>
<pingQuery>
qt=standard&q=solrpingquery
</pingQuery>
</admin>
</config>
{code}
and this schema.xml:
{code:xml}
<schema name="example core zero" version="1.1">
<types>
<!-- define a type name dynamically -->
<fieldtype name="${id_type:id_t}" class="solr.StrField"
sortMissingLast="true" omitNorms="true"/>
<fieldtype name="string" class="solr.StrField" sortMissingLast="true"
omitNorms="true"/>
</types>
<fields>
<!-- the type of unique key defined above -->
<field name="id" type="${id_type:id_t}" indexed="true" stored="true"
multiValued="false" required="true"/>
<field name="type" type="string" indexed="true" stored="true"
multiValued="false" />
<field name="name" type="string" indexed="true" stored="true"
multiValued="false" />
<field name="${solr.core.name:core}" type="string" indexed="true"
stored="true" multiValued="false" />
</fields>
<uniqueKey>id</uniqueKey>
<defaultSearchField>name</defaultSearchField>
<solrQueryParser defaultOperator="OR"/>
</schema>
{code}
Multicore.xml can define properties at the multicore & each core level.
Properties defined in the multicore scope can override system properties.
Properties defined in a core scope can override multicore & system properties.
Property definitions can use expressions to define their name & value; these
expressions are evaluated in their outer scope context .
Multicore serialization keeps properties as written (ie as expressions if they
were defined so).
The core descriptor properties are automatically defined in each core context,
namely:
solr.core.instanceDir
solr.core.instancePath
solr.core.name
solr.core.configName
solr.core.schemaName
The code itself refactored some of DOMUtil (the ant based property
substitution) into one added class (PropertyMap & PropertyMap.Evaluator).
The PropertyMap are chained (one link chain between core to multicore map);
those maps are owned by each core's ResourceLoader.
Config is modified a little to accommodate delaying & specializing property
expansions.
Multicore is modified so it properly parses & serializes.
Tested against the example above.
Reviews & comments more than welcome.
> Configuration properties in multicore.xml
> -----------------------------------------
>
> Key: SOLR-646
> URL: https://issues.apache.org/jira/browse/SOLR-646
> Project: Solr
> Issue Type: New Feature
> Affects Versions: 1.3
> Reporter: Henri Biestro
> Fix For: 1.3
>
> Attachments: solr-646.patch
>
>
> This patch refers to 'generalized configuration properties' as specified by
> [HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
> This means configuration & schema files can use expression based on
> properties defined in multicore.xml.
> h3. Use cases:
> Share the same schema and/or config file between multiple cores but point to
> different dataDirs using a <dataDir>${core.datadir}</dataDir>
> Change the general layout between data, config & schema directories.
> Define 'installation' wide properties (for replication for instance) in
> multicore.properties (different base/install/data directories on different
> hosts).
> h3. Syntax:
> h4. Defining properties:
> Either in the multicore.properties file (usual property format):
> {code:xml}
> env.value=an installation value
> env.other_value=another installation value
> {code}
> In the multicore.xml:
> {code:xml}
> <multicore'>
> <property name='mp0'>a value</property> <!-- a basic property -->
> <property name='mp1'>${env.value}</property> }<!-- a property whose value
> is an expression -->
> <core name='core_name'>
> <property name='p0'>some value</property>
> <property name='p1'>${mp0}-and some data</property>
> </core>
> </multicore>
> {code}
> h4. Using properties:
> Besides used defined properties, the following core descriptor properties are
> automatically defined in each core context, namely:
> {code}
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> {code}
> All properties can be used in any attribute or text node of a solrconfig.xml
> or schema.xml as in:
> {code:xml}
> <dataDir>${core.dataDir}</dataDir>
> {code}
> h4. Technical specifications
> Multicore.xml can define properties at the multicore & each core level.
> Properties defined in the multicore scope can override system properties.
> Properties defined in a core scope can override multicore & system properties.
> Property definitions can use expressions to define their name & value; these
> expressions are evaluated in their outer scope context .
> Multicore serialization keeps properties as written (ie as expressions if
> they were defined so).
> The core descriptor properties are automatically defined in each core
> context, namely:
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> h4. Test:
> The following (contrived) multicore.xml:
> {code:xml}
> <multicore adminPath='/admin/multicore' persistent='true'>
> <property name='revision'>33</property> <!-- a basic property -->
> <property name='zero'>0</property> <!-- used to expand the core0 name -->
> <property name='one'>1</property> <!-- used to expand the core1 name -->
> <property name='id_type'>bogus</property> <!-- a bogus type that will be
> overriden -->
> <property name='updateHandler'>bogus</property> <!-- a bogus updateHandler
> that will be overriden -->
> <core name='core${zero}' instanceDir='core0/'> <!-- the name is expanded
> -->
> <property name='id_type'>core${zero}_id</property> <!-- so is a text node
> -->
> <property name='updateHandler'>solr.DirectUpdateHandler2</property> <!--
> a property can be overriden -->
> <property name='revision'>11</property>
> </core>
> <core name='core${one}' instanceDir='core1/'>
> <property name='id_type'>core${one}_id</property>
> <property name='updateHandler'>solr.DirectUpdateHandler2</property>
> <property name='revision'>22</property>
> </core>
> </multicore>
> {code}
> allows this config.xml:
> {code:xml}
> <config>
> <!-- use the defined update handler property -->
> <updateHandler class="${updateHandler}" />
> <requestDispatcher handleSelect="true" >
> <requestParsers enableRemoteStreaming="false"
> multipartUploadLimitInKB="2048" />
> </requestDispatcher>
>
> <requestHandler name="standard" class="solr.StandardRequestHandler"
> default="true" />
> <requestHandler name="/update" class="solr.XmlUpdateRequestHandler" />
> <requestHandler name="/admin/luke"
> class="org.apache.solr.handler.admin.LukeRequestHandler" />
>
> <!-- config for the admin interface -->
> <admin>
> <defaultQuery>solr</defaultQuery>
> <gettableFiles>solrconfig.xml schema.xml admin-extra.html</gettableFiles>
> <pingQuery>
> qt=standard&q=solrpingquery
> </pingQuery>
> </admin>
> </config>
> {code}
> and this schema.xml:
> {code:xml}
> <schema name="example core zero" version="1.1">
> <types>
> <!-- define a type name dynamically -->
> <fieldtype name="${id_type:id_t}" class="solr.StrField"
> sortMissingLast="true" omitNorms="true"/>
> <fieldtype name="string" class="solr.StrField" sortMissingLast="true"
> omitNorms="true"/>
> </types>
> <fields>
> <!-- the type of unique key defined above -->
> <field name="id" type="${id_type:id_t}" indexed="true"
> stored="true" multiValued="false" required="true"/>
> <field name="type" type="string" indexed="true" stored="true"
> multiValued="false" />
> <field name="name" type="string" indexed="true" stored="true"
> multiValued="false" />
> <field name="${solr.core.name:core}" type="string" indexed="true"
> stored="true" multiValued="false" />
> </fields>
> <uniqueKey>id</uniqueKey>
> <defaultSearchField>name</defaultSearchField>
> <solrQueryParser defaultOperator="OR"/>
> </schema>
> {code}
> Allow the trunk test to work.
> h3. Coding notes:
> The code itself refactored some of DOMUtil (the ant based property
> substitution) into one added class (PropertyMap & PropertyMap.Evaluator).
> The PropertyMap are chained (one link chain between core to multicore map);
> those maps are owned by each core's ResourceLoader.
> Config is modified a little to accommodate delaying & specializing property
> expansions.
> Multicore is modified so it properly parses & serializes.
> Tested against the example above.
> Reviews & comments more than welcome.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
