[
https://issues.apache.org/jira/browse/SOLR-646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Henri Biestro updated SOLR-646:
-------------------------------
Description:
This patch refers to 'generalized configuration properties' as specified by
[HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
This means configuration & schema files can use expression based on properties
defined in multicore.xml.
h3. Use cases:
Describe core data directories from solr.xml as properties.
Share the same schema and/or config file between multiple cores.
Share reusable fragments of schemar & configuration between multiple cores.
h3. Usage:
h4. solr.xml
This*solr.xml* will be used to illustrates using properties for different
purpose.
{code:xml}
<solr persistent="true">
<property name="version">1.3</property>
<property name="lang">english, french</property>
<property name="en-cores">en,core0</property>
<property name="fr-cores">fr,core1</property>
<cores adminPath="/admin/cores">
<core name="${en-cores}" instanceDir="./">
<property name="version">3.5</property>
<property name="l10n">EN</property>
<property name="ctlField">core0</property>
<property name="comment">This is a sample</property>
</core>
<core name="${fr-cores}" instanceDir="./">
<property name="version">2.4</property>
<property name="l10n">FR</property>
<property name="ctlField">core1</property>
<property name="comment">Ceci est un exemple</property>
</core>
</cores>
</solr>
{code}
{{version}} : if you update your solr.xml or your cores for various motives, it
can be useful to track of a version. In this example, this will be used to
define the {{dataDir}} for each core.
{{en-cores}},{{fr-cores}}: with aliases, if the list is long or repetitive, it
might be convenient to use a property that can then be used to describe the
Solr core name.
{{instanceDir}}: note that both cores will use the same instance directory,
sharing their configuration and schema. The {{dataDir}} will be set for each of
them from the *solrconfig.xml*.
h4. solrconfig.xml
This is where our *solr.xml* property are used to define the data directory as
a composition of, in our example, the language code {{l10n}} and the core
version stored in {{version}}.
{code:xml}
<config>
<dataDir>${solr.solr.home}/data/${l10n}-${version}</dataDir>
....
</config>
{code}
h5. schema.xml
The {{include}} allows to import a file within the schema (or a solrconfig);
this can help de-clutter long schemas.
The {{ctlField}} is just illustrating that a field & its type can be set
through properties as well; in our example, we will want the 'english' core to
refer to an 'english-configured' field and the 'french' core to a
'french-configured' one. The type for the field is defined as {{text-EN}} or
{{text-FR}} after expansion.
{code:xml}
<schema name="example core ${l10n}" version="1.1">
<types>
...
<include resource="text-l10n.xml"/>
</types>
<fields>
...
<field name="${ctlField}" type="text-${l10n}" indexed="true"
stored="true" multiValued="true" />
</fields>
{code}
This schema is importing this* text-l10n.xml* file which is a *fragment*; the
fragment tag must be present & indicates the file is to be included. Our
example only defines different stopwords for each language but you could of
course extend this to stemmers, synonyms, etc.
{code:xml}
<fragment>
<fieldType name="text-FR" class="solr.TextField"
positionIncrementGap="100">
...
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="stopwords-fr.txt"/>
...
</fieldType>
<fieldType name="text-EN" class="solr.TextField"
positionIncrementGap="100">
...
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="stopwords-en.txt"/>
...
</fieldType>
</fragment>
{code}
h4. Technical specifications
solr.xml can define properties at the multicore & each core level.
Properties defined in the multicore scope can override system properties.
Properties defined in a core scope can override multicore & system properties.
Property definitions can use expressions to define their name & value; these
expressions are evaluated in their outer scope context .
CoreContainer serialization keeps properties as defined; persistence is
idem-potent. (ie property expressions are written, not their evaluation).
The core descriptor properties are automatically defined in each core context,
namely:
solr.core.instanceDir
solr.core.name
solr.core.configName
solr.core.schemaName
h3. Coding notes:
- DOMUtil.java:
refactored substituteSystemProperties to use an Evaluator;
an Evaluator is a DOM visitor that expands property expressions "in place"
using a property map as an evaluation context
added an asString(node) method for logging purpose
- CoreDescriptor.java:
added an expression member to keep property expressions as defined in solr.xml
for persistence - allowing to write file as defined (not as expanded)
- CoreContainer.java:
add an expression member to keep property expression as defined in solr.xml for
persistence - allowing to write file as defined (not as expanded);
solrx.xml peristence is idem-potent
added a local DOMUtil.Evaluator that tracks property expressions to evaluate &
store them
*issues outlined through solr-646:*
fix in load:
CoreDescriptor p = new CoreDescriptor(this, names, ....);
was: CoreDescriptor p = new CoreDescriptor(this, name, ...);
fix in load;
register(aliases.get(a), core, false);
was of register(aliases.get(i), core, false);
- CoreAdminHandler.java
added an optional fileName to persist so it is possible to write the solr.xml
to a different file (for comparison purpose)
- CoreAdminRequest.java
added PersistRequest to allow passing optional fileName
- Config.java:
subsituteProperties has been moved out of constructor & doc member made
protected to allow override
added an IncludesEvaluator that deals with include/fragment
- SolrConfig.java & IndexSchema.ava
added explicit calls to substituteProperties to perform property/include
expansion
- SolrResourceLoader.java
added properties member to store CoreContainer & per-SolrCore properties
added constructor properties parameter & getter for properties
- SolrProperties.java:
test inspired by MulticoreExampleTestBase.java
loads 2 cores sharing a schema & config;
config define dataDir using a property
schema uses a localization (l10n) property to define an attribute
persists the file to check it keeps the expression properties
was:
This patch refers to 'generalized configuration properties' as specified by
[HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
This means configuration & schema files can use expression based on properties
defined in multicore.xml.
h3. Use cases:
Describe core data directories from solr.xml as properties.
Share the same schema and/or config file between multiple cores.
Share reusable fragments of schemar & configuration between multiple cores.
h3. Usage:
h4. solr.xml
This*solr.xml* will be used to illustrates using properties for different
purpose.
{code:xml}
<solr persistent="true">
<property name="version">1.3</property>
<property name="lang">english, french</property>
<property name="en-cores">en,core0</property>
<property name="fr-cores">fr,core1</property>
<cores adminPath="/admin/cores">
<core name="${en-cores}" instanceDir="./">
<property name="version">3.5</property>
<property name="l10n">EN</property>
<property name="ctlField">core0</property>
<property name="comment">This is a sample</property>
</core>
<core name="${fr-cores}" instanceDir="./">
<property name="version">2.4</property>
<property name="l10n">FR</property>
<property name="ctlField">core1</property>
<property name="comment">Ceci est un exemple</property>
</core>
</cores>
</solr>
{code}
{{version}} : if you update your solr.xml or your cores for various motives, it
can be useful to track of a version. In this example, this will be used to
define the {{dataDir}} for each core.
{{en-cores}},{{fr-cores}}: with aliases, if the list is long or repetitive, it
might be convenient to use a property that can then be used to describe the
Solr core name.
{{instanceDir}}: note that both cores will use the same instance directory,
sharing their configuration and schema. The {{dataDir}} will be set for each of
them from the *solrconfig.xml*.
h4. solrconfig.xml
This is where our *solr.xml* property are used to define the data directory as
a composition of, in our example, the language code {{l10n}} and the core
version stored in {{version}}.
{code:xml}
<config>
<dataDir>${solr.solr.home}/data/${l10n}-${version}</dataDir>
....
</config>
{code}
h5. schema.xml
The {{include}} allows to import a file within the schema (or a solrconfig);
this can help de-clutter long schemas.
The {{ctlField}} is just illustrating that a field & its type can be set
through properties as well; in our example, we will want the 'english' core to
refer to an 'english-configured' field and the 'french' core to a
'french-configured' one. The type for the field is defined as {{text-EN}} or
{{text-FR}} after expansion.
{code:xml}
<schema name="example core ${l10n}" version="1.1">
<types>
...
<include resource="text-l10n.xml"/>
</types>
<fields>
...
<field name="${ctlField}" type="text-${l10n}" indexed="true"
stored="true" multiValued="true" />
</fields>
{code}
This schema is importing this* text-l10n.xml* file which is a *fragment*; the
fragment tag must be present & indicates the file is to be included. Our
example only defines different stopwords for each language but you could of
course extend this to stemmers, synonyms, etc.
{code:xml}
<fragment>
<fieldType name="text-FR" class="solr.TextField"
positionIncrementGap="100">
...
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="stopwords-fr.txt"/>
...
</fieldType>
<fieldType name="text-EN" class="solr.TextField"
positionIncrementGap="100">
...
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="stopwords-en.txt"/>
...
</fieldType>
</fragment>
{code}
h4. Technical specifications
solr.xml can define properties at the multicore & each core level.
Properties defined in the multicore scope can override system properties.
Properties defined in a core scope can override multicore & system properties.
Property definitions can use expressions to define their name & value; these
expressions are evaluated in their outer scope context .
CoreContainer serialization keeps properties as defined; persistence is
idem-potent. (ie property expressions are written, not their evaluation).
The core descriptor properties are automatically defined in each core context,
namely:
solr.core.instanceDir
solr.core.name
solr.core.configName
solr.core.schemaName
h3. Coding notes:
DOMUtil.java:
refactored substituteSystemProperties to use an Evaluator;
an Evaluator is a DOM visitor that expands property expressions "in place"
using a property map as an evaluation context
added an asString(node) method for logging purpose
CoreDescriptor.java:
added an expression member to keep property expressions as defined in solr.xml
for persistence - allowing to write file as defined (not as expanded)
CoreContainer.java:
add an expression member to keep property expression as defined in solr.xml for
persistence - allowing to write file as defined (not as expanded);
solrx.xml peristence is idem-potent
added a local DOMUtil.Evaluator that tracks property expressions to evaluate &
store them
issues outlined through solr-646:
fix in load:
CoreDescriptor p = new CoreDescriptor(this, names, ....);
was: CoreDescriptor p = new CoreDescriptor(this, name, ...);
fix in load;
register(aliases.get(a), core, false);
was of register(aliases.get(i), core, false);
CoreAdminHandler.java
added an optional fileName to persist so it is possible to write the solr.xml
to a different file (for comparison purpose)
CoreAdminRequest.java
added PersistRequest to allow passing optional fileName
Config.java:
subsituteProperties has been moved out of constructor & doc member made
protected to allow override
added an IncludesEvaluator that deals with include/fragment
SolrConfig.java & IndexSchema.ava
added explicit calls to substituteProperties to perform property/include
expansion
SolrResourceLoader.java
added properties member to store CoreContainer & per-SolrCore properties
added constructor properties parameter & getter for properties
added initial property loading allowing installation wide properties &
solr.solr.home to be defined in a 'solr.properties' file
SolrProperties.java:
test inspired by MulticoreExampleTestBase.java
loads 2 cores sharing a schema & config;
config define dataDir using a property
schema uses a localization (l10n) property to define an attribute
persists the file to check it keeps the expression properties
> Configuration properties in multicore.xml
> -----------------------------------------
>
> Key: SOLR-646
> URL: https://issues.apache.org/jira/browse/SOLR-646
> Project: Solr
> Issue Type: New Feature
> Affects Versions: 1.3
> Reporter: Henri Biestro
> Assignee: Shalin Shekhar Mangar
> Fix For: 1.3
>
> Attachments: solr-646.patch, SOLR-646.patch, solr-646.patch,
> solr-646.patch, solr-646.patch, solr-646.patch, solr-646.patch
>
>
> This patch refers to 'generalized configuration properties' as specified by
> [HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
> This means configuration & schema files can use expression based on
> properties defined in multicore.xml.
> h3. Use cases:
> Describe core data directories from solr.xml as properties.
> Share the same schema and/or config file between multiple cores.
> Share reusable fragments of schemar & configuration between multiple cores.
> h3. Usage:
> h4. solr.xml
> This*solr.xml* will be used to illustrates using properties for different
> purpose.
> {code:xml}
> <solr persistent="true">
> <property name="version">1.3</property>
> <property name="lang">english, french</property>
> <property name="en-cores">en,core0</property>
> <property name="fr-cores">fr,core1</property>
> <cores adminPath="/admin/cores">
> <core name="${en-cores}" instanceDir="./">
> <property name="version">3.5</property>
> <property name="l10n">EN</property>
> <property name="ctlField">core0</property>
> <property name="comment">This is a sample</property>
> </core>
> <core name="${fr-cores}" instanceDir="./">
> <property name="version">2.4</property>
> <property name="l10n">FR</property>
> <property name="ctlField">core1</property>
> <property name="comment">Ceci est un exemple</property>
> </core>
> </cores>
> </solr>
> {code}
> {{version}} : if you update your solr.xml or your cores for various motives,
> it can be useful to track of a version. In this example, this will be used to
> define the {{dataDir}} for each core.
> {{en-cores}},{{fr-cores}}: with aliases, if the list is long or repetitive,
> it might be convenient to use a property that can then be used to describe
> the Solr core name.
> {{instanceDir}}: note that both cores will use the same instance directory,
> sharing their configuration and schema. The {{dataDir}} will be set for each
> of them from the *solrconfig.xml*.
> h4. solrconfig.xml
> This is where our *solr.xml* property are used to define the data directory
> as a composition of, in our example, the language code {{l10n}} and the core
> version stored in {{version}}.
> {code:xml}
> <config>
> <dataDir>${solr.solr.home}/data/${l10n}-${version}</dataDir>
> ....
> </config>
> {code}
> h5. schema.xml
> The {{include}} allows to import a file within the schema (or a solrconfig);
> this can help de-clutter long schemas.
> The {{ctlField}} is just illustrating that a field & its type can be set
> through properties as well; in our example, we will want the 'english' core
> to refer to an 'english-configured' field and the 'french' core to a
> 'french-configured' one. The type for the field is defined as {{text-EN}} or
> {{text-FR}} after expansion.
> {code:xml}
> <schema name="example core ${l10n}" version="1.1">
> <types>
> ...
> <include resource="text-l10n.xml"/>
> </types>
> <fields>
> ...
> <field name="${ctlField}" type="text-${l10n}" indexed="true"
> stored="true" multiValued="true" />
> </fields>
> {code}
> This schema is importing this* text-l10n.xml* file which is a *fragment*; the
> fragment tag must be present & indicates the file is to be included. Our
> example only defines different stopwords for each language but you could of
> course extend this to stemmers, synonyms, etc.
> {code:xml}
> <fragment>
> <fieldType name="text-FR" class="solr.TextField"
> positionIncrementGap="100">
> ...
> <filter class="solr.StopFilterFactory" ignoreCase="true"
> words="stopwords-fr.txt"/>
> ...
> </fieldType>
> <fieldType name="text-EN" class="solr.TextField"
> positionIncrementGap="100">
> ...
> <filter class="solr.StopFilterFactory" ignoreCase="true"
> words="stopwords-en.txt"/>
> ...
> </fieldType>
> </fragment>
> {code}
> h4. Technical specifications
> solr.xml can define properties at the multicore & each core level.
> Properties defined in the multicore scope can override system properties.
> Properties defined in a core scope can override multicore & system properties.
> Property definitions can use expressions to define their name & value; these
> expressions are evaluated in their outer scope context .
> CoreContainer serialization keeps properties as defined; persistence is
> idem-potent. (ie property expressions are written, not their evaluation).
> The core descriptor properties are automatically defined in each core
> context, namely:
> solr.core.instanceDir
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> h3. Coding notes:
> - DOMUtil.java:
> refactored substituteSystemProperties to use an Evaluator;
> an Evaluator is a DOM visitor that expands property expressions "in place"
> using a property map as an evaluation context
> added an asString(node) method for logging purpose
> - CoreDescriptor.java:
> added an expression member to keep property expressions as defined in
> solr.xml for persistence - allowing to write file as defined (not as expanded)
> - CoreContainer.java:
> add an expression member to keep property expression as defined in solr.xml
> for persistence - allowing to write file as defined (not as expanded);
> solrx.xml peristence is idem-potent
> added a local DOMUtil.Evaluator that tracks property expressions to evaluate
> & store them
> *issues outlined through solr-646:*
> fix in load:
> CoreDescriptor p = new CoreDescriptor(this, names, ....);
> was: CoreDescriptor p = new CoreDescriptor(this, name, ...);
> fix in load;
> register(aliases.get(a), core, false);
> was of register(aliases.get(i), core, false);
> - CoreAdminHandler.java
> added an optional fileName to persist so it is possible to write the solr.xml
> to a different file (for comparison purpose)
> - CoreAdminRequest.java
> added PersistRequest to allow passing optional fileName
> - Config.java:
> subsituteProperties has been moved out of constructor & doc member made
> protected to allow override
> added an IncludesEvaluator that deals with include/fragment
> - SolrConfig.java & IndexSchema.ava
> added explicit calls to substituteProperties to perform property/include
> expansion
> - SolrResourceLoader.java
> added properties member to store CoreContainer & per-SolrCore properties
> added constructor properties parameter & getter for properties
> - SolrProperties.java:
> test inspired by MulticoreExampleTestBase.java
> loads 2 cores sharing a schema & config;
> config define dataDir using a property
> schema uses a localization (l10n) property to define an attribute
> persists the file to check it keeps the expression properties
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
