[jira] Updated: (SOLR-646) Configuration properties in multicore.xml
[
https://issues.apache.org/jira/browse/SOLR-646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Henri Biestro updated SOLR-646:
---
Description:
This patch refers to 'generalized configuration properties' as specified by
[HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
This means configuration & schema files can use expression based on properties
defined in *solr.xml*.
h3. Use cases:
Describe core data directories from solr.xml as properties.
Share the same schema and/or config file between multiple cores.
Share reusable fragments of schema & configuration between multiple cores.
h3. Usage:
h4. solr.xml
This *solr.xml* will be used to illustrates using properties for different
purpose.
{code:xml}
{code}
{{version}} : if you update your solr.xml or your cores for various motives, it
can be useful to track of a version. In this example, this will be used to
define the {{dataDir}} for each core.
{{en-cores}},{{fr-cores}}: with aliases, if the list is long or repetitive, it
might be convenient to use a property that can then be used to describe the
Solr core name.
{{instanceDir}}: note that both cores will use the same instance directory,
sharing their configuration and schema. The {{dataDir}} will be set for each of
them from the *solrconfig.xml*.
h4. solrconfig.xml
This is where our *solr.xml* property are used to define the data directory as
a composition of, in our example, the language code {{l10n}} and the core
version stored in {{version}}.
{code:xml}
${solr.solr.home}/data/${l10n}-${version}
{code}
h5. schema.xml
The {{include}} allows to import a file within the schema (or a solrconfig);
this can help de-clutter long schemas or reuse parts.
The {{ctlField}} is just illustrating that a field & its type can be set
through properties as well; in our example, we will want the 'english' core to
refer to an 'english-configured' field and the 'french' core to a
'french-configured' one. The type for the field is defined as {{text-EN}} or
{{text-FR}} after expansion.
{code:xml}
...
...
{code}
This schema is importing this *text-l10n.xml* file which is a *fragment*; the
fragment tag must be present & indicates the file is to be included. Our
example only defines different stopwords for each language but you could of
course extend this to stemmers, synonyms, etc.
{code:xml}
...
...
...
...
{code}
Alternatively, one can use XML entities using the 'solr:' protocol to the same
end as in:
{code:xml}
]>
&textL10n;
...
{code}
h4. Technical specifications
solr.xml can define properties at the multicore & each core level.
Properties defined in the multicore scope can override system properties.
Properties defined in a core scope can override multicore & system properties.
Property definitions can use expressions to define their name & value; these
expressions are evaluated in their outer scope context .
CoreContainer serialization keeps properties as defined; persistence is
idem-potent. (ie property expressions are written, not their evaluation).
The core descriptor properties are automatically defined in each core context,
namely:
solr.core.instanceDir
solr.core.name
solr.core.configName
solr.core.schemaName
h3. Coding notes:
- DOMUtil.java:
cosmetic changes
toMapExcept systematically skips 'xml:base" attributes (which may come from
entity resolving)
- CoreDescriptor.java:
The core descriptor does not store properties as values but as expressions (and
all its members can be property expressions as well) allowing to write file as
defined (not as evaluated)
The public getCoreProperties is removed for that reason. (too bad we were in
such a rush...)
- CoreContainer.java:
changes related to extracting the core names before they are evaluated in load()
changes related to evaluating core descriptor member before adding them to the
core's loader properties
fix in persistFile which was not interpreting relative pathes correctly
fix in persist because properties were not written at the right place
changes in persist to write expressions (and core name when it is one)
- Config.java:
subsituteProperties has been moved out of constructor so calls must be explicit.
added the entity resolver
added subsituteIncludes which processes
- SolrConfig.java & IndexSchema.java
added explicit calls to substituteIncludesto perform property/include expansion
- SolrResourceLoader.java
cosmetic, changed getCoreProperties to getProperties (since they may come from
the CoreContainer)
- SolrProperties.java:
schema uses a localization (l10n) property to define an attribute
persists the file to check it keeps the expression properties
[jira] Updated: (SOLR-646) Configuration properties in multicore.xml
[
https://issues.apache.org/jira/browse/SOLR-646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Henri Biestro updated SOLR-646:
---
Attachment: solr-646.patch
Updated to be based on SOLR-716 commit. Thanks Shalin.
Some improvement over SOLR--716 :
- in solr.xml, /Solr/cores/core attributes can use properties defined in /solr
or system (including name/aliases)
- persistence of property expressions (rather than values)
- All core descriptor members can be expressions (related to SOLR-715 )
- fixes a bug in persistence related to SOLR-718 commit (will have to create an
issue for this)
- If a core was declared with an expression as its name and its aliases haven't
changed, persistence will rewrite the property expression.
That is, if you solr.xml does something like:
{code:xml}
...
...
{code}
The expression '${en-cores}' will be written back (if you did not realias the
core of course).
Some improvements over previous version:
There are now 2 ways to include files
One is a tribute to SOLR-712 (Thanks Amit) which means to include through
entities with the twist that an entity resolver exploit properties in URIs in
the 'solr:' protocol:
{code:xml}
]>
&textL10n;
...
{code}
Or the easier to read way (imho):
{code:xml}
...
{code}
There are tests for both versions.
Now, we've got options. :-)
> Configuration properties in multicore.xml
> -
>
> Key: SOLR-646
> URL: https://issues.apache.org/jira/browse/SOLR-646
> Project: Solr
> Issue Type: New Feature
>Affects Versions: 1.3
>Reporter: Henri Biestro
>Assignee: Shalin Shekhar Mangar
> Fix For: 1.4
>
> Attachments: solr-646.patch, solr-646.patch, solr-646.patch,
> SOLR-646.patch, solr-646.patch, solr-646.patch, solr-646.patch,
> solr-646.patch, solr-646.patch
>
>
> This patch refers to 'generalized configuration properties' as specified by
> [HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
> This means configuration & schema files can use expression based on
> properties defined in *solr.xml*.
> h3. Use cases:
> Describe core data directories from solr.xml as properties.
> Share the same schema and/or config file between multiple cores.
> Share reusable fragments of schema & configuration between multiple cores.
> h3. Usage:
> h4. solr.xml
> This *solr.xml* will be used to illustrates using properties for different
> purpose.
> {code:xml}
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> {code}
> {{version}} : if you update your solr.xml or your cores for various motives,
> it can be useful to track of a version. In this example, this will be used to
> define the {{dataDir}} for each core.
> {{en-cores}},{{fr-cores}}: with aliases, if the list is long or repetitive,
> it might be convenient to use a property that can then be used to describe
> the Solr core name.
> {{instanceDir}}: note that both cores will use the same instance directory,
> sharing their configuration and schema. The {{dataDir}} will be set for each
> of them from the *solrconfig.xml*.
> h4. solrconfig.xml
> This is where our *solr.xml* property are used to define the data directory
> as a composition of, in our example, the language code {{l10n}} and the core
> version stored in {{version}}.
> {code:xml}
>
> ${solr.solr.home}/data/${l10n}-${version}
>
>
> {code}
> h5. schema.xml
> The {{include}} allows to import a file within the schema (or a solrconfig);
> this can help de-clutter long schemas or reuse parts.
> {color:red}This is an experimental feature that may not be kept in the
> future.{color}
> The {{ctlField}} is just illustrating that a field & its type can be set
> through properties as well; in our example, we will want the 'english' core
> to refer to an 'english-configured' field and the 'french' core to a
> 'french-configured' one. The type for the field is defined as {{text-EN}} or
> {{text-FR}} after expansion.
> {code:xml}
>
>
> ...
>
>
>
> ...
>stored="true" multiValued="true" />
>
> {code}
> This schema is importing this *text-l10n.xml* file which is a *fragment*; the
> fragment tag must be present & indicates the file is to be included. Our
> example only defines different stopwords for each language but you could of
> course extend this to stemmers, synonyms, etc.
> {code:xml}
>
>positionIncrementGap="100">
> ...
>words="stopwords-fr.txt"/>
> ...
>
>positionIncrementGap="100">
> ...
>words="stopwords-en.txt"/>
> ...
>
>
> {code}
> h4. Technical specifications
> solr.xml can define properties at the multicore & each core level.
> Properti
[jira] Updated: (SOLR-646) Configuration properties in multicore.xml
[
https://issues.apache.org/jira/browse/SOLR-646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shalin Shekhar Mangar updated SOLR-646:
---
Fix Version/s: (was: 1.3)
1.4
Per Henri's suggestion, marking this for 1.4 -- the core issues will be taken
care of in SOLR-716 in 1.3 itself
> Configuration properties in multicore.xml
> -
>
> Key: SOLR-646
> URL: https://issues.apache.org/jira/browse/SOLR-646
> Project: Solr
> Issue Type: New Feature
>Affects Versions: 1.3
>Reporter: Henri Biestro
>Assignee: Shalin Shekhar Mangar
> Fix For: 1.4
>
> Attachments: solr-646.patch, solr-646.patch, SOLR-646.patch,
> solr-646.patch, solr-646.patch, solr-646.patch, solr-646.patch, solr-646.patch
>
>
> This patch refers to 'generalized configuration properties' as specified by
> [HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
> This means configuration & schema files can use expression based on
> properties defined in *solr.xml*.
> h3. Use cases:
> Describe core data directories from solr.xml as properties.
> Share the same schema and/or config file between multiple cores.
> Share reusable fragments of schema & configuration between multiple cores.
> h3. Usage:
> h4. solr.xml
> This *solr.xml* will be used to illustrates using properties for different
> purpose.
> {code:xml}
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> {code}
> {{version}} : if you update your solr.xml or your cores for various motives,
> it can be useful to track of a version. In this example, this will be used to
> define the {{dataDir}} for each core.
> {{en-cores}},{{fr-cores}}: with aliases, if the list is long or repetitive,
> it might be convenient to use a property that can then be used to describe
> the Solr core name.
> {{instanceDir}}: note that both cores will use the same instance directory,
> sharing their configuration and schema. The {{dataDir}} will be set for each
> of them from the *solrconfig.xml*.
> h4. solrconfig.xml
> This is where our *solr.xml* property are used to define the data directory
> as a composition of, in our example, the language code {{l10n}} and the core
> version stored in {{version}}.
> {code:xml}
>
> ${solr.solr.home}/data/${l10n}-${version}
>
>
> {code}
> h5. schema.xml
> The {{include}} allows to import a file within the schema (or a solrconfig);
> this can help de-clutter long schemas or reuse parts.
> {color:red}This is an experimental feature that may not be kept in the
> future.{color}
> The {{ctlField}} is just illustrating that a field & its type can be set
> through properties as well; in our example, we will want the 'english' core
> to refer to an 'english-configured' field and the 'french' core to a
> 'french-configured' one. The type for the field is defined as {{text-EN}} or
> {{text-FR}} after expansion.
> {code:xml}
>
>
> ...
>
>
>
> ...
>stored="true" multiValued="true" />
>
> {code}
> This schema is importing this *text-l10n.xml* file which is a *fragment*; the
> fragment tag must be present & indicates the file is to be included. Our
> example only defines different stopwords for each language but you could of
> course extend this to stemmers, synonyms, etc.
> {code:xml}
>
>positionIncrementGap="100">
> ...
>words="stopwords-fr.txt"/>
> ...
>
>positionIncrementGap="100">
> ...
>words="stopwords-en.txt"/>
> ...
>
>
> {code}
> h4. Technical specifications
> solr.xml can define properties at the multicore & each core level.
> Properties defined in the multicore scope can override system properties.
> Properties defined in a core scope can override multicore & system properties.
> Property definitions can use expressions to define their name & value; these
> expressions are evaluated in their outer scope context .
> CoreContainer serialization keeps properties as defined; persistence is
> idem-potent. (ie property expressions are written, not their evaluation).
> The core descriptor properties are automatically defined in each core
> context, namely:
> solr.core.instanceDir
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> h3. Coding notes:
> - DOMUtil.java:
> refactored substituteSystemProperties to use an Evaluator;
> an Evaluator is a DOM visitor that expands property expressions "in place"
> using a property map as an evaluation context
> added an asString(node) method for logging purpose
> - CoreDescriptor.java:
> added an expression member to keep property expressions as defined in
> solr.xml for persistence - allowing to write file as defined (not as expanded)
> -
[jira] Updated: (SOLR-646) Configuration properties in multicore.xml
[
https://issues.apache.org/jira/browse/SOLR-646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Henri Biestro updated SOLR-646:
---
Description:
This patch refers to 'generalized configuration properties' as specified by
[HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
This means configuration & schema files can use expression based on properties
defined in *solr.xml*.
h3. Use cases:
Describe core data directories from solr.xml as properties.
Share the same schema and/or config file between multiple cores.
Share reusable fragments of schema & configuration between multiple cores.
h3. Usage:
h4. solr.xml
This *solr.xml* will be used to illustrates using properties for different
purpose.
{code:xml}
{code}
{{version}} : if you update your solr.xml or your cores for various motives, it
can be useful to track of a version. In this example, this will be used to
define the {{dataDir}} for each core.
{{en-cores}},{{fr-cores}}: with aliases, if the list is long or repetitive, it
might be convenient to use a property that can then be used to describe the
Solr core name.
{{instanceDir}}: note that both cores will use the same instance directory,
sharing their configuration and schema. The {{dataDir}} will be set for each of
them from the *solrconfig.xml*.
h4. solrconfig.xml
This is where our *solr.xml* property are used to define the data directory as
a composition of, in our example, the language code {{l10n}} and the core
version stored in {{version}}.
{code:xml}
${solr.solr.home}/data/${l10n}-${version}
{code}
h5. schema.xml
The {{include}} allows to import a file within the schema (or a solrconfig);
this can help de-clutter long schemas or reuse parts.
{color:red}This is an experimental feature that may not be kept in the
future.{color}
The {{ctlField}} is just illustrating that a field & its type can be set
through properties as well; in our example, we will want the 'english' core to
refer to an 'english-configured' field and the 'french' core to a
'french-configured' one. The type for the field is defined as {{text-EN}} or
{{text-FR}} after expansion.
{code:xml}
...
...
{code}
This schema is importing this *text-l10n.xml* file which is a *fragment*; the
fragment tag must be present & indicates the file is to be included. Our
example only defines different stopwords for each language but you could of
course extend this to stemmers, synonyms, etc.
{code:xml}
...
...
...
...
{code}
h4. Technical specifications
solr.xml can define properties at the multicore & each core level.
Properties defined in the multicore scope can override system properties.
Properties defined in a core scope can override multicore & system properties.
Property definitions can use expressions to define their name & value; these
expressions are evaluated in their outer scope context .
CoreContainer serialization keeps properties as defined; persistence is
idem-potent. (ie property expressions are written, not their evaluation).
The core descriptor properties are automatically defined in each core context,
namely:
solr.core.instanceDir
solr.core.name
solr.core.configName
solr.core.schemaName
h3. Coding notes:
- DOMUtil.java:
refactored substituteSystemProperties to use an Evaluator;
an Evaluator is a DOM visitor that expands property expressions "in place"
using a property map as an evaluation context
added an asString(node) method for logging purpose
- CoreDescriptor.java:
added an expression member to keep property expressions as defined in solr.xml
for persistence - allowing to write file as defined (not as expanded)
- CoreContainer.java:
add an expression member to keep property expression as defined in solr.xml for
persistence - allowing to write file as defined (not as expanded);
solrx.xml peristence is idem-potent
added a local DOMUtil.Evaluator that tracks property expressions to evaluate &
store them
*issues outlined through solr-646:*
fix in load:
CoreDescriptor p = new CoreDescriptor(this, names, );
was: CoreDescriptor p = new CoreDescriptor(this, name, ...);
fix in load;
register(aliases.get(a), core, false);
was of register(aliases.get(i), core, false);
- CoreAdminHandler.java
added an optional fileName to persist so it is possible to write the solr.xml
to a different file (for comparison purpose)
- CoreAdminRequest.java
added PersistRequest to allow passing optional fileName
- Config.java:
subsituteProperties has been moved out of constructor & doc member made
protected to allow override
added an IncludesEvaluator that deals with include/fragment
- SolrConfig.java & IndexSchema.ava
added explicit calls to substituteProperties to pe
[jira] Updated: (SOLR-646) Configuration properties in multicore.xml
[
https://issues.apache.org/jira/browse/SOLR-646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Henri Biestro updated SOLR-646:
---
Description:
This patch refers to 'generalized configuration properties' as specified by
[HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
This means configuration & schema files can use expression based on properties
defined in *solr.xml*.
h3. Use cases:
Describe core data directories from solr.xml as properties.
Share the same schema and/or config file between multiple cores.
Share reusable fragments of schema & configuration between multiple cores.
h3. Usage:
h4. solr.xml
This *solr.xml* will be used to illustrates using properties for different
purpose.
{code:xml}
{code}
{{version}} : if you update your solr.xml or your cores for various motives, it
can be useful to track of a version. In this example, this will be used to
define the {{dataDir}} for each core.
{{en-cores}},{{fr-cores}}: with aliases, if the list is long or repetitive, it
might be convenient to use a property that can then be used to describe the
Solr core name.
{{instanceDir}}: note that both cores will use the same instance directory,
sharing their configuration and schema. The {{dataDir}} will be set for each of
them from the *solrconfig.xml*.
h4. solrconfig.xml
This is where our *solr.xml* property are used to define the data directory as
a composition of, in our example, the language code {{l10n}} and the core
version stored in {{version}}.
{code:xml}
${solr.solr.home}/data/${l10n}-${version}
{code}
h5. schema.xml
The {{include}} allows to import a file within the schema (or a solrconfig);
this can help de-clutter long schemas or reuse parts.
{color:red}This is an experimental feature thant may not be kept in the
future.{color}
The {{ctlField}} is just illustrating that a field & its type can be set
through properties as well; in our example, we will want the 'english' core to
refer to an 'english-configured' field and the 'french' core to a
'french-configured' one. The type for the field is defined as {{text-EN}} or
{{text-FR}} after expansion.
{code:xml}
...
...
{code}
This schema is importing this *text-l10n.xml* file which is a *fragment*; the
fragment tag must be present & indicates the file is to be included. Our
example only defines different stopwords for each language but you could of
course extend this to stemmers, synonyms, etc.
{code:xml}
...
...
...
...
{code}
h4. Technical specifications
solr.xml can define properties at the multicore & each core level.
Properties defined in the multicore scope can override system properties.
Properties defined in a core scope can override multicore & system properties.
Property definitions can use expressions to define their name & value; these
expressions are evaluated in their outer scope context .
CoreContainer serialization keeps properties as defined; persistence is
idem-potent. (ie property expressions are written, not their evaluation).
The core descriptor properties are automatically defined in each core context,
namely:
solr.core.instanceDir
solr.core.name
solr.core.configName
solr.core.schemaName
h3. Coding notes:
- DOMUtil.java:
refactored substituteSystemProperties to use an Evaluator;
an Evaluator is a DOM visitor that expands property expressions "in place"
using a property map as an evaluation context
added an asString(node) method for logging purpose
- CoreDescriptor.java:
added an expression member to keep property expressions as defined in solr.xml
for persistence - allowing to write file as defined (not as expanded)
- CoreContainer.java:
add an expression member to keep property expression as defined in solr.xml for
persistence - allowing to write file as defined (not as expanded);
solrx.xml peristence is idem-potent
added a local DOMUtil.Evaluator that tracks property expressions to evaluate &
store them
*issues outlined through solr-646:*
fix in load:
CoreDescriptor p = new CoreDescriptor(this, names, );
was: CoreDescriptor p = new CoreDescriptor(this, name, ...);
fix in load;
register(aliases.get(a), core, false);
was of register(aliases.get(i), core, false);
- CoreAdminHandler.java
added an optional fileName to persist so it is possible to write the solr.xml
to a different file (for comparison purpose)
- CoreAdminRequest.java
added PersistRequest to allow passing optional fileName
- Config.java:
subsituteProperties has been moved out of constructor & doc member made
protected to allow override
added an IncludesEvaluator that deals with include/fragment
- SolrConfig.java & IndexSchema.ava
added explicit calls to substituteProperties to p
[jira] Updated: (SOLR-646) Configuration properties in multicore.xml
[
https://issues.apache.org/jira/browse/SOLR-646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Henri Biestro updated SOLR-646:
---
Attachment: solr-646.patch
Modified property definition syntax to follow ant's.
Renamed test class.
Made the include an experimental feature.
> Configuration properties in multicore.xml
> -
>
> Key: SOLR-646
> URL: https://issues.apache.org/jira/browse/SOLR-646
> Project: Solr
> Issue Type: New Feature
>Affects Versions: 1.3
>Reporter: Henri Biestro
>Assignee: Shalin Shekhar Mangar
> Fix For: 1.3
>
> Attachments: solr-646.patch, solr-646.patch, SOLR-646.patch,
> solr-646.patch, solr-646.patch, solr-646.patch, solr-646.patch, solr-646.patch
>
>
> This patch refers to 'generalized configuration properties' as specified by
> [HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
> This means configuration & schema files can use expression based on
> properties defined in *solr.xml*.
> h3. Use cases:
> Describe core data directories from solr.xml as properties.
> Share the same schema and/or config file between multiple cores.
> Share reusable fragments of schema & configuration between multiple cores.
> h3. Usage:
> h4. solr.xml
> This *solr.xml* will be used to illustrates using properties for different
> purpose.
> {code:xml}
>
> 1.3
> english, french
> en,core0
> fr,core1
>
>
> 3.5
> EN
> core0
> This is a sample
>
>
> 2.4
> FR
> core1
> Ceci est un exemple
>
>
>
> {code}
> {{version}} : if you update your solr.xml or your cores for various motives,
> it can be useful to track of a version. In this example, this will be used to
> define the {{dataDir}} for each core.
> {{en-cores}},{{fr-cores}}: with aliases, if the list is long or repetitive,
> it might be convenient to use a property that can then be used to describe
> the Solr core name.
> {{instanceDir}}: note that both cores will use the same instance directory,
> sharing their configuration and schema. The {{dataDir}} will be set for each
> of them from the *solrconfig.xml*.
> h4. solrconfig.xml
> This is where our *solr.xml* property are used to define the data directory
> as a composition of, in our example, the language code {{l10n}} and the core
> version stored in {{version}}.
> {code:xml}
>
> ${solr.solr.home}/data/${l10n}-${version}
>
>
> {code}
> h5. schema.xml
> The {{include}} allows to import a file within the schema (or a solrconfig);
> this can help de-clutter long schemas.
> The {{ctlField}} is just illustrating that a field & its type can be set
> through properties as well; in our example, we will want the 'english' core
> to refer to an 'english-configured' field and the 'french' core to a
> 'french-configured' one. The type for the field is defined as {{text-EN}} or
> {{text-FR}} after expansion.
> {code:xml}
>
>
> ...
>
>
>
> ...
>stored="true" multiValued="true" />
>
> {code}
> This schema is importing this *text-l10n.xml* file which is a *fragment*; the
> fragment tag must be present & indicates the file is to be included. Our
> example only defines different stopwords for each language but you could of
> course extend this to stemmers, synonyms, etc.
> {code:xml}
>
>positionIncrementGap="100">
> ...
>words="stopwords-fr.txt"/>
> ...
>
>positionIncrementGap="100">
> ...
>words="stopwords-en.txt"/>
> ...
>
>
> {code}
> h4. Technical specifications
> solr.xml can define properties at the multicore & each core level.
> Properties defined in the multicore scope can override system properties.
> Properties defined in a core scope can override multicore & system properties.
> Property definitions can use expressions to define their name & value; these
> expressions are evaluated in their outer scope context .
> CoreContainer serialization keeps properties as defined; persistence is
> idem-potent. (ie property expressions are written, not their evaluation).
> The core descriptor properties are automatically defined in each core
> context, namely:
> solr.core.instanceDir
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> h3. Coding notes:
> - DOMUtil.java:
> refactored substituteSystemProperties to use an Evaluator;
> an Evaluator is a DOM visitor that expands property expressions "in place"
> using a property map as an evaluation context
> added an asString(node) method for logging purpose
> - CoreDescriptor.java:
> added an expression member to keep property expressions as defined in
> solr.xml for persistence - allowing to write file as defined (not as expanded)
> - CoreContainer.java:
> add an expression member to keep property ex
[jira] Updated: (SOLR-646) Configuration properties in multicore.xml
[
https://issues.apache.org/jira/browse/SOLR-646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Henri Biestro updated SOLR-646:
---
Description:
This patch refers to 'generalized configuration properties' as specified by
[HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
This means configuration & schema files can use expression based on properties
defined in *solr.xml*.
h3. Use cases:
Describe core data directories from solr.xml as properties.
Share the same schema and/or config file between multiple cores.
Share reusable fragments of schema & configuration between multiple cores.
h3. Usage:
h4. solr.xml
This *solr.xml* will be used to illustrates using properties for different
purpose.
{code:xml}
1.3
english, french
en,core0
fr,core1
3.5
EN
core0
This is a sample
2.4
FR
core1
Ceci est un exemple
{code}
{{version}} : if you update your solr.xml or your cores for various motives, it
can be useful to track of a version. In this example, this will be used to
define the {{dataDir}} for each core.
{{en-cores}},{{fr-cores}}: with aliases, if the list is long or repetitive, it
might be convenient to use a property that can then be used to describe the
Solr core name.
{{instanceDir}}: note that both cores will use the same instance directory,
sharing their configuration and schema. The {{dataDir}} will be set for each of
them from the *solrconfig.xml*.
h4. solrconfig.xml
This is where our *solr.xml* property are used to define the data directory as
a composition of, in our example, the language code {{l10n}} and the core
version stored in {{version}}.
{code:xml}
${solr.solr.home}/data/${l10n}-${version}
{code}
h5. schema.xml
The {{include}} allows to import a file within the schema (or a solrconfig);
this can help de-clutter long schemas.
The {{ctlField}} is just illustrating that a field & its type can be set
through properties as well; in our example, we will want the 'english' core to
refer to an 'english-configured' field and the 'french' core to a
'french-configured' one. The type for the field is defined as {{text-EN}} or
{{text-FR}} after expansion.
{code:xml}
...
...
{code}
This schema is importing this *text-l10n.xml* file which is a *fragment*; the
fragment tag must be present & indicates the file is to be included. Our
example only defines different stopwords for each language but you could of
course extend this to stemmers, synonyms, etc.
{code:xml}
...
...
...
...
{code}
h4. Technical specifications
solr.xml can define properties at the multicore & each core level.
Properties defined in the multicore scope can override system properties.
Properties defined in a core scope can override multicore & system properties.
Property definitions can use expressions to define their name & value; these
expressions are evaluated in their outer scope context .
CoreContainer serialization keeps properties as defined; persistence is
idem-potent. (ie property expressions are written, not their evaluation).
The core descriptor properties are automatically defined in each core context,
namely:
solr.core.instanceDir
solr.core.name
solr.core.configName
solr.core.schemaName
h3. Coding notes:
- DOMUtil.java:
refactored substituteSystemProperties to use an Evaluator;
an Evaluator is a DOM visitor that expands property expressions "in place"
using a property map as an evaluation context
added an asString(node) method for logging purpose
- CoreDescriptor.java:
added an expression member to keep property expressions as defined in solr.xml
for persistence - allowing to write file as defined (not as expanded)
- CoreContainer.java:
add an expression member to keep property expression as defined in solr.xml for
persistence - allowing to write file as defined (not as expanded);
solrx.xml peristence is idem-potent
added a local DOMUtil.Evaluator that tracks property expressions to evaluate &
store them
*issues outlined through solr-646:*
fix in load:
CoreDescriptor p = new CoreDescriptor(this, names, );
was: CoreDescriptor p = new CoreDescriptor(this, name, ...);
fix in load;
register(aliases.get(a), core, false);
was of register(aliases.get(i), core, false);
- CoreAdminHandler.java
added an optional fileName to persist so it is possible to write the solr.xml
to a different file (for comparison purpose)
- CoreAdminRequest.java
added PersistRequest to allow passing optional fileName
- Config.java:
subsituteProperties has been moved out of constructor & doc member made
protected to allow override
added an IncludesEvaluator that deals with include/fragment
- SolrConfig.java & IndexSchema.ava
added explicit calls to substituteProperties to perform property/in
[jira] Updated: (SOLR-646) Configuration properties in multicore.xml
[
https://issues.apache.org/jira/browse/SOLR-646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Henri Biestro updated SOLR-646:
---
Description:
This patch refers to 'generalized configuration properties' as specified by
[HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
This means configuration & schema files can use expression based on properties
defined in *solr.xml*.
h3. Use cases:
Describe core data directories from solr.xml as properties.
Share the same schema and/or config file between multiple cores.
Share reusable fragments of schemar & configuration between multiple cores.
h3. Usage:
h4. solr.xml
This *solr.xml* will be used to illustrates using properties for different
purpose.
{code:xml}
1.3
english, french
en,core0
fr,core1
3.5
EN
core0
This is a sample
2.4
FR
core1
Ceci est un exemple
{code}
{{version}} : if you update your solr.xml or your cores for various motives, it
can be useful to track of a version. In this example, this will be used to
define the {{dataDir}} for each core.
{{en-cores}},{{fr-cores}}: with aliases, if the list is long or repetitive, it
might be convenient to use a property that can then be used to describe the
Solr core name.
{{instanceDir}}: note that both cores will use the same instance directory,
sharing their configuration and schema. The {{dataDir}} will be set for each of
them from the *solrconfig.xml*.
h4. solrconfig.xml
This is where our *solr.xml* property are used to define the data directory as
a composition of, in our example, the language code {{l10n}} and the core
version stored in {{version}}.
{code:xml}
${solr.solr.home}/data/${l10n}-${version}
{code}
h5. schema.xml
The {{include}} allows to import a file within the schema (or a solrconfig);
this can help de-clutter long schemas.
The {{ctlField}} is just illustrating that a field & its type can be set
through properties as well; in our example, we will want the 'english' core to
refer to an 'english-configured' field and the 'french' core to a
'french-configured' one. The type for the field is defined as {{text-EN}} or
{{text-FR}} after expansion.
{code:xml}
...
...
{code}
This schema is importing this* text-l10n.xml* file which is a *fragment*; the
fragment tag must be present & indicates the file is to be included. Our
example only defines different stopwords for each language but you could of
course extend this to stemmers, synonyms, etc.
{code:xml}
...
...
...
...
{code}
h4. Technical specifications
solr.xml can define properties at the multicore & each core level.
Properties defined in the multicore scope can override system properties.
Properties defined in a core scope can override multicore & system properties.
Property definitions can use expressions to define their name & value; these
expressions are evaluated in their outer scope context .
CoreContainer serialization keeps properties as defined; persistence is
idem-potent. (ie property expressions are written, not their evaluation).
The core descriptor properties are automatically defined in each core context,
namely:
solr.core.instanceDir
solr.core.name
solr.core.configName
solr.core.schemaName
h3. Coding notes:
- DOMUtil.java:
refactored substituteSystemProperties to use an Evaluator;
an Evaluator is a DOM visitor that expands property expressions "in place"
using a property map as an evaluation context
added an asString(node) method for logging purpose
- CoreDescriptor.java:
added an expression member to keep property expressions as defined in solr.xml
for persistence - allowing to write file as defined (not as expanded)
- CoreContainer.java:
add an expression member to keep property expression as defined in solr.xml for
persistence - allowing to write file as defined (not as expanded);
solrx.xml peristence is idem-potent
added a local DOMUtil.Evaluator that tracks property expressions to evaluate &
store them
*issues outlined through solr-646:*
fix in load:
CoreDescriptor p = new CoreDescriptor(this, names, );
was: CoreDescriptor p = new CoreDescriptor(this, name, ...);
fix in load;
register(aliases.get(a), core, false);
was of register(aliases.get(i), core, false);
- CoreAdminHandler.java
added an optional fileName to persist so it is possible to write the solr.xml
to a different file (for comparison purpose)
- CoreAdminRequest.java
added PersistRequest to allow passing optional fileName
- Config.java:
subsituteProperties has been moved out of constructor & doc member made
protected to allow override
added an IncludesEvaluator that deals with include/fragment
- SolrConfig.java & IndexSchema.ava
added explicit calls to substituteProperties to perform property/i
[jira] Updated: (SOLR-646) Configuration properties in multicore.xml
[
https://issues.apache.org/jira/browse/SOLR-646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Henri Biestro updated SOLR-646:
---
Description:
This patch refers to 'generalized configuration properties' as specified by
[HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
This means configuration & schema files can use expression based on properties
defined in multicore.xml.
h3. Use cases:
Describe core data directories from solr.xml as properties.
Share the same schema and/or config file between multiple cores.
Share reusable fragments of schemar & configuration between multiple cores.
h3. Usage:
h4. solr.xml
This*solr.xml* will be used to illustrates using properties for different
purpose.
{code:xml}
1.3
english, french
en,core0
fr,core1
3.5
EN
core0
This is a sample
2.4
FR
core1
Ceci est un exemple
{code}
{{version}} : if you update your solr.xml or your cores for various motives, it
can be useful to track of a version. In this example, this will be used to
define the {{dataDir}} for each core.
{{en-cores}},{{fr-cores}}: with aliases, if the list is long or repetitive, it
might be convenient to use a property that can then be used to describe the
Solr core name.
{{instanceDir}}: note that both cores will use the same instance directory,
sharing their configuration and schema. The {{dataDir}} will be set for each of
them from the *solrconfig.xml*.
h4. solrconfig.xml
This is where our *solr.xml* property are used to define the data directory as
a composition of, in our example, the language code {{l10n}} and the core
version stored in {{version}}.
{code:xml}
${solr.solr.home}/data/${l10n}-${version}
{code}
h5. schema.xml
The {{include}} allows to import a file within the schema (or a solrconfig);
this can help de-clutter long schemas.
The {{ctlField}} is just illustrating that a field & its type can be set
through properties as well; in our example, we will want the 'english' core to
refer to an 'english-configured' field and the 'french' core to a
'french-configured' one. The type for the field is defined as {{text-EN}} or
{{text-FR}} after expansion.
{code:xml}
...
...
{code}
This schema is importing this* text-l10n.xml* file which is a *fragment*; the
fragment tag must be present & indicates the file is to be included. Our
example only defines different stopwords for each language but you could of
course extend this to stemmers, synonyms, etc.
{code:xml}
...
...
...
...
{code}
h4. Technical specifications
solr.xml can define properties at the multicore & each core level.
Properties defined in the multicore scope can override system properties.
Properties defined in a core scope can override multicore & system properties.
Property definitions can use expressions to define their name & value; these
expressions are evaluated in their outer scope context .
CoreContainer serialization keeps properties as defined; persistence is
idem-potent. (ie property expressions are written, not their evaluation).
The core descriptor properties are automatically defined in each core context,
namely:
solr.core.instanceDir
solr.core.name
solr.core.configName
solr.core.schemaName
h3. Coding notes:
- DOMUtil.java:
refactored substituteSystemProperties to use an Evaluator;
an Evaluator is a DOM visitor that expands property expressions "in place"
using a property map as an evaluation context
added an asString(node) method for logging purpose
- CoreDescriptor.java:
added an expression member to keep property expressions as defined in solr.xml
for persistence - allowing to write file as defined (not as expanded)
- CoreContainer.java:
add an expression member to keep property expression as defined in solr.xml for
persistence - allowing to write file as defined (not as expanded);
solrx.xml peristence is idem-potent
added a local DOMUtil.Evaluator that tracks property expressions to evaluate &
store them
*issues outlined through solr-646:*
fix in load:
CoreDescriptor p = new CoreDescriptor(this, names, );
was: CoreDescriptor p = new CoreDescriptor(this, name, ...);
fix in load;
register(aliases.get(a), core, false);
was of register(aliases.get(i), core, false);
- CoreAdminHandler.java
added an optional fileName to persist so it is possible to write the solr.xml
to a different file (for comparison purpose)
- CoreAdminRequest.java
added PersistRequest to allow passing optional fileName
- Config.java:
subsituteProperties has been moved out of constructor & doc member made
protected to allow override
added an IncludesEvaluator that deals with include/fragment
- SolrConfig.java & IndexSchema.ava
added explicit calls to substituteProperties to perform property
[jira] Updated: (SOLR-646) Configuration properties in multicore.xml
[
https://issues.apache.org/jira/browse/SOLR-646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Henri Biestro updated SOLR-646:
---
Description:
This patch refers to 'generalized configuration properties' as specified by
[HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
This means configuration & schema files can use expression based on properties
defined in multicore.xml.
h3. Use cases:
Describe core data directories from solr.xml as properties.
Share the same schema and/or config file between multiple cores.
Share reusable fragments of schemar & configuration between multiple cores.
h3. Usage:
h4. solr.xml
This*solr.xml* will be used to illustrates using properties for different
purpose.
{code:xml}
1.3
english, french
en,core0
fr,core1
3.5
EN
core0
This is a sample
2.4
FR
core1
Ceci est un exemple
{code}
{{version}} : if you update your solr.xml or your cores for various motives, it
can be useful to track of a version. In this example, this will be used to
define the {{dataDir}} for each core.
{{en-cores}},{{fr-cores}}: with aliases, if the list is long or repetitive, it
might be convenient to use a property that can then be used to describe the
Solr core name.
{{instanceDir}}: note that both cores will use the same instance directory,
sharing their configuration and schema. The {{dataDir}} will be set for each of
them from the *solrconfig.xml*.
h4. solrconfig.xml
This is where our *solr.xml* property are used to define the data directory as
a composition of, in our example, the language code {{l10n}} and the core
version stored in {{version}}.
{code:xml}
${solr.solr.home}/data/${l10n}-${version}
{code}
h5. schema.xml
The {{include}} allows to import a file within the schema (or a solrconfig);
this can help de-clutter long schemas.
The {{ctlField}} is just illustrating that a field & its type can be set
through properties as well; in our example, we will want the 'english' core to
refer to an 'english-configured' field and the 'french' core to a
'french-configured' one. The type for the field is defined as {{text-EN}} or
{{text-FR}} after expansion.
{code:xml}
...
...
{code}
This schema is importing this* text-l10n.xml* file which is a *fragment*; the
fragment tag must be present & indicates the file is to be included. Our
example only defines different stopwords for each language but you could of
course extend this to stemmers, synonyms, etc.
{code:xml}
...
...
...
...
{code}
h4. Technical specifications
solr.xml can define properties at the multicore & each core level.
Properties defined in the multicore scope can override system properties.
Properties defined in a core scope can override multicore & system properties.
Property definitions can use expressions to define their name & value; these
expressions are evaluated in their outer scope context .
CoreContainer serialization keeps properties as defined; persistence is
idem-potent. (ie property expressions are written, not their evaluation).
The core descriptor properties are automatically defined in each core context,
namely:
solr.core.instanceDir
solr.core.name
solr.core.configName
solr.core.schemaName
h3. Coding notes:
DOMUtil.java:
refactored substituteSystemProperties to use an Evaluator;
an Evaluator is a DOM visitor that expands property expressions "in place"
using a property map as an evaluation context
added an asString(node) method for logging purpose
CoreDescriptor.java:
added an expression member to keep property expressions as defined in solr.xml
for persistence - allowing to write file as defined (not as expanded)
CoreContainer.java:
add an expression member to keep property expression as defined in solr.xml for
persistence - allowing to write file as defined (not as expanded);
solrx.xml peristence is idem-potent
added a local DOMUtil.Evaluator that tracks property expressions to evaluate &
store them
issues outlined through solr-646:
fix in load:
CoreDescriptor p = new CoreDescriptor(this, names, );
was: CoreDescriptor p = new CoreDescriptor(this, name, ...);
fix in load;
register(aliases.get(a), core, false);
was of register(aliases.get(i), core, false);
CoreAdminHandler.java
added an optional fileName to persist so it is possible to write the solr.xml
to a different file (for comparison purpose)
CoreAdminRequest.java
added PersistRequest to allow passing optional fileName
Config.java:
subsituteProperties has been moved out of constructor & doc member made
protected to allow override
added an IncludesEvaluator that deals with include/fragment
SolrConfig.java & IndexSchema.ava
added explicit calls to substituteProperties to perform property/include
expans
[jira] Updated: (SOLR-646) Configuration properties in multicore.xml
[
https://issues.apache.org/jira/browse/SOLR-646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Henri Biestro updated SOLR-646:
---
Attachment: solr-646.patch
New stripped down version since there was indeed a leaner way to achieve the
same (reviewing does help to improve, thanks Shalin).
Added a specific persistence verification & a multicore inspired test to verify
shared schema/config configuration.
I'll update the issue description to reflect the patch code.
> Configuration properties in multicore.xml
> -
>
> Key: SOLR-646
> URL: https://issues.apache.org/jira/browse/SOLR-646
> Project: Solr
> Issue Type: New Feature
>Affects Versions: 1.3
>Reporter: Henri Biestro
>Assignee: Shalin Shekhar Mangar
> Fix For: 1.3
>
> Attachments: solr-646.patch, SOLR-646.patch, solr-646.patch,
> solr-646.patch, solr-646.patch, solr-646.patch, solr-646.patch
>
>
> This patch refers to 'generalized configuration properties' as specified by
> [HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
> This means configuration & schema files can use expression based on
> properties defined in multicore.xml.
> h3. Use cases:
> Share the same schema and/or config file between multiple cores but point to
> different dataDirs using a ${core.datadir}
> Change the general layout between data, config & schema directories.
> Define 'installation' wide properties (for replication for instance) in
> multicore.properties (different base/install/data directories on different
> hosts).
> h3. Syntax:
> h4. Defining properties:
> Either in the multicore.properties file (usual property format):
> {code:xml}
> env.value=an installation value
> env.other_value=another installation value
> {code}
> In the multicore.xml:
> {code:xml}
>
> a value
> ${env.value} }
>
> some value
> ${mp0}-and some data
>
>
> {code}
> h4. Using properties:
> Besides used defined properties, the following core descriptor properties are
> automatically defined in each core context, namely:
> {code}
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> {code}
> All properties can be used in any attribute or text node of a solrconfig.xml
> or schema.xml as in:
> {code:xml}
> ${core.dataDir}
> {code}
> h4. Technical specifications
> Multicore.xml can define properties at the multicore & each core level.
> Properties defined in the multicore scope can override system properties.
> Properties defined in a core scope can override multicore & system properties.
> Property definitions can use expressions to define their name & value; these
> expressions are evaluated in their outer scope context .
> Multicore serialization keeps properties as written (ie as expressions if
> they were defined so).
> The core descriptor properties are automatically defined in each core
> context, namely:
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> h4. Test:
> The following (contrived) multicore.xml:
> {code:xml}
>
> 33
> 0
> 1
> bogus
> bogus
>
> core${zero}_id
> solr.DirectUpdateHandler2
> 11
>
>
> core${one}_id
> solr.DirectUpdateHandler2
> 22
>
>
> {code}
> allows this config.xml:
> {code:xml}
>
>
>
>
> multipartUploadLimitInKB="2048" />
>
>
>default="true" />
>
>class="org.apache.solr.handler.admin.LukeRequestHandler" />
>
>
>
> solr
> solrconfig.xml schema.xml admin-extra.html
>
> qt=standard&q=solrpingquery
>
>
>
> {code}
> and this schema.xml:
> {code:xml}
>
>
>
> sortMissingLast="true" omitNorms="true"/>
> omitNorms="true"/>
>
>
>
>stored="true" multiValued="false" required="true"/>
>multiValued="false" />
>multiValued="false" />
>stored="true" multiValued="false" />
>
> id
> name
>
>
> {code}
> Allow the trunk test to work.
> h3. Coding notes:
> The code itself refactored some of DOMUtil (the ant based property
> substitution) into one added class (PropertyMap & PropertyMap.Evaluator).
> The PropertyMap are chained (one link chain between core to multicore map);
> those maps are owned by each core's ResourceLoader.
> Config is modified a little to accommodate delaying & specializing property
> expansions.
> Multicore is modified so it properly parses & serializes.
> Tested against the example above.
> Reviews & comments more than welcome.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-646) Configuration properties in multicore.xml
[
https://issues.apache.org/jira/browse/SOLR-646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shalin Shekhar Mangar updated SOLR-646:
---
Attachment: SOLR-646.patch
Here's another attempt on this issue.
# Three types of properties -- core specific, global and system (evaluated in
that order)
# Syntax for properties in solr.xml is modified slightly to use attributes
instead of element values
{code:xml}
{code}
# Properties supported are:
{code}
solr.core.instanceDir
solr.core.name
solr.core.configName
solr.core.schemaName
{code}
# SolrResourceLoader keeps a reference to the CoreDescriptor to get access to
the core specific properties.
# PropertyReplacer is heavily borrowed from TemplateString in
DataImportHandler. It is trimmed down to our needs and enhanced to support
default values. TestPropertyReplacer is a unit test for this class. They could
be merged together in the future but right now I don't want us to have any
regressions.
# It doesn't support escaping variables like the previous DOMUtil code e.g.
$${literal} gives you $${literal} but it used to give ${literal}. I don't think
we need that capability anyway.
# No support for fragment and imports -- I'm strongly against it. I don't think
there's anything which we can't do without them and they make for poor
readability.
Review comments and suggestions are invited. As soon as we are done with this
issue we can start the release process for 1.3
> Configuration properties in multicore.xml
> -
>
> Key: SOLR-646
> URL: https://issues.apache.org/jira/browse/SOLR-646
> Project: Solr
> Issue Type: New Feature
>Affects Versions: 1.3
>Reporter: Henri Biestro
>Assignee: Shalin Shekhar Mangar
> Fix For: 1.3
>
> Attachments: SOLR-646.patch, solr-646.patch, solr-646.patch,
> solr-646.patch, solr-646.patch, solr-646.patch
>
>
> This patch refers to 'generalized configuration properties' as specified by
> [HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
> This means configuration & schema files can use expression based on
> properties defined in multicore.xml.
> h3. Use cases:
> Share the same schema and/or config file between multiple cores but point to
> different dataDirs using a ${core.datadir}
> Change the general layout between data, config & schema directories.
> Define 'installation' wide properties (for replication for instance) in
> multicore.properties (different base/install/data directories on different
> hosts).
> h3. Syntax:
> h4. Defining properties:
> Either in the multicore.properties file (usual property format):
> {code:xml}
> env.value=an installation value
> env.other_value=another installation value
> {code}
> In the multicore.xml:
> {code:xml}
>
> a value
> ${env.value} }
>
> some value
> ${mp0}-and some data
>
>
> {code}
> h4. Using properties:
> Besides used defined properties, the following core descriptor properties are
> automatically defined in each core context, namely:
> {code}
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> {code}
> All properties can be used in any attribute or text node of a solrconfig.xml
> or schema.xml as in:
> {code:xml}
> ${core.dataDir}
> {code}
> h4. Technical specifications
> Multicore.xml can define properties at the multicore & each core level.
> Properties defined in the multicore scope can override system properties.
> Properties defined in a core scope can override multicore & system properties.
> Property definitions can use expressions to define their name & value; these
> expressions are evaluated in their outer scope context .
> Multicore serialization keeps properties as written (ie as expressions if
> they were defined so).
> The core descriptor properties are automatically defined in each core
> context, namely:
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> h4. Test:
> The following (contrived) multicore.xml:
> {code:xml}
>
> 33
> 0
> 1
> bogus
> bogus
>
> core${zero}_id
> solr.DirectUpdateHandler2
> 11
>
>
> core${one}_id
> solr.DirectUpdateHandler2
> 22
>
>
> {code}
> allows this config.xml:
> {code:xml}
>
>
>
>
> multipartUploadLimitInKB="2048" />
>
>
>default="true" />
>
>class="org.apache.solr.handler.admin.LukeRequestHandler" />
>
>
>
> solr
> solrconfig.xml schema.xml admin-extra.html
>
> qt=standard&q=solrpingquery
>
>
>
> {code}
> and this schema.xml:
> {code:xml}
>
>
>
> sortMissingLast="true" omitNorms="true"/>
> omitNorms="true"/>
>
>
>
>stored="t
[jira] Updated: (SOLR-646) Configuration properties in multicore.xml
[
https://issues.apache.org/jira/browse/SOLR-646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan McKinley updated SOLR-646:
---
Assignee: (was: Ryan McKinley)
I am leaving on vacation... and will be unable to give this any attention, so i
have removed myself as the "Assignee"
> Configuration properties in multicore.xml
> -
>
> Key: SOLR-646
> URL: https://issues.apache.org/jira/browse/SOLR-646
> Project: Solr
> Issue Type: New Feature
>Affects Versions: 1.3
>Reporter: Henri Biestro
> Fix For: 1.3
>
> Attachments: solr-646.patch, solr-646.patch, solr-646.patch,
> solr-646.patch, solr-646.patch
>
>
> This patch refers to 'generalized configuration properties' as specified by
> [HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
> This means configuration & schema files can use expression based on
> properties defined in multicore.xml.
> h3. Use cases:
> Share the same schema and/or config file between multiple cores but point to
> different dataDirs using a ${core.datadir}
> Change the general layout between data, config & schema directories.
> Define 'installation' wide properties (for replication for instance) in
> multicore.properties (different base/install/data directories on different
> hosts).
> h3. Syntax:
> h4. Defining properties:
> Either in the multicore.properties file (usual property format):
> {code:xml}
> env.value=an installation value
> env.other_value=another installation value
> {code}
> In the multicore.xml:
> {code:xml}
>
> a value
> ${env.value} }
>
> some value
> ${mp0}-and some data
>
>
> {code}
> h4. Using properties:
> Besides used defined properties, the following core descriptor properties are
> automatically defined in each core context, namely:
> {code}
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> {code}
> All properties can be used in any attribute or text node of a solrconfig.xml
> or schema.xml as in:
> {code:xml}
> ${core.dataDir}
> {code}
> h4. Technical specifications
> Multicore.xml can define properties at the multicore & each core level.
> Properties defined in the multicore scope can override system properties.
> Properties defined in a core scope can override multicore & system properties.
> Property definitions can use expressions to define their name & value; these
> expressions are evaluated in their outer scope context .
> Multicore serialization keeps properties as written (ie as expressions if
> they were defined so).
> The core descriptor properties are automatically defined in each core
> context, namely:
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> h4. Test:
> The following (contrived) multicore.xml:
> {code:xml}
>
> 33
> 0
> 1
> bogus
> bogus
>
> core${zero}_id
> solr.DirectUpdateHandler2
> 11
>
>
> core${one}_id
> solr.DirectUpdateHandler2
> 22
>
>
> {code}
> allows this config.xml:
> {code:xml}
>
>
>
>
> multipartUploadLimitInKB="2048" />
>
>
>default="true" />
>
>class="org.apache.solr.handler.admin.LukeRequestHandler" />
>
>
>
> solr
> solrconfig.xml schema.xml admin-extra.html
>
> qt=standard&q=solrpingquery
>
>
>
> {code}
> and this schema.xml:
> {code:xml}
>
>
>
> sortMissingLast="true" omitNorms="true"/>
> omitNorms="true"/>
>
>
>
>stored="true" multiValued="false" required="true"/>
>multiValued="false" />
>multiValued="false" />
>stored="true" multiValued="false" />
>
> id
> name
>
>
> {code}
> Allow the trunk test to work.
> h3. Coding notes:
> The code itself refactored some of DOMUtil (the ant based property
> substitution) into one added class (PropertyMap & PropertyMap.Evaluator).
> The PropertyMap are chained (one link chain between core to multicore map);
> those maps are owned by each core's ResourceLoader.
> Config is modified a little to accommodate delaying & specializing property
> expansions.
> Multicore is modified so it properly parses & serializes.
> Tested against the example above.
> Reviews & comments more than welcome.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-646) Configuration properties in multicore.xml
[
https://issues.apache.org/jira/browse/SOLR-646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Henri Biestro updated SOLR-646:
---
Attachment: solr-646.patch
Refactored following HossMan recommendations.
This still includes the "include" (was import) feature but removing it is easy
(in Config.java) & will reduce modifications in ResourceLoader.java.
Added a specific test that loads a shared schema & config but uses properties
to differentiate behavior (different locales, different dataDirs).
Includes a one letter fix in CoreContainer.load for aliases (get(a) instead of
get(i)).
More tests will follow (persist verification is "manual") & I'll try to cut the
patch in 2 (properties - include).
I might be busier in the coming days but will try not to slow down 1.3 release
if next review says this is on the right track.
> Configuration properties in multicore.xml
> -
>
> Key: SOLR-646
> URL: https://issues.apache.org/jira/browse/SOLR-646
> Project: Solr
> Issue Type: New Feature
>Affects Versions: 1.3
>Reporter: Henri Biestro
>Assignee: Ryan McKinley
> Fix For: 1.3
>
> Attachments: solr-646.patch, solr-646.patch, solr-646.patch,
> solr-646.patch, solr-646.patch
>
>
> This patch refers to 'generalized configuration properties' as specified by
> [HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
> This means configuration & schema files can use expression based on
> properties defined in multicore.xml.
> h3. Use cases:
> Share the same schema and/or config file between multiple cores but point to
> different dataDirs using a ${core.datadir}
> Change the general layout between data, config & schema directories.
> Define 'installation' wide properties (for replication for instance) in
> multicore.properties (different base/install/data directories on different
> hosts).
> h3. Syntax:
> h4. Defining properties:
> Either in the multicore.properties file (usual property format):
> {code:xml}
> env.value=an installation value
> env.other_value=another installation value
> {code}
> In the multicore.xml:
> {code:xml}
>
> a value
> ${env.value} }
>
> some value
> ${mp0}-and some data
>
>
> {code}
> h4. Using properties:
> Besides used defined properties, the following core descriptor properties are
> automatically defined in each core context, namely:
> {code}
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> {code}
> All properties can be used in any attribute or text node of a solrconfig.xml
> or schema.xml as in:
> {code:xml}
> ${core.dataDir}
> {code}
> h4. Technical specifications
> Multicore.xml can define properties at the multicore & each core level.
> Properties defined in the multicore scope can override system properties.
> Properties defined in a core scope can override multicore & system properties.
> Property definitions can use expressions to define their name & value; these
> expressions are evaluated in their outer scope context .
> Multicore serialization keeps properties as written (ie as expressions if
> they were defined so).
> The core descriptor properties are automatically defined in each core
> context, namely:
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> h4. Test:
> The following (contrived) multicore.xml:
> {code:xml}
>
> 33
> 0
> 1
> bogus
> bogus
>
> core${zero}_id
> solr.DirectUpdateHandler2
> 11
>
>
> core${one}_id
> solr.DirectUpdateHandler2
> 22
>
>
> {code}
> allows this config.xml:
> {code:xml}
>
>
>
>
> multipartUploadLimitInKB="2048" />
>
>
>default="true" />
>
>class="org.apache.solr.handler.admin.LukeRequestHandler" />
>
>
>
> solr
> solrconfig.xml schema.xml admin-extra.html
>
> qt=standard&q=solrpingquery
>
>
>
> {code}
> and this schema.xml:
> {code:xml}
>
>
>
> sortMissingLast="true" omitNorms="true"/>
> omitNorms="true"/>
>
>
>
>stored="true" multiValued="false" required="true"/>
>multiValued="false" />
>multiValued="false" />
>stored="true" multiValued="false" />
>
> id
> name
>
>
> {code}
> Allow the trunk test to work.
> h3. Coding notes:
> The code itself refactored some of DOMUtil (the ant based property
> substitution) into one added class (PropertyMap & PropertyMap.Evaluator).
> The PropertyMap are chained (one link chain between core to multicore map);
> those maps are owned by each core's ResourceLoader.
> Config is modified a little to accommodate delaying & specializing property
> expansions.
> Multicore is modified so it properly parses & serializes.
> Tes
[jira] Updated: (SOLR-646) Configuration properties in multicore.xml
[
https://issues.apache.org/jira/browse/SOLR-646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Henri Biestro updated SOLR-646:
---
Attachment: solr-646.patch
updated to trunk 684310
> Configuration properties in multicore.xml
> -
>
> Key: SOLR-646
> URL: https://issues.apache.org/jira/browse/SOLR-646
> Project: Solr
> Issue Type: New Feature
>Affects Versions: 1.3
>Reporter: Henri Biestro
>Assignee: Ryan McKinley
> Fix For: 1.3
>
> Attachments: solr-646.patch, solr-646.patch, solr-646.patch,
> solr-646.patch
>
>
> This patch refers to 'generalized configuration properties' as specified by
> [HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
> This means configuration & schema files can use expression based on
> properties defined in multicore.xml.
> h3. Use cases:
> Share the same schema and/or config file between multiple cores but point to
> different dataDirs using a ${core.datadir}
> Change the general layout between data, config & schema directories.
> Define 'installation' wide properties (for replication for instance) in
> multicore.properties (different base/install/data directories on different
> hosts).
> h3. Syntax:
> h4. Defining properties:
> Either in the multicore.properties file (usual property format):
> {code:xml}
> env.value=an installation value
> env.other_value=another installation value
> {code}
> In the multicore.xml:
> {code:xml}
>
> a value
> ${env.value} }
>
> some value
> ${mp0}-and some data
>
>
> {code}
> h4. Using properties:
> Besides used defined properties, the following core descriptor properties are
> automatically defined in each core context, namely:
> {code}
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> {code}
> All properties can be used in any attribute or text node of a solrconfig.xml
> or schema.xml as in:
> {code:xml}
> ${core.dataDir}
> {code}
> h4. Technical specifications
> Multicore.xml can define properties at the multicore & each core level.
> Properties defined in the multicore scope can override system properties.
> Properties defined in a core scope can override multicore & system properties.
> Property definitions can use expressions to define their name & value; these
> expressions are evaluated in their outer scope context .
> Multicore serialization keeps properties as written (ie as expressions if
> they were defined so).
> The core descriptor properties are automatically defined in each core
> context, namely:
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> h4. Test:
> The following (contrived) multicore.xml:
> {code:xml}
>
> 33
> 0
> 1
> bogus
> bogus
>
> core${zero}_id
> solr.DirectUpdateHandler2
> 11
>
>
> core${one}_id
> solr.DirectUpdateHandler2
> 22
>
>
> {code}
> allows this config.xml:
> {code:xml}
>
>
>
>
> multipartUploadLimitInKB="2048" />
>
>
>default="true" />
>
>class="org.apache.solr.handler.admin.LukeRequestHandler" />
>
>
>
> solr
> solrconfig.xml schema.xml admin-extra.html
>
> qt=standard&q=solrpingquery
>
>
>
> {code}
> and this schema.xml:
> {code:xml}
>
>
>
> sortMissingLast="true" omitNorms="true"/>
> omitNorms="true"/>
>
>
>
>stored="true" multiValued="false" required="true"/>
>multiValued="false" />
>multiValued="false" />
>stored="true" multiValued="false" />
>
> id
> name
>
>
> {code}
> Allow the trunk test to work.
> h3. Coding notes:
> The code itself refactored some of DOMUtil (the ant based property
> substitution) into one added class (PropertyMap & PropertyMap.Evaluator).
> The PropertyMap are chained (one link chain between core to multicore map);
> those maps are owned by each core's ResourceLoader.
> Config is modified a little to accommodate delaying & specializing property
> expansions.
> Multicore is modified so it properly parses & serializes.
> Tested against the example above.
> Reviews & comments more than welcome.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-646) Configuration properties in multicore.xml
[
https://issues.apache.org/jira/browse/SOLR-646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Henri Biestro updated SOLR-646:
---
Attachment: solr-646.patch
Fixed dependency on Java 6; my bad, sorry Jeremy.
Solved 2 questions pending from the previous patch:
1 - Added schemaDir & configDir to solr core descriptors; this means all
'automatic' properties come as representations of values coming from the core
descriptor. I removed the solr.core.instancePath and homogenized the
solr.core.{instance,schema,config}Dir properties to be expanded relative to the
multicore instance dir if not absolute.
2 - Added some checking/warning if some user defined properties collide with
automatic ones during core creation; the system computed value is the one kept.
The current list of automated properties (& example values):
solr.core.configDir = ..\..\..\example\multicore\core0\conf/
solr.core.configName = solrconfig.xml
solr.core.name = core0
solr.core.instanceDir = ..\..\..\example\multicore\core0/
solr.core.schemaName = schema.xml
solr.core.schemaDir = ..\..\..\example\multicore\core0\conf/
I'll udpate the issue description to reflect the current choices.
New (sub) issue/choice to make:
There is no way yet to specify properties through solrj when creating a core
(nor a way to specify the core {config, schema}{Dir, Name}). Any preferred
syntax?
property.=&property.=
is my current idea to pass them in a http request but this is heavy...
> Configuration properties in multicore.xml
> -
>
> Key: SOLR-646
> URL: https://issues.apache.org/jira/browse/SOLR-646
> Project: Solr
> Issue Type: New Feature
>Affects Versions: 1.3
>Reporter: Henri Biestro
> Fix For: 1.3
>
> Attachments: solr-646.patch, solr-646.patch, solr-646.patch
>
>
> This patch refers to 'generalized configuration properties' as specified by
> [HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
> This means configuration & schema files can use expression based on
> properties defined in multicore.xml.
> h3. Use cases:
> Share the same schema and/or config file between multiple cores but point to
> different dataDirs using a ${core.datadir}
> Change the general layout between data, config & schema directories.
> Define 'installation' wide properties (for replication for instance) in
> multicore.properties (different base/install/data directories on different
> hosts).
> h3. Syntax:
> h4. Defining properties:
> Either in the multicore.properties file (usual property format):
> {code:xml}
> env.value=an installation value
> env.other_value=another installation value
> {code}
> In the multicore.xml:
> {code:xml}
>
> a value
> ${env.value} }
>
> some value
> ${mp0}-and some data
>
>
> {code}
> h4. Using properties:
> Besides used defined properties, the following core descriptor properties are
> automatically defined in each core context, namely:
> {code}
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> {code}
> All properties can be used in any attribute or text node of a solrconfig.xml
> or schema.xml as in:
> {code:xml}
> ${core.dataDir}
> {code}
> h4. Technical specifications
> Multicore.xml can define properties at the multicore & each core level.
> Properties defined in the multicore scope can override system properties.
> Properties defined in a core scope can override multicore & system properties.
> Property definitions can use expressions to define their name & value; these
> expressions are evaluated in their outer scope context .
> Multicore serialization keeps properties as written (ie as expressions if
> they were defined so).
> The core descriptor properties are automatically defined in each core
> context, namely:
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> h4. Test:
> The following (contrived) multicore.xml:
> {code:xml}
>
> 33
> 0
> 1
> bogus
> bogus
>
> core${zero}_id
> solr.DirectUpdateHandler2
> 11
>
>
> core${one}_id
> solr.DirectUpdateHandler2
> 22
>
>
> {code}
> allows this config.xml:
> {code:xml}
>
>
>
>
> multipartUploadLimitInKB="2048" />
>
>
>default="true" />
>
>class="org.apache.solr.handler.admin.LukeRequestHandler" />
>
>
>
> solr
> solrconfig.xml schema.xml admin-extra.html
>
> qt=standard&q=solrpingquery
>
>
>
> {code}
> and this schema.xml:
> {code:xml}
>
>
>
> sortMissingLast="true" omitNorms="true"/>
> omitNorms="true"/>
>
>
>
>stored="true" multiValued="false" required="true"/>
>multiValued="false" />
>multiValued="false" />
>stored="t
[jira] Updated: (SOLR-646) Configuration properties in multicore.xml
[
https://issues.apache.org/jira/browse/SOLR-646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Henri Biestro updated SOLR-646:
---
Attachment: solr-646.patch
Besides the syntax, the latest patch version adds the (where resource can be a constant or a variable).
This means config and schemas can be chunked in multiple pieces that can be
reused in different cores. Combined with properties, this completely frees up
core configuration.
Also added a ... tag to allow importing XML chunks that
don't have a natural root node.
The PropertyMap.imports() allows to enumerate which expressions were resolved
as resources locally (aka config, schema, import) and their logical location
when implicit. (Map.Entry e; e.getValue()[0] is the canonical
resource value, getValue()[1] the 'symbolic' one. When value[0] == value[1], it
means this was absolute or found in a JAR.
Shalin, thanks, good to know; I've only been using/refactoring DOMUtil since
this patch mainly deals with property expansion (besides definition &
book-keeping).
> Configuration properties in multicore.xml
> -
>
> Key: SOLR-646
> URL: https://issues.apache.org/jira/browse/SOLR-646
> Project: Solr
> Issue Type: New Feature
>Affects Versions: 1.3
>Reporter: Henri Biestro
> Fix For: 1.3
>
> Attachments: solr-646.patch, solr-646.patch
>
>
> This patch refers to 'generalized configuration properties' as specified by
> [HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
> This means configuration & schema files can use expression based on
> properties defined in multicore.xml.
> h3. Use cases:
> Share the same schema and/or config file between multiple cores but point to
> different dataDirs using a ${core.datadir}
> Change the general layout between data, config & schema directories.
> Define 'installation' wide properties (for replication for instance) in
> multicore.properties (different base/install/data directories on different
> hosts).
> h3. Syntax:
> h4. Defining properties:
> Either in the multicore.properties file (usual property format):
> {code:xml}
> env.value=an installation value
> env.other_value=another installation value
> {code}
> In the multicore.xml:
> {code:xml}
>
> a value
> ${env.value} }
>
> some value
> ${mp0}-and some data
>
>
> {code}
> h4. Using properties:
> Besides used defined properties, the following core descriptor properties are
> automatically defined in each core context, namely:
> {code}
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> {code}
> All properties can be used in any attribute or text node of a solrconfig.xml
> or schema.xml as in:
> {code:xml}
> ${core.dataDir}
> {code}
> h4. Technical specifications
> Multicore.xml can define properties at the multicore & each core level.
> Properties defined in the multicore scope can override system properties.
> Properties defined in a core scope can override multicore & system properties.
> Property definitions can use expressions to define their name & value; these
> expressions are evaluated in their outer scope context .
> Multicore serialization keeps properties as written (ie as expressions if
> they were defined so).
> The core descriptor properties are automatically defined in each core
> context, namely:
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> h4. Test:
> The following (contrived) multicore.xml:
> {code:xml}
>
> 33
> 0
> 1
> bogus
> bogus
>
> core${zero}_id
> solr.DirectUpdateHandler2
> 11
>
>
> core${one}_id
> solr.DirectUpdateHandler2
> 22
>
>
> {code}
> allows this config.xml:
> {code:xml}
>
>
>
>
> multipartUploadLimitInKB="2048" />
>
>
>default="true" />
>
>class="org.apache.solr.handler.admin.LukeRequestHandler" />
>
>
>
> solr
> solrconfig.xml schema.xml admin-extra.html
>
> qt=standard&q=solrpingquery
>
>
>
> {code}
> and this schema.xml:
> {code:xml}
>
>
>
> sortMissingLast="true" omitNorms="true"/>
> omitNorms="true"/>
>
>
>
>stored="true" multiValued="false" required="true"/>
>multiValued="false" />
>multiValued="false" />
>stored="true" multiValued="false" />
>
> id
> name
>
>
> {code}
> Allow the trunk test to work.
> h3. Coding notes:
> The code itself refactored some of DOMUtil (the ant based property
> substitution) into one added class (PropertyMap & PropertyMap.Evaluator).
> The PropertyMap are chained (one link chain between core to multicore map);
> those maps are owned by each core's ResourceLoader.
> Config is modified a little to accommod
[jira] Updated: (SOLR-646) Configuration properties in multicore.xml
[
https://issues.apache.org/jira/browse/SOLR-646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Henri Biestro updated SOLR-646:
---
Description:
This patch refers to 'generalized configuration properties' as specified by
[HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
This means configuration & schema files can use expression based on properties
defined in multicore.xml.
h3. Use cases:
Share the same schema and/or config file between multiple cores but point to
different dataDirs using a ${core.datadir}
Change the general layout between data, config & schema directories.
Define 'installation' wide properties (for replication for instance) in
multicore.properties (different base/install/data directories on different
hosts).
h3. Syntax:
h4. Defining properties:
Either in the multicore.properties file (usual property format):
{code:xml}
env.value=an installation value
env.other_value=another installation value
{code}
In the multicore.xml:
{code:xml}
a value
${env.value} }
some value
${mp0}-and some data
{code}
h4. Using properties:
Besides used defined properties, the following core descriptor properties are
automatically defined in each core context, namely:
{code}
solr.core.instanceDir
solr.core.instancePath
solr.core.name
solr.core.configName
solr.core.schemaName
{code}
All properties can be used in any attribute or text node of a solrconfig.xml or
schema.xml as in:
{code:xml}
${core.dataDir}
{code}
h4. Technical specifications
Multicore.xml can define properties at the multicore & each core level.
Properties defined in the multicore scope can override system properties.
Properties defined in a core scope can override multicore & system properties.
Property definitions can use expressions to define their name & value; these
expressions are evaluated in their outer scope context .
Multicore serialization keeps properties as written (ie as expressions if they
were defined so).
The core descriptor properties are automatically defined in each core context,
namely:
solr.core.instanceDir
solr.core.instancePath
solr.core.name
solr.core.configName
solr.core.schemaName
h4. Test:
The following (contrived) multicore.xml:
{code:xml}
33
0
1
bogus
bogus
core${zero}_id
solr.DirectUpdateHandler2
11
core${one}_id
solr.DirectUpdateHandler2
22
{code}
allows this config.xml:
{code:xml}
solr
solrconfig.xml schema.xml admin-extra.html
qt=standard&q=solrpingquery
{code}
and this schema.xml:
{code:xml}
id
name
{code}
Allow the trunk test to work.
h3. Coding notes:
The code itself refactored some of DOMUtil (the ant based property
substitution) into one added class (PropertyMap & PropertyMap.Evaluator).
The PropertyMap are chained (one link chain between core to multicore map);
those maps are owned by each core's ResourceLoader.
Config is modified a little to accommodate delaying & specializing property
expansions.
Multicore is modified so it properly parses & serializes.
Tested against the example above.
Reviews & comments more than welcome.
was:
This patch refers to 'generalized configuration properties' as specified by
[HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
This means configuration & schema files can use expression based on properties
defined in multicore.xml.
Use cases:
Share the same schema and/or config file between multiple cores but point to
different dataDirs using a ${core.datadir}
Change the general layout between data, config & schema directories.
Define 'installation' wide properties (for replication for instance) in
multicore.properties (different base/install/data directories on different
hosts).
The following (contrived) multicore.xml:
{code:xml}
33
0
1
bogus
bogus
core${zero}_id
solr.DirectUpdateHandler2
11
core${one}_id
solr.DirectUpdateHandler2
22
{code}
allows this config.xml:
{code:xml}
solr
solrconfig.xml schema.xml admin-extra.html
qt=standard&q=solrpingquery
{code}
and this schema.xml:
{code:xml}
id
name
{code}
Multicore.xml can define properties at the multicore & each core level.
Properties defined in the multicore scope can override system properties.
Properties defined in a core scope can override multicore & system properties.
Property definitions can use expressions to define their name & value; these
expressions are evaluated in their outer scope context .
Multicore serialization keeps properties as written (ie as expressions if they
were defined so).
The core descriptor properties are automatically
[jira] Updated: (SOLR-646) Configuration properties in multicore.xml
[
https://issues.apache.org/jira/browse/SOLR-646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Henri Biestro updated SOLR-646:
---
Description:
This patch refers to 'generalized configuration properties' as specified by
[HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
This means configuration & schema files can use expression based on properties
defined in multicore.xml.
Use cases:
Share the same schema and/or config file between multiple cores but point to
different dataDirs using a ${core.datadir}
Change the general layout between data, config & schema directories.
Define 'installation' wide properties (for replication for instance) in
multicore.properties (different base/install/data directories on different
hosts).
The following (contrived) multicore.xml:
{code:xml}
33
0
1
bogus
bogus
core${zero}_id
solr.DirectUpdateHandler2
11
core${one}_id
solr.DirectUpdateHandler2
22
{code}
allows this config.xml:
{code:xml}
solr
solrconfig.xml schema.xml admin-extra.html
qt=standard&q=solrpingquery
{code}
and this schema.xml:
{code:xml}
id
name
{code}
Multicore.xml can define properties at the multicore & each core level.
Properties defined in the multicore scope can override system properties.
Properties defined in a core scope can override multicore & system properties.
Property definitions can use expressions to define their name & value; these
expressions are evaluated in their outer scope context .
Multicore serialization keeps properties as written (ie as expressions if they
were defined so).
The core descriptor properties are automatically defined in each core context,
namely:
solr.core.instanceDir
solr.core.instancePath
solr.core.name
solr.core.configName
solr.core.schemaName
The code itself refactored some of DOMUtil (the ant based property
substitution) into one added class (PropertyMap & PropertyMap.Evaluator).
The PropertyMap are chained (one link chain between core to multicore map);
those maps are owned by each core's ResourceLoader.
Config is modified a little to accommodate delaying & specializing property
expansions.
Multicore is modified so it properly parses & serializes.
Tested against the example above.
Reviews & comments more than welcome.
was:
This patch refers to 'generalized configuration properties' as specified by
[HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
This means configuration & schema files can use expression based on properties
defined in multicore.xml.
For instance, the following multicore.xml:
{code:xml}
33
0
1
bogus
bogus
core${zero}_id
solr.DirectUpdateHandler2
11
core${one}_id
solr.DirectUpdateHandler2
22
{code}
allows this config.xml:
{code:xml}
solr
solrconfig.xml schema.xml admin-extra.html
qt=standard&q=solrpingquery
{code}
and this schema.xml:
{code:xml}
id
name
{code}
Multicore.xml can define properties at the multicore & each core level.
Properties defined in the multicore scope can override system properties.
Properties defined in a core scope can override multicore & system properties.
Property definitions can use expressions to define their name & value; these
expressions are evaluated in their outer scope context .
Multicore serialization keeps properties as written (ie as expressions if they
were defined so).
The core descriptor properties are automatically defined in each core context,
namely:
solr.core.instanceDir
solr.core.instancePath
solr.core.name
solr.core.configName
solr.core.schemaName
The code itself refactored some of DOMUtil (the ant based property
substitution) into one added class (PropertyMap & PropertyMap.Evaluator).
The PropertyMap are chained (one link chain between core to multicore map);
those maps are owned by each core's ResourceLoader.
Config is modified a little to accommodate delaying & specializing property
expansions.
Multicore is modified so it properly parses & serializes.
Tested against the example above.
Reviews & comments more than welcome.
> Configuration properties in multicore.xml
> -
>
> Key: SOLR-646
> URL: https://issues.apache.org/jira/browse/SOLR-646
> Project: Solr
> Issue Type: New Feature
>Affects Versions: 1.3
>Reporter: Henri Biestro
> Fix For: 1.3
>
> Attachments: solr-646.patch
>
>
> This patch refers to 'generalized configuration properties' as specified by
> [HossMan|https://issues.apache.org/jira/browse/SOLR-350?focu
[jira] Updated: (SOLR-646) Configuration properties in multicore.xml
[
https://issues.apache.org/jira/browse/SOLR-646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Henri Biestro updated SOLR-646:
---
Attachment: solr-646.patch
supersedes the version in solr-350 with some improvements & bug fixes.
Multicore will attempt to load the bundle "multicore" (aka tries to load a
multicore.properties) allowing to define installation wide properties.
Unrelated but convenient to test/verify persistence, solrj now allows to
persist the file with an explicit name.
> Configuration properties in multicore.xml
> -
>
> Key: SOLR-646
> URL: https://issues.apache.org/jira/browse/SOLR-646
> Project: Solr
> Issue Type: New Feature
>Affects Versions: 1.3
>Reporter: Henri Biestro
> Fix For: 1.3
>
> Attachments: solr-646.patch
>
>
> This patch refers to 'generalized configuration properties' as specified by
> [HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
> This means configuration & schema files can use expression based on
> properties defined in multicore.xml.
> For instance, the following multicore.xml:
> {code:xml}
>
> 33
> 0
> 1
> bogus
> bogus
>
> core${zero}_id
> solr.DirectUpdateHandler2
> 11
>
>
> core${one}_id
> solr.DirectUpdateHandler2
> 22
>
>
> {code}
> allows this config.xml:
> {code:xml}
>
>
>
>
> multipartUploadLimitInKB="2048" />
>
>
>default="true" />
>
>class="org.apache.solr.handler.admin.LukeRequestHandler" />
>
>
>
> solr
> solrconfig.xml schema.xml admin-extra.html
>
> qt=standard&q=solrpingquery
>
>
>
> {code}
> and this schema.xml:
> {code:xml}
>
>
>
> sortMissingLast="true" omitNorms="true"/>
> omitNorms="true"/>
>
>
>
>stored="true" multiValued="false" required="true"/>
>multiValued="false" />
>multiValued="false" />
>stored="true" multiValued="false" />
>
> id
> name
>
>
> {code}
> Multicore.xml can define properties at the multicore & each core level.
> Properties defined in the multicore scope can override system properties.
> Properties defined in a core scope can override multicore & system properties.
> Property definitions can use expressions to define their name & value; these
> expressions are evaluated in their outer scope context .
> Multicore serialization keeps properties as written (ie as expressions if
> they were defined so).
> The core descriptor properties are automatically defined in each core
> context, namely:
> solr.core.instanceDir
> solr.core.instancePath
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> The code itself refactored some of DOMUtil (the ant based property
> substitution) into one added class (PropertyMap & PropertyMap.Evaluator).
> The PropertyMap are chained (one link chain between core to multicore map);
> those maps are owned by each core's ResourceLoader.
> Config is modified a little to accommodate delaying & specializing property
> expansions.
> Multicore is modified so it properly parses & serializes.
> Tested against the example above.
> Reviews & comments more than welcome.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
