[
https://issues.apache.org/jira/browse/SOLR-646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12624467#action_12624467
]
henrib edited comment on SOLR-646 at 8/21/08 2:06 PM:
-------------------------------------------------------------
0 - I *do* use today an include mechanism of some sort so my field types are
not defined in the schema; this allows to automate the schema generation from
another tool & not have to deal with copying content.
1 - the CoreContainer evaluator is a "transient" being; it keeps track of
properties as they are "read" and how they are evaluated during configuration.
It is not stored nor kept beyond the scope of load().
I dont see what you mean by house keeping. The things we keep afterwards are
property expressions (1 map in CoreContainer & 1 map per CoreDescriptor) and 1
Properties instance per SolrResourceLoader.
2 - The CoreContainer Evaluator is the part that declares properties, assign
them a scope, etc. The code is fairly well commented so I dont really get your
comment. Anyway, it is necessary, for persistence to be idempotent, ie we
should be able to write what we (just) read as in SolrProperties test, we need
to keep these around.
The Config evaluator is dealing with the include so I wont get there; what
about the idea of an "experimental" state?
3 - To keep the evaluator so we can extract the pieces of information it
collected (cf 1). The loaderProperties are filled in by the evaluator as the
last step of its subsituteProperties method. We have to do it at the end so the
original evaluation scope is preserved during the whole evaluation.
4 - You misinterpret my example; when I'm changing some indexing/schema
configuration in an heavy way, I like to be able to test it before, etc; these
changes (like changing stemmer, synonyms, etc) require reindexing. So, the
version I'm talking about is the 'index data' version. To be strict, I do
consider modifyng the schema a "major" version and modifying config a "minor"
one wrt to index reindex/reload. But that would make the example a bit terse.
5 - I believe I did not remove any public method in the patch, did I?
6 - The visitor pattern used in the original substituteSystemProperties has
just been 'methodified'. The code is no different than what it was and it
allows to implement just what's needed by derivation.
was (Author: henrib):
0 - I *do* use today an include mechanism of some sort so my field types
are not defined in the schema; this allows to automate the schema generation
from another tool & not have to deal with copying content.
1 - the CoreContainer evaluator is a "transient" being; it keeps track of
properties as they are "read" and how they are evaluated during configuration.
It is not stored nor kept beyond the scope of load().
I dont see what you mean by house keeping. The things we keep afterwards are
property expressions (1 map in CoreContainer & 1 map per core) and 1 Properties
instance per CoreContainer & per map.
2 - The CoreContainer Evaluator is the part that declares properties, assign
them a scope, etc. The code is fairly well commented so I dont really get your
comment. Anyway, it is necessary, for persistence to be idempotent, ie we
should be able to write what we (just) read as in SolrProperties test, we need
to keep these around.
The Config evaluator is dealing with the include so I wont get there; what
about the idea of an "experimental" state?
3 - To keep the evaluator so we can extract the pieces of information it
collected (cf 1). The loaderProperties are filled in by the evaluator as the
last step of its subsituteProperties method. We have to do it at the end so the
original evaluation scope is preserved during the whole evaluation.
4 - You misinterpret my example; when I'm changing some indexing/schema
configuration in an heavy way, I like to be able to test it before, etc; these
changes (like changing stemmer, synonyms, etc) require reindexing. So, the
version I'm talking about is the 'index data' version. To be strict, I do
consider modifyng the schema a "major" version and modifying config a "minor"
one wrt to index reindex/reload. But that would make the example a bit terse.
5 - I believe I did not remove any public method in the patch, did I?
6 - The visitor pattern used in the original substituteSystemProperties has
just been 'methodified'. The code is no different than what it was and it
allows to implement just what's needed by derivation.
> Configuration properties in multicore.xml
> -----------------------------------------
>
> Key: SOLR-646
> URL: https://issues.apache.org/jira/browse/SOLR-646
> Project: Solr
> Issue Type: New Feature
> Affects Versions: 1.3
> Reporter: Henri Biestro
> Assignee: Shalin Shekhar Mangar
> Fix For: 1.3
>
> Attachments: solr-646.patch, SOLR-646.patch, solr-646.patch,
> solr-646.patch, solr-646.patch, solr-646.patch, solr-646.patch
>
>
> This patch refers to 'generalized configuration properties' as specified by
> [HossMan|https://issues.apache.org/jira/browse/SOLR-350?focusedCommentId=12562834#action_12562834]
> This means configuration & schema files can use expression based on
> properties defined in *solr.xml*.
> h3. Use cases:
> Describe core data directories from solr.xml as properties.
> Share the same schema and/or config file between multiple cores.
> Share reusable fragments of schema & configuration between multiple cores.
> h3. Usage:
> h4. solr.xml
> This *solr.xml* will be used to illustrates using properties for different
> purpose.
> {code:xml}
> <solr persistent="true">
> <property name="version">1.3</property>
> <property name="lang">english, french</property>
> <property name="en-cores">en,core0</property>
> <property name="fr-cores">fr,core1</property>
> <cores adminPath="/admin/cores">
> <core name="${en-cores}" instanceDir="./">
> <property name="version">3.5</property>
> <property name="l10n">EN</property>
> <property name="ctlField">core0</property>
> <property name="comment">This is a sample</property>
> </core>
> <core name="${fr-cores}" instanceDir="./">
> <property name="version">2.4</property>
> <property name="l10n">FR</property>
> <property name="ctlField">core1</property>
> <property name="comment">Ceci est un exemple</property>
> </core>
> </cores>
> </solr>
> {code}
> {{version}} : if you update your solr.xml or your cores for various motives,
> it can be useful to track of a version. In this example, this will be used to
> define the {{dataDir}} for each core.
> {{en-cores}},{{fr-cores}}: with aliases, if the list is long or repetitive,
> it might be convenient to use a property that can then be used to describe
> the Solr core name.
> {{instanceDir}}: note that both cores will use the same instance directory,
> sharing their configuration and schema. The {{dataDir}} will be set for each
> of them from the *solrconfig.xml*.
> h4. solrconfig.xml
> This is where our *solr.xml* property are used to define the data directory
> as a composition of, in our example, the language code {{l10n}} and the core
> version stored in {{version}}.
> {code:xml}
> <config>
> <dataDir>${solr.solr.home}/data/${l10n}-${version}</dataDir>
> ....
> </config>
> {code}
> h5. schema.xml
> The {{include}} allows to import a file within the schema (or a solrconfig);
> this can help de-clutter long schemas.
> The {{ctlField}} is just illustrating that a field & its type can be set
> through properties as well; in our example, we will want the 'english' core
> to refer to an 'english-configured' field and the 'french' core to a
> 'french-configured' one. The type for the field is defined as {{text-EN}} or
> {{text-FR}} after expansion.
> {code:xml}
> <schema name="example core ${l10n}" version="1.1">
> <types>
> ...
> <include resource="text-l10n.xml"/>
> </types>
> <fields>
> ...
> <field name="${ctlField}" type="text-${l10n}" indexed="true"
> stored="true" multiValued="true" />
> </fields>
> {code}
> This schema is importing this *text-l10n.xml* file which is a *fragment*; the
> fragment tag must be present & indicates the file is to be included. Our
> example only defines different stopwords for each language but you could of
> course extend this to stemmers, synonyms, etc.
> {code:xml}
> <fragment>
> <fieldType name="text-FR" class="solr.TextField"
> positionIncrementGap="100">
> ...
> <filter class="solr.StopFilterFactory" ignoreCase="true"
> words="stopwords-fr.txt"/>
> ...
> </fieldType>
> <fieldType name="text-EN" class="solr.TextField"
> positionIncrementGap="100">
> ...
> <filter class="solr.StopFilterFactory" ignoreCase="true"
> words="stopwords-en.txt"/>
> ...
> </fieldType>
> </fragment>
> {code}
> h4. Technical specifications
> solr.xml can define properties at the multicore & each core level.
> Properties defined in the multicore scope can override system properties.
> Properties defined in a core scope can override multicore & system properties.
> Property definitions can use expressions to define their name & value; these
> expressions are evaluated in their outer scope context .
> CoreContainer serialization keeps properties as defined; persistence is
> idem-potent. (ie property expressions are written, not their evaluation).
> The core descriptor properties are automatically defined in each core
> context, namely:
> solr.core.instanceDir
> solr.core.name
> solr.core.configName
> solr.core.schemaName
> h3. Coding notes:
> - DOMUtil.java:
> refactored substituteSystemProperties to use an Evaluator;
> an Evaluator is a DOM visitor that expands property expressions "in place"
> using a property map as an evaluation context
> added an asString(node) method for logging purpose
> - CoreDescriptor.java:
> added an expression member to keep property expressions as defined in
> solr.xml for persistence - allowing to write file as defined (not as expanded)
> - CoreContainer.java:
> add an expression member to keep property expression as defined in solr.xml
> for persistence - allowing to write file as defined (not as expanded);
> solrx.xml peristence is idem-potent
> added a local DOMUtil.Evaluator that tracks property expressions to evaluate
> & store them
> *issues outlined through solr-646:*
> fix in load:
> CoreDescriptor p = new CoreDescriptor(this, names, ....);
> was: CoreDescriptor p = new CoreDescriptor(this, name, ...);
> fix in load;
> register(aliases.get(a), core, false);
> was of register(aliases.get(i), core, false);
> - CoreAdminHandler.java
> added an optional fileName to persist so it is possible to write the solr.xml
> to a different file (for comparison purpose)
> - CoreAdminRequest.java
> added PersistRequest to allow passing optional fileName
> - Config.java:
> subsituteProperties has been moved out of constructor & doc member made
> protected to allow override
> added an IncludesEvaluator that deals with include/fragment
> - SolrConfig.java & IndexSchema.ava
> added explicit calls to substituteProperties to perform property/include
> expansion
> - SolrResourceLoader.java
> added properties member to store CoreContainer & per-SolrCore properties
> added constructor properties parameter & getter for properties
> - SolrProperties.java:
> test inspired by MulticoreExampleTestBase.java
> loads 2 cores sharing a schema & config;
> config define dataDir using a property
> schema uses a localization (l10n) property to define an attribute
> persists the file to check it keeps the expression properties
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
