[
https://issues.apache.org/jira/browse/SOLR-433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12570712#action_12570712
]
dsteigerwald edited comment on SOLR-433 at 2/20/08 7:19 AM:
----------------------------------------------------------------
We have a ctl script that controls all of the functions and makes sure we don't
run some things on the slaves (ie, snappuller, snapinstaller, rsyncd stuff).
We pass it a core and an index type:
{noformat}
./snapctl -a [rsyncd-enable/snappuller/snapinstaller/etc] -c core0 -i spell
{noformat}
spell is the name of our spellcheck index. Default index is 'index'.
This is testing well in QA right now. Hopefully this will help others out and
maybe we'll have something similar committed soon.
Here's the basics of our snapctl script. It's really tailored to our
environment, so I probably won't post it as is.
{noformat}
$SOLR_BIN=/home/dsteiger/apps/solr/solr/bin
$CORES_PATH=/home/dsteiger/local/solr/cores
$CORE=core0 # from -c arg, default is 'null'
$INDEX=spell # from -i arg, default is 'index'
$SOLR_LOGS=/home/dsteiger/apps/solr/logs # symlinked to somewhere else
# $MASTER_HOST is determined based on environment (devel/qa/prod) from the
scripts.conf
$SOLR_BIN/rsyncd-enable
$SOLR_BIN/rsyncd-disable
$SOLR_BIN/rsyncd-start -d $CORES_PATH
$SOLR_BIN/rsyncd-stop
$SOLR_BIN/snappuller-enable
$SOLR_BIN/snappuller-disable
$SOLR_BIN/snapshooter -d $CORES_PATH/$CORE/data -i $INDEX
$SOLR_BIN/snappuller -M $MASTER_HOST -S $SOLR_LOGS/clients -D
$CORES_PATH/$CORE/data -d $CORES_PATH/$CORE/data -z -c $CORE -i $INDEX
$SOLR_BIN/snapinstaller -M $MASTER_HOST -S $SOLR_LOGS/clients -d
$CORES_PATH/$CORE/data -c $CORE -i $INDEX
$SOLR_BIN/snapcleaner -D 1 -d $CORES_PATH/$CORE/data -i $INDEX
{noformat}
We also modified core.RunExecutableListener to be able to pass the core name to
out snapctl script.
{code:xml}
<listener event="postCommit" class="core.RunExecutableListener">
<str name="exe">./solr/bin/snapctl</str>
<str name="dir">.</str>
<bool name="wait">true</bool>
<bool name="coreName">true</bool>
<arr name="args"><str>-a snapshooter</str><str>-i index</str></arr>
</listener>
<listener event="postOptimize" class="core.RunExecutableListener">
<str name="exe">./solr/bin/snapctl</str>
<str name="dir">.</str>
<bool name="wait">true</bool>
<bool name="coreName">true</bool>
<arr name="args"> <str>-a snapshooter</str> </arr>
</listener>
{code}
Going to attach patch to RunExecutableListener we're using.
was (Author: dsteigerwald):
We have a ctl script that controls all of the functions and makes sure we
don't run some things on the slaves (ie, snappuller, snapinstaller, rsyncd
stuff).
We pass it a core and an index type:
{noformat}
./snapctl -a [rsyncd-enable/snappuller/snapinstaller/etc] -c core0 -i spell
{noformat}
spell is the name of our spellcheck index. Default index is 'index'.
This is testing well in QA right now. Hopefully this will help others out and
maybe we'll have something similar committed soon.
Here's the basics of our snapctl script. It's really tailored to our
environment, so I probably won't post it as is.
{noformat}
$SOLR_BIN=/home/dsteiger/apps/solr/solr/bin
$CORES_PATH=/home/dsteiger/local/solr/cores
$CORE=core0 # from -c arg, default is 'null'
$INDEX=spell # from -i arg, default is 'index'
$SOLR_LOGS=/home/dsteiger/apps/solr/logs # symlinked to somewhere else
# $MASTER_HOST is determined based on environment (devel/qa/prod) from the
scripts.conf
$SOLR_BIN/rsyncd-enable
$SOLR_BIN/rsyncd-disable
$SOLR_BIN/rsyncd-start -d $CORES_PATH
$SOLR_BIN/rsyncd-stop
$SOLR_BIN/snappuller-enable
$SOLR_BIN/snappuller-disable
$SOLR_BIN/snapshooter -d $CORES_PATH/$CORE/data -i $INDEX
$SOLR_BIN/snappuller -M $MASTER_HOST -S $SOLR_LOGS/clients -D
$CORES_PATH/$CORE/data -d $CORES_PATH/$CORE/data -z -c $CORE -i $INDEX
$SOLR_BIN/snapinstaller -M $MASTER_HOST -S $SOLR_LOGS/clients -d
$CORES_PATH/$CORE/data -c $CORE -i $INDEX
$SOLR_BIN/snapcleaner -D 1 -d $CORES_PATH/$CORE/data -i $INDEX
{noformat}
We also modified core.RunExecutableListener to be able to pass the core name to
out snapctl script.
{code:xml}
<listener event="postCommit" class="core.RunExecutableListener">
<str name="exe">./solr/bin/snapctl</str>
<str name="dir">.</str>
<bool name="wait">true</bool>
<bool name="coreName">true</bool>
<arr name="args"><str>-a snapshooter</str><str>-i index</str></arr>
</listener>
<listener event="postOptimize" class="core.RunExecutableListener">
<str name="exe">./solr/bin/snapctl</str>
<str name="dir">.</str>
<bool name="wait">true</bool>
<bool name="coreName">true</bool>
<arr name="args"> <str>-a snapshooter</str> </arr>
</listener>
{code}
I think this is all I changed for RunExecutableListener:
{code:title=core.RunExecutableListener.java|borderStyle=solid}
// in init()
// Add the core name to the command.
if ("true".equals(args.get("coreName")) ||
Boolean.TRUE.equals(args.get("coreName"))) {
cmdlist.add("-c " + core.getName());
}
{code}
> MultiCore and SpellChecker replication
> --------------------------------------
>
> Key: SOLR-433
> URL: https://issues.apache.org/jira/browse/SOLR-433
> Project: Solr
> Issue Type: Improvement
> Components: replication, spellchecker
> Affects Versions: 1.3
> Reporter: Otis Gospodnetic
> Fix For: 1.3
>
> Attachments: solr-433.patch
>
>
> With MultiCore functionality coming along, it looks like we'll need to be
> able to:
> A) snapshot each core's index directory, and
> B) replicate any and all cores' complete data directories, not just their
> index directories.
> Pulled from the "spellchecker and multi-core index replication" thread -
> http://markmail.org/message/pj2rjzegifd6zm7m
> Otis:
> I think that makes sense - distribute everything for a given core, not just
> its index. And the spellchecker could then also have its data dir (and only
> index/ underneath really) and be replicated in the same fashion.
> Right?
> Ryan:
> Yes, that was my thought. If an arbitrary directory could be distributed,
> then you could have
> /path/to/dist/index/...
> /path/to/dist/spelling-index/...
> /path/to/dist/foo
> and that would all get put into a snapshot. This would also let you put
> multiple cores within a single distribution:
> /path/to/dist/core0/index/...
> /path/to/dist/core0/spelling-index/...
> /path/to/dist/core0/foo
> /path/to/dist/core1/index/...
> /path/to/dist/core1/spelling-index/...
> /path/to/dist/core1/foo
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
