[
https://issues.apache.org/jira/browse/SOLR-561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12641495#action_12641495
]
Shalin Shekhar Mangar commented on SOLR-561:
--------------------------------------------
Thanks for going through this Yonik.
bq. Snappuller should use getNewestSearcher() rather than getSearcher() to
avoid pulling the same snapshot more than once if warming takes a long time.
The SnapPuller calls commit with waitSearcher=true, so the call will wait for
the searcher to get registered and warmed. The reentrant lock in SnapPuller
will be released only after the commit call returns. So it should be OK, right?
bq. It seems like renewing a lease (a short term reservation) whenever an
access is done would solve both of these problems (and is what I initially had
in mind). All requests should indicate what commit point is being copied so
that the lease can be extended.
Since files are transferred in one go, the master knows about the access time
but it does not know if the transfer has ended so the lease may expire in
between the transfer leading to a failure. We'll need to track the transfers
individually as well. If the slave dies in between the transfer, we'll need to
track that as well and time-out the lease appropriately. If I compare the state
of things to the old way of replication, not sure if this feature is worth the
effort. What do you think?
bq. Has anyone tested this with a large files (say 5G or more)...
We have been testing it with a large index (wikipedia articles, around 7-8GB
index on disk) with Tomcat across networks (transfer rate between servers is
around 700-800 KB/sec). We haven't seen any problem yet. We'll continue to test
this with Tomcat and other containers and report performance numbers and
problems, if any.
> Solr replication by Solr (for windows also)
> -------------------------------------------
>
> Key: SOLR-561
> URL: https://issues.apache.org/jira/browse/SOLR-561
> Project: Solr
> Issue Type: New Feature
> Components: replication
> Affects Versions: 1.4
> Environment: All
> Reporter: Noble Paul
> Assignee: Shalin Shekhar Mangar
> Fix For: 1.4
>
> Attachments: deletion_policy.patch, SOLR-561-core.patch,
> SOLR-561-full.patch, SOLR-561-full.patch, SOLR-561-full.patch,
> SOLR-561-full.patch, SOLR-561.patch, SOLR-561.patch, SOLR-561.patch,
> SOLR-561.patch, SOLR-561.patch, SOLR-561.patch, SOLR-561.patch,
> SOLR-561.patch, SOLR-561.patch, SOLR-561.patch, SOLR-561.patch
>
>
> The current replication strategy in solr involves shell scripts . The
> following are the drawbacks with the approach
> * It does not work with windows
> * Replication works as a separate piece not integrated with solr.
> * Cannot control replication from solr admin/JMX
> * Each operation requires manual telnet to the host
> Doing the replication in java has the following advantages
> * Platform independence
> * Manual steps can be completely eliminated. Everything can be driven from
> solrconfig.xml .
> ** Adding the url of the master in the slaves should be good enough to enable
> replication. Other things like frequency of
> snapshoot/snappull can also be configured . All other information can be
> automatically obtained.
> * Start/stop can be triggered from solr/admin or JMX
> * Can get the status/progress while replication is going on. It can also
> abort an ongoing replication
> * No need to have a login into the machine
> * From a development perspective, we can unit test it
> This issue can track the implementation of solr replication in java
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
