[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15585936#comment-15585936 ] Hrishikesh Gadre commented on SOLR-5750: [~TimOwen] [~dsmiley] I think this is not yet implemented (due to some unit test failure ?). https://github.com/apache/lucene-solr/commit/70bcd562f98ede21dfc93a1ba002c61fac888b29#diff-e864a6be5b98b5340273c1db4f4677a6R107 I am not sure why this problem exists just for restore operation (and not for create). > Backup/Restore API for SolrCloud > > > Key: SOLR-5750 > URL: https://issues.apache.org/jira/browse/SOLR-5750 > Project: Solr > Issue Type: Sub-task > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Varun Thacker > Fix For: 6.1 > > Attachments: SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, > SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch > > > We should have an easy way to do backups and restores in SolrCloud. The > ReplicationHandler supports a backup command which can create snapshots of > the index but that is too little. > The command should be able to backup: > # Snapshots of all indexes or indexes from the leader or the shards > # Config set > # Cluster state > # Cluster properties > # Aliases > # Overseer work queue? > A restore should be able to completely restore the cloud i.e. no manual steps > required other than bringing nodes back up or setting up a new cloud cluster. > SOLR-5340 will be a part of this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15585826#comment-15585826 ] Tim Owen commented on SOLR-5750: [~dsmiley] you mentioned in the mailing list back in March that you'd fixed the situation where restored collections are created using the old stateFormat=1 but it still seems to be doing that ... did that fix not make it into this ticket before merging? We've been trying out the backup/restore and noticed it's putting the collection's state into the global clusterstate.json instead of where it should be. > Backup/Restore API for SolrCloud > > > Key: SOLR-5750 > URL: https://issues.apache.org/jira/browse/SOLR-5750 > Project: Solr > Issue Type: Sub-task > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Varun Thacker > Fix For: 6.1 > > Attachments: SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, > SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch > > > We should have an easy way to do backups and restores in SolrCloud. The > ReplicationHandler supports a backup command which can create snapshots of > the index but that is too little. > The command should be able to backup: > # Snapshots of all indexes or indexes from the leader or the shards > # Config set > # Cluster state > # Cluster properties > # Aliases > # Overseer work queue? > A restore should be able to completely restore the cloud i.e. no manual steps > required other than bringing nodes back up or setting up a new cloud cluster. > SOLR-5340 will be a part of this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15585823#comment-15585823 ] Tim Owen commented on SOLR-5750: [~dsmiley] you mentioned in the mailing list back in March that you'd fixed the situation where restored collections are created using the old stateFormat=1 but it still seems to be doing that ... did that fix not make it into this ticket before merging? We've been trying out the backup/restore and noticed it's putting the collection's state into the global clusterstate.json instead of where it should be. > Backup/Restore API for SolrCloud > > > Key: SOLR-5750 > URL: https://issues.apache.org/jira/browse/SOLR-5750 > Project: Solr > Issue Type: Sub-task > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Varun Thacker > Fix For: 6.1 > > Attachments: SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, > SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch > > > We should have an easy way to do backups and restores in SolrCloud. The > ReplicationHandler supports a backup command which can create snapshots of > the index but that is too little. > The command should be able to backup: > # Snapshots of all indexes or indexes from the leader or the shards > # Config set > # Cluster state > # Cluster properties > # Aliases > # Overseer work queue? > A restore should be able to completely restore the cloud i.e. no manual steps > required other than bringing nodes back up or setting up a new cloud cluster. > SOLR-5340 will be a part of this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15276630#comment-15276630 ] Hrishikesh Gadre commented on SOLR-5750: [~varunthacker] Thanks for the documentation. I have few comments, bq. Backup and Restore Solr collections and it's associated configurations to a shared filesystem - for example HDFS or a Network File System Currently we don't support integration with HDFS directly. This is being done as part of SOLR-9055. bq. location/string/no/The location on the shared drive for the backup command to write to. Alternately it can be set as a cluster property (hyperlink to CLUSTERPROP and document it as a supported property) I don't think we have added support for configuring location via CLUSTERPROP API. We have discussed an alternate approach of configuring file-systems (or Backup repositories) via solr.xml. So I think it would make more sense to configure "default" location as part of that change (instead of enabling it via CLUSTERPROP API). This is being done as part of SOLR-9055. > Backup/Restore API for SolrCloud > > > Key: SOLR-5750 > URL: https://issues.apache.org/jira/browse/SOLR-5750 > Project: Solr > Issue Type: Sub-task > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Varun Thacker > Fix For: 6.1 > > Attachments: SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, > SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch > > > We should have an easy way to do backups and restores in SolrCloud. The > ReplicationHandler supports a backup command which can create snapshots of > the index but that is too little. > The command should be able to backup: > # Snapshots of all indexes or indexes from the leader or the shards > # Config set > # Cluster state > # Cluster properties > # Aliases > # Overseer work queue? > A restore should be able to completely restore the cloud i.e. no manual steps > required other than bringing nodes back up or setting up a new cloud cluster. > SOLR-5340 will be a part of this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15275476#comment-15275476 ] Varun Thacker commented on SOLR-5750: - Some docs on the feature . Any thoughts on this otherwise I'll add it over to the ref guide SolrCloud Backup and Restore of Collections Backup and Restore Solr collections and it's associated configurations to a shared filesystem - for example HDFS or a Network File System Backup command: /admin/collections?action=BACKUP&name=myBackupName&collection=myCollectionName&location=/path/to/my/shared/drive The backup command will backup Solr indexes and configurations for a specified collection. The backup command takes one copy from each shard for the indexes. For configurations it backs up the configSet that was associated with the collection and other meta information. key/Type/Required/Default/Description name/string/yes//The backup name collection/string/yes//The name of the collection that needs to be backed up location/string/no/The location on the shared drive for the backup command to write to. Alternately it can be set as a cluster property (hyperlink to CLUSTERPROP and document it as a supported property) async ( copy over from existing docs) Restore command: /admin/collections?action=RESTORE&name=myBackupName&location=/path/to/my/sharded/drive&collection=myRestoredCollectionName Restores Solr indexes and associated configurations. The restore operation will create a collection with the specified name from the collection parameter. You cannot restore into the same collection and the collection should not be present at the time of restoring the collection, Solr will create it for you. The collection created will be of the same number of shards and replicas as the original colleciton, preserving routing information etc. Optionally you can overide some parameters documented below. For restoring the associated configSet if a configSet with the same name exists in ZK then Solr will reuse that else it will upload the backed up configSet in ZooKeeper and use that for the restored collection. You can use the Collection ALIAS (hyperlink) feature to make sure client's don't need to change the endpoint to query or index against the restored collection. key/Type/Required/Default/Description name/string/yes//The backup name that needs to be restored collection/string/yes//The collection where the indexes will be restored to. location/string/no/The location on the shared drive for the restore command to read from. Alternately it can be set as a cluster property (hyperlink to CLUSTERPROP and document it as a supported property) (copy over from existing docs) async collection.configName replicationFactor maxShardsPerNode autoAddReplicas property.Param stateFormat > Backup/Restore API for SolrCloud > > > Key: SOLR-5750 > URL: https://issues.apache.org/jira/browse/SOLR-5750 > Project: Solr > Issue Type: Sub-task > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Varun Thacker > Fix For: 6.1 > > Attachments: SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, > SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch > > > We should have an easy way to do backups and restores in SolrCloud. The > ReplicationHandler supports a backup command which can create snapshots of > the index but that is too little. > The command should be able to backup: > # Snapshots of all indexes or indexes from the leader or the shards > # Config set > # Cluster state > # Cluster properties > # Aliases > # Overseer work queue? > A restore should be able to completely restore the cloud i.e. no manual steps > required other than bringing nodes back up or setting up a new cloud cluster. > SOLR-5340 will be a part of this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15272334#comment-15272334 ] ASF subversion and git services commented on SOLR-5750: --- Commit 18d933ee65320ab1cae92a79d6635996fee9e818 in lucene-solr's branch refs/heads/master from [~dsmiley] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=18d933e ] SOLR-5750: Fix test to specify the collection on add > Backup/Restore API for SolrCloud > > > Key: SOLR-5750 > URL: https://issues.apache.org/jira/browse/SOLR-5750 > Project: Solr > Issue Type: Sub-task > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Varun Thacker > Fix For: 6.1 > > Attachments: SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, > SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch > > > We should have an easy way to do backups and restores in SolrCloud. The > ReplicationHandler supports a backup command which can create snapshots of > the index but that is too little. > The command should be able to backup: > # Snapshots of all indexes or indexes from the leader or the shards > # Config set > # Cluster state > # Cluster properties > # Aliases > # Overseer work queue? > A restore should be able to completely restore the cloud i.e. no manual steps > required other than bringing nodes back up or setting up a new cloud cluster. > SOLR-5340 will be a part of this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15272336#comment-15272336 ] ASF subversion and git services commented on SOLR-5750: --- Commit 83640278860d58ac01a7233ea4a96a49a3b6843d in lucene-solr's branch refs/heads/branch_6x from [~dsmiley] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=8364027 ] SOLR-5750: Fix test to specify the collection on add (cherry picked from commit 18d933e) > Backup/Restore API for SolrCloud > > > Key: SOLR-5750 > URL: https://issues.apache.org/jira/browse/SOLR-5750 > Project: Solr > Issue Type: Sub-task > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Varun Thacker > Fix For: 6.1 > > Attachments: SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, > SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch > > > We should have an easy way to do backups and restores in SolrCloud. The > ReplicationHandler supports a backup command which can create snapshots of > the index but that is too little. > The command should be able to backup: > # Snapshots of all indexes or indexes from the leader or the shards > # Config set > # Cluster state > # Cluster properties > # Aliases > # Overseer work queue? > A restore should be able to completely restore the cloud i.e. no manual steps > required other than bringing nodes back up or setting up a new cloud cluster. > SOLR-5340 will be a part of this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15271341#comment-15271341 ] ASF subversion and git services commented on SOLR-5750: --- Commit 0a20dd47d1abbf0036896fac03dc7d801ebcd5bd in lucene-solr's branch refs/heads/master from [~dsmiley] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=0a20dd4 ] SOLR-5750: Fix test to specify the collection on commit > Backup/Restore API for SolrCloud > > > Key: SOLR-5750 > URL: https://issues.apache.org/jira/browse/SOLR-5750 > Project: Solr > Issue Type: Sub-task > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Varun Thacker > Fix For: 6.1 > > Attachments: SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, > SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch > > > We should have an easy way to do backups and restores in SolrCloud. The > ReplicationHandler supports a backup command which can create snapshots of > the index but that is too little. > The command should be able to backup: > # Snapshots of all indexes or indexes from the leader or the shards > # Config set > # Cluster state > # Cluster properties > # Aliases > # Overseer work queue? > A restore should be able to completely restore the cloud i.e. no manual steps > required other than bringing nodes back up or setting up a new cloud cluster. > SOLR-5340 will be a part of this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15271339#comment-15271339 ] ASF subversion and git services commented on SOLR-5750: --- Commit 13832b4f857bc6f726a4764aa446a487b847bfee in lucene-solr's branch refs/heads/branch_6x from [~dsmiley] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=13832b4 ] SOLR-5750: Fix test to specify the collection on commit (cherry picked from commit 0a20dd4) > Backup/Restore API for SolrCloud > > > Key: SOLR-5750 > URL: https://issues.apache.org/jira/browse/SOLR-5750 > Project: Solr > Issue Type: Sub-task > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Varun Thacker > Fix For: 6.1 > > Attachments: SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, > SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch > > > We should have an easy way to do backups and restores in SolrCloud. The > ReplicationHandler supports a backup command which can create snapshots of > the index but that is too little. > The command should be able to backup: > # Snapshots of all indexes or indexes from the leader or the shards > # Config set > # Cluster state > # Cluster properties > # Aliases > # Overseer work queue? > A restore should be able to completely restore the cloud i.e. no manual steps > required other than bringing nodes back up or setting up a new cloud cluster. > SOLR-5340 will be a part of this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15271173#comment-15271173 ] David Smiley commented on SOLR-5750: Thanks for reporting; I'll dig. > Backup/Restore API for SolrCloud > > > Key: SOLR-5750 > URL: https://issues.apache.org/jira/browse/SOLR-5750 > Project: Solr > Issue Type: Sub-task > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Varun Thacker > Fix For: 6.1 > > Attachments: SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, > SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch > > > We should have an easy way to do backups and restores in SolrCloud. The > ReplicationHandler supports a backup command which can create snapshots of > the index but that is too little. > The command should be able to backup: > # Snapshots of all indexes or indexes from the leader or the shards > # Config set > # Cluster state > # Cluster properties > # Aliases > # Overseer work queue? > A restore should be able to completely restore the cloud i.e. no manual steps > required other than bringing nodes back up or setting up a new cloud cluster. > SOLR-5340 will be a part of this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[
https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270796#comment-15270796
]
Shalin Shekhar Mangar commented on SOLR-5750:
-
I am seeing a reproducible failure in TestCloudBackupRestore
{code}
2> NOTE: reproduce with: ant test -Dtestcase=TestCloudBackupRestore
-Dtests.method=test -Dtests.seed=4AD582D4894F28CF -Dtests.slow=true
-Dtests.locale=fr-BE -Dtests.timezone=Europe/Simferopol -Dtests.asserts=true
-Dtests.file.encoding=ISO-8859-1
[19:07:22.801] ERROR 28.2s J0 | TestCloudBackupRestore.test <<<
> Throwable #1: org.apache.solr.client.solrj.SolrServerException: No
collection param specified on request and no default collection has been set.
>at
__randomizedtesting.SeedInfo.seed([4AD582D4894F28CF:C281BD0E27B34537]:0)
>at
org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:590)
>at
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1073)
>at
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:962)
>at
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:898)
>at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)
>at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:484)
>at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:463)
>at
org.apache.solr.cloud.TestCloudBackupRestore.test(TestCloudBackupRestore.java:105)
{code}
> Backup/Restore API for SolrCloud
>
>
> Key: SOLR-5750
> URL: https://issues.apache.org/jira/browse/SOLR-5750
> Project: Solr
> Issue Type: Sub-task
> Components: SolrCloud
>Reporter: Shalin Shekhar Mangar
>Assignee: Varun Thacker
> Fix For: 6.1
>
> Attachments: SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch,
> SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch
>
>
> We should have an easy way to do backups and restores in SolrCloud. The
> ReplicationHandler supports a backup command which can create snapshots of
> the index but that is too little.
> The command should be able to backup:
> # Snapshots of all indexes or indexes from the leader or the shards
> # Config set
> # Cluster state
> # Cluster properties
> # Aliases
> # Overseer work queue?
> A restore should be able to completely restore the cloud i.e. no manual steps
> required other than bringing nodes back up or setting up a new cloud cluster.
> SOLR-5340 will be a part of this issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15268021#comment-15268021 ] ASF subversion and git services commented on SOLR-5750: --- Commit dac044c94a33ebd655c1d5f5c628c83c75bf8697 in lucene-solr's branch refs/heads/branch_6x from [~dsmiley] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=dac044c ] SOLR-5750: Add /admin/collections?action=BACKUP and RESTORE (cherry picked from commit 70bcd56) > Backup/Restore API for SolrCloud > > > Key: SOLR-5750 > URL: https://issues.apache.org/jira/browse/SOLR-5750 > Project: Solr > Issue Type: Sub-task > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: David Smiley > Fix For: 5.2, master > > Attachments: SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, > SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch > > > We should have an easy way to do backups and restores in SolrCloud. The > ReplicationHandler supports a backup command which can create snapshots of > the index but that is too little. > The command should be able to backup: > # Snapshots of all indexes or indexes from the leader or the shards > # Config set > # Cluster state > # Cluster properties > # Aliases > # Overseer work queue? > A restore should be able to completely restore the cloud i.e. no manual steps > required other than bringing nodes back up or setting up a new cloud cluster. > SOLR-5340 will be a part of this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15267951#comment-15267951 ] ASF subversion and git services commented on SOLR-5750: --- Commit 70bcd562f98ede21dfc93a1ba002c61fac888b29 in lucene-solr's branch refs/heads/master from [~dsmiley] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=70bcd56 ] SOLR-5750: Add /admin/collections?action=BACKUP and RESTORE > Backup/Restore API for SolrCloud > > > Key: SOLR-5750 > URL: https://issues.apache.org/jira/browse/SOLR-5750 > Project: Solr > Issue Type: Sub-task > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: David Smiley > Fix For: 5.2, master > > Attachments: SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, > SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch > > > We should have an easy way to do backups and restores in SolrCloud. The > ReplicationHandler supports a backup command which can create snapshots of > the index but that is too little. > The command should be able to backup: > # Snapshots of all indexes or indexes from the leader or the shards > # Config set > # Cluster state > # Cluster properties > # Aliases > # Overseer work queue? > A restore should be able to completely restore the cloud i.e. no manual steps > required other than bringing nodes back up or setting up a new cloud cluster. > SOLR-5340 will be a part of this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15266708#comment-15266708 ] Hrishikesh Gadre commented on SOLR-5750: [~dsmiley] sure. I have filed SOLR-9055 and uploaded the patch. Please take a look and let me have your feedback. > Backup/Restore API for SolrCloud > > > Key: SOLR-5750 > URL: https://issues.apache.org/jira/browse/SOLR-5750 > Project: Solr > Issue Type: Sub-task > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: David Smiley > Fix For: 5.2, master > > Attachments: SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, > SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch > > > We should have an easy way to do backups and restores in SolrCloud. The > ReplicationHandler supports a backup command which can create snapshots of > the index but that is too little. > The command should be able to backup: > # Snapshots of all indexes or indexes from the leader or the shards > # Config set > # Cluster state > # Cluster properties > # Aliases > # Overseer work queue? > A restore should be able to completely restore the cloud i.e. no manual steps > required other than bringing nodes back up or setting up a new cloud cluster. > SOLR-5340 will be a part of this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15266098#comment-15266098 ] David Smiley commented on SOLR-5750: I made some small updates to the branch like I said I would. I elected _not_ to switch Snapshooter to use mkdir instead of mkdirs for back-compat concerns so I have the backup caller make this check instead. [~hgadre] I took a look. As this issue finally nears the end, I don't feel good about rushing into the multiple abstractions introduced in the P/R (not to say I don't like them). Please file a new issue and add me as a Watcher. I have some comments but I'll wait to put them there. > Backup/Restore API for SolrCloud > > > Key: SOLR-5750 > URL: https://issues.apache.org/jira/browse/SOLR-5750 > Project: Solr > Issue Type: Sub-task > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: David Smiley > Fix For: 5.2, master > > Attachments: SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, > SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch > > > We should have an easy way to do backups and restores in SolrCloud. The > ReplicationHandler supports a backup command which can create snapshots of > the index but that is too little. > The command should be able to backup: > # Snapshots of all indexes or indexes from the leader or the shards > # Config set > # Cluster state > # Cluster properties > # Aliases > # Overseer work queue? > A restore should be able to completely restore the cloud i.e. no manual steps > required other than bringing nodes back up or setting up a new cloud cluster. > SOLR-5340 will be a part of this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15264910#comment-15264910 ] Hrishikesh Gadre commented on SOLR-5750: [~dsmiley] Please take a look at following pull request https://github.com/apache/lucene-solr/pull/36 Unfortunately I wasn't able to apply your latest patch. Hence I had to send a PR. I am still working on refactoring the "restore" API. But in the mean time any feedback would be great. > Backup/Restore API for SolrCloud > > > Key: SOLR-5750 > URL: https://issues.apache.org/jira/browse/SOLR-5750 > Project: Solr > Issue Type: Sub-task > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: David Smiley > Fix For: 5.2, master > > Attachments: SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, > SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch > > > We should have an easy way to do backups and restores in SolrCloud. The > ReplicationHandler supports a backup command which can create snapshots of > the index but that is too little. > The command should be able to backup: > # Snapshots of all indexes or indexes from the leader or the shards > # Config set > # Cluster state > # Cluster properties > # Aliases > # Overseer work queue? > A restore should be able to completely restore the cloud i.e. no manual steps > required other than bringing nodes back up or setting up a new cloud cluster. > SOLR-5340 will be a part of this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15264704#comment-15264704 ] Hrishikesh Gadre commented on SOLR-5750: [~dsmiley] I am almost done with refactoring the patch. I will submit the patch in next couple of hours. >>to avoid risk of confusion with "snapshot" possibly being a named commit >>(SOLR-9038) in the log statements and backup.properites I'll call it a >>backupName, not snapshotName. I have already fixed this in my patch. > Backup/Restore API for SolrCloud > > > Key: SOLR-5750 > URL: https://issues.apache.org/jira/browse/SOLR-5750 > Project: Solr > Issue Type: Sub-task > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: David Smiley > Fix For: 5.2, master > > Attachments: SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, > SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch > > > We should have an easy way to do backups and restores in SolrCloud. The > ReplicationHandler supports a backup command which can create snapshots of > the index but that is too little. > The command should be able to backup: > # Snapshots of all indexes or indexes from the leader or the shards > # Config set > # Cluster state > # Cluster properties > # Aliases > # Overseer work queue? > A restore should be able to completely restore the cloud i.e. no manual steps > required other than bringing nodes back up or setting up a new cloud cluster. > SOLR-5340 will be a part of this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[
https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15264676#comment-15264676
]
David Smiley commented on SOLR-5750:
I'll give some more time for review, like maybe Monday unless there are further
changes to be done from any review/feedback. Some things I think I want to
change (which I will do today):
* simply remove the Overseer.processMessage case statements for RESTORE &
BACKUP as they simply aren't used. This resolves a nocommit.
* to avoid risk of confusion with "snapshot" possibly being a named commit
(SOLR-9038) in the log statements and backup.properites I'll call it a
backupName, not snapshotName.
Tentative CHANGES.txt is as follows:
{noformat}
* SOLR-5750: Add /admin/collections?action=BACKUP and RESTORE assuming access
to a shared file system.
(Varun Thacker, David Smiley)
{noformat}
About the "shared file system" requirement, it occurred to me this isn't really
tested; it'd be nice it if failed fast if not all shards can see the backup
location's ZK backup export. I'm working on ensuring the backup fails if all
slices don't see the backup directory that should be created at the start of
the backup process. This seems a small matter of ensuring that
SnapShooter.validateCreateSnapshot call mkdir (which will fail if the parent
dir isn't there) and not mkdirs but I'm testing to ensure the replication
handler's use of SnapShooter is fine with this; I think it is.
> Backup/Restore API for SolrCloud
>
>
> Key: SOLR-5750
> URL: https://issues.apache.org/jira/browse/SOLR-5750
> Project: Solr
> Issue Type: Sub-task
> Components: SolrCloud
>Reporter: Shalin Shekhar Mangar
>Assignee: David Smiley
> Fix For: 5.2, master
>
> Attachments: SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch,
> SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch
>
>
> We should have an easy way to do backups and restores in SolrCloud. The
> ReplicationHandler supports a backup command which can create snapshots of
> the index but that is too little.
> The command should be able to backup:
> # Snapshots of all indexes or indexes from the leader or the shards
> # Config set
> # Cluster state
> # Cluster properties
> # Aliases
> # Overseer work queue?
> A restore should be able to completely restore the cloud i.e. no manual steps
> required other than bringing nodes back up or setting up a new cloud cluster.
> SOLR-5340 will be a part of this issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15261542#comment-15261542 ] David Smiley commented on SOLR-5750: I commented on SOLR-9038. Should that issue come to pass, you're right that we need to differentiate a "backup" from a "snapshot" in our terminology, assuming we even want these names. As it is, in some of the code I used "snapshotName" as a nod to the existing snapshooter code using such terminology. > Backup/Restore API for SolrCloud > > > Key: SOLR-5750 > URL: https://issues.apache.org/jira/browse/SOLR-5750 > Project: Solr > Issue Type: Sub-task > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Varun Thacker > Fix For: 5.2, master > > Attachments: SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, > SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch > > > We should have an easy way to do backups and restores in SolrCloud. The > ReplicationHandler supports a backup command which can create snapshots of > the index but that is too little. > The command should be able to backup: > # Snapshots of all indexes or indexes from the leader or the shards > # Config set > # Cluster state > # Cluster properties > # Aliases > # Overseer work queue? > A restore should be able to completely restore the cloud i.e. no manual steps > required other than bringing nodes back up or setting up a new cloud cluster. > SOLR-5340 will be a part of this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15260892#comment-15260892 ] Hrishikesh Gadre commented on SOLR-5750: Ok please ignore the last point regarding the API naming. I see that we are already using BACKUP/RESTORE as API names... > Backup/Restore API for SolrCloud > > > Key: SOLR-5750 > URL: https://issues.apache.org/jira/browse/SOLR-5750 > Project: Solr > Issue Type: Sub-task > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Varun Thacker > Fix For: 5.2, master > > Attachments: SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, > SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch > > > We should have an easy way to do backups and restores in SolrCloud. The > ReplicationHandler supports a backup command which can create snapshots of > the index but that is too little. > The command should be able to backup: > # Snapshots of all indexes or indexes from the leader or the shards > # Config set > # Cluster state > # Cluster properties > # Aliases > # Overseer work queue? > A restore should be able to completely restore the cloud i.e. no manual steps > required other than bringing nodes back up or setting up a new cloud cluster. > SOLR-5340 will be a part of this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15260789#comment-15260789 ] Hrishikesh Gadre commented on SOLR-5750: [~dsmiley] [~shalinmangar] I am refactoring the the backup/restore implementation in the Overseer primarily to make it extensible. Although we can do that as part of a separate JIRA, it would be nice if we can get it done as part of this patch itself. I should be able to complete this in next day or two. Also I filed [SOLR-9038|https://issues.apache.org/jira/browse/SOLR-9038] to provide a lightweight alternative to the "full" backup solution implemented by this JIRA. Both these JIRAs need following functionality - Backing up collection metadata - Restore the collection (both index data and collection metadata). Since the APIs defined in this patch use the name "snapshot", I wonder how should we reconcile these two features? I could think of following options, - provide a way to override the behavior (e.g. as an API parameter OR some other global configuration) - rename the APIs here to indicate that its a "full" backup (e.g. CREATEBACKUP/DELETEBACKUP). This will be consistent with the current core level operations (BACKUP/RESTORE). Please let me know your thoughts on this... > Backup/Restore API for SolrCloud > > > Key: SOLR-5750 > URL: https://issues.apache.org/jira/browse/SOLR-5750 > Project: Solr > Issue Type: Sub-task > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Varun Thacker > Fix For: 5.2, master > > Attachments: SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, > SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch > > > We should have an easy way to do backups and restores in SolrCloud. The > ReplicationHandler supports a backup command which can create snapshots of > the index but that is too little. > The command should be able to backup: > # Snapshots of all indexes or indexes from the leader or the shards > # Config set > # Cluster state > # Cluster properties > # Aliases > # Overseer work queue? > A restore should be able to completely restore the cloud i.e. no manual steps > required other than bringing nodes back up or setting up a new cloud cluster. > SOLR-5340 will be a part of this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15240649#comment-15240649 ] David Smiley commented on SOLR-5750: I pushed some new commits to the branch: https://github.com/apache/lucene-solr/commits/solr-5750 Some important bits: * Restored conf name is configurable; won't overwrite existing. Defaults to original. * replicationFactor and some other settings are customizable on restoration. * Shard/slice info is restored instead of reconstituted. Thus shard hash ranges (from e.g. shard split) is restored. I still want to: * test asyncId * test property.customKey=customVal passes through * resolve nocommits on how to pass config name & replicationFactor on the SolrJ side Beyond that there are just some nocommits of a cleanup/document nature. I code review would be much appreciated. Should we just use GitHub branches (do I need my own fork?) for that or ReviewBoard? Possibly in this issue but more likely in another, it would be great if collection restoration could honor CREATE_NODE_SET, snitches, and rules. I need to see how much extra work that'd be but I don't want to delay this too much. > Backup/Restore API for SolrCloud > > > Key: SOLR-5750 > URL: https://issues.apache.org/jira/browse/SOLR-5750 > Project: Solr > Issue Type: Sub-task > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Varun Thacker > Fix For: 5.2, master > > Attachments: SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, > SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch > > > We should have an easy way to do backups and restores in SolrCloud. The > ReplicationHandler supports a backup command which can create snapshots of > the index but that is too little. > The command should be able to backup: > # Snapshots of all indexes or indexes from the leader or the shards > # Config set > # Cluster state > # Cluster properties > # Aliases > # Overseer work queue? > A restore should be able to completely restore the cloud i.e. no manual steps > required other than bringing nodes back up or setting up a new cloud cluster. > SOLR-5340 will be a part of this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15227620#comment-15227620 ] Hrishikesh Gadre commented on SOLR-5750: [~dsmiley] Yes collection "aliases" is a nice option to satisfy above requirement. I have made few comments in SOLR-8950, mostly from usability perspective. Please share your thoughts there (just to keep all relevant discussion in one thread). > Backup/Restore API for SolrCloud > > > Key: SOLR-5750 > URL: https://issues.apache.org/jira/browse/SOLR-5750 > Project: Solr > Issue Type: Sub-task > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Varun Thacker > Fix For: 5.2, master > > Attachments: SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, > SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch > > > We should have an easy way to do backups and restores in SolrCloud. The > ReplicationHandler supports a backup command which can create snapshots of > the index but that is too little. > The command should be able to backup: > # Snapshots of all indexes or indexes from the leader or the shards > # Config set > # Cluster state > # Cluster properties > # Aliases > # Overseer work queue? > A restore should be able to completely restore the cloud i.e. no manual steps > required other than bringing nodes back up or setting up a new cloud cluster. > SOLR-5340 will be a part of this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15227555#comment-15227555 ] David Smiley commented on SOLR-5750: (I'm working moving this forward still; expect another update in a day or so) [~hgadre] the current functionality in the patch allows one to specify a restored collection name. In the event one is trying to snapshot/restore, I think this can and should be accomplished with collection aliases. So if you intend on using this feature, you would create the collection with a sequence number (or say a/b) and then restore to the next sequence number and when done then switch the alias to a new one, and then delete the old one. I'll grant you that it's less ideal in that one has to plan for this in advance and do this little dance. > Backup/Restore API for SolrCloud > > > Key: SOLR-5750 > URL: https://issues.apache.org/jira/browse/SOLR-5750 > Project: Solr > Issue Type: Sub-task > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Varun Thacker > Fix For: 5.2, master > > Attachments: SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, > SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch > > > We should have an easy way to do backups and restores in SolrCloud. The > ReplicationHandler supports a backup command which can create snapshots of > the index but that is too little. > The command should be able to backup: > # Snapshots of all indexes or indexes from the leader or the shards > # Config set > # Cluster state > # Cluster properties > # Aliases > # Overseer work queue? > A restore should be able to completely restore the cloud i.e. no manual steps > required other than bringing nodes back up or setting up a new cloud cluster. > SOLR-5340 will be a part of this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15227182#comment-15227182 ] Hrishikesh Gadre commented on SOLR-5750: The current patch assumes that we are restoring on a cluster which does not contain the specified "collection". In case user wants to restore a snapshot on the same cluster, we would need to replace the index state (after verifying that the zk metadata state is valid). In this scenario, we would need to "disable" solr collection so that we can safely restore the snapshot. Please refer to SOLR-8950 for details. > Backup/Restore API for SolrCloud > > > Key: SOLR-5750 > URL: https://issues.apache.org/jira/browse/SOLR-5750 > Project: Solr > Issue Type: Sub-task > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Varun Thacker > Fix For: 5.2, master > > Attachments: SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, > SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch > > > We should have an easy way to do backups and restores in SolrCloud. The > ReplicationHandler supports a backup command which can create snapshots of > the index but that is too little. > The command should be able to backup: > # Snapshots of all indexes or indexes from the leader or the shards > # Config set > # Cluster state > # Cluster properties > # Aliases > # Overseer work queue? > A restore should be able to completely restore the cloud i.e. no manual steps > required other than bringing nodes back up or setting up a new cloud cluster. > SOLR-5340 will be a part of this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15220368#comment-15220368 ] Hrishikesh Gadre commented on SOLR-5750: [~varunthacker] Thanks for the info. That make sense. I am currently investigating Solr Backup/restore changes for HDFS. I found this old email thread which discusses different approaches, http://qnalist.com/questions/5433155/proper-way-to-backup-solr I think we settled on "copying" behavior for portability reason. I think we should make this a configurable behavior so that we can make the best use of capabilities of underlying file-system. May be we introduce a higher level API (may be at the Directory interface level) ? Any thoughts? CC [~elyograg] [~markrmil...@gmail.com] [~yo...@apache.org] > Backup/Restore API for SolrCloud > > > Key: SOLR-5750 > URL: https://issues.apache.org/jira/browse/SOLR-5750 > Project: Solr > Issue Type: Sub-task > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Varun Thacker > Fix For: 5.2, master > > Attachments: SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, > SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch > > > We should have an easy way to do backups and restores in SolrCloud. The > ReplicationHandler supports a backup command which can create snapshots of > the index but that is too little. > The command should be able to backup: > # Snapshots of all indexes or indexes from the leader or the shards > # Config set > # Cluster state > # Cluster properties > # Aliases > # Overseer work queue? > A restore should be able to completely restore the cloud i.e. no manual steps > required other than bringing nodes back up or setting up a new cloud cluster. > SOLR-5340 will be a part of this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15219901#comment-15219901 ] David Smiley commented on SOLR-5750: Agreed; I noted this as a nocommit in my last comment. "option of not restoring the config set; assume there's one there that is suitable". Or to be more precise, restore the configset but don't overwrite if it's already there. > Backup/Restore API for SolrCloud > > > Key: SOLR-5750 > URL: https://issues.apache.org/jira/browse/SOLR-5750 > Project: Solr > Issue Type: Sub-task > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Varun Thacker > Fix For: 5.2, master > > Attachments: SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, > SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch > > > We should have an easy way to do backups and restores in SolrCloud. The > ReplicationHandler supports a backup command which can create snapshots of > the index but that is too little. > The command should be able to backup: > # Snapshots of all indexes or indexes from the leader or the shards > # Config set > # Cluster state > # Cluster properties > # Aliases > # Overseer work queue? > A restore should be able to completely restore the cloud i.e. no manual steps > required other than bringing nodes back up or setting up a new cloud cluster. > SOLR-5340 will be a part of this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[
https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15219645#comment-15219645
]
Jan Høydahl commented on SOLR-5750:
---
bq. Creates a core-less collection with the config set from the backup ( it
appends a restore.configSetName to it for avoiding collissions )
What if we backup two collections using the same config "foo"? After restore of
the first collection, we will have a config "restore.foo", but what happens
during restore of the second collection? Have not looked at the patch code.
Should it not be a goal that a backup + nuke + restore leaves you with a state
as close to the original as possible, i.e. with one config called "foo", used
by both collections? Perhaps a restore option {{shareConfig=true}} which
creates the config if missing, otherwise reuses what is there?
> Backup/Restore API for SolrCloud
>
>
> Key: SOLR-5750
> URL: https://issues.apache.org/jira/browse/SOLR-5750
> Project: Solr
> Issue Type: Sub-task
> Components: SolrCloud
>Reporter: Shalin Shekhar Mangar
>Assignee: Varun Thacker
> Fix For: 5.2, master
>
> Attachments: SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch,
> SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch
>
>
> We should have an easy way to do backups and restores in SolrCloud. The
> ReplicationHandler supports a backup command which can create snapshots of
> the index but that is too little.
> The command should be able to backup:
> # Snapshots of all indexes or indexes from the leader or the shards
> # Config set
> # Cluster state
> # Cluster properties
> # Aliases
> # Overseer work queue?
> A restore should be able to completely restore the cloud i.e. no manual steps
> required other than bringing nodes back up or setting up a new cloud cluster.
> SOLR-5340 will be a part of this issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15219600#comment-15219600 ] Varun Thacker commented on SOLR-5750: - Hi Hrishikesh, bq. Is there a reason why we have introduced a core Admin API for Backup/restore instead of reusing the replication handler? When we issue a backup command and it's with an async param then the core admin handler already has the necessary hooks to deal with the async requests. Hence in my patch I added core admin api's for backup/restore . > Backup/Restore API for SolrCloud > > > Key: SOLR-5750 > URL: https://issues.apache.org/jira/browse/SOLR-5750 > Project: Solr > Issue Type: Sub-task > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Varun Thacker > Fix For: 5.2, master > > Attachments: SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, > SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch > > > We should have an easy way to do backups and restores in SolrCloud. The > ReplicationHandler supports a backup command which can create snapshots of > the index but that is too little. > The command should be able to backup: > # Snapshots of all indexes or indexes from the leader or the shards > # Config set > # Cluster state > # Cluster properties > # Aliases > # Overseer work queue? > A restore should be able to completely restore the cloud i.e. no manual steps > required other than bringing nodes back up or setting up a new cloud cluster. > SOLR-5340 will be a part of this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15218993#comment-15218993 ] Hrishikesh Gadre commented on SOLR-5750: [~varunthacker] Is there a reason why we have introduced a core Admin API for Backup/restore instead of reusing the replication handler? > Backup/Restore API for SolrCloud > > > Key: SOLR-5750 > URL: https://issues.apache.org/jira/browse/SOLR-5750 > Project: Solr > Issue Type: Sub-task > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Varun Thacker > Fix For: 5.2, master > > Attachments: SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, > SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch > > > We should have an easy way to do backups and restores in SolrCloud. The > ReplicationHandler supports a backup command which can create snapshots of > the index but that is too little. > The command should be able to backup: > # Snapshots of all indexes or indexes from the leader or the shards > # Config set > # Cluster state > # Cluster properties > # Aliases > # Overseer work queue? > A restore should be able to completely restore the cloud i.e. no manual steps > required other than bringing nodes back up or setting up a new cloud cluster. > SOLR-5340 will be a part of this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15200515#comment-15200515 ] Hrishikesh Gadre commented on SOLR-5750: [~varunthacker] Is it possible that when the backup command is sent, one of the shards (or specifically the shard leader) is in "recovering" state ? If yes what happens with the current implementation? > Backup/Restore API for SolrCloud > > > Key: SOLR-5750 > URL: https://issues.apache.org/jira/browse/SOLR-5750 > Project: Solr > Issue Type: Sub-task > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Varun Thacker > Fix For: 5.2, master > > Attachments: SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, > SOLR-5750.patch, SOLR-5750.patch > > > We should have an easy way to do backups and restores in SolrCloud. The > ReplicationHandler supports a backup command which can create snapshots of > the index but that is too little. > The command should be able to backup: > # Snapshots of all indexes or indexes from the leader or the shards > # Config set > # Cluster state > # Cluster properties > # Aliases > # Overseer work queue? > A restore should be able to completely restore the cloud i.e. no manual steps > required other than bringing nodes back up or setting up a new cloud cluster. > SOLR-5340 will be a part of this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[
https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15184501#comment-15184501
]
David Smiley commented on SOLR-5750:
Hello Varun. I've been kicking the tires on this. When I run it explicitly,
it seems to work fine -- no errors. I also tested changing my IP address
between backup & restore, simply by switching my network connection, and I'm
glad to see the restored collection with the correct IPs.
Unfortunately, the test has never succeeded for me. Half the time I get an NPE
from Overseer, the other half the time the test fails its asserts at the end
but before then I connection failure problems that I have yet to look closer
at. The Overseer NPE is curious... clusterState.json is null causing an NPE
when a log message is printed around ~line 214, log.info("processMessage: .
I started to debug that a bit but moved on. I plan to look further into these
things but I wanted to report this progress.
BTW I see no RestoreStatus in the patch.
I was thinking maybe I'll throw up an updated patch in ReviewBoard, but I'm
waiting on INFRA-11152
> Backup/Restore API for SolrCloud
>
>
> Key: SOLR-5750
> URL: https://issues.apache.org/jira/browse/SOLR-5750
> Project: Solr
> Issue Type: Sub-task
> Components: SolrCloud
>Reporter: Shalin Shekhar Mangar
>Assignee: Varun Thacker
> Fix For: 5.2, master
>
> Attachments: SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch,
> SOLR-5750.patch, SOLR-5750.patch
>
>
> We should have an easy way to do backups and restores in SolrCloud. The
> ReplicationHandler supports a backup command which can create snapshots of
> the index but that is too little.
> The command should be able to backup:
> # Snapshots of all indexes or indexes from the leader or the shards
> # Config set
> # Cluster state
> # Cluster properties
> # Aliases
> # Overseer work queue?
> A restore should be able to completely restore the cloud i.e. no manual steps
> required other than bringing nodes back up or setting up a new cloud cluster.
> SOLR-5340 will be a part of this issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15126396#comment-15126396 ] David Smiley commented on SOLR-5750: What's left to do on this issue? Does it actually work? The feedback I've seen seem to be around improvements to make it better -- i.e. using a shared filesystem resulting in no actual copying of files. I looked at the patch. On the restore side I noticed a loop of slices and then a loop of replicas starting with this comment: "//Copy data from backed up index to each replica". Shouldn't there be just one replica per shard to restore, and then later the replicationFactor will expand to the desired level? Another question is I wonder if any of these loops should be done in parallel or if they are issuing asynchronous requests so it isn't necessary. It would help to document the pertinent loops with this information, and possibly do some in parallel if they should be done so. > Backup/Restore API for SolrCloud > > > Key: SOLR-5750 > URL: https://issues.apache.org/jira/browse/SOLR-5750 > Project: Solr > Issue Type: Sub-task > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Varun Thacker > Fix For: 5.2, Trunk > > Attachments: SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, > SOLR-5750.patch > > > We should have an easy way to do backups and restores in SolrCloud. The > ReplicationHandler supports a backup command which can create snapshots of > the index but that is too little. > The command should be able to backup: > # Snapshots of all indexes or indexes from the leader or the shards > # Config set > # Cluster state > # Cluster properties > # Aliases > # Overseer work queue? > A restore should be able to completely restore the cloud i.e. no manual steps > required other than bringing nodes back up or setting up a new cloud cluster. > SOLR-5340 will be a part of this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15014991#comment-15014991 ] Mark Miller commented on SOLR-5750: --- Might be a bit hackey and certainly not necessary for first impl, but would be great if we could do hard snapshots if we detect a UNIX like OS. > Backup/Restore API for SolrCloud > > > Key: SOLR-5750 > URL: https://issues.apache.org/jira/browse/SOLR-5750 > Project: Solr > Issue Type: Sub-task > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Varun Thacker > Fix For: 5.2, Trunk > > Attachments: SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, > SOLR-5750.patch > > > We should have an easy way to do backups and restores in SolrCloud. The > ReplicationHandler supports a backup command which can create snapshots of > the index but that is too little. > The command should be able to backup: > # Snapshots of all indexes or indexes from the leader or the shards > # Config set > # Cluster state > # Cluster properties > # Aliases > # Overseer work queue? > A restore should be able to completely restore the cloud i.e. no manual steps > required other than bringing nodes back up or setting up a new cloud cluster. > SOLR-5340 will be a part of this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15014972#comment-15014972 ] Gregory Chanan commented on SOLR-5750: -- bq. I think the best approach is to make sure "snapshotDirectory" is mandatory if DirectoryFactory.isShared() is false . If it is a shared drive we can have the path default to a directory relative to dataDir. sgtm, thanks. > Backup/Restore API for SolrCloud > > > Key: SOLR-5750 > URL: https://issues.apache.org/jira/browse/SOLR-5750 > Project: Solr > Issue Type: Sub-task > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Varun Thacker > Fix For: 5.2, Trunk > > Attachments: SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, > SOLR-5750.patch > > > We should have an easy way to do backups and restores in SolrCloud. The > ReplicationHandler supports a backup command which can create snapshots of > the index but that is too little. > The command should be able to backup: > # Snapshots of all indexes or indexes from the leader or the shards > # Config set > # Cluster state > # Cluster properties > # Aliases > # Overseer work queue? > A restore should be able to completely restore the cloud i.e. no manual steps > required other than bringing nodes back up or setting up a new cloud cluster. > SOLR-5340 will be a part of this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[
https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15014960#comment-15014960
]
Varun Thacker commented on SOLR-5750:
-
I think the best approach is to make sure "snapshotDirectory" is mandatory if
{{DirectoryFactory.isShared()}} is {{false}}
> Backup/Restore API for SolrCloud
>
>
> Key: SOLR-5750
> URL: https://issues.apache.org/jira/browse/SOLR-5750
> Project: Solr
> Issue Type: Sub-task
> Components: SolrCloud
>Reporter: Shalin Shekhar Mangar
>Assignee: Varun Thacker
> Fix For: 5.2, Trunk
>
> Attachments: SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch,
> SOLR-5750.patch
>
>
> We should have an easy way to do backups and restores in SolrCloud. The
> ReplicationHandler supports a backup command which can create snapshots of
> the index but that is too little.
> The command should be able to backup:
> # Snapshots of all indexes or indexes from the leader or the shards
> # Config set
> # Cluster state
> # Cluster properties
> # Aliases
> # Overseer work queue?
> A restore should be able to completely restore the cloud i.e. no manual steps
> required other than bringing nodes back up or setting up a new cloud cluster.
> SOLR-5340 will be a part of this issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15014946#comment-15014946 ] Gregory Chanan commented on SOLR-5750: -- bq. When we do a restore, the current impl. restores into a fresh collection. So the backed up data can be on another node not accessible to the restoring collection shard. Hard times on a non-shared file system :). In that case, sure, require it to be set for non-shared file systems (can do it for all filesystems for now, we can change in the future). Or, restore on the same node as the existing shard where you took the snapshot to remove the shared mount requirement? > Backup/Restore API for SolrCloud > > > Key: SOLR-5750 > URL: https://issues.apache.org/jira/browse/SOLR-5750 > Project: Solr > Issue Type: Sub-task > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Varun Thacker > Fix For: 5.2, Trunk > > Attachments: SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, > SOLR-5750.patch > > > We should have an easy way to do backups and restores in SolrCloud. The > ReplicationHandler supports a backup command which can create snapshots of > the index but that is too little. > The command should be able to backup: > # Snapshots of all indexes or indexes from the leader or the shards > # Config set > # Cluster state > # Cluster properties > # Aliases > # Overseer work queue? > A restore should be able to completely restore the cloud i.e. no manual steps > required other than bringing nodes back up or setting up a new cloud cluster. > SOLR-5340 will be a part of this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15014918#comment-15014918 ] Varun Thacker commented on SOLR-5750: - bq. We can't default to some path relative to the shard's dataDir? When we do a restore, the current impl. restores into a fresh collection. So the backed up data can be on another node not accessible to the restoring collection shard. > Backup/Restore API for SolrCloud > > > Key: SOLR-5750 > URL: https://issues.apache.org/jira/browse/SOLR-5750 > Project: Solr > Issue Type: Sub-task > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Varun Thacker > Fix For: 5.2, Trunk > > Attachments: SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, > SOLR-5750.patch > > > We should have an easy way to do backups and restores in SolrCloud. The > ReplicationHandler supports a backup command which can create snapshots of > the index but that is too little. > The command should be able to backup: > # Snapshots of all indexes or indexes from the leader or the shards > # Config set > # Cluster state > # Cluster properties > # Aliases > # Overseer work queue? > A restore should be able to completely restore the cloud i.e. no manual steps > required other than bringing nodes back up or setting up a new cloud cluster. > SOLR-5340 will be a part of this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15014909#comment-15014909 ] Gregory Chanan commented on SOLR-5750: -- bq. I don't think it should default to anything and if not specified snapshots should fail. The reason being users not on a shared filesystem need to specify a shared mount for this to work. We can't default to some path relative to the shard's dataDir? > Backup/Restore API for SolrCloud > > > Key: SOLR-5750 > URL: https://issues.apache.org/jira/browse/SOLR-5750 > Project: Solr > Issue Type: Sub-task > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Varun Thacker > Fix For: 5.2, Trunk > > Attachments: SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch, > SOLR-5750.patch > > > We should have an easy way to do backups and restores in SolrCloud. The > ReplicationHandler supports a backup command which can create snapshots of > the index but that is too little. > The command should be able to backup: > # Snapshots of all indexes or indexes from the leader or the shards > # Config set > # Cluster state > # Cluster properties > # Aliases > # Overseer work queue? > A restore should be able to completely restore the cloud i.e. no manual steps > required other than bringing nodes back up or setting up a new cloud cluster. > SOLR-5340 will be a part of this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[
https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15014898#comment-15014898
]
Varun Thacker commented on SOLR-5750:
-
bq. 1) I know we already have "location" via
https://cwiki.apache.org/confluence/display/solr/Making+and+Restoring+Backups+of+SolrCores
but it just seems needlessly risky / error prone. What if a user purposefully
or accidentally overwrites important data? You are giving anyone making a
snapshot call solr's permissions. Beyond that, making "location" a required
param is not the greatest interface. Most of the time when I'm taking a
snapshot I don't even care where it is, I expect the system to just do
something sensible and let me interact with the API with some id (i.e. name).
HDFS and HBase snapshots work in this way, for example. Why not just have a
backup location specified in solr.xml with some sensible default?
bq. 2) On the above point: "I expect the system to just do something sensible
and let me interact with the API with some id (i.e. name)" – why do I pass in a
location for RESTORE? Can't the system just remember that from the backup call?
Nice idea. We could let users specify it within the solr.xml solrcloud tag .
I don't think it should default to anything and if not specified snapshots
should fail. The reason being users not on a shared filesystem need to specify
a shared mount for this to work.
And by specifying it within the solr.xml file , even the restore command
wouldn't need the user to specify the location which will solve your concern in
the 2nd point
{code}
${snapshotDirectory:}
{code}
> Backup/Restore API for SolrCloud
>
>
> Key: SOLR-5750
> URL: https://issues.apache.org/jira/browse/SOLR-5750
> Project: Solr
> Issue Type: Sub-task
> Components: SolrCloud
>Reporter: Shalin Shekhar Mangar
>Assignee: Varun Thacker
> Fix For: 5.2, Trunk
>
> Attachments: SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch,
> SOLR-5750.patch
>
>
> We should have an easy way to do backups and restores in SolrCloud. The
> ReplicationHandler supports a backup command which can create snapshots of
> the index but that is too little.
> The command should be able to backup:
> # Snapshots of all indexes or indexes from the leader or the shards
> # Config set
> # Cluster state
> # Cluster properties
> # Aliases
> # Overseer work queue?
> A restore should be able to completely restore the cloud i.e. no manual steps
> required other than bringing nodes back up or setting up a new cloud cluster.
> SOLR-5340 will be a part of this issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[
https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15014824#comment-15014824
]
Gregory Chanan commented on SOLR-5750:
--
Did a cursory look at this. Some questions/comments:
1) I know we already have "location" via
https://cwiki.apache.org/confluence/display/solr/Making+and+Restoring+Backups+of+SolrCores
but it just seems needlessly risky / error prone. What if a user purposefully
or accidentally overwrites important data? You are giving anyone making a
snapshot call solr's permissions. Beyond that, making "location" a required
param is not the greatest interface. Most of the time when I'm taking a
snapshot I don't even _care_ where it is, I expect the system to just do
something sensible and let me interact with the API with some id (i.e. name).
HDFS and HBase snapshots work in this way, for example. Why not just have a
backup location specified in solr.xml with some sensible default?
2) On the above point: "I expect the system to just do something sensible and
let me interact with the API with some id (i.e. name)" -- why do I pass in a
location for RESTORE? Can't the system just remember that from the backup call?
3) There's no API for deleting a snapshot?
4) There's no API for listing snapshots? (I don't think this needs to be in an
initial version necessarily)
3) From {quote}So the idea is the location that you give should be a shared
file system so that all the replica backup along with the ZK information stay
in one place. Then during restore the same location can be used.
We can then support storing to other locations such as s3, hdfs etc as separate
Jiras then{quote}
I'm not sure the shard-at-a-time makes sense for shared file systems. For
example, it's much more efficient to take an hdfs snapshot of the entire
collection directory than of each individual shard. I haven't fully thought
through how we support both, e.g. we could do something different based on the
underlying storage of the collection (though that wouldn't let you backup a
local FS collection to a shared FS), or allow a "snapshotType" parameter or
something. I _think_ we can just make whatever you have the default here, so I
don't think we strictly need to do anything in this patch. Just pointing that
out.
> Backup/Restore API for SolrCloud
>
>
> Key: SOLR-5750
> URL: https://issues.apache.org/jira/browse/SOLR-5750
> Project: Solr
> Issue Type: Sub-task
> Components: SolrCloud
>Reporter: Shalin Shekhar Mangar
>Assignee: Varun Thacker
> Fix For: 5.2, Trunk
>
> Attachments: SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch,
> SOLR-5750.patch
>
>
> We should have an easy way to do backups and restores in SolrCloud. The
> ReplicationHandler supports a backup command which can create snapshots of
> the index but that is too little.
> The command should be able to backup:
> # Snapshots of all indexes or indexes from the leader or the shards
> # Config set
> # Cluster state
> # Cluster properties
> # Aliases
> # Overseer work queue?
> A restore should be able to completely restore the cloud i.e. no manual steps
> required other than bringing nodes back up or setting up a new cloud cluster.
> SOLR-5340 will be a part of this issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15009198#comment-15009198 ] Varun Thacker commented on SOLR-5750: - HI Jan, Thanks for looking into this. Let me bring up the patch to date and post it again. > Backup/Restore API for SolrCloud > > > Key: SOLR-5750 > URL: https://issues.apache.org/jira/browse/SOLR-5750 > Project: Solr > Issue Type: Sub-task > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Varun Thacker > Fix For: 5.2, Trunk > > Attachments: SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch > > > We should have an easy way to do backups and restores in SolrCloud. The > ReplicationHandler supports a backup command which can create snapshots of > the index but that is too little. > The command should be able to backup: > # Snapshots of all indexes or indexes from the leader or the shards > # Config set > # Cluster state > # Cluster properties > # Aliases > # Overseer work queue? > A restore should be able to completely restore the cloud i.e. no manual steps > required other than bringing nodes back up or setting up a new cloud cluster. > SOLR-5340 will be a part of this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15007595#comment-15007595 ] Gregory Chanan commented on SOLR-5750: -- I'm planning to take a look at getting something like that up for shared file systems (hdfs). That's an easier problem in that the underlying storage mechanism already supports snapshots and taking a snapshot doesn't require going to every shard (assuming reasonable data dir settings). > Backup/Restore API for SolrCloud > > > Key: SOLR-5750 > URL: https://issues.apache.org/jira/browse/SOLR-5750 > Project: Solr > Issue Type: Sub-task > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Varun Thacker > Fix For: 5.2, Trunk > > Attachments: SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch > > > We should have an easy way to do backups and restores in SolrCloud. The > ReplicationHandler supports a backup command which can create snapshots of > the index but that is too little. > The command should be able to backup: > # Snapshots of all indexes or indexes from the leader or the shards > # Config set > # Cluster state > # Cluster properties > # Aliases > # Overseer work queue? > A restore should be able to completely restore the cloud i.e. no manual steps > required other than bringing nodes back up or setting up a new cloud cluster. > SOLR-5340 will be a part of this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15007546#comment-15007546 ] Jan Høydahl commented on SOLR-5750: --- Reviving this. What about getting a basic, even if unperfect, cloud backup command into the hands of users, then followup with improvements? Have not reviewed the patch but looks like it can add great value already as is. > Backup/Restore API for SolrCloud > > > Key: SOLR-5750 > URL: https://issues.apache.org/jira/browse/SOLR-5750 > Project: Solr > Issue Type: Sub-task > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Varun Thacker > Fix For: 5.2, Trunk > > Attachments: SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch > > > We should have an easy way to do backups and restores in SolrCloud. The > ReplicationHandler supports a backup command which can create snapshots of > the index but that is too little. > The command should be able to backup: > # Snapshots of all indexes or indexes from the leader or the shards > # Config set > # Cluster state > # Cluster properties > # Aliases > # Overseer work queue? > A restore should be able to completely restore the cloud i.e. no manual steps > required other than bringing nodes back up or setting up a new cloud cluster. > SOLR-5340 will be a part of this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[
https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14611757#comment-14611757
]
Noble Paul commented on SOLR-5750:
--
2 design suggestions
1) Move the operations to {{CollectionsHandler}} . When the process starts add
the backup name and the node that is processing the task to ZK
2) Do not restore all the replicas at once . Just create one replica of a shard
first and then do ADDREPLICA till you have enough replicas
> Backup/Restore API for SolrCloud
>
>
> Key: SOLR-5750
> URL: https://issues.apache.org/jira/browse/SOLR-5750
> Project: Solr
> Issue Type: Sub-task
> Components: SolrCloud
>Reporter: Shalin Shekhar Mangar
>Assignee: Varun Thacker
> Fix For: 5.2, Trunk
>
> Attachments: SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch
>
>
> We should have an easy way to do backups and restores in SolrCloud. The
> ReplicationHandler supports a backup command which can create snapshots of
> the index but that is too little.
> The command should be able to backup:
> # Snapshots of all indexes or indexes from the leader or the shards
> # Config set
> # Cluster state
> # Cluster properties
> # Aliases
> # Overseer work queue?
> A restore should be able to completely restore the cloud i.e. no manual steps
> required other than bringing nodes back up or setting up a new cloud cluster.
> SOLR-5340 will be a part of this issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602533#comment-14602533 ] Varun Thacker commented on SOLR-5750: - Hi Jan, If you look at the patch most of the backup/restore work is being offloaded to the ReplicationHandler. So to make the underlying storage pluggable we just need to make the changes there. The only change required with respect to this patch is the backup of zk files. > Backup/Restore API for SolrCloud > > > Key: SOLR-5750 > URL: https://issues.apache.org/jira/browse/SOLR-5750 > Project: Solr > Issue Type: Sub-task > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Varun Thacker > Fix For: 5.2, Trunk > > Attachments: SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch > > > We should have an easy way to do backups and restores in SolrCloud. The > ReplicationHandler supports a backup command which can create snapshots of > the index but that is too little. > The command should be able to backup: > # Snapshots of all indexes or indexes from the leader or the shards > # Config set > # Cluster state > # Cluster properties > # Aliases > # Overseer work queue? > A restore should be able to completely restore the cloud i.e. no manual steps > required other than bringing nodes back up or setting up a new cloud cluster. > SOLR-5340 will be a part of this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14601771#comment-14601771 ] Jan Høydahl commented on SOLR-5750: --- See my comment on SOLR-7374 - we should have something pluggable here. > Backup/Restore API for SolrCloud > > > Key: SOLR-5750 > URL: https://issues.apache.org/jira/browse/SOLR-5750 > Project: Solr > Issue Type: Sub-task > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Varun Thacker > Fix For: 5.2, Trunk > > Attachments: SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch > > > We should have an easy way to do backups and restores in SolrCloud. The > ReplicationHandler supports a backup command which can create snapshots of > the index but that is too little. > The command should be able to backup: > # Snapshots of all indexes or indexes from the leader or the shards > # Config set > # Cluster state > # Cluster properties > # Aliases > # Overseer work queue? > A restore should be able to completely restore the cloud i.e. no manual steps > required other than bringing nodes back up or setting up a new cloud cluster. > SOLR-5340 will be a part of this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14601709#comment-14601709 ] Varun Thacker commented on SOLR-5750: - So the idea is the location that you give should be a shared file system so that all the replica backup along with the ZK information stay in one place. Then during restore the same location can be used. We can then support storing to other locations such as s3, hdfs etc as separate Jiras then > Backup/Restore API for SolrCloud > > > Key: SOLR-5750 > URL: https://issues.apache.org/jira/browse/SOLR-5750 > Project: Solr > Issue Type: Sub-task > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Varun Thacker > Fix For: 5.2, Trunk > > Attachments: SOLR-5750.patch, SOLR-5750.patch, SOLR-5750.patch > > > We should have an easy way to do backups and restores in SolrCloud. The > ReplicationHandler supports a backup command which can create snapshots of > the index but that is too little. > The command should be able to backup: > # Snapshots of all indexes or indexes from the leader or the shards > # Config set > # Cluster state > # Cluster properties > # Aliases > # Overseer work queue? > A restore should be able to completely restore the cloud i.e. no manual steps > required other than bringing nodes back up or setting up a new cloud cluster. > SOLR-5340 will be a part of this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14551169#comment-14551169 ] Greg Solovyev commented on SOLR-5750: - [~varunthacker] if I am understanding the code correctly, when a collection has multiple shards spread over multiple nodes, data from each shard will be backed up only on that shard's node, metadata from ZK will be saved on whichever node happens to handle the backup request. Am I missing something here? > Backup/Restore API for SolrCloud > > > Key: SOLR-5750 > URL: https://issues.apache.org/jira/browse/SOLR-5750 > Project: Solr > Issue Type: Sub-task > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Varun Thacker > Fix For: Trunk, 5.2 > > Attachments: SOLR-5750.patch, SOLR-5750.patch > > > We should have an easy way to do backups and restores in SolrCloud. The > ReplicationHandler supports a backup command which can create snapshots of > the index but that is too little. > The command should be able to backup: > # Snapshots of all indexes or indexes from the leader or the shards > # Config set > # Cluster state > # Cluster properties > # Aliases > # Overseer work queue? > A restore should be able to completely restore the cloud i.e. no manual steps > required other than bringing nodes back up or setting up a new cloud cluster. > SOLR-5340 will be a part of this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14522594#comment-14522594 ] Damien Kamerman commented on SOLR-5750: --- Only snapshot if the index version has changed. > Backup/Restore API for SolrCloud > > > Key: SOLR-5750 > URL: https://issues.apache.org/jira/browse/SOLR-5750 > Project: Solr > Issue Type: Sub-task > Components: SolrCloud >Reporter: Shalin Shekhar Mangar >Assignee: Varun Thacker > Fix For: Trunk, 5.2 > > > We should have an easy way to do backups and restores in SolrCloud. The > ReplicationHandler supports a backup command which can create snapshots of > the index but that is too little. > The command should be able to backup: > # Snapshots of all indexes or indexes from the leader or the shards > # Config set > # Cluster state > # Cluster properties > # Aliases > # Overseer work queue? > A restore should be able to completely restore the cloud i.e. no manual steps > required other than bringing nodes back up or setting up a new cloud cluster. > SOLR-5340 will be a part of this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196367#comment-14196367 ] Mark Bennett commented on SOLR-5750: At least 2 clients that I know of have recently asked about this. Would be awesome! > Backup/Restore API for SolrCloud > > > Key: SOLR-5750 > URL: https://issues.apache.org/jira/browse/SOLR-5750 > Project: Solr > Issue Type: Sub-task > Components: SolrCloud >Reporter: Shalin Shekhar Mangar > Fix For: 4.9, Trunk > > > We should have an easy way to do backups and restores in SolrCloud. The > ReplicationHandler supports a backup command which can create snapshots of > the index but that is too little. > The command should be able to backup: > # Snapshots of all indexes or indexes from the leader or the shards > # Config set > # Cluster state > # Cluster properties > # Aliases > # Overseer work queue? > A restore should be able to completely restore the cloud i.e. no manual steps > required other than bringing nodes back up or setting up a new cloud cluster. > SOLR-5340 will be a part of this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13937924#comment-13937924 ] Robert Parker commented on SOLR-5750: - You should have the option of backing up/replicating a live searchable collection on SolrCloud A to a live searchable collection across a WAN on SolrCloud B, each with their own separate ZooKeeper ensemble. You should also be able to rename the collection on the fly so that the live searchable collection on SolrCloud A is called "collectionA" and its live updated searchable replication copy is known as "collectionB" so as to allow a single remote instance of SolrCloud to be multi-homed to act as a replication target for multiple other Solr instances' collections, even if those collections happen to have the same name on each of their source instances. Also, WAN compression/optimization would be helpful as well. > Backup/Restore API for SolrCloud > > > Key: SOLR-5750 > URL: https://issues.apache.org/jira/browse/SOLR-5750 > Project: Solr > Issue Type: Sub-task > Components: SolrCloud >Reporter: Shalin Shekhar Mangar > Fix For: 4.8, 5.0 > > > We should have an easy way to do backups and restores in SolrCloud. The > ReplicationHandler supports a backup command which can create snapshots of > the index but that is too little. > The command should be able to backup: > # Snapshots of all indexes or indexes from the leader or the shards > # Config set > # Cluster state > # Cluster properties > # Aliases > # Overseer work queue? > A restore should be able to completely restore the cloud i.e. no manual steps > required other than bringing nodes back up or setting up a new cloud cluster. > SOLR-5340 will be a part of this issue. -- This message was sent by Atlassian JIRA (v6.2#6252) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[
https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13910268#comment-13910268
]
Shalin Shekhar Mangar commented on SOLR-5750:
-
{quote}
It should also be able to take a backup of a specific collection only
Option of compressing data which is backed up
{quote}
+1
> Backup/Restore API for SolrCloud
>
>
> Key: SOLR-5750
> URL: https://issues.apache.org/jira/browse/SOLR-5750
> Project: Solr
> Issue Type: Sub-task
> Components: SolrCloud
>Reporter: Shalin Shekhar Mangar
> Fix For: 4.8, 5.0
>
>
> We should have an easy way to do backups and restores in SolrCloud. The
> ReplicationHandler supports a backup command which can create snapshots of
> the index but that is too little.
> The command should be able to backup:
> # Snapshots of all indexes or indexes from the leader or the shards
> # Config set
> # Cluster state
> # Cluster properties
> # Aliases
> # Overseer work queue?
> A restore should be able to completely restore the cloud i.e. no manual steps
> required other than bringing nodes back up or setting up a new cloud cluster.
> SOLR-5340 will be a part of this issue.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5750) Backup/Restore API for SolrCloud
[ https://issues.apache.org/jira/browse/SOLR-5750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13907174#comment-13907174 ] Noble Paul commented on SOLR-5750: -- * It should also be able to take a backup of a specific collection only * Option of compressing data which is backed up > Backup/Restore API for SolrCloud > > > Key: SOLR-5750 > URL: https://issues.apache.org/jira/browse/SOLR-5750 > Project: Solr > Issue Type: Sub-task > Components: SolrCloud >Reporter: Shalin Shekhar Mangar > Fix For: 4.8, 5.0 > > > We should have an easy way to do backups and restores in SolrCloud. The > ReplicationHandler supports a backup command which can create snapshots of > the index but that is too little. > The command should be able to backup: > # Snapshots of all indexes or indexes from the leader or the shards > # Config set > # Cluster state > # Cluster properties > # Aliases > # Overseer work queue? > A restore should be able to completely restore the cloud i.e. no manual steps > required other than bringing nodes back up or setting up a new cloud cluster. > SOLR-5340 will be a part of this issue. -- This message was sent by Atlassian JIRA (v6.1.5#6160) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
