Although it might not be clear in the documentation, I think the strong inference is that you need to set up the new cluster with the same requirements for creating the snapshots in the first place. This means... - Shared File Repository. You need to configure every new node with the same access (maybe even same path) as the old cluster. - I think you need to create your new cluster using the same exact same cluster and node configuration. This probably means copying your old elasticsearch.yml on each node to the new corresponding node. - After doing some snapshots myself, I can see how all the data is stored in the backup, so although you <can> copy data to your new cluster, I can't see how it would be any benefit. Assuming you intend to restore the full backup and not individual indices, everything will be over-written. I'm also going to guess (maybe needs verification) there may be problems naming your cluster something different (needs verification but it makes sense). So, I would guess that after you do your snapshots you should just shutdown your cluster unless it's a Production cluster. If you run both clusters simultaneously, then they need to be physically isolated. Then, after either switching or connecting your shared storage where the snapshots are located, execute a restore to the new cluster. If the original cluster is still running, then you can always make a snapshot of the changes that happened after the data and state has been restored to the new cluster and migrate the new, incremental data the same way. IMO, Based on what I've observed so far... Like doing anything else this new, IMO it would be prudent to plan for incremental steps in the whole process which can be back-tracked if you run into something unexpected. Tony
On Thursday, February 20, 2014 4:42:11 AM UTC-8, Attila Bukor wrote: > Hi, > > I've read this one, but it's not clear to me how to do it on a different > machine. > > If I do a backup on oldcluster with > > curl -XPUT oldcluster:9200/_snapshot/my_backup/snapshot_1 (assuming I've > already created the my_backup repo), there will be metadata-snapshot_1, > snapshot-snapshot_1 and some other files under > /var/lib/elasticsearch/my_backup or some other place which is configured. > > Should I just tar the whole directory, scp it to node1 on newcluster and > call > > curl -XPOST newcluster-node1:9200/_snapshot/my_backup/snapshot_1/restore ? > > I don't really want to break things and this situation is not really > explained in the docs. > > Cheers, > Attila > > On 02/20/2014 01:14 PM, Yann Barraud wrote: > > > http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-snapshots.html#_restore > > > :-) > > > > > > Cordialement, > > Yann Barraud > > > > > > 2014-02-20 12:03 GMT+01:00 Attila Bukor > > <r1pp3...@w4it.eu <javascript:> > > <mailto:r1pp3...@w4it.eu <javascript:>>>: > > > > Hi Tony, > > > > The problem is the hardware, that is why we need to move it to the > new > > cluster. > > > > Yann, how can I restore a snapshot on a different machine? > > > > Cheers, > > Attila > > > > > > On 02/19/2014 07:00 PM, Tony Su wrote: > > > > I was wondering which backup/restore method you intended to use. > > > > Snapshot/restore requires 1.0. > > If you intend to upgrade your 0.9.x cluster to 1.0 first, it > > looks like > > possibly a really good option (I intend to start testing > > snapshot/restore sometime soon. I've already created snapshots > > and am > > impressed how it seems to take roughly the same amount to create > a > > snapshot no matter how much data is being backed up, probably > > within reason) > > > > If your chosen path requires upgrading the existing cluster to > 1.0 > > anyway, why migrate to the new cluster? > > > > Tony > > > > > > > > On Wednesday, February 19, 2014 8:16:02 AM UTC-8, Attila Bukor > > wrote: > > > > Hi Yann, > > > > Thank you for your response, you saved me from a *lot* of > > headache. > > So the > > revised flow: > > > > - Upgrade oldcluster to 1.0.0. > > - Implement a logger in the application which dumps all > > index API calls > > into a file. > > - Backup my index on oldcluster following these docs: > > > > > http://www.elasticsearch.org/__guide/en/elasticsearch/__reference/master/modules-__snapshots.html > > > < > http://www.elasticsearch.org/guide/en/elasticsearch/reference/master/modules-snapshots.html> > > > > > > < > http://www.elasticsearch.org/__guide/en/elasticsearch/__reference/master/modules-__snapshots.html > > > < > http://www.elasticsearch.org/guide/en/elasticsearch/reference/master/modules-snapshots.html>> > > > > > > - Restore the backed up index on newcluster following the > > same docs. > > - Run the log from after the last _id in the snapshot. > > Repeat until > > they're the same. > > - Change the IP to newcluster in the application. > > - Repeat step 5 one more time. > > - Remove logging, delete logs. > > - Shut down oldcluster. > > > > In this case, the switch should not be noticed by anyone > > and the data > > integrity is guaranteed, right? > > > > Cheers, > > Attila > > > > On Wed, 19 Feb 2014, Yann Barraud wrote: > > > > > Hi, > > > > > > If I get you well, you'll face an issue. Renaming > > oldcluster will > > make it "forget" previous indexes. (Take a look at data > > you'll find a > > dir named "oldcluster"). > > > > > > The step you're missing is to backup data fro oldcluster > > > Then shtudown, rename > > > Then restore... > > > > > > Or even shutdown and restore on new cluster... > > > > > > Cheers, > > > Yann > > > > > > > > > > > > Le mercredi 19 février 2014 16:01:03 UTC+1, Attila Bukor > > a écrit : > > > Hey everybody, > > > > > > I have a question regarding the migration of the > > indices to a > > new cluster > > > seamlessly. Let me first describe the situation: > > > > > > I have a project which uses Elasticsearch since a > > few weeks > > ago. This is our > > > first Elasticsearch project, and as we are > > satisfied with it, > > we decided to > > > dedicate a cluster of 3 nodes to run Elasticsearch > > (lets > > call it > > > "newcluster"). It is up and running separately > > from our old > > 1-node > > > Elasticsearch installation ("oldcluster"). We want > > to migrate > > everything to > > > newcluster, and turn oldcluster off. All 4 servers > > are running > > Ubuntu Server, > > > Elasticsearch is installed from the official deb > > package, > > oldcluster is > > > 0.90.10 and newcluster 1.0.0. > > > > > > I came up with the following solution, but I'm not > > sure it > > will > > work: > > > > > > - Upgrade oldcluster to 1.0.0 (tested it in dev, > > should work) > > > - Backup the index (we have only one index right > > now) in case > > something goes > > > wrong. > > > - Change oldcluster's name to newcluster. > > > - Wait until the nodes synchronize themselves (how > > can I check > > if they're in > > > sync?) > > > - Shut down oldcluster. > > > > > > Thank you very much, > > > Attila > > > > > > > > > > > > > > > -- > > You received this message because you are subscribed to the > Google > > Groups "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from > > it, send > > > > an email to > > elasticsearch+unsubscribe@__googlegroups.com > > <mailto:elasticsearch%2bunsubscr...@googlegroups.com<javascript:>>. > > > To view this discussion on the web visit > > > https://groups.google.com/d/__msgid/elasticsearch/f375faef-__2dd4-416d-8150-75afe2318f0c%__40googlegroups.com > > > < > https://groups.google.com/d/msgid/elasticsearch/f375faef-2dd4-416d-8150-75afe2318f0c%40googlegroups.com>. > > > > For more options, visit > > https://groups.google.com/__groups/opt_out > > <https://groups.google.com/groups/opt_out>. > > > > > > > > -- > > You received this message because you are subscribed to a topic in > > the Google Groups "elasticsearch" group. > > To unsubscribe from this topic, visit > > > https://groups.google.com/d/__topic/elasticsearch/__aOVumOxmsuQ/unsubscribe > > < > https://groups.google.com/d/topic/elasticsearch/aOVumOxmsuQ/unsubscribe>. > > To unsubscribe from this group and all its topics, send an email to > > elasticsearch+unsubscribe@__googlegroups.com > > <mailto:elasticsearch%2bunsubscr...@googlegroups.com <javascript:>>. > > > To view this discussion on the web visit > > > https://groups.google.com/d/__msgid/elasticsearch/le4ne3%__24cg5%241%40ger.gmane.org > > > < > https://groups.google.com/d/msgid/elasticsearch/le4ne3%24cg5%241%40ger.gmane.org>. > > > > > > For more options, visit https://groups.google.com/__groups/opt_out > > <https://groups.google.com/groups/opt_out>. > > > > > > -- > > You received this message because you are subscribed to the Google > > Groups "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send > > an email to > > elasticsearc...@googlegroups.com <javascript:>. > > To view this discussion on the web visit > > > https://groups.google.com/d/msgid/elasticsearch/CA%2BhvuXc%2BuQWV5ySXa0407rHCmafEef4x4OU0qdBJ5FNLzMy0yQ%40mail.gmail.com. > > > > For more options, visit https://groups.google.com/groups/opt_out. > > > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/c1e1c3e8-f2fc-43a6-bb9c-526be656eb73%40googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
