I will include my response to the original post:
Snapshots are at the segment level. The more segments stored in the
> repository, the more segments will have to be compared to those in each
> successive snapshot. With merges taking place continually in an active
> index, you may end up with
Hello,
Sorry for hijacking this thread, but I'm currently also pondering the best
way to perform periodic snapshots in AWS.
My main concern is that we are using blue-green deployment with ephemeral
storage on EC2, so if for some reason there is a problem with the cluster,
we might lose a lot o
Yes, I am now seeing the snapshots complete in about 2 minutes after
switching to a new, empty bucket.
I'm not sure why the initial request to snapshot to the empty repo was
hanging because the snapshot did in fact complete in about 2 minutes,
according to the S3 timestamp.
Time to automate dele
Having too many snapshots is problematic. Each snapshot is done in
incremental manner, so in order to figure out what changes and what is
available all snapshots in the repository needs to be scanned, which takes
time as number of snapshots growing. I would recommend pruning old
snapshots as ti
I am also interested in this topic.
We were snapshotting our cluster of two nodes every 2 hours (invoked via a
cron job) to an S3 repository (we were running ES 1.2.2 with
cloud-aws-plugin version 2.2.0, then we upgraded to ES 1.4.0 with
cloud-aws-plugin 2.4.0 but are still seeing issues describ
Hi Vineeth,
Thanks for the reply.
I am aware of how to create and delete snapshots using cloud-aws.
What I wanted to know was how should the work flow of periodic snapshot
be?especially how to deal with old snapshots? having too many old
snapshots- will this impact something?
On Friday, Novemb
Hi ,
There is a s3 repository plugin -
https://github.com/elasticsearch/elasticsearch-cloud-aws#s3-repository
Use this.
The snapshots are incremental , so it should fit your purpose perfectly.
Thanks
Vineeth
On Fri, Nov 7, 2014 at 3:22 PM, Pradeep Reddy <
pradeepreddy.manu.iit...@gm
I want to backup the data every 15-30 min. I will be storing the snapshots
in S3.
DELETE old and then PUT new snapshot many not be the best practice as you
may end up with nothing if something goes wrong.
Using timestamp for snapshot names may be one option, but how to delete old
snapshots th
