Thank you for your suggestion. I tried the stream2es library but I get a OutOfMemoryError when trying to use that.
On Friday, June 6, 2014 5:13:19 PM UTC-4, Antonio Augusto Santos wrote: > > Take a look at stream2es https://github.com/elasticsearch/stream2es > > On Friday, June 6, 2014 2:13:06 PM UTC-3, Brian Lamb wrote: >> >> I should also point out that I had to edit a file in the >> metadata-snapshot file to change around the s3 keys and bucket name to >> match what development was expecting. >> >> On Friday, June 6, 2014 1:11:57 PM UTC-4, Brian Lamb wrote: >>> >>> Hi all, >>> >>> I want to do a one time copy of the data on my production elastic search >>> instance to my development elastic search instance. Both are managed by AWS >>> if that makes this easier. Here is what I tried: >>> >>> On production: >>> >>> curl -XPUT 'http://localhost:9200/_snapshot/my_s3_repository' -d '{ >>> "type": "s3", >>> "settings": { >>> "access_key": "productionAccessKey", >>> "bucket": "productionBucketName", >>> "region": "region", >>> "secret_key": "productionSecretKey" >>> } >>> }' >>> curl -XPUT " >>> http://localhost:9200/_snapshot/my_s3_repository/snapshot_2014_06_02"; >>> >>> What this does is upload the instance to a production level s3 bucket. >>> >>> Then in the aws console, I copy all of it to a development level s3 >>> bucket. >>> >>> Next on development: >>> >>> curl -XPUT 'http://localhost:9200/_snapshot/my_s3_repository' -d '{ >>> "type": "s3", >>> "settings": { >>> "access_key": "developmentAccessKey", >>> "bucket": "developmentBucketName", >>> "region": "region", >>> "secret_key": "developmentSecretKey" >>> } >>> }' >>> curl -XPOST " >>> http://localhost:9200/_snapshot/my_s3_repository/snapshot_2014_06_02/_restore >>> " >>> >>> This gives me the following message: >>> >>> $ curl -XPOST " >>> http://localhost:9200/_snapshot/my_s3_repository/snapshot_2014_06_02/_restore?pretty=true >>> " >>> { >>> "error" : "SnapshotException[[my_s3_repository:snapshot_2014_06_02] >>> failed to get snapshots]; nested: IOException[Failed to get >>> [snapshot-snapshot_2014_06_02]]; nested: AmazonS3Exception[Status Code: >>> 404, AWS Service: Amazon S3, AWS Request ID: RequestId, AWS Error Code: >>> NoSuchKey, AWS Error Message: The specified key does not exist.]; ", >>> "status" : 500 >>> } >>> >>> Also, when I try to get the snapshots, I get the following: >>> >>> $ curl -XGET "localhost:9200/_snapshot/_status?pretty=true" >>> { >>> "snapshots" : [ ] >>> } >>> >>> This leads me to believe that I am not connecting the snapshot correctly >>> but I'm not sure what I am doing incorrectly. Regenerating the index on >>> development is not really a possibility as it took a few months to generate >>> the index the first time around. If there is a better way to do this, I'm >>> all for it. >>> >>> Thanks, >>> >>> Brian Lamb >>> >> -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/0ed279b6-599f-4c90-917a-d377622e12cd%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
