I recently have been testing the S3 snapshot and restore performance. I'm able to pull down ~25GB across all 4 data nodes in 10 mins on a restore. On a given node, I seem to be only getting about 70-80 MBit/s, CPU utilization is near zero. On the one node that has two shards being restored to it, the through put is nearly double.
Is there anything I can do to increase the parallelism of the restore process? So I assume there is something perhaps limited at the shard level? My Repo Settings ------------------------------------ { "type": "s3", "settings": { "bucket": "mybucket", "region": "us-east", "protocol": "https", "base_path": "/elasticsearch", "secret_key": "SECRET", "access_key": "KEY", "max_snapshot_bytes_per_sec": "150mb", "max_restore_bytes_per_sec": "500mb" } } Cluster Info ------------------------------------ ES 1.4.1 4 Data Nodes r3.2xlarge (8 core, 30GB JVM heap, SSD's) 5 Shards, 1 replica 80GB primary store size (160GB w/ replica) -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/3cd6348a-9738-4857-9af9-01c3e0b0bafb%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.