Finally, I fixed my problem. There was a mistake for the field "discovery.ec2.groups". Instead of a string, I had to put an array of string. And I also forgot to add the tag "platform:prod" to CloudFormation when launching my stack.
Fixed! On Friday, 7 February 2014 14:54:05 UTC+1, Thomas FATTAL wrote: > > Hi, > > I'm trying to configure two Elasticsearch nodes in AWS in the same > autoscaling group (CloudFormation). > I am having some problems with them discovering each other. > > The following shows the elasticsearch.log I have on the first machine with > the instance-id "i-2db5db03". > The second machine has an instance-id "i-324e6612". > > It seems that both nodes recognize each other, thanks to > discovery.ec2.tag.* field I added but then there are some problems that > make them not to join together: > > [2014-02-07 13:17:08,852][INFO ][node ] > [ip-10-238-225-133.ec2.internal] version[1.0.0.Beta2], pid[15342], > build[296cfbe/2013-12-02T15:46:27Z] > [2014-02-07 13:17:08,853][INFO ][node ] > [ip-10-238-225-133.ec2.internal] initializing ... > [2014-02-07 13:17:08,917][INFO ][plugins ] > [ip-10-238-225-133.ec2.internal] loaded [cloud-aws], sites [paramedic] > [2014-02-07 13:17:15,452][DEBUG][discovery.zen.ping.unicast] > [ip-10-238-225-133.ec2.internal] using initial hosts [], with > concurrent_connects [10] > [2014-02-07 13:17:15,455][DEBUG][discovery.ec2 ] > [ip-10-238-225-133.ec2.internal] using ping.timeout [3s], > master_election.filter_client [true], master_election.filter_data [false] > [2014-02-07 13:17:15,456][DEBUG][discovery.zen.elect ] > [ip-10-238-225-133.ec2.internal] using minimum_master_nodes [1] > [2014-02-07 13:17:15,457][DEBUG][discovery.zen.fd ] > [ip-10-238-225-133.ec2.internal] [master] uses ping_interval [1s], > ping_timeout [30s], ping_retries [3] > [2014-02-07 13:17:15,500][DEBUG][discovery.zen.fd ] > [ip-10-238-225-133.ec2.internal] [node ] uses ping_interval [1s], > ping_timeout [30s], ping_retries [3] > [2014-02-07 13:17:16,769][DEBUG][discovery.ec2 ] > [ip-10-238-225-133.ec2.internal] using host_type [PRIVATE_IP], tags > [{platform=prod}], groups [[]] with any_group [true], availability_zones > [[]] > [2014-02-07 13:17:19,930][INFO ][node ] > [ip-10-238-225-133.ec2.internal] initialized > [2014-02-07 13:17:19,931][INFO ][node ] > [ip-10-238-225-133.ec2.internal] starting ... > [2014-02-07 13:17:20,455][INFO ][transport ] > [ip-10-238-225-133.ec2.internal] bound_address {inet[/0.0.0.0:9300]}, > publish_address {inet[/10.238.225.133:9300]} > [2014-02-07 13:17:20,527][TRACE][discovery ] > [ip-10-238-225-133.ec2.internal] waiting for 30s for the initial state to > be set by the discovery > [2014-02-07 13:17:21,981][TRACE][discovery.ec2 ] > [ip-10-238-225-133.ec2.internal] building dynamic unicast discovery nodes... > [2014-02-07 13:17:21,982][TRACE][discovery.ec2 ] > [ip-10-238-225-133.ec2.internal] filtering out instance i-2db5db03 based > tags {platform=prod}, not part of [{Key: aws:cloudformation:stack-id, > Value: > arn:aws:cloudformation:us-east-1:876119091332:stack/ES-10/daf53050-8ff8-11e3-bdce-50e241629418, > > }, {Key: aws:cloudformation:stack-name, Value: ES-10, }, {Key: > aws:cloudformation:logical-id, Value: ESASG, }, {Key: > aws:autoscaling:groupName, Value: ES-10-ESASG-BHGX7KKQ9QPR, }] > [2014-02-07 13:17:21,983][TRACE][discovery.ec2 ] > [ip-10-238-225-133.ec2.internal] filtering out instance i-324e6612 based > tags {platform=prod}, not part of [{Key: aws:cloudformation:logical-id, > Value: ESASG, }, {Key: aws:cloudformation:stack-id, Value: > arn:aws:cloudformation:us-east-1:876119091332:stack/ES-10/daf53050-8ff8-11e3-bdce-50e241629418, > > }, {Key: aws:cloudformation:stack-name, Value: ES-10, }, {Key: > aws:autoscaling:groupName, Value: ES-10-ESASG-BHGX7KKQ9QPR, }] > [2014-02-07 13:17:21,983][DEBUG][discovery.ec2 ] > [ip-10-238-225-133.ec2.internal] using dynamic discovery nodes [] > [2014-02-07 13:17:23,744][TRACE][discovery.ec2 ] > [ip-10-238-225-133.ec2.internal] building dynamic unicast discovery nodes... > [2014-02-07 13:17:23,745][TRACE][discovery.ec2 ] > [ip-10-238-225-133.ec2.internal] filtering out instance i-2db5db03 based > tags {platform=prod}, not part of [{Key: aws:cloudformation:stack-id, > Value: > arn:aws:cloudformation:us-east-1:876119091332:stack/ES-10/daf53050-8ff8-11e3-bdce-50e241629418, > > }, {Key: aws:cloudformation:stack-name, Value: ES-10, }, {Key: > aws:cloudformation:logical-id, Value: ESASG, }, {Key: > aws:autoscaling:groupName, Value: ES-10-ESASG-BHGX7KKQ9QPR, }] > [2014-02-07 13:17:23,745][TRACE][discovery.ec2 ] > [ip-10-238-225-133.ec2.internal] filtering out instance i-324e6612 based > tags {platform=prod}, not part of [{Key: aws:cloudformation:logical-id, > Value: ESASG, }, {Key: aws:cloudformation:stack-id, Value: > arn:aws:cloudformation:us-east-1:876119091332:stack/ES-10/daf53050-8ff8-11e3-bdce-50e241629418, > > }, {Key: aws:cloudformation:stack-name, Value: ES-10, }, {Key: > aws:autoscaling:groupName, Value: ES-10-ESASG-BHGX7KKQ9QPR, }] > [2014-02-07 13:17:25,604][DEBUG][discovery.ec2 ] > [ip-10-238-225-133.ec2.internal] using dynamic discovery nodes [] > [2014-02-07 13:17:25,604][TRACE][discovery.ec2 ] > [ip-10-238-225-133.ec2.internal] full ping responses: {none} > [2014-02-07 13:17:25,605][DEBUG][discovery.ec2 ] > [ip-10-238-225-133.ec2.internal] filtered ping responses: > (filter_client[true], filter_data[false]) {none} > [2014-02-07 13:17:25,618][INFO ][cluster.service ] > [ip-10-238-225-133.ec2.internal] new_master > [ip-10-238-225-133.ec2.internal][tUBe7JNdTquROI6UWVLNpw][inet[/10.238.225.133:9300]]{aws_availability_zone=us-east-1d, > > max_local_storage_nodes=1}, reason: zen-disco-join (elected_as_master) > [2014-02-07 13:17:25,752][TRACE][discovery ] > [ip-10-238-225-133.ec2.internal] initial state set from discovery > [2014-02-07 13:17:25,752][INFO ][discovery ] > [ip-10-238-225-133.ec2.internal] elasticsearch/tUBe7JNdTquROI6UWVLNpw > [2014-02-07 13:17:25,892][INFO ][http ] > [ip-10-238-225-133.ec2.internal] bound_address {inet[/0.0.0.0:9200]}, > publish_address {inet[/10.238.225.133:9200]} > [2014-02-07 13:17:25,893][INFO ][node ] > [ip-10-238-225-133.ec2.internal] started > [2014-02-07 13:17:25,924][INFO ][gateway ] > [ip-10-238-225-133.ec2.internal] recovered [0] indices into cluster_state > > > > I'm using Chef to configure my node with the cookbook of Karmi ( > https://github.com/elasticsearch/cookbook-elasticsearch). > The node.json is the following: > > "/etc/chef/node.json": { > "content": { > "run_list": [ > "recipe[java]", > "recipe[monit]", > "recipe[elasticsearch]", > "recipe[elasticsearch::plugins]", > "recipe[elasticsearch::aws]", > "recipe[elasticsearch::monit]" > ], > "java": { > "install_flavor": "oracle", > "oracle": { > "accept_oracle_download_terms": true > }, > "jdk_version": "7" > > }, > "elasticsearch": { > "version": "1.0.0.Beta2", > "logging": { > "discovery": "TRACE" > }, > "cluster": { > "name": "elasticsearch" > }, > "bootstrap": { > "mlockall": false > }, > "discovery": { > "type": "ec2", > "ec2": { > "groups": {"Ref": "ESSecurityGroup"}, > "tag": { > "platform": "prod" > } > }, > "zen": { > "ping": { > "multicast": { > "enabled": false > } > } > } > }, > "cloud": { > "aws": { > "access_key": {"Ref": "ESHostKeys"}, > "secret_key": {"Fn::GetAtt": [ > "ESHostKeys", "SecretAccessKey"]}, > "region": "us-east-1" > }, > "ec2": { > "security_group": {"Ref": > "ESSecurityGroup"} > } > }, > "monit": { > "notify_email": "my_email_address", > "mail_format": { > "from": "from_email_address", > "subject": > "[monit]$SERVICE$EVENTon$HOST", > "message": "$SERVICE$ACTION: > $DESCRIPTION" > } > }, > "plugins": { > "karmi/elasticsearch-paramedic" : {} > } > } > }, > "mode": "000644", > "owner": "root", > "group": "root" > } > } > > > I'm using Oracle 7 for ElasticSearch. The machines are Ubuntu 12.04 and > I'm using Elasticsearch 1.0.0 RC2. > > It seems that the first node is seeing the second node but refuse to join > it because the tags "aws:cloudformation:stack-id", " > aws:cloudformation:stack-name", "aws:cloudformation:logical-id" and > "aws:autoscaling:groupName" > are not the same. However, I checked these tags for both instances and it's > the same (as the two instances have been created in the same Autoscaling > group). > > Any ideas? > Thanks! > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/754216cd-c396-4396-a1e6-b1e6a9cc63c3%40googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.