Re: EC2 Discovery
Don't try ec2 discovery until you have tested that: - you can connect from one machine to another on port 9300 ( nc as client and server, basic networking/ firewalling) - run a simple aws ec2 describe instances call with the API key you plan to use, and you can see the machines you need there. Bonus points for filtering based on the rules you intense to use ( sec group, tags). This is to ensure your API keys have the correct access needed. Once you have those basic steps working, use them on es config. Make sure you enable ec2 discovery and disable the zen discovery ( it will run first and likely time out and ec2 disco won't get to exec). The other thing to watch out for is contacting nodes which are too busy to ack your new nodes request for cluster info...but that would be a problem with zen disco too. On 21/03/2014 12:31 PM, Raphael Miranda raphaelmira...@gmail.com wrote: are both machines in the same security group? -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/eb8bb939-3b9d-4f5b-a45c-3d529f75983e%40googlegroups.com . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CACj2-4LjvKAW9cnkrjQUR6%3Dk8FRf%3DKzmDUAUopHLVUMNc1ixOw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: EC2 Discovery
I am not sure if I missed something, but what you mentioned I believe I already tried as showing in my original post. I can connect to each machine individually and I am able ti index and query it fine with default configuration without any zen or ec2 settings. But, when I turned them on like I show on the post, I get this Request failed to get to the server (status code: 0): when trying to query the instance. Did you mean I should try to see if I can access one instance from the other? This I didn't try yet. On Friday, March 21, 2014 4:46:40 AM UTC-4, Norberto Meijome wrote: Don't try ec2 discovery until you have tested that: - you can connect from one machine to another on port 9300 ( nc as client and server, basic networking/ firewalling) - run a simple aws ec2 describe instances call with the API key you plan to use, and you can see the machines you need there. Bonus points for filtering based on the rules you intense to use ( sec group, tags). This is to ensure your API keys have the correct access needed. Once you have those basic steps working, use them on es config. Make sure you enable ec2 discovery and disable the zen discovery ( it will run first and likely time out and ec2 disco won't get to exec). The other thing to watch out for is contacting nodes which are too busy to ack your new nodes request for cluster info...but that would be a problem with zen disco too. On 21/03/2014 12:31 PM, Raphael Miranda raphael...@gmail.comjavascript: wrote: are both machines in the same security group? -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com javascript:. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/eb8bb939-3b9d-4f5b-a45c-3d529f75983e%40googlegroups.com . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/7c9e9da8-6efe-4005-8a69-c00daa6ec711%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: EC2 Discovery
I am not sure if I missed something, but what you mentioned I believe I already tried as showing in my original post. I can connect from one instance to another. I can connect to each machine individually and I am able to index and query it fine with default configuration without any zen or ec2 settings. But, when I turned them on like I show on the post, I get this Request failed to get to the server (status code: 0): when trying to query the instance, and when I do this, it won't even log anything, it is not getting that far. On Friday, March 21, 2014 4:46:40 AM UTC-4, Norberto Meijome wrote: Don't try ec2 discovery until you have tested that: - you can connect from one machine to another on port 9300 ( nc as client and server, basic networking/ firewalling) - run a simple aws ec2 describe instances call with the API key you plan to use, and you can see the machines you need there. Bonus points for filtering based on the rules you intense to use ( sec group, tags). This is to ensure your API keys have the correct access needed. Once you have those basic steps working, use them on es config. Make sure you enable ec2 discovery and disable the zen discovery ( it will run first and likely time out and ec2 disco won't get to exec). The other thing to watch out for is contacting nodes which are too busy to ack your new nodes request for cluster info...but that would be a problem with zen disco too. On 21/03/2014 12:31 PM, Raphael Miranda raphael...@gmail.comjavascript: wrote: are both machines in the same security group? -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com javascript:. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/eb8bb939-3b9d-4f5b-a45c-3d529f75983e%40googlegroups.com . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/48571118-fd84-45da-9aaf-0314c936336b%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: EC2 Discovery
Ok, we are seeing this error in the log, any clues? Error injecting constructor, java.lang.IllegalStateException: This is a proxy used to support circular references involving constructors. The object we're proxying is not constructed yet. Please wait until after injection has completed to use this object. at org.elasticsearch.cache.NodeCache.init(Unknown Source) while locating org.elasticsearch.cache.NodeCache Caused by: java.lang.IllegalStateException: This is a proxy used to support circular references involving constructors. On Thursday, March 20, 2014 3:18:11 PM UTC-4, IronMan2014 wrote: I can't seem to make my EC2 cluster of 2 nodes/instances work. - If I comment out the section below, I can connect to each instance individually and query it, I am using Sense plugin to query Elastic Search Instance. - With the sections included as below, I cannot query neither node, I get Request failed to get to the server (status code: 0): In /instance 1/ elasticsearch.yml cluster.name: mycluster node.name: node_1 cloud: aws: access_key: MY ACCESSKEY secret_key: My SecretKEY discovery: type: ec2 In /instance 2/ elasticsearch.yml cluster.name: mycluster node.name: node_2 cloud: aws: access_key: MY ACCESSKEY secret_key: My SecretKEY discovery: type: ec2 I also tried with: discovery.zen.ping.multicast.enabled: false discovery.zen.ping.unicast.hosts: [Instance_1 IP:9300, instance_2 IP:9300] Here is mycluster.log from both instances: $ more /var/log/elasticsearch/mycluster.log [2014-03-20 19:00:37,635][INFO ][node ] [node_1]version [0.90.10], pid[3520], build[0a5781f/2014-01-10T10:18:37Z] [2014-03-20 19:00:37,635][INFO ][node ] [node_1]initializing ... [2014-03-20 19:00:37,698][INFO ][plugins ] [node_1]loaded [mapper-attachments, cloud-aws], sites [] [2014-03-20 19:01:17,898][INFO ][node ] [node_1]version [0.90.10], pid[3594], build[0a5781f/2014-01-10T10:18:37Z] [2014-03-20 19:01:17,898][INFO ][node ] [node_1]initializing ... [2014-03-20 19:01:17,961][INFO ][plugins ] [node_1]loaded [mapper-attachments, cloud-aws], sites [] [2014-03-20 19:03:50,048][INFO ][node ] [node_1]version [0.90.10], pid[3671], build[0a5781f/2014-01-10T10:18:37Z] [2014-03-20 19:03:50,048][INFO ][node ] [node_1]initializing ... [2014-03-20 19:03:50,111][INFO ][plugins ] [node_1]loaded [mapper-attachments, cloud-aws], sites [] $more /var/log/elasticsearch/mycluster.log [2014-03-20 19:00:29,465][INFO ][node ] [node_2]version [0.90.10], pid[2800], build[0a5781f/2014-01-10T10:18:37Z] [2014-03-20 19:00:29,466][INFO ][node ] [node_2]initializing ... [2014-03-20 19:00:29,528][INFO ][plugins ] [node_2]loaded [mapper-attachments, cloud-aws], sites [] [2014-03-20 19:01:26,645][INFO ][node ] [node_2]version [0.90.10], pid[2874], build[0a5781f/2014-01-10T10:18:37Z] [2014-03-20 19:01:26,646][INFO ][node ] [node_2]initializing ... [2014-03-s ... -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/0ed56a05-1538-4a4d-91d7-8a9880ef7508%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: EC2 Discovery
/etc/elasticsearch$ more /var/log/elasticsearch/mycluster.log [2014-03-20 19:00:37,635][INFO ][node ] [node_1] version [0.90.10], pid[3520], build[0a5781f/2014-01-10T10:18:37Z] [2014-03-20 19:00:37,635][INFO ][node ] [node_1]initializing ... [2014-03-20 19:00:37,698][INFO ][plugins ] [node_1] loaded [mapper-attachments, cloud-aws], sites [] [2014-03-20 19:01:17,898][INFO ][node ] [node_1] version [0.90.10], pid[3594], build[0a5781f/2014-01-10T10:18:37Z] [2014-03-20 19:01:17,898][INFO ][node ] [node_1]initializing ... [2014-03-20 19:01:17,961][INFO ][plugins ] [node_1] loaded [mapper-attachments, cloud-aws], sites [] [2014-03-20 19:03:50,048][INFO ][node ] [node_1] version [0.90.10], pid[3671], build[0a5781f/2014-01-10T10:18:37Z] [2014-03-20 19:03:50,048][INFO ][node ] [node_1]initializing ... [2014-03-20 19:03:50,111][INFO ][plugins ] [node_1] loaded [mapper-attachments, cloud-aws], sites [] more /var/log/elasticsearch/mycluster.log [2014-03-20 19:00:29,465][INFO ][node ] [node_2] version [0.90.10], pid[2800], build[0a5781f/2014-01-10T10:18:37Z] [2014-03-20 19:00:29,466][INFO ][node ] [node_2]initializing ... [2014-03-20 19:00:29,528][INFO ][plugins ] [node_2] loaded [mapper-attachments, cloud-aws], sites [] [2014-03-20 19:01:26,645][INFO ][node ] [node_2] version [0.90.10], pid[2874], build[0a5781f/2014-01-10T10:18:37Z] [2014-03-20 19:01:26,646][INFO ][node ] [node_2]initializing ... [2014-03-20 19:01:26,709][INFO ][plugins ] [node_2] loaded [mapper-attachments, cloud-aws], sites [] [2014-03-20 19:03:58,357][INFO ][node ] [node_2] version [0.90.10], pid[2951], build[0a5781f/2014-01-10T10:18:37Z] [2014-03-20 19:03:58,357][INFO ][node ] [node_2]initializing ... [2014-03-20 19:03:58,420][INFO ][plugins ] [node_2] loaded [mapper-attachments, cloud-aws], sites [] On Thursday, March 20, 2014 2:50:46 PM UTC-4, IronMan2014 wrote: can't seem to make my EC2 cluster of 2 nodes work. - If I comment out the section below, I can connect to each instance individually and query it. - With the sections included as below, I cannot query neither node, I get Request failed to get to the server (status code: 0): cluster.name: myCluster node.name: node 1 cloud: aws: access_key: MY ACCESSKEY secret_key: My SecretKEY discovery: type: ec2 cluster.name: myCluster node.name: node 2 cloud: aws: access_key: MY ACCESSKEY secret_key: My SecretKEY discovery: type: ec2 I also tried with: discovery.zen.ping.multicast.enabled: false discovery.zen.ping.unicast.hosts: [Instance_1 IP:9300, instance_2 IP:9300] -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a73dfef8-23e6-4c6a-a780-0b7722ae45b3%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: EC2 discovery issues with 2 local instances
I think I understand what is happening here. Wondering if giving another elasticsearch.yml file as à configuration for the second node with all defaults (except cluster name) could help. -- David ;-) Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 18 mars 2014 à 15:41, Bastien Chong bastien...@gmail.com a écrit : Hi, I have a server with 2 ES instances. The first one is the master one, used to store documents. The second one is just there to receive requests from Kibana (I call it ES read-only), it has : node.master: false node.data: false Both are configured with cloud-aws plugin, and the http/java port are left by default for automatic assignment. When master start first, it's binded to 9200/9300, then the second one to 9201/9300. When I do : curl -XGET 'http://localhost:9200/_cluster/health?pretty=true' everything is working as expected and both nodes are in the same cluster. But, if I start the read-only instance first, discovery stop working. I have enabled DEBUG and TRACE but I didn't found what's the issue. I also tried to hardcode the port allocation : transport.tcp.port: 930(0/1) http.port: 920(0-1) And after that, It's actually worse, whatever the order I start the instances, EC2 discovery is broken. So it's sort of a race-condition that's happening. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8851834f-cefd-4732-8c34-f3410cec0c99%40googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/914F1C15-3832-4C45-BD7C-095286011D23%40pilato.fr. For more options, visit https://groups.google.com/d/optout.
Re: EC2 discovery issues with 2 local instances
I'm not sure what I can change since I have the bare minimum config : The only difference between the 2 elasticsearch.yml config file are: node.master: false node.data: false path.conf: /etc/elasticsearchro path.data: /var/lib/elasticsearchro Both config share this part: cluster.name: ESCluster cloud: aws: access_key: xx secret_key: x region: us-west-2 discovery: type: ec2 ec2: tag: elasticsearch: true On Tuesday, March 18, 2014 11:00:15 AM UTC-4, David Pilato wrote: I think I understand what is happening here. Wondering if giving another elasticsearch.yml file as à configuration for the second node with all defaults (except cluster name) could help. -- David ;-) Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 18 mars 2014 à 15:41, Bastien Chong basti...@gmail.com javascript: a écrit : Hi, I have a server with 2 ES instances. The first one is the master one, used to store documents. The second one is just there to receive requests from Kibana (I call it ES read-only), it has : node.master: false node.data: false Both are configured with cloud-aws plugin, and the http/java port are left by default for automatic assignment. When master start first, it's binded to 9200/9300, then the second one to 9201/9300. When I do : *curl -XGET 'http://localhost:9200/_cluster/health?pretty=true' http://localhost:9200/_cluster/health?pretty=true'* everything is working as expected and both nodes are in the same cluster. *But*, if I start the read-only instance first, discovery stop working. I have enabled DEBUG and TRACE but I didn't found what's the issue. I also tried to hardcode the port allocation : transport.tcp.port: 930(0/1) http.port: 920(0-1) And after that, It's actually worse, whatever the order I start the instances, EC2 discovery is broken. So it's sort of a race-condition that's happening. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com javascript:. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8851834f-cefd-4732-8c34-f3410cec0c99%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/8851834f-cefd-4732-8c34-f3410cec0c99%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/f31f79ec-ad39-440f-b2cd-36b642e18d48%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: EC2 discovery issues with 2 local instances
For the second node, remove aws: access_key: xx secret_key: x region: us-west-2 discovery: type: ec2 ec2: tag: elasticsearch: true It should work I think. Or set unicast to localhost and disable multicast -- David Pilato | Technical Advocate | Elasticsearch.com @dadoonet | @elasticsearchfr Le 18 mars 2014 à 16:25:52, Bastien Chong (bastien...@gmail.com) a écrit: I'm not sure what I can change since I have the bare minimum config : The only difference between the 2 elasticsearch.yml config file are: node.master: false node.data: false path.conf: /etc/elasticsearchro path.data: /var/lib/elasticsearchro Both config share this part: cluster.name: ESCluster cloud: aws: access_key: xx secret_key: x region: us-west-2 discovery: type: ec2 ec2: tag: elasticsearch: true On Tuesday, March 18, 2014 11:00:15 AM UTC-4, David Pilato wrote: I think I understand what is happening here. Wondering if giving another elasticsearch.yml file as à configuration for the second node with all defaults (except cluster name) could help. -- David ;-) Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 18 mars 2014 à 15:41, Bastien Chong basti...@gmail.com a écrit : Hi, I have a server with 2 ES instances. The first one is the master one, used to store documents. The second one is just there to receive requests from Kibana (I call it ES read-only), it has : node.master: false node.data: false Both are configured with cloud-aws plugin, and the http/java port are left by default for automatic assignment. When master start first, it's binded to 9200/9300, then the second one to 9201/9300. When I do : curl -XGET 'http://localhost:9200/_cluster/health?pretty=true' everything is working as expected and both nodes are in the same cluster. But, if I start the read-only instance first, discovery stop working. I have enabled DEBUG and TRACE but I didn't found what's the issue. I also tried to hardcode the port allocation : transport.tcp.port: 930(0/1) http.port: 920(0-1) And after that, It's actually worse, whatever the order I start the instances, EC2 discovery is broken. So it's sort of a race-condition that's happening. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8851834f-cefd-4732-8c34-f3410cec0c99%40googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/f31f79ec-ad39-440f-b2cd-36b642e18d48%40googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/etPan.5328835c.737b8ddc.97ca%40MacBook-Air-de-David.local. For more options, visit https://groups.google.com/d/optout.
Re: EC2 discovery issues with 2 local instances
I tried without luck. I changed : discovery.zen.ping.multicast.enabled: false discovery.zen.ping.unicast.hosts: [localhost] and commented the plugin part. So by starting the 'read-only' instance first, it get 9200/9300. Then I start the master instance (9201/9301) In the Read-only log : sending to [#zen_unicast_1#][inet[localhost/127.0.0.1:9300]] It's not finding that I have an instance on port 9301. On Tuesday, March 18, 2014 2:07:38 PM UTC-4, David Pilato wrote: May be I was unclear. I know you need ec2 discovery. but you have two nodes per machine. Set the first one to use ec2 discovery and the second one to use regular discovery. And could you open an issue in elasticsearch-cloud-aws plugin? -- *David Pilato* | *Technical Advocate* | *Elasticsearch.com* @dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr Le 18 mars 2014 à 19:03:04, Bastien Chong (basti...@gmail.comjavascript:) a écrit: What I didn't mention, is that this node won't be alone, there will be about 6 other identical node, so I need the EC2 discovery feature. On Tuesday, March 18, 2014 1:33:16 PM UTC-4, David Pilato wrote: For the second node, remove aws: access_key: xx secret_key: x region: us-west-2 discovery: type: ec2 ec2: tag: elasticsearch: true It should work I think. Or set unicast to localhost and disable multicast -- *David Pilato* | *Technical Advocate* | *Elasticsearch.com* @dadoonet https://twitter.com/dadoonet | @elasticsearchfrhttps://twitter.com/elasticsearchfr Le 18 mars 2014 à 16:25:52, Bastien Chong (basti...@gmail.com) a écrit: I'm not sure what I can change since I have the bare minimum config : The only difference between the 2 elasticsearch.yml config file are: node.master: false node.data: false path.conf: /etc/elasticsearchro path.data: /var/lib/elasticsearchro Both config share this part: cluster.name: ESCluster cloud: aws: access_key: xx secret_key: x region: us-west-2 discovery: type: ec2 ec2: tag: elasticsearch: true On Tuesday, March 18, 2014 11:00:15 AM UTC-4, David Pilato wrote: I think I understand what is happening here. Wondering if giving another elasticsearch.yml file as à configuration for the second node with all defaults (except cluster name) could help. -- David ;-) Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 18 mars 2014 à 15:41, Bastien Chong basti...@gmail.com a écrit : Hi, I have a server with 2 ES instances. The first one is the master one, used to store documents. The second one is just there to receive requests from Kibana (I call it ES read-only), it has : node.master: false node.data: false Both are configured with cloud-aws plugin, and the http/java port are left by default for automatic assignment. When master start first, it's binded to 9200/9300, then the second one to 9201/9300. When I do : *curl -XGET 'http://localhost:9200/_cluster/health?pretty=true' http://localhost:9200/_cluster/health?pretty=true'* everything is working as expected and both nodes are in the same cluster. *But*, if I start the read-only instance first, discovery stop working. I have enabled DEBUG and TRACE but I didn't found what's the issue. I also tried to hardcode the port allocation : transport.tcp.port: 930(0/1) http.port: 920(0-1) And after that, It's actually worse, whatever the order I start the instances, EC2 discovery is broken. So it's sort of a race-condition that's happening. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8851834f-cefd-4732-8c34-f3410cec0c99%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/8851834f-cefd-4732-8c34-f3410cec0c99%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/f31f79ec-ad39-440f-b2cd-36b642e18d48%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/f31f79ec-ad39-440f-b2cd-36b642e18d48%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit
Re: EC2 discovery issues with 2 local instances
I also tried to comment the EC2 config on the master and enable unicast discovery. Same issue ! On Tuesday, March 18, 2014 10:41:39 AM UTC-4, Bastien Chong wrote: Hi, I have a server with 2 ES instances. The first one is the master one, used to store documents. The second one is just there to receive requests from Kibana (I call it ES read-only), it has : node.master: false node.data: false Both are configured with cloud-aws plugin, and the http/java port are left by default for automatic assignment. When master start first, it's binded to 9200/9300, then the second one to 9201/9300. When I do : *curl -XGET 'http://localhost:9200/_cluster/health?pretty=true http://localhost:9200/_cluster/health?pretty=true'* everything is working as expected and both nodes are in the same cluster. *But*, if I start the read-only instance first, discovery stop working. I have enabled DEBUG and TRACE but I didn't found what's the issue. I also tried to hardcode the port allocation : transport.tcp.port: 930(0/1) http.port: 920(0-1) And after that, It's actually worse, whatever the order I start the instances, EC2 discovery is broken. So it's sort of a race-condition that's happening. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/7778b165-f596-4638-82c2-523ee13c9857%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: EC2 discovery issues with 2 local instances
Opening a new thread. https://groups.google.com/forum/#!topic/elasticsearch/tJi7iJhU9ZU On Tuesday, March 18, 2014 3:29:16 PM UTC-4, Bastien Chong wrote: The common denominator is that when I start a [*node.master: false] [node.data: false] *node fist, the cluster is not created properly, no matter how the discovery is configured, EC2 or unicast localhost. On Tuesday, March 18, 2014 10:41:39 AM UTC-4, Bastien Chong wrote: Hi, I have a server with 2 ES instances. The first one is the master one, used to store documents. The second one is just there to receive requests from Kibana (I call it ES read-only), it has : node.master: false node.data: false Both are configured with cloud-aws plugin, and the http/java port are left by default for automatic assignment. When master start first, it's binded to 9200/9300, then the second one to 9201/9300. When I do : *curl -XGET 'http://localhost:9200/_cluster/health?pretty=true http://localhost:9200/_cluster/health?pretty=true'* everything is working as expected and both nodes are in the same cluster. *But*, if I start the read-only instance first, discovery stop working. I have enabled DEBUG and TRACE but I didn't found what's the issue. I also tried to hardcode the port allocation : transport.tcp.port: 930(0/1) http.port: 920(0-1) And after that, It's actually worse, whatever the order I start the instances, EC2 discovery is broken. So it's sort of a race-condition that's happening. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ce057522-32e5-429f-a069-ccee522f6d1c%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: EC2 Discovery is not working with AutoScaling group (AWS)
Finally, I fixed my problem.
There was a mistake for the field discovery.ec2.groups. Instead of a
string, I had to put an array of string.
And I also forgot to add the tag platform:prod to CloudFormation when
launching my stack.
Fixed!
On Friday, 7 February 2014 14:54:05 UTC+1, Thomas FATTAL wrote:
Hi,
I'm trying to configure two Elasticsearch nodes in AWS in the same
autoscaling group (CloudFormation).
I am having some problems with them discovering each other.
The following shows the elasticsearch.log I have on the first machine with
the instance-id i-2db5db03.
The second machine has an instance-id i-324e6612.
It seems that both nodes recognize each other, thanks to
discovery.ec2.tag.* field I added but then there are some problems that
make them not to join together:
[2014-02-07 13:17:08,852][INFO ][node ]
[ip-10-238-225-133.ec2.internal] version[1.0.0.Beta2], pid[15342],
build[296cfbe/2013-12-02T15:46:27Z]
[2014-02-07 13:17:08,853][INFO ][node ]
[ip-10-238-225-133.ec2.internal] initializing ...
[2014-02-07 13:17:08,917][INFO ][plugins ]
[ip-10-238-225-133.ec2.internal] loaded [cloud-aws], sites [paramedic]
[2014-02-07 13:17:15,452][DEBUG][discovery.zen.ping.unicast]
[ip-10-238-225-133.ec2.internal] using initial hosts [], with
concurrent_connects [10]
[2014-02-07 13:17:15,455][DEBUG][discovery.ec2]
[ip-10-238-225-133.ec2.internal] using ping.timeout [3s],
master_election.filter_client [true], master_election.filter_data [false]
[2014-02-07 13:17:15,456][DEBUG][discovery.zen.elect ]
[ip-10-238-225-133.ec2.internal] using minimum_master_nodes [1]
[2014-02-07 13:17:15,457][DEBUG][discovery.zen.fd ]
[ip-10-238-225-133.ec2.internal] [master] uses ping_interval [1s],
ping_timeout [30s], ping_retries [3]
[2014-02-07 13:17:15,500][DEBUG][discovery.zen.fd ]
[ip-10-238-225-133.ec2.internal] [node ] uses ping_interval [1s],
ping_timeout [30s], ping_retries [3]
[2014-02-07 13:17:16,769][DEBUG][discovery.ec2]
[ip-10-238-225-133.ec2.internal] using host_type [PRIVATE_IP], tags
[{platform=prod}], groups [[]] with any_group [true], availability_zones
[[]]
[2014-02-07 13:17:19,930][INFO ][node ]
[ip-10-238-225-133.ec2.internal] initialized
[2014-02-07 13:17:19,931][INFO ][node ]
[ip-10-238-225-133.ec2.internal] starting ...
[2014-02-07 13:17:20,455][INFO ][transport]
[ip-10-238-225-133.ec2.internal] bound_address {inet[/0.0.0.0:9300]},
publish_address {inet[/10.238.225.133:9300]}
[2014-02-07 13:17:20,527][TRACE][discovery]
[ip-10-238-225-133.ec2.internal] waiting for 30s for the initial state to
be set by the discovery
[2014-02-07 13:17:21,981][TRACE][discovery.ec2]
[ip-10-238-225-133.ec2.internal] building dynamic unicast discovery nodes...
[2014-02-07 13:17:21,982][TRACE][discovery.ec2]
[ip-10-238-225-133.ec2.internal] filtering out instance i-2db5db03 based
tags {platform=prod}, not part of [{Key: aws:cloudformation:stack-id,
Value:
arn:aws:cloudformation:us-east-1:876119091332:stack/ES-10/daf53050-8ff8-11e3-bdce-50e241629418,
}, {Key: aws:cloudformation:stack-name, Value: ES-10, }, {Key:
aws:cloudformation:logical-id, Value: ESASG, }, {Key:
aws:autoscaling:groupName, Value: ES-10-ESASG-BHGX7KKQ9QPR, }]
[2014-02-07 13:17:21,983][TRACE][discovery.ec2]
[ip-10-238-225-133.ec2.internal] filtering out instance i-324e6612 based
tags {platform=prod}, not part of [{Key: aws:cloudformation:logical-id,
Value: ESASG, }, {Key: aws:cloudformation:stack-id, Value:
arn:aws:cloudformation:us-east-1:876119091332:stack/ES-10/daf53050-8ff8-11e3-bdce-50e241629418,
}, {Key: aws:cloudformation:stack-name, Value: ES-10, }, {Key:
aws:autoscaling:groupName, Value: ES-10-ESASG-BHGX7KKQ9QPR, }]
[2014-02-07 13:17:21,983][DEBUG][discovery.ec2]
[ip-10-238-225-133.ec2.internal] using dynamic discovery nodes []
[2014-02-07 13:17:23,744][TRACE][discovery.ec2]
[ip-10-238-225-133.ec2.internal] building dynamic unicast discovery nodes...
[2014-02-07 13:17:23,745][TRACE][discovery.ec2]
[ip-10-238-225-133.ec2.internal] filtering out instance i-2db5db03 based
tags {platform=prod}, not part of [{Key: aws:cloudformation:stack-id,
Value:
arn:aws:cloudformation:us-east-1:876119091332:stack/ES-10/daf53050-8ff8-11e3-bdce-50e241629418,
}, {Key: aws:cloudformation:stack-name, Value: ES-10, }, {Key:
aws:cloudformation:logical-id, Value: ESASG, }, {Key:
aws:autoscaling:groupName, Value: ES-10-ESASG-BHGX7KKQ9QPR, }]
[2014-02-07 13:17:23,745][TRACE][discovery.ec2]
[ip-10-238-225-133.ec2.internal] filtering out instance i-324e6612 based
tags {platform=prod}, not part of [{Key: aws:cloudformation:logical-id,
Value: ESASG, }, {Key: aws:cloudformation:stack-id, Value:
Re: ec2 discovery
As I understand it, the ec2 plugin simply does ec2 API calls to list instances, filtered as per your config. It plays no actual part in the connectivity or clustering part - just discovery. So yes it makes sense what you saw. On 23/01/2014 7:43 PM, barak barak.ya...@gmail.com wrote: Hi, I've 3 nodes on ec2, each of them run 0.90.9. With default config, none of them discover each other. Now I want 2 of them to form a cluster, so I installed on those 2 nodes aws-cloud plugin, changed the related configuration and restarted. The 2 nodes indeed discovered each other as expected, but also the 3rd node... Only after changing the cluster name of that 3rd node, it left the cluster. Is it OK that node without aws-cloud plugin got discovered by the others? Thanks! -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/f4cf931c-b492-4333-8454-29013c557213%40googlegroups.com . For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CACj2-4JFoXitTDSFiNrvLCBf1ycjpg4U0-9YJ-gsFp5Gn%2BvCwg%40mail.gmail.com. For more options, visit https://groups.google.com/groups/opt_out.
