Re: This version of Kibana requires at least Elasticsearch 1.4.0.Beta1 but using 1.4.1

2014-12-03 Thread David Montgomery

I added the below to elasticsearch.ymk config.  Still kibana provides the 
same error

http.cors.enabled: true
http.cors.allow-origin: http://monitor-development-east.test.com:5601

For those at elastiseaerch..can you provide me color on what my be going 
on?  

Thanks







On Thursday, December 4, 2014 7:53:50 AM UTC+8, David Montgomery wrote:
>
> Hi,
>
>  have no agents as of yet.  Just logstash server with the below config
>
> input {
>   redis {
> host => "<%=@redis_host%>"
> data_type => "list"
> key => "logstash"
> codec => json
>   }
> }
>
> output {
> stdout { }
> elasticsearch { 
> host => "<%=node[:ipaddress]%>" 
> }
> }
>
> On Thursday, December 4, 2014 6:19:36 AM UTC+8, Jack Judge wrote:
>>
>> This tripped me up too.
>>
>> Are your logstash agents using the elasticsearch output ? If so, they'll 
>> be running the embedded version of ES that comes with Logstash, and that's 
>> the version that's stopping kibana from working. Basically *any* ES node 
>> that connects to your cluster in any form must be 1.4.x
>> My solution was to change the logstash output to elasticsearch_http
>>
>> JJ
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/caf7c2cf-3a02-4b31-8d3c-1b2c9ce52ed2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: This version of Kibana requires at least Elasticsearch 1.4.0.Beta1 but using 1.4.1

2014-12-03 Thread David Montgomery
Hi,

 have no agents as of yet.  Just logstash server with the below config

input {
  redis {
host => "<%=@redis_host%>"
data_type => "list"
key => "logstash"
codec => json
  }
}

output {
stdout { }
elasticsearch { 
host => "<%=node[:ipaddress]%>" 
}
}

On Thursday, December 4, 2014 6:19:36 AM UTC+8, Jack Judge wrote:
>
> This tripped me up too.
>
> Are your logstash agents using the elasticsearch output ? If so, they'll 
> be running the embedded version of ES that comes with Logstash, and that's 
> the version that's stopping kibana from working. Basically *any* ES node 
> that connects to your cluster in any form must be 1.4.x
> My solution was to change the logstash output to elasticsearch_http
>
> JJ
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/1980ac96-edef-4df6-8ec5-38671718fafc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: This version of Kibana requires at least Elasticsearch 1.4.0.Beta1 but using 1.4.1

2014-12-03 Thread David Montgomery
I am kibana 4.0.0-BETA2

On Wednesday, December 3, 2014 7:30:03 PM UTC+8, Mark Walkom wrote:
>
> What version of Kibana?
>
> On 3 December 2014 at 21:48, David Montgomery  > wrote:
>
>> Hi,
>>
>>
>> This version of Kibana requires at least Elasticsearch 1.4.0.Beta1
>>
>> SetupError@
>> http://monitor-development-east.test.com:5601/index.js?_b=3998:42905:51
>> checkEsVersion/<@
>> http://monitor-development-east.test.com:5601/index.js?_b=3998:43091:14
>> qFactory/defer/deferred.promise.then/wrappedCallback@
>> http://monitor-development-east.test.com:5601/index.js?_b=3998:20764:15
>> qFactory/ref/<.then/<@
>> http://monitor-development-east.test.com:5601/index.js?_b=3998:20850:11
>> $RootScopeProvider/this.$get> http://monitor-development-east.test.com:5601/index.js?_b=3998:21893:9
>> $RootScopeProvider/this.$get> http://monitor-development-east.test.com:5601/index.js?_b=3998:21705:15
>> $RootScopeProvider/this.$get> http://monitor-development-east.test.com:5601/index.js?_b=3998:21997:13
>> done@
>> http://monitor-development-east.test.com:5601/index.js?_b=3998:17570:34
>> completeRequest@
>> http://monitor-development-east.test.com:5601/index.js?_b=3998:17784:7
>> createHttpBackend/> http://monitor-development-east.test.com:5601/index.js?_b=3998:17727:1
>>
>> I am using 1.4.1.  Clearly kibana is not working.  Why?
>>
>>
>>
>> Here is my ES server
>> {
>>   "status" : 200,
>>   "name" : "Controller",
>>   "cluster_name" : "elasticsearch",
>>   "version" : {
>> "number" : "1.4.1",
>> "build_hash" : "89d3241d670db65f994242c8e8383b169779e2d4",
>> "build_timestamp" : "2014-11-26T15:49:29Z",
>> "build_snapshot" : false,
>> "lucene_version" : "4.10.2"
>>   },
>>   "tagline" : "You Know, for Search"
>> }
>>
>>
>> Thaks
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to elasticsearc...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/elasticsearch/CACF%2B8roTzNMtt5FJGQqGQSUQBaMXvVSPs4phpcLDCbM-Baek0g%40mail.gmail.com
>>  
>> <https://groups.google.com/d/msgid/elasticsearch/CACF%2B8roTzNMtt5FJGQqGQSUQBaMXvVSPs4phpcLDCbM-Baek0g%40mail.gmail.com?utm_medium=email&utm_source=footer>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/35e05f7f-4275-4a15-bfca-518afc4e1cc7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: This version of Kibana requires at least Elasticsearch 1.4.0.Beta1 but using 1.4.1

2014-12-03 Thread David Montgomery
PS

I installed 1.4.0.Beta1

{
  "status" : 200,
  "name" : "Discus",
  "cluster_name" : "elasticsearch",
  "version" : {
"number" : "1.4.0.Beta1",
"build_hash" : "1f25669f3299b0680b266c3acaece43774fb59ae",
"build_timestamp" : "2014-10-01T14:58:15Z",
"build_snapshot" : false,
"lucene_version" : "4.10.1"
  },
  "tagline" : "You Know, for Search"
}




Wow... get the same error.  Is kibana not stable?




On Wed, Dec 3, 2014 at 6:48 PM, David Montgomery 
wrote:

> Hi,
>
>
> This version of Kibana requires at least Elasticsearch 1.4.0.Beta1
>
> SetupError@
> http://monitor-development-east.test.com:5601/index.js?_b=3998:42905:51
> checkEsVersion/<@
> http://monitor-development-east.test.com:5601/index.js?_b=3998:43091:14
> qFactory/defer/deferred.promise.then/wrappedCallback@
> http://monitor-development-east.test.com:5601/index.js?_b=3998:20764:15
> qFactory/ref/<.then/<@
> http://monitor-development-east.test.com:5601/index.js?_b=3998:20850:11
> $RootScopeProvider/this.$get http://monitor-development-east.test.com:5601/index.js?_b=3998:21893:9
> $RootScopeProvider/this.$get http://monitor-development-east.test.com:5601/index.js?_b=3998:21705:15
> $RootScopeProvider/this.$get http://monitor-development-east.test.com:5601/index.js?_b=3998:21997:13
> done@
> http://monitor-development-east.test.com:5601/index.js?_b=3998:17570:34
> completeRequest@
> http://monitor-development-east.test.com:5601/index.js?_b=3998:17784:7
> createHttpBackend/ http://monitor-development-east.test.com:5601/index.js?_b=3998:17727:1
>
> I am using 1.4.1.  Clearly kibana is not working.  Why?
>
>
>
> Here is my ES server
> {
>   "status" : 200,
>   "name" : "Controller",
>   "cluster_name" : "elasticsearch",
>   "version" : {
> "number" : "1.4.1",
> "build_hash" : "89d3241d670db65f994242c8e8383b169779e2d4",
> "build_timestamp" : "2014-11-26T15:49:29Z",
> "build_snapshot" : false,
> "lucene_version" : "4.10.2"
>   },
>   "tagline" : "You Know, for Search"
> }
>
>
> Thaks
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CACF%2B8rrM_h6-WjLHMwHLdt78j71ZarhRj433jsvRkgtBkRXQZg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


This version of Kibana requires at least Elasticsearch 1.4.0.Beta1 but using 1.4.1

2014-12-03 Thread David Montgomery
Hi,


This version of Kibana requires at least Elasticsearch 1.4.0.Beta1

SetupError@
http://monitor-development-east.test.com:5601/index.js?_b=3998:42905:51
checkEsVersion/<@
http://monitor-development-east.test.com:5601/index.js?_b=3998:43091:14
qFactory/defer/deferred.promise.then/wrappedCallback@
http://monitor-development-east.test.com:5601/index.js?_b=3998:20764:15
qFactory/ref/<.then/<@
http://monitor-development-east.test.com:5601/index.js?_b=3998:20850:11
$RootScopeProvider/this.$gethttp://monitor-development-east.test.com:5601/index.js?_b=3998:21893:9
$RootScopeProvider/this.$gethttp://monitor-development-east.test.com:5601/index.js?_b=3998:21705:15
$RootScopeProvider/this.$gethttp://monitor-development-east.test.com:5601/index.js?_b=3998:21997:13
done@http://monitor-development-east.test.com:5601/index.js?_b=3998:17570:34
completeRequest@
http://monitor-development-east.test.com:5601/index.js?_b=3998:17784:7
createHttpBackend/http://monitor-development-east.test.com:5601/index.js?_b=3998:17727:1

I am using 1.4.1.  Clearly kibana is not working.  Why?



Here is my ES server
{
  "status" : 200,
  "name" : "Controller",
  "cluster_name" : "elasticsearch",
  "version" : {
"number" : "1.4.1",
"build_hash" : "89d3241d670db65f994242c8e8383b169779e2d4",
"build_timestamp" : "2014-11-26T15:49:29Z",
"build_snapshot" : false,
"lucene_version" : "4.10.2"
  },
  "tagline" : "You Know, for Search"
}


Thaks

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CACF%2B8roTzNMtt5FJGQqGQSUQBaMXvVSPs4phpcLDCbM-Baek0g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


How to do a rolling downgrade of servers

2014-10-20 Thread David Montgomery
 Hi,

I have 3 ES servers that are m1.large.  I need to change the machines types 
and reduce to 1 node.

The indexing and the shards were left at there default values and I am 
using unicast.


So  if I want to go to one node can I just stop 2 ES severs?

And on the last reaming node, that is e.g. m1.large, can  just add another 
1.medium node then when done syncing then stop the first m1.large?

I should have 1 m1.medium then

Thanks 

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/c0b52869-5758-499e-b401-cbbb763f778d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


ES reports 4 nodes but only 3 severs. One node is used twice

2014-08-01 Thread David Montgomery
Hi,

ES is showing 4 nodes but I only have 3.  How do I resolve?  One
address is used twice
172.31.6.204


{
  "ok" : true,
  "cluster_name" : "elasticsearch",
  "nodes" : {
"Fi5sM17uTku9LJYYDtKglQ" : {
  "name" : "Beautiful Dreamer",
  "transport_address" : "inet[/172.31.6.204:9301]",
  "hostname" : "ip-172-31-6-204",
  "version" : "0.90.5",
  "http_address" : "inet[/172.31.6.204:9201]"
},
"9K0XXGu1STaXfW0pupkbVg" : {
  "name" : "Beetle",
  "transport_address" : "inet[/172.31.6.93:9300]",
  "hostname" : "ip-172-31-6-93",
  "version" : "0.90.5",
  "http_address" : "inet[/172.31.6.93:9200]"
},
"HcTcSy1YTR22YkTVy9TrgA" : {
  "name" : "Arize",
  "transport_address" : "inet[/172.31.12.204:9300]",
  "hostname" : "ip-172-31-12-204",
  "version" : "0.90.5",
  "http_address" : "inet[/172.31.12.204:9200]"
},
"xz7aOp9zQcGinK5iJNrSYQ" : {
  "name" : "Obituary",
  "transport_address" : "inet[/172.31.6.204:9300]",
  "hostname" : "ip-172-31-6-204",
  "version" : "0.90.5",
  "http_address" : "inet[/172.31.6.204:9200]"
}
  }
}

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CACF%2B8rpzOsvWm_SkDyTSRf_O21T%2BnaWnYpGrvAzOSS8ENKH-QA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


Re: ES and logstash are nit working well with each other

2014-05-29 Thread David Montgomery
Before I give up on logstash..I now placed elastic search on the same
server using the below.


input {
  redis {
host => "redis.queue.do.development.sf.test.com"
data_type => "list"
key => "logstash"
codec => json
  }
}


output {
stdout { }
elasticsearch {
bind_host => "127.0.0.1"
}
}


I get this erorr   I am using 1.4.1for logstash and
0.90.9 for es.


'/usr/local/share/logstash-1.4.1/bin/logstash -f
/usr/local/share/logstash.indexer.config
Using milestone 2 input plugin 'redis'. This plugin should be stable, but
if you see strange behavior, please let us know! For more information on
plugin milestones, see
http://logstash.net/docs/1.4.1/plugin-milestones{:level=>:warn}
log4j, [2014-05-29T11:18:31.923]  WARN: org.elasticsearch.discovery:
[logstash-do-logstash-sf-development-20140527082230-16645-2010] waited for
30s and no initial state was set by the discovery
Exception in thread ">output"
org.elasticsearch.discovery.MasterNotDiscoveredException: waited for [30s]
at
org.elasticsearch.action.support.master.TransportMasterNodeOperationAction$3.onTimeout(org/elasticsearch/action/support/master/TransportMasterNodeOperationAction.java:180)
at
org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(org/elasticsearch/cluster/service/InternalClusterService.java:492)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(java/util/concurrent/ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(java/util/concurrent/ThreadPoolExecutor.java:615)
at java.lang.Thread.run(java/lang/Thread.java:744)















On Thu, May 29, 2014 at 1:15 PM, David Montgomery  wrote:

> PS...and how is this possible?  I feel so bad I bought the kindle
> logstash book:(
>
> I changed to host rather than bind host.  I mean..wow..I have ports open.
> See?  ufw satus ===> 9200:9400/tcp  ALLOW
> my.logstash.ipaddress
>
> I have ES running
> service elasticsearch status
>  * elasticsearch is running
>
>
>
>  /usr/local/share/logstash-1.4.1/bin/logstash -f
> /usr/local/share/logstash.indexer.configUsing milestone 2 input plugin
> 'redis'. This plugin should be stable, but if you see strange behavior,
> please let us know! For more information on plugin milestones, see
> http://logstash.net/docs/1.4.1/plugin-milestones {:level=>:warn}
> Exception in thread ">output"
> org.elasticsearch.transport.BindTransportException: Failed to bind to
> [9300-9400]
> at
> org.elasticsearch.transport.netty.NettyTransport.doStart(org/elasticsearch/transport/netty/NettyTransport.java:380)
> at
> org.elasticsearch.common.component.AbstractLifecycleComponent.start(org/elasticsearch/common/component/AbstractLifecycleComponent.java:85)
> at
> org.elasticsearch.transport.TransportService.doStart(org/elasticsearch/transport/TransportService.java:92)
> at
> org.elasticsearch.common.component.AbstractLifecycleComponent.start(org/elasticsearch/common/component/AbstractLifecycleComponent.java:85)
> at
> org.elasticsearch.node.internal.InternalNode.start(org/elasticsearch/node/internal/InternalNode.java:229)
> at
> org.elasticsearch.node.NodeBuilder.node(org/elasticsearch/node/NodeBuilder.java:166)
> at java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:606)
> at
> RUBY.build_client(/usr/local/share/logstash-1.4.1/lib/logstash/outputs/elasticsearch/protocol.rb:198)
> at
> RUBY.client(/usr/local/share/logstash-1.4.1/lib/logstash/outputs/elasticsearch/protocol.rb:15)
> at
> RUBY.initialize(/usr/local/share/logstash-1.4.1/lib/logstash/outputs/elasticsearch/protocol.rb:157)
> at
> RUBY.register(/usr/local/share/logstash-1.4.1/lib/logstash/outputs/elasticsearch.rb:250)
> at org.jruby.RubyArray.each(org/jruby/RubyArray.java:1613)
> at
> RUBY.outputworker(/usr/local/share/logstash-1.4.1/lib/logstash/pipeline.rb:220)
> at
> RUBY.start_outputs(/usr/local/share/logstash-1.4.1/lib/logstash/pipeline.rb:152)
> at java.lang.Thread.run(java/lang/Thread.java:744)
>
>
>
> On Thu, May 29, 2014 at 12:43 PM, David Montgomery <
> davidmontgom...@gmail.com> wrote:
>
>> Hi,
>>
>> I am, rather concerned that ES is not working with logstash index server.
>>
>> I start logstash index server like this:
>>
>> /usr/local/share/logstash-1.4.1/bin/logstash -f
>> /usr/local/share/logstash.indexer.config
>>
>> Using milestone 2 input plugin 'redis'. This plugin should be stable, but
>> if you see strange behavior, please let us know! For more information on
>> plugin milestones, see 
>> http://logstash

Re: ES and logstash are nit working well with each other

2014-05-28 Thread David Montgomery
PS...and how is this possible?  I feel so bad I bought the kindle
logstash book:(

I changed to host rather than bind host.  I mean..wow..I have ports open.
See?  ufw satus ===> 9200:9400/tcp  ALLOW
my.logstash.ipaddress

I have ES running
service elasticsearch status
 * elasticsearch is running



 /usr/local/share/logstash-1.4.1/bin/logstash -f
/usr/local/share/logstash.indexer.configUsing milestone 2 input plugin
'redis'. This plugin should be stable, but if you see strange behavior,
please let us know! For more information on plugin milestones, see
http://logstash.net/docs/1.4.1/plugin-milestones {:level=>:warn}
Exception in thread ">output"
org.elasticsearch.transport.BindTransportException: Failed to bind to
[9300-9400]
at
org.elasticsearch.transport.netty.NettyTransport.doStart(org/elasticsearch/transport/netty/NettyTransport.java:380)
at
org.elasticsearch.common.component.AbstractLifecycleComponent.start(org/elasticsearch/common/component/AbstractLifecycleComponent.java:85)
at
org.elasticsearch.transport.TransportService.doStart(org/elasticsearch/transport/TransportService.java:92)
at
org.elasticsearch.common.component.AbstractLifecycleComponent.start(org/elasticsearch/common/component/AbstractLifecycleComponent.java:85)
at
org.elasticsearch.node.internal.InternalNode.start(org/elasticsearch/node/internal/InternalNode.java:229)
at
org.elasticsearch.node.NodeBuilder.node(org/elasticsearch/node/NodeBuilder.java:166)
at java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:606)
at
RUBY.build_client(/usr/local/share/logstash-1.4.1/lib/logstash/outputs/elasticsearch/protocol.rb:198)
at
RUBY.client(/usr/local/share/logstash-1.4.1/lib/logstash/outputs/elasticsearch/protocol.rb:15)
at
RUBY.initialize(/usr/local/share/logstash-1.4.1/lib/logstash/outputs/elasticsearch/protocol.rb:157)
at
RUBY.register(/usr/local/share/logstash-1.4.1/lib/logstash/outputs/elasticsearch.rb:250)
at org.jruby.RubyArray.each(org/jruby/RubyArray.java:1613)
at
RUBY.outputworker(/usr/local/share/logstash-1.4.1/lib/logstash/pipeline.rb:220)
at
RUBY.start_outputs(/usr/local/share/logstash-1.4.1/lib/logstash/pipeline.rb:152)
at java.lang.Thread.run(java/lang/Thread.java:744)



On Thu, May 29, 2014 at 12:43 PM, David Montgomery <
davidmontgom...@gmail.com> wrote:

> Hi,
>
> I am, rather concerned that ES is not working with logstash index server.
>
> I start logstash index server like this:
>
> /usr/local/share/logstash-1.4.1/bin/logstash -f
> /usr/local/share/logstash.indexer.config
>
> Using milestone 2 input plugin 'redis'. This plugin should be stable, but
> if you see strange behavior, please let us know! For more information on
> plugin milestones, see 
> http://logstash.net/docs/1.4.1/plugin-milestones{:level=>:warn}
> log4j, [2014-05-29T00:33:12.473]  WARN: org.elasticsearch.discovery:
> [logstash-do-logstash-sf-development-20140527082230-2162-2010] waited for
> 30s and no initial state was set by the discovery
> Exception in thread ">output"
> org.elasticsearch.discovery.MasterNotDiscoveredException: waited for [30s]
> at
> org.elasticsearch.action.support.master.TransportMasterNodeOperationAction$3.onTimeout(org/elasticsearch/action/support/master/TransportMasterNodeOperationAction.java:180)
> at
> org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(org/elasticsearch/cluster/service/InternalClusterService.java:492)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(java/util/concurrent/ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(java/util/concurrent/ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(java/lang/Thread.java:744)
>
>
> Here is my config:
>
> input {
>   redis {
> host => "redis.queue.do.development.sf.test.com"
> data_type => "list"
> key => "logstash"
> codec => json
>   }
> }
>
>
> output {
> stdout { }
> elasticsearch {
> host => "eslogs.do.development.sf.test.com"
> }
> }
>
> Yes..ES us running and Yes ports are open for logstash
>
> So..whats the deal?
>
> Thanks
>
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CACF%2B8rpwG%3D3n3PPpsqBhsDFTm9t9tbMTkhwNSAx%2BuForbGHgrg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


ES and logstash are nit working well with each other

2014-05-28 Thread David Montgomery
Hi,

I am, rather concerned that ES is not working with logstash index server.

I start logstash index server like this:

/usr/local/share/logstash-1.4.1/bin/logstash -f
/usr/local/share/logstash.indexer.config

Using milestone 2 input plugin 'redis'. This plugin should be stable, but
if you see strange behavior, please let us know! For more information on
plugin milestones, see
http://logstash.net/docs/1.4.1/plugin-milestones{:level=>:warn}
log4j, [2014-05-29T00:33:12.473]  WARN: org.elasticsearch.discovery:
[logstash-do-logstash-sf-development-20140527082230-2162-2010] waited for
30s and no initial state was set by the discovery
Exception in thread ">output"
org.elasticsearch.discovery.MasterNotDiscoveredException: waited for [30s]
at
org.elasticsearch.action.support.master.TransportMasterNodeOperationAction$3.onTimeout(org/elasticsearch/action/support/master/TransportMasterNodeOperationAction.java:180)
at
org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(org/elasticsearch/cluster/service/InternalClusterService.java:492)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(java/util/concurrent/ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(java/util/concurrent/ThreadPoolExecutor.java:615)
at java.lang.Thread.run(java/lang/Thread.java:744)


Here is my config:

input {
  redis {
host => "redis.queue.do.development.sf.test.com"
data_type => "list"
key => "logstash"
codec => json
  }
}


output {
stdout { }
elasticsearch {
host => "eslogs.do.development.sf.test.com"
}
}

Yes..ES us running and Yes ports are open for logstash

So..whats the deal?

Thanks

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CACF%2B8rpLhKKFPv%2BZ_JpwiR%3DuNUJLrjm7PGQ0z6dR%2BF04dBytyQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


Re: Why will unicast will not work?

2014-02-19 Thread David Montgomery
sorry.  I changed to port 9300.  It worked!  Yeay.  Multicast did not work
though.  Now life is good:)


On Wed, Feb 19, 2014 at 4:41 PM, David Montgomery  wrote:

>  Why will unicast will not work?
>
>
> discovery.zen.ping.multicast.enabled: false
> discovery.zen.ping.unicast.hosts:
> ["198.199.94.xxx:9200","192.241.198.yyy:9200"]
>
> Both nodes have been restarted.  Both have the same config file.  Both can
> telent into each other.  Yet I get the below.  I am using version 1.0.0
>
> root@oXdevelopmentXsfXelasticsearchX11:~# curl -XGET '
> http://198.199.94.xxx:9200/_cluster/health?pretty=true'
> {
>   "cluster_name" : "test",
>   "status" : "green",
>   "timed_out" : false,
>   "number_of_nodes" : 1,
>   "number_of_data_nodes" : 1,
>   "active_primary_shards" : 0,
>   "active_shards" : 0,
>   "relocating_shards" : 0,
>   "initializing_shards" : 0,
>   "unassigned_shards" : 0
> }
>
> I am on digital ocean.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CACF%2B8rrxHst_HbP2bJksezANYOExAGCm96GF%2BtPUh-JH9Y9Ofg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


Why will unicast will not work?

2014-02-19 Thread David Montgomery
 Why will unicast will not work?


discovery.zen.ping.multicast.enabled: false
discovery.zen.ping.unicast.hosts:
["198.199.94.xxx:9200","192.241.198.yyy:9200"]

Both nodes have been restarted.  Both have the same config file.  Both can
telent into each other.  Yet I get the below.  I am using version 1.0.0

root@oXdevelopmentXsfXelasticsearchX11:~# curl -XGET '
http://198.199.94.xxx:9200/_cluster/health?pretty=true'
{
  "cluster_name" : "test",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 1,
  "number_of_data_nodes" : 1,
  "active_primary_shards" : 0,
  "active_shards" : 0,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0
}

I am on digital ocean.

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CACF%2B8rq5Nyq4%2B8ejE19m5vUiAwLECdfHmp8pzo2f_KRsHeshEg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


Re: cant create cluster on ec2

2014-02-03 Thread David Montgomery
Just tried 0.90.11.  Same issue.  I have two running nodes on ec2..and
wow...they cant talk.




On Tue, Feb 4, 2014 at 6:51 AM, David Montgomery
wrote:

> Hi Jonathan
>
> Just be clear...you could not cluster?  I wili try the lastest stable
> release cuase I am giving up on 1.0.0RC.
>
> Thanks
>
>
> On Sun, Feb 2, 2014 at 10:10 AM, Jonathan Goodwin <
> jonathanwgood...@gmail.com> wrote:
>
>> Hello David and David,
>>
>> I took a look at the logs / configs posted and mine are mostly similar
>> (though i'm not using any rivers) and I'm having the same issue.
>> I've tested on 0.90.9 and 0.90.7, I hadn't tried the 1.0.0RC yet because
>> the logstash website suggested I stick to 0.90.9
>> If you need any more logs / config samples I'd be happy to provide those.
>>
>> Best,
>> JG
>>
>> On Friday, January 31, 2014 3:46:07 PM UTC-5, David Montgomery wrote:
>>
>>> Hi,
>>>
>>> In chef I used -N flag which changes the host name when using
>>> chef-client.  I removed the flag and built again.  No difference.  I have
>>> two nodes running.  Both have the name cluster name, ports are 100% wide
>>> open...and wow..ES cant talk despite both running in the same region.
>>>
>>> So...ES...any suggestions?
>>>
>>> root@ip-10-238-131-236:/var/log/elasticsearch# hostname
>>> ip-10-238-131-236
>>>
>>>
>>> root@ip-10-238-131-236:/var/log/elasticsearch# curl -XGET '
>>> http://127.0.0.1:9200/_cluster/health?pretty=true'
>>> {
>>>   "cluster_name" : "mizilaelasticsearch",
>>>   "status" : "green",
>>>   "timed_out" : false,
>>>   "number_of_nodes" : 1,
>>>   "number_of_data_nodes" : 1,
>>>   "active_primary_shards" : 0,
>>>   "active_shards" : 0,
>>>   "relocating_shards" : 0,
>>>   "initializing_shards" : 0,
>>>   "unassigned_shards" : 0
>>> }
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Fri, Jan 31, 2014 at 12:20 AM, Ivan Brusic  wrote:
>>>
>>>> Did you override your network.host configuration parameter by any
>>>> chance?
>>>>
>>>> --
>>>> Ivan
>>>>
>>>>
>>>> On Wed, Jan 29, 2014 at 1:56 PM, David Montgomery >>> > wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> Details
>>>>>
>>>>> 1) both servers are sing the same sec group
>>>>> 2 all ports between 9200 and 9300 are open 0.0.0.0/0
>>>>> 3) ec2 instacances in us-east region
>>>>> 4) Did a fresh boot on 2 servers both servers are running.  Both are
>>>>> not talking
>>>>> 5) Attached are the logs for both servers and the common
>>>>> elasticsearch.yml file
>>>>> 6) Below is he chef recipe I wrote.  I am using 1.0.0.RC1 with
>>>>> elasticsearch-cloud-aws/2.0.0.RC1
>>>>> /usr/share/elasticsearch/plugins/cloud-aws
>>>>> ubuntu@ip-10-73-142-187:/usr/share/elasticsearch/plugins/cloud-aws$ ls
>>>>> aws-java-sdk-1.3.32.jar  commons-codec-1.4.jar
>>>>> commons-logging-1.1.1.jar  elasticsearch-cloud-aws-2.0.0.RC1.jar
>>>>> httpclient-4.1.1.jar  httpcore-4.1.jar
>>>>>
>>>>>
>>>>> data_bag("my_data_bag")
>>>>> db = data_bag_item("my_data_bag", "my")
>>>>> AWS_ACCESS_KEY_ID = db[node.chef_environment]['
>>>>> aws']['AWS_ACCESS_KEY_ID']
>>>>> AWS_SECRET_ACCESS_KEY = db[node.chef_environment]['
>>>>> aws']['AWS_SECRET_ACCESS_KEY']
>>>>>
>>>>>
>>>>> version = '1.0.0.RC1'
>>>>> remote_file 
>>>>> "#{Chef::Config[:file_cache_path]}/elasticsearch-#{version}.deb"
>>>>> do
>>>>>   source "https://download.elasticsearch.org/
>>>>> elasticsearch/elasticsearch/elasticsearch-#{version}.deb"
>>>>>   action :create_if_missing
>>>>> end
>>>>>
>>>>> dpkg_package 
>>>>> "#{Chef::Config[:file_cache_path]}/elasticsearch-#{version}.deb"
>>>>> do
>>>>>   action :install
>>>>> end
>>>>> bash "add_aw

Re: cant create cluster on ec2

2014-02-03 Thread David Montgomery
Hi Jonathan

Just be clear...you could not cluster?  I wili try the lastest stable
release cuase I am giving up on 1.0.0RC.

Thanks


On Sun, Feb 2, 2014 at 10:10 AM, Jonathan Goodwin <
jonathanwgood...@gmail.com> wrote:

> Hello David and David,
>
> I took a look at the logs / configs posted and mine are mostly similar
> (though i'm not using any rivers) and I'm having the same issue.
> I've tested on 0.90.9 and 0.90.7, I hadn't tried the 1.0.0RC yet because
> the logstash website suggested I stick to 0.90.9
> If you need any more logs / config samples I'd be happy to provide those.
>
> Best,
> JG
>
> On Friday, January 31, 2014 3:46:07 PM UTC-5, David Montgomery wrote:
>
>> Hi,
>>
>> In chef I used -N flag which changes the host name when using
>> chef-client.  I removed the flag and built again.  No difference.  I have
>> two nodes running.  Both have the name cluster name, ports are 100% wide
>> open...and wow..ES cant talk despite both running in the same region.
>>
>> So...ES...any suggestions?
>>
>> root@ip-10-238-131-236:/var/log/elasticsearch# hostname
>> ip-10-238-131-236
>>
>>
>> root@ip-10-238-131-236:/var/log/elasticsearch# curl -XGET '
>> http://127.0.0.1:9200/_cluster/health?pretty=true'
>> {
>>   "cluster_name" : "mizilaelasticsearch",
>>   "status" : "green",
>>   "timed_out" : false,
>>   "number_of_nodes" : 1,
>>   "number_of_data_nodes" : 1,
>>   "active_primary_shards" : 0,
>>   "active_shards" : 0,
>>   "relocating_shards" : 0,
>>   "initializing_shards" : 0,
>>   "unassigned_shards" : 0
>> }
>>
>>
>>
>>
>>
>>
>> On Fri, Jan 31, 2014 at 12:20 AM, Ivan Brusic  wrote:
>>
>>> Did you override your network.host configuration parameter by any chance?
>>>
>>> --
>>> Ivan
>>>
>>>
>>> On Wed, Jan 29, 2014 at 1:56 PM, David Montgomery 
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> Details
>>>>
>>>> 1) both servers are sing the same sec group
>>>> 2 all ports between 9200 and 9300 are open 0.0.0.0/0
>>>> 3) ec2 instacances in us-east region
>>>> 4) Did a fresh boot on 2 servers both servers are running.  Both are
>>>> not talking
>>>> 5) Attached are the logs for both servers and the common
>>>> elasticsearch.yml file
>>>> 6) Below is he chef recipe I wrote.  I am using 1.0.0.RC1 with
>>>> elasticsearch-cloud-aws/2.0.0.RC1
>>>> /usr/share/elasticsearch/plugins/cloud-aws
>>>> ubuntu@ip-10-73-142-187:/usr/share/elasticsearch/plugins/cloud-aws$ ls
>>>> aws-java-sdk-1.3.32.jar  commons-codec-1.4.jar
>>>> commons-logging-1.1.1.jar  elasticsearch-cloud-aws-2.0.0.RC1.jar
>>>> httpclient-4.1.1.jar  httpcore-4.1.jar
>>>>
>>>>
>>>> data_bag("my_data_bag")
>>>> db = data_bag_item("my_data_bag", "my")
>>>> AWS_ACCESS_KEY_ID = db[node.chef_environment]['
>>>> aws']['AWS_ACCESS_KEY_ID']
>>>> AWS_SECRET_ACCESS_KEY = db[node.chef_environment]['
>>>> aws']['AWS_SECRET_ACCESS_KEY']
>>>>
>>>>
>>>> version = '1.0.0.RC1'
>>>> remote_file 
>>>> "#{Chef::Config[:file_cache_path]}/elasticsearch-#{version}.deb"
>>>> do
>>>>   source "https://download.elasticsearch.org/
>>>> elasticsearch/elasticsearch/elasticsearch-#{version}.deb"
>>>>   action :create_if_missing
>>>> end
>>>>
>>>> dpkg_package 
>>>> "#{Chef::Config[:file_cache_path]}/elasticsearch-#{version}.deb"
>>>> do
>>>>   action :install
>>>> end
>>>> bash "add_aws" do
>>>>   cwd "/usr/share/elasticsearch"
>>>>   code <<-EOH
>>>> git clone https://github.com/elasticsearch/elasticsearch-
>>>> cloud-aws.git
>>>> /usr/share/elasticsearch/bin/plugin -install
>>>> elasticsearch/elasticsearch-cloud-aws/2.0.0.RC1
>>>>   EOH
>>>>   not_if {File.exists?("/usr/share/elasticsearch/elasticsearch-
>>>> cloud-aws")}
>>>> end
>>>>
>>>> service "elasticsearch" do
>>>>   supports :resta

Re: cant create cluster on ec2

2014-02-02 Thread David Montgomery
 ] [Hurricane] 
closed
[2014-01-27 19:54:56,193][INFO ][node ] [Cerise] 
version[1.0.0.RC1], pid[21694], build[c6155c5/2014-01-15T17:02:32Z]
[2014-01-27 19:54:56,193][INFO ][node ] [Cerise] 
initializing ...
[2014-01-27 19:54:56,208][INFO ][plugins  ] [Cerise] loaded 
[cloud-aws], sites []
[2014-01-27 19:55:01,703][DEBUG][discovery.zen.ping.multicast] [Cerise] 
using group [224.2.2.4], with port [54328], ttl [3], and address [null]
[2014-01-27 19:55:01,738][DEBUG][discovery.zen.ping.unicast] [Cerise] using 
initial hosts [], with concurrent_connects [10]
[2014-01-27 19:55:01,739][DEBUG][discovery.zen] [Cerise] using 
ping.timeout [3s], master_election.filter_client [true], 
master_election.filter_data [false]
[2014-01-27 19:55:01,740][DEBUG][discovery.zen.elect  ] [Cerise] using 
minimum_master_nodes [-1]
[2014-01-27 19:55:01,741][DEBUG][discovery.zen.fd ] [Cerise] 
[master] uses ping_interval [1s], ping_timeout [30s], ping_retries [3]
[2014-01-27 19:55:01,752][DEBUG][discovery.zen.fd ] [Cerise] [node  
] uses ping_interval [1s], ping_timeout [30s], ping_retries [3]
[2014-01-27 19:55:04,736][INFO ][node ] [Cerise] 
initialized
[2014-01-27 19:55:04,736][INFO ][node ] [Cerise] 
starting ...
[2014-01-27 19:55:04,981][INFO ][transport] [Cerise] 
bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address 
{inet[/10.208.51.152:9300]}
[2014-01-27 19:55:05,003][TRACE][discovery] [Cerise] 
waiting for 30s for the initial state to be set by the discovery
[2014-01-27 19:55:05,105][TRACE][discovery.zen.ping.multicast] [Cerise] [1] 
sending ping request
[2014-01-27 19:55:06,608][TRACE][discovery.zen.ping.multicast] [Cerise] [1] 
sending ping request
[2014-01-27 19:55:08,256][TRACE][discovery.zen] [Cerise] full 
ping responses: {none}
[2014-01-27 19:55:08,257][DEBUG][discovery.zen] [Cerise] 
filtered ping responses: (filter_client[true], filter_data[false]) {none}
[2014-01-27 19:55:08,262][INFO ][cluster.service  ] [Cerise] 
new_master 
[Cerise][TOid02a4RMqcyHzEpJEuPg][aws-elasticsearch-east-development-20140127160434][inet[/10.208.51.152:9300]],
 
reason: zen-disco-j$
[2014-01-27 19:55:08,297][TRACE][discovery] [Cerise] 
initial state set from discovery












On Tuesday, January 28, 2014 3:53:50 AM UTC+8, David Montgomery wrote:
>
> I assume that means this in the logging.yml fike?
>   # discovery
>   discovery: TRACE
>
> I do have it on...what should I be looking for in the logs?
>
> On Tuesday, January 28, 2014 3:47:10 AM UTC+8, David Montgomery wrote:
>>
>> Hi,
>>
>> As an fyi..I can telnet into the other machine.  Its only ES that does 
>> not work despite following the online docs.  
>>
>> If ES aws auto discovery  does not work  l assuming assume that 
>> http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-discovery-zen.htmlwill
>>  work?
>>
>> Thanks
>>
>>
>>
>>
>> On Tuesday, January 28, 2014 1:46:20 AM UTC+8, David Pilato wrote:
>>>
>>> Hi David,
>>>
>>>
>>> I think you should have nodes running under the same security group. In 
>>> that case, you don't have to open 9300-9400 ports to public.
>>> If they don't belong to the same group or if you want to reach them from 
>>> outside ec2 platform (public IP), then you need to set host_type to 
>>> public_ip.
>>>
>>> It sounds like your nodes are trying to connect with private ip here 
>>> (which is default).
>>>  
>>> My 2 cents
>>>
>>> -- 
>>> *David Pilato* | *Technical Advocate* | *Elasticsearch.com*
>>> @dadoonet <https://twitter.com/dadoonet> | 
>>> @elasticsearchfr<https://twitter.com/elasticsearchfr>
>>>
>>>
>>> Le 27 janvier 2014 at 18:07:48, David Montgomery (davidmo...@gmail.com) 
>>> a écrit:
>>>
>>> Hi,
>>>
>>> I have two nodes runnging on ec2. 
>>>
>>>  curl -X GET http://localhost:9200/_cluster/health?pretty=true
>>> {
>>>   "cluster_name" : "12345elasticsearch",
>>>   "status" : "green",
>>>   "timed_out" : false,
>>>   "number_of_nodes" : 1,
>>>   "number_of_data_nodes" : 1,
>>>   "active_primary_shards" : 0,
>>>   "active_shards" : 0,
>>>   "relocating_shards" : 0,
>>>   "initializing_shards" : 0,
>>>   "unassigned_shards" : 0
>>> }
>>>
>>> So..its fair to say that ec2 clu

Re: cant create cluster on ec2

2014-01-31 Thread David Montgomery
Hi,

In chef I used -N flag which changes the host name when using chef-client.
I removed the flag and built again.  No difference.  I have two nodes
running.  Both have the name cluster name, ports are 100% wide open...and
wow..ES cant talk despite both running in the same region.

So...ES...any suggestions?

root@ip-10-238-131-236:/var/log/elasticsearch# hostname
ip-10-238-131-236


root@ip-10-238-131-236:/var/log/elasticsearch# curl -XGET '
http://127.0.0.1:9200/_cluster/health?pretty=true'
{
  "cluster_name" : "mizilaelasticsearch",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 1,
  "number_of_data_nodes" : 1,
  "active_primary_shards" : 0,
  "active_shards" : 0,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0
}






On Fri, Jan 31, 2014 at 12:20 AM, Ivan Brusic  wrote:

> Did you override your network.host configuration parameter by any chance?
>
> --
> Ivan
>
>
> On Wed, Jan 29, 2014 at 1:56 PM, David Montgomery <
> davidmontgom...@gmail.com> wrote:
>
>> Hi,
>>
>> Details
>>
>> 1) both servers are sing the same sec group
>> 2 all ports between 9200 and 9300 are open 0.0.0.0/0
>> 3) ec2 instacances in us-east region
>> 4) Did a fresh boot on 2 servers both servers are running.  Both are not
>> talking
>> 5) Attached are the logs for both servers and the common
>> elasticsearch.yml file
>> 6) Below is he chef recipe I wrote.  I am using 1.0.0.RC1 with
>> elasticsearch-cloud-aws/2.0.0.RC1
>> /usr/share/elasticsearch/plugins/cloud-aws
>> ubuntu@ip-10-73-142-187:/usr/share/elasticsearch/plugins/cloud-aws$ ls
>> aws-java-sdk-1.3.32.jar  commons-codec-1.4.jar
>> commons-logging-1.1.1.jar  elasticsearch-cloud-aws-2.0.0.RC1.jar
>> httpclient-4.1.1.jar  httpcore-4.1.jar
>>
>>
>> data_bag("my_data_bag")
>> db = data_bag_item("my_data_bag", "my")
>> AWS_ACCESS_KEY_ID = db[node.chef_environment]['aws']['AWS_ACCESS_KEY_ID']
>> AWS_SECRET_ACCESS_KEY =
>> db[node.chef_environment]['aws']['AWS_SECRET_ACCESS_KEY']
>>
>>
>> version = '1.0.0.RC1'
>> remote_file
>> "#{Chef::Config[:file_cache_path]}/elasticsearch-#{version}.deb" do
>>   source "
>> https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-#{version}.deb
>> "
>>   action :create_if_missing
>> end
>>
>> dpkg_package
>> "#{Chef::Config[:file_cache_path]}/elasticsearch-#{version}.deb" do
>>   action :install
>> end
>> bash "add_aws" do
>>   cwd "/usr/share/elasticsearch"
>>   code <<-EOH
>> git clone
>> https://github.com/elasticsearch/elasticsearch-cloud-aws.git
>> /usr/share/elasticsearch/bin/plugin -install
>> elasticsearch/elasticsearch-cloud-aws/2.0.0.RC1
>>   EOH
>>   not_if
>> {File.exists?("/usr/share/elasticsearch/elasticsearch-cloud-aws")}
>> end
>>
>> service "elasticsearch" do
>>   supports :restart => true, :start => true, :stop => true
>>   action [ :enable, :start]
>> end
>>
>> template "/etc/elasticsearch/logging.yml" do
>>   path "/etc/elasticsearch/logging.yml"
>>   source "logging.yml.erb"
>>   owner "root"
>>   group "root"
>>   mode "0755"
>>   notifies :restart, resources(:service => "elasticsearch")
>> end
>>
>> template "/etc/elasticsearch/elasticsearch.yml" do
>>   path "/etc/elasticsearch/elasticsearch.yml"
>>   source "elasticsearch.yml.erb"
>>   owner "root"
>>   group "root"
>>   mode "0755"
>>   variables :AWS_ACCESS_KEY_ID => "#{AWS_ACCESS_KEY_ID}",
>> :AWS_SECRET_ACCESS_KEY => "#{AWS_SECRET_ACCESS_KEY}", :region => "us-east-1"
>>   notifies :restart, resources(:service => "elasticsearch")
>> end
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> On Tue, Jan 28, 2014 at 6:40 AM, David Montgomery <
>> davidmontgom...@gmail.com> wrote:
>>
>>> sure..let me the logs together and will post two files ..one log file
>>> for each of the servers.
>>>
>>> Thanks
>>>
>>>
>>> On Tue, Jan 28, 2014 at 6:25 AM, David Pilato  wrote:
>>>
>>>

Re: cant create cluster on ec2

2014-01-27 Thread David Montgomery
sure..let me the logs together and will post two files ..one log file for
each of the servers.

Thanks


On Tue, Jan 28, 2014 at 6:25 AM, David Pilato  wrote:

> Is it a known problem? Not yet. That's the reason I'm asking for logs. I
> would like to understand what is happening here. So full logs on both nodes
> would help.
>
> The way you set unicast is fine to me. But please, could you share with me
> (even through Direct mail) your elasticsearch.yml file and your logs in
> TRACE level?
> You can replace your credentials settings with . Please don't
> change anything else. I'd like to reproduce your case.
>
> Thanks!
>
>
> --
> David ;-)
> Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
>
>
> Le 27 janv. 2014 à 22:57, David Montgomery  a
> écrit :
>
> I have to say..it this a know problem?
>
> I removed the keys.
>
> So.. should the servers still cluster?  Wellits not.again
> given that the two machine can talk to each other via telent, in the same
> region, same sec group restarted services etcno go..no cluster
>
>
> Sohow do I manually configure nodes?  I have given up on auto
> discover.  Clearly does now work for me.
>
> Is the below an example?
>
> discovery.zen.ping.multicast.enabled: false
> discovery.zen.ping.unicast.hosts: ["0.test.com:9200", "1.test.com:9200"]
>
>
> Thanks
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Tuesday, January 28, 2014 4:19:25 AM UTC+8, David Pilato wrote:
>>
>> Yes. Could you gist it (and remove first keys / secrets of course)?
>>
>> --
>> David ;-)
>> Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
>>
>>
>> Le 27 janv. 2014 à 20:53, David Montgomery  a
>> écrit :
>>
>> I assume that means this in the logging.yml fike?
>>   # discovery
>>   discovery: TRACE
>>
>> I do have it on...what should I be looking for in the logs?
>>
>> On Tuesday, January 28, 2014 3:47:10 AM UTC+8, David Montgomery wrote:
>>>
>>> Hi,
>>>
>>> As an fyi..I can telnet into the other machine.  Its only ES that does
>>> not work despite following the online docs.
>>>
>>> If ES aws auto discovery  does not work  l assuming assume that
>>> http://www.elasticsearch.org/guide/en/elasticsearch/
>>> reference/current/modules-discovery-zen.html will work?
>>>
>>> Thanks
>>>
>>>
>>>
>>>
>>> On Tuesday, January 28, 2014 1:46:20 AM UTC+8, David Pilato wrote:
>>>>
>>>> Hi David,
>>>>
>>>>
>>>> I think you should have nodes running under the same security group. In
>>>> that case, you don't have to open 9300-9400 ports to public.
>>>> If they don't belong to the same group or if you want to reach them
>>>> from outside ec2 platform (public IP), then you need to set host_type
>>>> to public_ip.
>>>>
>>>> It sounds like your nodes are trying to connect with private ip here
>>>> (which is default).
>>>>
>>>> My 2 cents
>>>>
>>>> --
>>>> *David Pilato* | *Technical Advocate* | *Elasticsearch.com
>>>> <http://Elasticsearch.com>*
>>>> @dadoonet <https://twitter.com/dadoonet> | 
>>>> @elasticsearchfr<https://twitter.com/elasticsearchfr>
>>>>
>>>>
>>>> Le 27 janvier 2014 at 18:07:48, David Montgomery (davidmo...@gmail.com)
>>>> a écrit:
>>>>
>>>> Hi,
>>>>
>>>> I have two nodes runnging on ec2.
>>>>
>>>>  curl -X GET http://localhost:9200/_cluster/health?pretty=true
>>>> {
>>>>   "cluster_name" : "12345elasticsearch",
>>>>   "status" : "green",
>>>>   "timed_out" : false,
>>>>   "number_of_nodes" : 1,
>>>>   "number_of_data_nodes" : 1,
>>>>   "active_primary_shards" : 0,
>>>>   "active_shards" : 0,
>>>>   "relocating_shards" : 0,
>>>>   "initializing_shards" : 0,
>>>>   "unassigned_shards" : 0
>>>> }
>>>>
>>>> So..its fair to say that ec2 clustering did not work given the below
>>>> config.  Yes...the aws keys are correct and I have port 9200-9400 open.
>>>> What elasticsearch advertiseres is that all t

Re: cant create cluster on ec2

2014-01-27 Thread David Montgomery
I have to say..it this a know problem? 

I removed the keys.

So.. should the servers still cluster?  Wellits not.again given 
that the two machine can talk to each other via telent, in the same region, 
same sec group restarted services etcno go..no cluster 


Sohow do I manually configure nodes?  I have given up on auto 
discover.  Clearly does now work for me.  

Is the below an example?

discovery.zen.ping.multicast.enabled: false
discovery.zen.ping.unicast.hosts: ["0.test.com:9200", "1.test.com:9200"]


Thanks
























On Tuesday, January 28, 2014 4:19:25 AM UTC+8, David Pilato wrote:
>
> Yes. Could you gist it (and remove first keys / secrets of course)?
>
> --
> David ;-)
> Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
>
>
> Le 27 janv. 2014 à 20:53, David Montgomery 
> > 
> a écrit :
>
> I assume that means this in the logging.yml fike?
>   # discovery
>   discovery: TRACE
>
> I do have it on...what should I be looking for in the logs?
>
> On Tuesday, January 28, 2014 3:47:10 AM UTC+8, David Montgomery wrote:
>>
>> Hi,
>>
>> As an fyi..I can telnet into the other machine.  Its only ES that does 
>> not work despite following the online docs.  
>>
>> If ES aws auto discovery  does not work  l assuming assume that 
>> http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-discovery-zen.htmlwill
>>  work?
>>
>> Thanks
>>
>>
>>
>>
>> On Tuesday, January 28, 2014 1:46:20 AM UTC+8, David Pilato wrote:
>>>
>>> Hi David,
>>>
>>>
>>> I think you should have nodes running under the same security group. In 
>>> that case, you don't have to open 9300-9400 ports to public.
>>> If they don't belong to the same group or if you want to reach them from 
>>> outside ec2 platform (public IP), then you need to set host_type to 
>>> public_ip.
>>>
>>> It sounds like your nodes are trying to connect with private ip here 
>>> (which is default).
>>>  
>>> My 2 cents
>>>
>>> -- 
>>> *David Pilato* | *Technical Advocate* | *Elasticsearch.com 
>>> <http://Elasticsearch.com>*
>>> @dadoonet <https://twitter.com/dadoonet> | 
>>> @elasticsearchfr<https://twitter.com/elasticsearchfr>
>>>
>>>
>>> Le 27 janvier 2014 at 18:07:48, David Montgomery (davidmo...@gmail.com) 
>>> a écrit:
>>>
>>> Hi,
>>>
>>> I have two nodes runnging on ec2. 
>>>
>>>  curl -X GET http://localhost:9200/_cluster/health?pretty=true
>>> {
>>>   "cluster_name" : "12345elasticsearch",
>>>   "status" : "green",
>>>   "timed_out" : false,
>>>   "number_of_nodes" : 1,
>>>   "number_of_data_nodes" : 1,
>>>   "active_primary_shards" : 0,
>>>   "active_shards" : 0,
>>>   "relocating_shards" : 0,
>>>   "initializing_shards" : 0,
>>>   "unassigned_shards" : 0
>>> }
>>>
>>> So..its fair to say that ec2 clustering did not work given the below 
>>> config.  Yes...the aws keys are correct and I have port 9200-9400 open.  
>>> What elasticsearch advertiseres is that all that is needed. 
>>>
>>> I am using 1.0.0.RC1 with elasticsearch-cloud-aws= 2.0.0.RC1 on ubuntu 
>>> 12.04
>>>
>>>
>>> cloud.aws.access_key: <%=@AWS_ACCESS_KEY_ID%>
>>> cloud.aws.secret_key: <%=@AWS_SECRET_ACCESS_KEY%>
>>> discovery.type: ec2
>>>
>>>
>>>
>>> In the logs I see the below:
>>>
>>> [2014-01-27 16:58:49,708][TRACE][discovery.zen.ping.unicast] 
>>> [Administrator] [1] failed to connect to 
>>> [#cloud-i-1079ac6b-0][aws-elasticsearch-east-development-20140127160434][inet[/
>>> 10.211.29.210:9300]]
>>> org.elasticsearch.transport.ConnectTransportException: [][inet[/
>>> 10.211.29.210:9300]] connect_timeout[30s]
>>> at 
>>> org.elasticsearch.transport.netty.NettyTransport.connectToChannelsLight(NettyTransport.java:676)
>>> at 
>>> org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:636)
>>> at 
>>> org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:603)
>>> at 
>>> org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:

Re: cant create cluster on ec2

2014-01-27 Thread David Montgomery
I assume that means this in the logging.yml fike?
  # discovery
  discovery: TRACE

I do have it on...what should I be looking for in the logs?

On Tuesday, January 28, 2014 3:47:10 AM UTC+8, David Montgomery wrote:
>
> Hi,
>
> As an fyi..I can telnet into the other machine.  Its only ES that does not 
> work despite following the online docs.  
>
> If ES aws auto discovery  does not work  l assuming assume that 
> http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-discovery-zen.htmlwill
>  work?
>
> Thanks
>
>
>
>
> On Tuesday, January 28, 2014 1:46:20 AM UTC+8, David Pilato wrote:
>>
>> Hi David,
>>
>>
>> I think you should have nodes running under the same security group. In 
>> that case, you don't have to open 9300-9400 ports to public.
>> If they don't belong to the same group or if you want to reach them from 
>> outside ec2 platform (public IP), then you need to set host_type to 
>> public_ip.
>>
>> It sounds like your nodes are trying to connect with private ip here 
>> (which is default).
>>  
>> My 2 cents
>>
>> -- 
>> *David Pilato* | *Technical Advocate* | *Elasticsearch.com*
>> @dadoonet <https://twitter.com/dadoonet> | 
>> @elasticsearchfr<https://twitter.com/elasticsearchfr>
>>
>>
>> Le 27 janvier 2014 at 18:07:48, David Montgomery (davidmo...@gmail.com) 
>> a écrit:
>>
>> Hi,
>>
>> I have two nodes runnging on ec2. 
>>
>>  curl -X GET http://localhost:9200/_cluster/health?pretty=true
>> {
>>   "cluster_name" : "12345elasticsearch",
>>   "status" : "green",
>>   "timed_out" : false,
>>   "number_of_nodes" : 1,
>>   "number_of_data_nodes" : 1,
>>   "active_primary_shards" : 0,
>>   "active_shards" : 0,
>>   "relocating_shards" : 0,
>>   "initializing_shards" : 0,
>>   "unassigned_shards" : 0
>> }
>>
>> So..its fair to say that ec2 clustering did not work given the below 
>> config.  Yes...the aws keys are correct and I have port 9200-9400 open.  
>> What elasticsearch advertiseres is that all that is needed. 
>>
>> I am using 1.0.0.RC1 with elasticsearch-cloud-aws= 2.0.0.RC1 on ubuntu 
>> 12.04
>>
>>
>> cloud.aws.access_key: <%=@AWS_ACCESS_KEY_ID%>
>> cloud.aws.secret_key: <%=@AWS_SECRET_ACCESS_KEY%>
>> discovery.type: ec2
>>
>>
>>
>> In the logs I see the below:
>>
>> [2014-01-27 16:58:49,708][TRACE][discovery.zen.ping.unicast] 
>> [Administrator] [1] failed to connect to 
>> [#cloud-i-1079ac6b-0][aws-elasticsearch-east-development-20140127160434][inet[/
>> 10.211.29.210:9300]]
>> org.elasticsearch.transport.ConnectTransportException: [][inet[/
>> 10.211.29.210:9300]] connect_timeout[30s]
>> at 
>> org.elasticsearch.transport.netty.NettyTransport.connectToChannelsLight(NettyTransport.java:676)
>> at 
>> org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:636)
>> at 
>> org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:603)
>> at 
>> org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:133)
>> at 
>> org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$3.run(UnicastZenPing.java:278)
>> at 
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
>> at 
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> at java.lang.Thread.run(Thread.java:701)
>> Caused by: 
>> org.elasticsearch.common.netty.channel.ConnectTimeoutException: connection 
>> timed out: /10.239.36.98:9300
>> at 
>> org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.processConnectTimeout(NioClientBoss.java:137)
>> at 
>> org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:83)
>> at 
>> org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318)
>> at 
>> org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42)
>> at 
>> org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
>> at 
>> org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
>> ... 3 mo

Re: cant create cluster on ec2

2014-01-27 Thread David Montgomery
Hi,

As an fyi..I can telnet into the other machine.  Its only ES that does not 
work despite following the online docs.  

If ES aws auto discovery  does not work  l assuming assume that 
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-discovery-zen.html
 
will work?

Thanks




On Tuesday, January 28, 2014 1:46:20 AM UTC+8, David Pilato wrote:
>
> Hi David,
>
>
> I think you should have nodes running under the same security group. In 
> that case, you don't have to open 9300-9400 ports to public.
> If they don't belong to the same group or if you want to reach them from 
> outside ec2 platform (public IP), then you need to set host_type to 
> public_ip.
>
> It sounds like your nodes are trying to connect with private ip here 
> (which is default).
>  
> My 2 cents
>
> -- 
> *David Pilato* | *Technical Advocate* | *Elasticsearch.com*
> @dadoonet <https://twitter.com/dadoonet> | 
> @elasticsearchfr<https://twitter.com/elasticsearchfr>
>
>
> Le 27 janvier 2014 at 18:07:48, David Montgomery 
> (davidmo...@gmail.com) 
> a écrit:
>
> Hi,
>
> I have two nodes runnging on ec2. 
>
>  curl -X GET http://localhost:9200/_cluster/health?pretty=true
> {
>   "cluster_name" : "12345elasticsearch",
>   "status" : "green",
>   "timed_out" : false,
>   "number_of_nodes" : 1,
>   "number_of_data_nodes" : 1,
>   "active_primary_shards" : 0,
>   "active_shards" : 0,
>   "relocating_shards" : 0,
>   "initializing_shards" : 0,
>   "unassigned_shards" : 0
> }
>
> So..its fair to say that ec2 clustering did not work given the below 
> config.  Yes...the aws keys are correct and I have port 9200-9400 open.  
> What elasticsearch advertiseres is that all that is needed. 
>
> I am using 1.0.0.RC1 with elasticsearch-cloud-aws= 2.0.0.RC1 on ubuntu 
> 12.04
>
>
> cloud.aws.access_key: <%=@AWS_ACCESS_KEY_ID%>
> cloud.aws.secret_key: <%=@AWS_SECRET_ACCESS_KEY%>
> discovery.type: ec2
>
>
>
> In the logs I see the below:
>
> [2014-01-27 16:58:49,708][TRACE][discovery.zen.ping.unicast] 
> [Administrator] [1] failed to connect to 
> [#cloud-i-1079ac6b-0][aws-elasticsearch-east-development-20140127160434][inet[/
> 10.211.29.210:9300]]
> org.elasticsearch.transport.ConnectTransportException: [][inet[/
> 10.211.29.210:9300]] connect_timeout[30s]
> at 
> org.elasticsearch.transport.netty.NettyTransport.connectToChannelsLight(NettyTransport.java:676)
> at 
> org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:636)
> at 
> org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:603)
> at 
> org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:133)
> at 
> org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$3.run(UnicastZenPing.java:278)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:701)
> Caused by: org.elasticsearch.common.netty.channel.ConnectTimeoutException: 
> connection timed out: /10.239.36.98:9300
> at 
> org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.processConnectTimeout(NioClientBoss.java:137)
> at 
> org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:83)
> at 
> org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318)
> at 
> org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42)
> at 
> org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
> at 
> org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
> ... 3 more
>
>
>
>
>
> Thanks
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> --
> You received this message because you are subscribed to the Google Groups 
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to elasticsearc...@googlegroups.com .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/elasticsearch/ec028f53-c6b2-48e1-b714-a6f1ac7ae1cd%40googlegroups.com
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/ea43ded6-cb4a-45d4-b120-980718476983%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


Re: cant create cluster on ec2

2014-01-27 Thread David Montgomery
Hi,

I have the same sec group.  Just staying that not having ports open is not
an issue.  ES is running on both nodes.  Both nodes cant talk.  Those fro
ES..anyu suggestions?  Do those from ES follow this group?

thanks


On Tue, Jan 28, 2014 at 1:46 AM, David Pilato  wrote:

> Hi David,
>
>
> I think you should have nodes running under the same security group. In
> that case, you don't have to open 9300-9400 ports to public.
> If they don't belong to the same group or if you want to reach them from
> outside ec2 platform (public IP), then you need to set host_type to
> public_ip.
>
> It sounds like your nodes are trying to connect with private ip here
> (which is default).
>
> My 2 cents
>
> --
> *David Pilato* | *Technical Advocate* | *Elasticsearch.com*
> @dadoonet <https://twitter.com/dadoonet> | 
> @elasticsearchfr<https://twitter.com/elasticsearchfr>
>
>
> Le 27 janvier 2014 at 18:07:48, David Montgomery (
> davidmontgom...@gmail.com) a écrit:
>
> Hi,
>
> I have two nodes runnging on ec2.
>
>  curl -X GET http://localhost:9200/_cluster/health?pretty=true
> {
>   "cluster_name" : "12345elasticsearch",
>   "status" : "green",
>   "timed_out" : false,
>   "number_of_nodes" : 1,
>   "number_of_data_nodes" : 1,
>   "active_primary_shards" : 0,
>   "active_shards" : 0,
>   "relocating_shards" : 0,
>   "initializing_shards" : 0,
>   "unassigned_shards" : 0
> }
>
> So..its fair to say that ec2 clustering did not work given the below
> config.  Yes...the aws keys are correct and I have port 9200-9400 open.
> What elasticsearch advertiseres is that all that is needed.
>
> I am using 1.0.0.RC1 with elasticsearch-cloud-aws= 2.0.0.RC1 on ubuntu
> 12.04
>
>
> cloud.aws.access_key: <%=@AWS_ACCESS_KEY_ID%>
> cloud.aws.secret_key: <%=@AWS_SECRET_ACCESS_KEY%>
> discovery.type: ec2
>
>
>
> In the logs I see the below:
>
> [2014-01-27 16:58:49,708][TRACE][discovery.zen.ping.unicast]
> [Administrator] [1] failed to connect to
> [#cloud-i-1079ac6b-0][aws-elasticsearch-east-development-20140127160434][inet[/
> 10.211.29.210:9300]]
> org.elasticsearch.transport.ConnectTransportException: [][inet[/
> 10.211.29.210:9300]] connect_timeout[30s]
> at
> org.elasticsearch.transport.netty.NettyTransport.connectToChannelsLight(NettyTransport.java:676)
> at
> org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:636)
> at
> org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:603)
> at
> org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:133)
> at
> org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$3.run(UnicastZenPing.java:278)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:701)
> Caused by: org.elasticsearch.common.netty.channel.ConnectTimeoutException:
> connection timed out: /10.239.36.98:9300
> at
> org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.processConnectTimeout(NioClientBoss.java:137)
> at
> org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:83)
> at
> org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318)
> at
> org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42)
> at
> org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
> at
> org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
> ... 3 more
>
>
>
>
>
> Thanks
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
>
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/ec028f53-c6b2-48e1-b714-a6f1ac7ae1cd%40googlegroups.com
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CACF%2B8rqw5_jPrdenE9crYHGZX-7QEtWDKnWhv0M6ch3durjO%2BQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


cant create cluster on ec2

2014-01-27 Thread David Montgomery
Hi,

I have two nodes runnging on ec2.  

 curl -X GET http://localhost:9200/_cluster/health?pretty=true
{
  "cluster_name" : "12345elasticsearch",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 1,
  "number_of_data_nodes" : 1,
  "active_primary_shards" : 0,
  "active_shards" : 0,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0
}

So..its fair to say that ec2 clustering did not work given the below 
config.  Yes...the aws keys are correct and I have port 9200-9400 open.  
What elasticsearch advertiseres is that all that is needed.  

I am using 1.0.0.RC1 with elasticsearch-cloud-aws= 2.0.0.RC1 on ubuntu 12.04


cloud.aws.access_key: <%=@AWS_ACCESS_KEY_ID%>
cloud.aws.secret_key: <%=@AWS_SECRET_ACCESS_KEY%>
discovery.type: ec2



In the logs I see the below:

[2014-01-27 16:58:49,708][TRACE][discovery.zen.ping.unicast] 
[Administrator] [1] failed to connect to 
[#cloud-i-1079ac6b-0][aws-elasticsearch-east-development-20140127160434][inet[/10.211.29.210:9300]]
org.elasticsearch.transport.ConnectTransportException: 
[][inet[/10.211.29.210:9300]] connect_timeout[30s]
at 
org.elasticsearch.transport.netty.NettyTransport.connectToChannelsLight(NettyTransport.java:676)
at 
org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:636)
at 
org.elasticsearch.transport.netty.NettyTransport.connectToNodeLight(NettyTransport.java:603)
at 
org.elasticsearch.transport.TransportService.connectToNodeLight(TransportService.java:133)
at 
org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$3.run(UnicastZenPing.java:278)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:701)
Caused by: org.elasticsearch.common.netty.channel.ConnectTimeoutException: 
connection timed out: /10.239.36.98:9300
at 
org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.processConnectTimeout(NioClientBoss.java:137)
at 
org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:83)
at 
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318)
at 
org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42)
at 
org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at 
org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
... 3 more





Thanks















-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/ec028f53-c6b2-48e1-b714-a6f1ac7ae1cd%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


Re: ES not working on ec2 with auto discover

2014-01-26 Thread David Montgomery
Hi,

Okproblem solvedwas not discovery.,  I was the attachment of 
the volume.  This brings up another issue.why did I not get an error?

logger:
  # log action execution errors for easier debugging
  action: DEBUG

I just removed this line form elasticsearch.yml: path.data: /data

I attachment is formated as mkfs.ext3 and the volume exists.

is not path.data: /data is method to specify where the data should be 
stored?


Thanks









On Monday, January 27, 2014 12:32:01 AM UTC+8, Karel Minařík wrote:
>
> > 4) I did not have a problem with ES when not using aws for auto 
> discovery.  Is there another way to cluster machines on aws?  Clearly there 
> is an issues somewhere.  If I have 2 or more machines..is there another 
> way? 
>
> The AWS discovery mechanism is alive and kicking in most cases. Based on 
> your description, you seem to not be able to *start* Elasticsearch with the 
> plugin installed. Is that the case? 
>
> For the AWS discovery to work, you have to open port 9300-9400 among the 
> Elasticsearch instances. Have you done so? 
>
> Best way how to achieve that in EC2 is to have a dedicated security group, 
> which opens these ports to "itself". Screenshot (of the old AWS console): 
> http://www.elasticsearch.org/content/themes/elasticsearch-org/images/content/chef-solo/create-security-group.png
>  
>
> Karel 
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/e636e0fd-406e-4286-8633-cf8db252be0c%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


Re: ES not working on ec2 with auto discover

2014-01-26 Thread David Montgomery
Hi,

1) the credentials are valid.  Its the same creds that I use attached a 
volume and create tags using the aws cookbook where the are stored in a 
data bag.

2) I am in the east region and using a m1.small

3) The file exists
sudo find / -name elasticsearch-cloud-aws-2.0.0.RC1.jar
/usr/share/elasticsearch/plugins/cloud-aws/elasticsearch-cloud-aws-2.0.0.RC1.jar

4) I did not have a problem with ES when not using aws for auto discovery.  
Is there another way to cluster machines on aws?  Clearly there is an 
issues somewhere.  If I have 2 or more machines..is there another way?

Thanks


On Sunday, January 26, 2014 5:38:30 PM UTC+8, Karel Minařík wrote:
>
>
> I use the 1.0.0.RC1 this is what I get.  ES still not working.  What is 
>> missing?  I installed elasticsearch-cloud-aws.   
>>
>
> I would double check, that you're using the matching versions of 
> Elasticsearch and the Cloud AWS plugin. Check the files actually installed, 
> you should see `elasticsearch-cloud-aws-2.0.0.RC1.jar` for 1.x there.
>
> Also, you can increase the logging for AWS related events to have more 
> visibility into the process: 
> https://github.com/elasticsearch/elasticsearch/blob/master/config/logging.yml#L8
>
> If the problems stay, can you provide info on which region you're using, 
> check if the credentials are valid, what type of instance you're using, how 
> do you set ES_HEAP_SIZE?
>
> Finally, in your Chef recipe, you don't have to clone the repository, 
> `bin/plugin -install elasticsearch/elasticsearch-cloud-aws/2.0.0.RC1` 
> should be enough...
>
> Karel
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/9fbbd022-a77c-4783-b21d-22b73dcad592%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


Re: ES not working on ec2 with auto discover

2014-01-25 Thread David Montgomery


Well.

I use the 1.0.0.RC1 this is what I get.  ES still not working.  What is 
missing?  I installed elasticsearch-cloud-aws.   

/var/log/elasticsearch# service elasticsearch start
 * Starting Elasticsearch Server
   ...done.
root@aws-elasticsearch-east-development:/var/log/elasticsearch# service 
elasticsearch status
 * elasticsearch is not running


this is what is in the log

[2014-01-25 15:28:59,363][INFO ][node ] [Novs] 
version[1.0.0.RC1], pid[18046], build[c6155c5/2014-01-15T17:02:32Z]
[2014-01-25 15:28:59,364][INFO ][node ] [Novs] 
initializing ...
[2014-01-25 15:28:59,452][INFO ][plugins  ] [Novs] loaded 
[cloud-aws], sites []
[2014-01-25 16:44:34,376][INFO ][node ] 
[Amatsu-Mikaboshi] version[1.0.0.RC1], pid[18564], 
build[c6155c5/2014-01-15T17:02:32Z]
[2014-01-25 16:44:34,377][INFO ][node ] 
[Amatsu-Mikaboshi] initializing ...
[2014-01-25 16:44:34,422][INFO ][plugins  ] 
[Amatsu-Mikaboshi] loaded [cloud-aws], sites []
[2014-01-25 16:54:00,859][INFO ][node ] [Bob] 
version[1.0.0.RC1], pid[18826], build[c6155c5/2014-01-15T17:02:32Z]
[2014-01-25 16:54:00,860][INFO ][node ] [Bob] 
initializing ...
[2014-01-25 16:54:00,904][INFO ][plugins  ] [Bob] loaded 
[cloud-aws], sites []


Here is the elasticserch.yml file

cluster.name: adfadfafafelasticsearch
path.data: /data/es/data
discovery.type: ec2
cloud.aws.access_key: <%=@AWS_ACCESS_KEY_ID %>
cloud.aws.secret_key: <%=@AWS_SECRET_ACCESS_KEY %>













On Saturday, January 25, 2014 2:34:35 PM UTC+8, David Pilato wrote:
>
> Aws plugin 2.0.0.RC1 works only with Elasticsearch 1.0.0.RC1 and >.
>
> Look at the README.
>
> --
> David ;-)
> Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
>
>
> Le 25 janv. 2014 à 03:37, David Montgomery 
> > 
> a écrit :
>
> Hi,
> Newbie for installing ES esp on ec2.
>
> I used the deb package to installing ubuntu 12.04
>
> anyway.
>
> here is the basics of how I install with my chef recipe
>
> version = '0.90.10'
> remote_file 
> "#{Chef::Config[:file_cache_path]}/elasticsearch-#{version}.deb" do
>   source "
> https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-#{version}.deb
> "
>   action :create_if_missing
> end
>
> dpkg_package 
> "#{Chef::Config[:file_cache_path]}/elasticsearch-#{version}.deb" do
>   action :install
> end
> bash "add_aws" do
>   cwd "/usr/share/elasticsearch"
>   code <<-EOH
> git clone https://github.com/elasticsearch/elasticsearch-cloud-aws.git
> bin/plugin -install elasticsearch/elasticsearch-cloud-aws/2.0.0.RC1
>   EOH
>   not_if {File.exists?("elasticsearch-cloud-aws")}
> end
>
>
> Here is my  log file additions 
> discovery.type: ec2
> cloud.aws.access_key: <%=@AWS_ACCESS_KEY_ID %>
> cloud.aws.secret_key: <%=@AWS_SECRET_ACCESS_KEY %>
>
>
>
>
> So...wow...ES will not work.
>
> this is what i get
>  sudo service elasticsearch status
>  * elasticsearch is not running
>
> I run for the command line this is what I get...
>
> /usr/share/elasticsearch/bin/elasticsearch -f
> log4j:WARN No appenders could be found for logger (node).
> log4j:WARN Please initialize the log4j system properly.
> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for 
> more info.
>
>
> I look at the logs..nothing that points to an error.
>
>
>
> So..how do I get auto discovery to work onb ec2?  
>
> Thanks
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>  -- 
> You received this message because you are subscribed to the Google Groups 
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to elasticsearc...@googlegroups.com .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/elasticsearch/a48c0e49-e956-402b-8ea3-cf7037dc9f0d%40googlegroups.com
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/8dfd0de7-53bd-49b6-87c9-3cc1abaeb654%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


ES not working on ec2 with auto discover

2014-01-24 Thread David Montgomery
Hi,
Newbie for installing ES esp on ec2.

I used the deb package to installing ubuntu 12.04

anyway.

here is the basics of how I install with my chef recipe

version = '0.90.10'
remote_file 
"#{Chef::Config[:file_cache_path]}/elasticsearch-#{version}.deb" do
  source 
"https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-#{version}.deb";
  action :create_if_missing
end

dpkg_package 
"#{Chef::Config[:file_cache_path]}/elasticsearch-#{version}.deb" do
  action :install
end
bash "add_aws" do
  cwd "/usr/share/elasticsearch"
  code <<-EOH
git clone https://github.com/elasticsearch/elasticsearch-cloud-aws.git
bin/plugin -install elasticsearch/elasticsearch-cloud-aws/2.0.0.RC1
  EOH
  not_if {File.exists?("elasticsearch-cloud-aws")}
end


Here is my  log file additions 
discovery.type: ec2
cloud.aws.access_key: <%=@AWS_ACCESS_KEY_ID %>
cloud.aws.secret_key: <%=@AWS_SECRET_ACCESS_KEY %>




So...wow...ES will not work.

this is what i get
 sudo service elasticsearch status
 * elasticsearch is not running

I run for the command line this is what I get...

/usr/share/elasticsearch/bin/elasticsearch -f
log4j:WARN No appenders could be found for logger (node).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for 
more info.


I look at the logs..nothing that points to an error.



So..how do I get auto discovery to work onb ec2?  

Thanks
















-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/a48c0e49-e956-402b-8ea3-cf7037dc9f0d%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.