Re: Master node keeps crashing

2014-06-10 Thread Gaurav Arora
The max file descriptors are all set to 64k. This is the output from one of the slave nodes - http://pastebin.com/RdmZsJbH On Tue, Jun 10, 2014 at 2:50 PM, sirkubax wrote: > curl -XGET 'http://localhost:9200/_nodes/_all/process?pretty=true' |less > > Have you been considering "max_file_descr

Re: Master node keeps crashing

2014-06-10 Thread sirkubax
curl -XGET 'http://localhost:9200/_nodes/_all/process?pretty=true' |less Have you been considering "max_file_descriptors"? W dniu wtorek, 10 czerwca 2014 09:36:35 UTC+2 użytkownik Gaurav Arora napisał: > > I am using the latest openjdk version 7 installed from ubuntu repos. > > ubuntu@es1:~$ ja

Re: Master node keeps crashing

2014-06-10 Thread Gaurav Arora
I am using the latest openjdk version 7 installed from ubuntu repos. ubuntu@es1:~$ java -version java version "1.7.0_51" OpenJDK Runtime Environment (IcedTea 2.4.6) (7u51-2.4.6-1ubuntu4) OpenJDK 64-Bit Server VM (build 24.51-b03, mixed mode) ES is set to run with -Xms14075m -Xmx14075m with bootst

Re: Master node keeps crashing

2014-06-10 Thread Mark Walkom
How much RAM per node, what java flavour and version, what ES version? Are the logs showing any OOM? Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com web: www.campaignmonitor.com On 10 June 2014 17:16, Gaurav Arora wrote: > I'm having a very odd

Master node keeps crashing

2014-06-10 Thread Gaurav Arora
I'm having a very odd problem with one of my elasticsearch clusters. The master node on the cluster crashes randomly. The cluster is running on 3 different ec2 instances. My cluster configuration is: 3 nodes (1 master, all data nodes) 600 GB of data (3k IOPS EBS volumes) 700 million documents T