Improving Elasticsearch performance on a single node by increasing shards

2014-05-13 Thread Rujuta Deshpande
Hi, I have set up an elasticsearch cluster with 1 shard and 0 replicas. My system has 16 GB RAM and I have allocated 8 GB to the ES Max/Min Heap. We are indexing a large number of logs everyday and the size of our daily index is approximately 3,500,000 documents. We are using Kibana to

Re: Improving Elasticsearch performance on a single node by increasing shards

2014-05-13 Thread Rujuta Deshpande
with what you can do. You'd be better off adding another node if you can, maybe someone else can comment on the rest. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com javascript: web: www.campaignmonitor.com On 13 May 2014 16:34, Rujuta

Re: Elasticsearch configuration for uninterrupted indexing

2014-03-25 Thread Rujuta Deshpande
. Is the 2GB per JVM or the total of the machine? For analytic applications, with multiples facets, 2 GBs might not be sufficient. -- Ivan On Sun, Mar 23, 2014 at 10:04 PM, Rujuta Deshpande ruj...@gmail.comjavascript: wrote: Hi, Thank you for the response. However, in our scenario, both

Re: Elasticsearch configuration for uninterrupted indexing

2014-03-23 Thread Rujuta Deshpande
, it is not doing much since the reduce phase is basically a pass-thru operation. With a two node cluster, I would say you are better off having both machines act as full nodes. Cheers, Ivan On Fri, Mar 21, 2014 at 5:04 AM, Rujuta Deshpande ruj...@gmail.comjavascript: wrote: Hi, I am

Elasticsearch configuration for uninterrupted indexing

2014-03-21 Thread Rujuta Deshpande
Hi, I am setting up a system consisting of elasticsearch-logstash-kibana for log analysis. I am using one machine (2 GB RAM, 2 CPUs) running logstash, kibana and two instances of elasticsearch. Two other machines, each running logstash-forwarder are pumping logs into the ELK system. The