Re: Heavy load on a small Elasticsearch cluster
What monitoring tool do you use? Try to reduce the frequency at least. >> >> The _nodes/stats?all url is VERY slow for an elasticsearch request, something like 1500 ms. Some tools like kopf poll it every 3 seconds. If your tool poll it too every minute, you can break something. The _nodes/stats/indices is the slowest sub part, and the most interesting sub part. Is it a regression in ES 1.3 ? I can't find any 1.1 ES in my network to bench the difference. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/62d15231-5ca6-49e1-90f9-f87b9ff76978%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: High cpu load but low memory usage.
Le vendredi 24 octobre 2014 10:43:21 UTC+2, Atrus a écrit : > > - There is 15 shards per index, is this too much or enough ? I've used the > default config. I know that this could be effect the load but dont know how > to figure out the exact number. > > It's a huge value. Shards can be split between nodes, do you target tu use 15 nodes? > - Is there any way to show the running queries ? something like mysql show > process list ? to show what queries have eat CPU alot. I have enable slow > log queries >1s but found nothing. > You can watch HTTP traffic, with pcap (I hack packetbeat, for that). It's from the outside, from the inside, use the hot thread. strace can help, too. > - Any suggestion is appreciate. > Do you poll the _nodes/stat url? a monitoring tool, or a web page like kopf? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/3fbc22df-c8d8-4472-8a1a-db90a130d795%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Heavy load on a small Elasticsearch cluster
Le vendredi 24 octobre 2014 19:59:01 UTC+2, Jörg Prante a écrit : > > You're doomed :) > > What monitoring tool do you use? Try to reduce the frequency at least. > > Jörg > > New Relic monitor the OS but don't touch ES. I used a patched version of Diamond. I unplugged it and I RTFM for specific frequency setup. Some browsers with kopf runnning. I can watch user agent, to know who is hurting the server. This tools was used for months, without breaking anything. I'm still suspicious about recovery status. M. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/0c979b7b-262b-4fdb-8a65-03396824c945%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Heavy load on a small Elasticsearch cluster
A simple cluster, 2 nodes, replica 1. Each node has 1.5Go RAM, 2 cores, SAS disks. With Elasticsearch 1.1 some deconnection appears, and some CPU load picks. Bad usage of logstash (lots of tiny bulk imports, with monthly indices). Logstash usage was fixed, and Elasticsearch upgraded to 1.3. 1.3.3, then 1.3.4, 10 minutes after. The CPU usage is now 100% (so one core used), LOTS of file descriptor opened, and memory usage is growing. RAM is upgraded to 2Go. Strace show that 5 threads use lots of CPU and 1 thread does 7000 stat()/s. Elasticsearch Hot thread show lots of FSDirectory.listAll. Disk usage is low, just a lots of stats. The shard is set to 9, and logstash opens lots of indices, 2286 shards for 7GB, 37487 files in the indices folder. In the recovery API, everything is "done" with strange percent score, all shards have "replica" states. Now, the load makes heavy waves, slowing the service. This is just a long migration from different version of Lucene (from ES 1.1 to 1.3), a misconfiguration, a real bug, or am I just doomed? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/2e7ad3ba-a1f6-44c8-b9e5-67b0c1ed8bc9%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
