Maybe index throttling is happening (ES would say so in the logs) because your merging is falling behind? Do you throttle IO for merges (it's throttled at paltry 20 MB / sec by default)? What does hot threads report? How about top/iostat?
We just got a blog post out about improving indexing throughput: http://www.elasticsearch.org/blog/performance-considerations-elasticsearch-indexing Also, you should upgrade to 1.3.2, which fixes a rare but nasty corruption issue in the upstream compression library ES embeds. And make sure you use at least JDK 1.7.0_55. Mike McCandless http://blog.mikemccandless.com On Fri, Sep 5, 2014 at 4:44 AM, Thomas <thomas.bo...@gmail.com> wrote: > Hi, > > I have been performing indexing operations in my elasticsearch cluster for > some time now. Suddenly, I have been facing some latency while indexing and > I'm trying to find the reason for it. > > Details: > > I have a custom process which is uploading every interval a number of logs > with bulk API. This process was taking about 5-7 minutes every time. For > some reason, the last days I noticed that the exact same procedure, same > volumes, takes about 15-20 minutes. While manipulating the data I run > update operations through scripting (groovy). My cluster is a set of 5 > nodes, my first impression was that I need to scale therefore I added an > extra node. The problem seemed that it was solved but after a day again I > face the same issue. > > Is it possible to give some ideas about what to check, or what seems to be > the issue? How is possible to check if a background process is running or > creating any issues (expunge etc.)? Does anyone has any similar problems? > > Any help appreciated, let me know what info to share > > ES version is 1.3.1 > JDK is 1.7 > > Thanks > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to elasticsearch+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/fed6d402-d4bb-44ea-8de7-d66c2ec5cb91%40googlegroups.com > <https://groups.google.com/d/msgid/elasticsearch/fed6d402-d4bb-44ea-8de7-d66c2ec5cb91%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAD7smReHr4qGpv2WcsFesvTb%3DFjYJ2NWrHXttuaWmWWnhwP%3D8w%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
