Hey, are you using HTTP keep alive connections? If not consider switching to them, as reopening a new TCP connection not only results in high latencies but also removes file handle resources from the elasticsearch process (the number of open files). if your client/language does not support this, you should either use nginx or at least try to use bulk operations in order to create less TCP connections, however this is just a workaround.
--Alex On Sat, Jul 19, 2014 at 5:23 AM, Brian Jones <tbrianjo...@gmail.com> wrote: > I'm trying to scale my indexing for the first time, and I'm running into > connections problems. I reach a scale where cURL connections from my > indexers start getting cURL7 errors ( connect failed ). It looks like ES > just stops accepting all HTTP connections for a period of time. I cannot > find the root cause. > > I'm running on an Amazon C3.4XL. The processors are not maxed, memory is > not maxed, IO is not showing issues. I'm not seeing problems in the ES > log, but I'm not sure I have logging fully enabled. I've tried increasing > the thread_pool for the indexer, and that doesn't help. I'm not seeing any > rejected connections there. I'm at a loss. > > The closest I can get is a guess using data from Bigdesk. When the number > HTTP channels starts exceeding the number of transport channels, I start to > see the problem emerge. I have no idea if this is related, but it's the > only metric I've traced that seems correlated. > > Thoughts? > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to elasticsearch+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/0fd75b96-3c00-47f1-9a59-3c9707f38734%40googlegroups.com > <https://groups.google.com/d/msgid/elasticsearch/0fd75b96-3c00-47f1-9a59-3c9707f38734%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGCwEM-QZCWND%2B0FuYNEZdjK_5izSaZUMFmB_TyBTTuFSyZB3g%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
