> Anything in the logs or slow logs? You're sure slow GCs aren't impacting > performance? >
There's nothing in the logs on the node other slow queries before or during the problematic period. The slow query logs show the same types of queries that we know to be slow (MLT queries with function rescores) and are shown. As a percentage of queries executed the there is no bump in number of slow queried before or during the problematic period. (Yes during the problematic period the broken node actually executes and returns query requests, it's not clear to me if it's simply routing queries to other nodes or if it's shards are actually executing queries as well.) In addition before the problem occurs there is no increase in ES threads, heap or non heap memory use and the number of GC cycles remained consistent at about once every 2 seconds on the node. There are no long GC cycles and the node never drops from the cluster. During the problematic period the cluster reports that it's in a green state however all of our logging indicates that no indexing operations complete cluster wide (we should be seeing 100-500 / sec under normal load). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/da65980b-d027-4ec2-9576-1f51a3dc6037%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
