This is caused by elasticsearch trying to load fielddata. Fielddata is used for sorting and faceting/aggregations. When a query has a sort parameter the node will try to load the fielddata for that field for all documents in the shard, not just those included in the query result. The breaker is tripped when ES estimates there is not enough heap available to load the fielddata so it just rejects the query rather than running the node out of heap space.
You should probably start by looking at the queries that are being run to determine what's triggering the error. To deal with it the options I'm aware of are to add heap space, more nodes or look at using doc_values to move fielddata off the heap. Kimbro On Wed, Oct 15, 2014 at 10:42 PM, Robin Clarke <robi...@gmail.com> wrote: > I'm still having this problem... has anybody got an idea what the cause / > solution might be? > > Thank you! :) > > On Tuesday, 7 October 2014 14:29:22 UTC+2, Robin Clarke wrote: >> >> I'm getting a lot of these errors in my Elasticsearch logs, and am also >> experiencing a lot of slowness on the cluster... >> >> New used memory 7670582710 [7.1gb] from field [machineName.raw] would be >> larger than configured breaker: 7666532352 [7.1gb], breaking >> ... >> New used memory 7674188379 [7.1gb] from field [@timestamp] would be >> larger than configured breaker: 7666532352 [7.1gb], breaking >> >> I've looked at the documentation about memory limits >> <http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/_limiting_memory_usage.html>, >> but I don't really understand what is causing this, and more importantly >> how to avoid this... >> >> My cluster is 10 machines @ 32GB memory and 8 CPU cores each. I have one >> ES node on each machine with 12GB memory allocated. On each machine there >> is additionally one logstash agent (1GB) and one redis server (2GB). >> I have 10 indexes open with one replication per shard (so each node >> should only be holding 22 shards (two more for kibana-int)). >> >> I'm using Elasticsearch 1.3.3, Logstash 1.4.2 >> >> Thanks for your help! >> >> -Robin- >> > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to elasticsearch+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/5935b1f4-809c-46ac-ba03-f1df33a8737e%40googlegroups.com > <https://groups.google.com/d/msgid/elasticsearch/5935b1f4-809c-46ac-ba03-f1df33a8737e%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAA0DmXZRMFsAMXCs9qmMk0KN%2B%2BuLh%3DCiEtP-r4vK3tZF0CRAmA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
