Thanks for the responses guys. We have an ES setup so that we have: hot, warm and cold ES nodes. Hot nodes are the only ones receiving realtime updates and have refresh intervals fairly low for indices there thus making a query cache pretty useless for data there.
Indices on warm nodes on the other hand are only updated every night and indices cold nodes are similar. Assuming we do have repetitive aggregation queries, sounds like bumping up query cache on warm/cold tier could have some significant speed ups for our more expensive aggregations. On Thursday, 21 May 2015 19:12:44 UTC-4, Adrien Grand wrote: > > On Thu, May 21, 2015 at 11:49 PM, James Macdonald < > james.m...@geofeedia.com <javascript:>> wrote: > >> Hi, I am a little confused by your response. Are you saying that >> query/filter caches are invalidated across all data in a shard every time >> the refresh interval ticks over? >> > > Sorry for the confusion: > - the query cache caches entire requests per index, and is competely > invalidated across all data every time the refresh interval ticks over AND > there have been changes since the last refresh > - the filter cache caches matching documents per segment, it is > invalidated per segment only when a segment goes away (typically because > it's been merged to a larger segment), which is unfrequent for large > segments > - the fielddata cache caches the document->value mapping per segment and > has the same invalidation rules as the filter cache > > >> I was under the impression that all field data and caching related >> operations were performed on a Lucene index segment level and that the >> caches would only be invalidated for a given segment if that segment had >> changed since the last refresh. Since most data is stored in large segments >> that don't take fresh writes and seldom merge this would mean that most >> caches are good for long periods of time; even if the shard is under >> constant indexing load. Am I mistaken? >> > > This is right for the fielddata and filter caches, but not for the query > cache. > > -- > Adrien > -- Please update your bookmarks! We have moved to https://discuss.elastic.co/ --- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/c9ca4e07-42c4-41d4-a919-50cb9c56f8eb%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
