Re: One large index vs. many smaller indexes

Thanks Adrien! Very much appreciate your time and help. Chris On Mon, Aug 25, 2014 at 3:44 AM, Adrien Grand adrien.gr...@elasticsearch.com wrote: I meant tens of shards per node. So if you have N nodes with I indices which have S shards and R replicas, that would be (I * S * (1 + R)) / N.

Re: One large index vs. many smaller indexes

Adrien, Thanks so much for the response. It was very helpful. I will check out those links on capacity planning for sure. One followup question. You mention that tens of shards per node would be ok. Are you meaning tens of shards from tens of indexes? Or tens of shards for a single index?

One large index vs. many smaller indexes

Hi all, As the subject says, I'm wondering about index size vs. number of indexes. I'm indexing many application log files, currently with an index by day for all logs, which will make a very large index. For just a few applications in Development, the index is 55GB a day (across 2 servers).

Re: One large index vs. many smaller indexes

Hi Chris, Usually, the problem is not that much in terms of indices but shards, which are the physical units of data storage (an index being a logical view over several shards). Something to beware of is that shards typically have some constant overhead (disk space, file descriptors, memory