Thanks Adrien!
Very much appreciate your time and help.
Chris
On Mon, Aug 25, 2014 at 3:44 AM, Adrien Grand <
adrien.gr...@elasticsearch.com> wrote:
> I meant tens of shards per node. So if you have N nodes with I indices
> which have S shards and R replicas, that would be (I * S * (1 + R)) /
I meant tens of shards per node. So if you have N nodes with I indices
which have S shards and R replicas, that would be (I * S * (1 + R)) / N.
One shard per node is optimal but doesn't allows for growth: if you add one
more node, you cannot spread the indexing work load, that is why it is
common
Adrien,
Thanks so much for the response. It was very helpful. I will check out
those links on capacity planning for sure.
One followup question. You mention that tens of shards per node would be
ok. Are you meaning tens of shards from tens of indexes? Or tens of
shards for a single index? R
Hi Chris,
Usually, the problem is not that much in terms of indices but shards, which
are the physical units of data storage (an index being a logical view over
several shards).
Something to beware of is that shards typically have some constant overhead
(disk space, file descriptors, memory usage
Hi all,
As the subject says, I'm wondering about index size vs. number of indexes.
I'm indexing many application log files, currently with an index by day for
all logs, which will make a very large index. For just a few applications
in Development, the index is 55GB a day (across 2 servers). In