Hey all, I have 30 machines to build my ES cluster. I am going to need to build 2000 indexes with 14400 shards total. Should I build two clusters or one?
I have a multi customer data set. 65% of the customers have very little data, "small customers". The other 35% have 100x more data, "big customers". My thought was to build one cluster where all the "small customers" share a set of 60 "monthly" indexes each with 100 shards. I would build a second cluster where each customer gets their own set of 60 "monthly" indexes each with 6 shards. This configuration is so that I can keep shard sizes below 5gb across both clusters. The small cluster is routed by customerId. The big cluster uses the default routing strategy. questions: 1) Should I make two clusters or just one? 2) Do I need to keep shard sizes below 5gb? 3) Is management of one cluster with 2000 indexes and 14400 shards more difficult than 2 clusters where "small" cluster has 60 indexes and 600 shards and "big" cluster has 1900 indexes and 13000 shards? Thanks, Brad -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/9e14329b-ebb3-4948-9fd3-eaf10cce7477%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
