Hi, If I have calculated correctly, that corresponds to about 238TB of raw data. If this is the size of JSON documents being indexed in Elasticsearch, you will definitely need more than 2 nodes.
The good thing about using aliases the way David describes is that you will not need to put all users in the same index as the aliases hides the underlying index and makes that transparent. You can therefore e.g. put your first 100 customers in one index and then add new indices as the number of customers grow. This makes it easier to incrementally handle growth. Best regards, Christian On Thursday, April 23, 2015 at 8:30:52 AM UTC+1, Zaid Amir wrote: > > Hi, > > I am trying to figure out the best way to design my ES cluster. Currently > my search service is subscription based and each user can only search his > own data. > > So looking around I found several examples about users data flow and the > way of using aliases and its all straight forward. > > One thing that I am struggling to understand is the routing setup. Now > lets assume that I started an index named "accounts" with 100 primary > shards and 1 replica. Now users started subscribing so I start creating an > alias per user and routing each alias to a specific shard (1, 2, 3, ..., > 100). > > No if 100 users have already subscribed and a new user comes along, can I > route the new user to the first shard? Or should I start another index for > the next 100 users? > > My concern with this also is performance. Lets say that both nodes are > running on a Quad core CPU with 32 GB of ram? Is there a good indicator of > how many shards I should allocate per index assuming that each document is > around 512KB in size? > > > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a432734a-e840-4e24-9c7a-21f6efed8e97%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
