There is a whole bunch of good stuff in the docs so I'd suggest you start there - http://www.elastic.co/guide/en/elasticsearch/reference/current/docs-index_.html#index-routing
Don't play with the hashing/sharding algorithm unless you know exactly what you are doing! On 20 March 2015 at 11:34, Vladi Feigin <vla...@liveperson.com> wrote: > Thank you Mark > Can you please elaborate regarding the routing? Are you meaning using > customer id as a routing value? > Can you give an example? Link? > Should I override the shard calculation function? > בתאריך 20 במרץ 2015 19:43, מאת "Mark Walkom" <markwal...@gmail.com>: > >> This is where you use routing and aliases. >> >> Use routing to send each customers documents to a specific shard, you can >> then query using the same routing value and reduce your exposure. Then use >> aliases so you can easily move larger customers out to their own index if >> need be. >> >> On 20 March 2015 at 09:53, Matías Waisgold <mwaisg...@gmail.com> wrote: >> >>> I had a project with the same context. We decided to increase the # of >>> shards as it was impossible to have one index for each customer. >>> Another approach is to have only some customers (hardcoded) separated >>> from the rest. If you can, in advance, detect this users it might be a good >>> idea and then have a "Rest of the world" index for non important ones. >>> >>> Also when we increased the # of shards, we incremented the amount of >>> servers but with smaller ones, that improved a lot our failure resiliency. >>> Hope that helps. >>> >>> >>> On Friday, March 20, 2015 at 5:28:55 PM UTC+1, Vladi Feigin wrote: >>>> >>>> Hello, >>>> >>>> Please share your thoughts >>>> We have one big ES index and 18 shards (9 primary and 9 replicas) >>>> We have thousands of customers and each customer could have millions or >>>> as opposite very small number of documents >>>> We never search across all customers but within a specific customer. In >>>> other words all our queries have a customer id filter. >>>> The big disadvantage of having one big index is we always search the >>>> data of all customers rather than looking in one customer >>>> Obviously it hurts our queries performance. >>>> We're thinking to create multiple indexes : an index per customer. But >>>> in our case it means having hundreds or maybe thousands indexes >>>> In terms of the maintenance is a big overhead >>>> Other approach is create many shards >>>> Could you, please share your experience and thoughts? >>>> What would you recommend in this scenario >>>> Thank you in advance, >>>> Vladi Feigin >>>> >>>> >>>> >>>> >>>> >>>> This message may contain confidential and/or privileged information. >>>> If you are not the addressee or authorized to receive this on behalf of >>>> the addressee you must not use, copy, disclose or take action based on this >>>> message or any information herein. >>>> If you have received this message in error, please advise the sender >>>> immediately by reply email and delete this message. Thank you. >>>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "elasticsearch" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to elasticsearch+unsubscr...@googlegroups.com. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/elasticsearch/974e95e4-8c25-4500-8823-853806bd5cbb%40googlegroups.com >>> <https://groups.google.com/d/msgid/elasticsearch/974e95e4-8c25-4500-8823-853806bd5cbb%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "elasticsearch" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/elasticsearch/OhfFUiygbMM/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> elasticsearch+unsubscr...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_qoqsjdsp8YyVgHH1ieOC8aTnqpvHSdL3WPMHsnCv7OA%40mail.gmail.com >> <https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_qoqsjdsp8YyVgHH1ieOC8aTnqpvHSdL3WPMHsnCv7OA%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> For more options, visit https://groups.google.com/d/optout. >> > > This message may contain confidential and/or privileged information. > If you are not the addressee or authorized to receive this on behalf of > the addressee you must not use, copy, disclose or take action based on this > message or any information herein. > If you have received this message in error, please advise the sender > immediately by reply email and delete this message. Thank you. > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to elasticsearch+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/CACvWdiogwAwtJFQBpXF9Hxa6m7_XDjW5vqN0XCunADC8A-O1ow%40mail.gmail.com > <https://groups.google.com/d/msgid/elasticsearch/CACvWdiogwAwtJFQBpXF9Hxa6m7_XDjW5vqN0XCunADC8A-O1ow%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEYi1X9t3_ApV%3DGC5hBT%3D%2BNYOiLnZAAGMAAxi-uoTO3YTH9BBQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
