Having a separate cluster is definitely a better way to go. OR, you can control the shard, replica placement so that they are always placed in the same DC. In this way, you can avoid interDC issues still having a single cluster. I have the similar issue and I am looking at it as one of the alternative.
On Saturday, May 10, 2014 1:05:08 AM UTC-7, Sebastian Łaskawiec wrote: > > Thanks for the answer! We've been talking with several other teams in our > company and it looks like this is the most recommended and stable setup. > > Regards > Sebastian > > W dniu środa, 7 maja 2014 03:23:43 UTC+2 użytkownik Mark Walkom napisał: >> >> Go the latter method and have two clusters, ES can be very sensitive to >> network latency and you'll likely end up with more problems than it is >> worth. >> Given you already have the data source of truth being replicated, it's >> the sanest option to just read that locally. >> >> Regards, >> Mark Walkom >> >> Infrastructure Engineer >> Campaign Monitor >> email: ma...@campaignmonitor.com >> web: www.campaignmonitor.com >> >> >> On 6 May 2014 23:51, Sebastian Łaskawiec <sebastian...@gmail.com> wrote: >> >>> Hi! >>> >>> I'd like to ask for advice about deployment in multi DC scenario. >>> >>> Currently we operate on 2 Data Centers in active/standby mode. like to >>> opeIn case of ES we'd like to have different approach - we'drate in >>> active-active mode (we want to optimize our resources especially for >>> querying). >>> Here are some details about target configuration: >>> >>> - 4 ES instances per DC. Full cluster will have 8 instances. >>> - Up to 1 TB of data >>> - Data pulled from database using JDBC River >>> - Database is replicated asynchronously between DCs. Each DC will >>> have its own database instance to pull data. >>> - Average latency between DCs is about several miliseconds >>> - We need to operate when passive DC is down >>> >>> We know that multi DC configuration might end with Split Brain issue. >>> Here is how we want to prevent it: >>> >>> - Set node.master: true only in 4 nodes in active DC >>> - Set node.master: false in passive DC >>> - This way we'll be sure that new cluster will not be created in >>> passive DC >>> - Additionally we'd like to set discovery.zen.minimum_master_nodes: >>> 3 (to avoid Split Brain in active DC) >>> >>> Additionally there is problem with switchover (passive DC becomes active >>> and active becomes passive). In our system it takes about 20 minutes and >>> this is the maximum length of our maintenance window. We were thinking of >>> shutting down whole ES cluster and switch node.master setting in >>> configuration files (as far as I know this settings can not be changed via >>> REST api). Then we'd need to start whole cluster. >>> >>> So my question is: is it better to have one big ES cluster operating on >>> both DCs or should we change our approach and create 2 separate clusters >>> (and rely on database replication)? I'd be grateful for advice. >>> >>> Regards >>> Sebastian >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "elasticsearch" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to elasticsearc...@googlegroups.com. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/elasticsearch/6be53754-63fd-4202-b940-750a3e0c1a8f%40googlegroups.com<https://groups.google.com/d/msgid/elasticsearch/6be53754-63fd-4202-b940-750a3e0c1a8f%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5875ae02-0cdd-4ce7-bce0-18e01bf0877a%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
