Re: ElasticSearch across multiple data center architecture design options
Ok. Got it. Thank you! On Thu, Mar 12, 2015 at 8:56 PM, aa...@definemg.com wrote: Perhaps you are misunderstanding me. ElasticSearch does not provide a load balancer for this purpose. You would use a typical HTTP load balancer which could be anything as simple as Nginx, to something costly and expensive like a NetScalar. Configuring such a loadbalancer I believe is outside the scope of this list. On Thursday, March 12, 2015 at 11:16:12 AM UTC-6, Abigail wrote: Yes, that is what I meant. Is there any reference for set up the load balance for Kibana 4? Or if it is easier for Kibana 3? On Thu, Mar 12, 2015 at 12:26 PM, aa...@definemg.com wrote: Why not load balance multiple tribe nodes, if you need multiple. On Wednesday, March 11, 2015 at 9:41:39 AM UTC-6, Abigail wrote: Hi Mark, Thank you for your reply. Is there any existing approach for kibana to communicate with multiple tribe nodes? Or is it something we should implement by ourselves by customizing kibana? Thank you! Abigail On Tuesday, March 10, 2015 at 8:56:25 PM UTC-4, Mark Walkom wrote: 1 - It's pretty simple and has been used before. 2 - it can be yes. You can have multiple tribe nodes though. 3 - This may be possible but you'd have to hack a fair bit of code, so it's not really practical. On 10 March 2015 at 13:00, Alex nay...@gmail.com wrote: Hi all, We are planning to use ELK for our log analysis. We have multiple data centers. Since it is not recommended to have across data center cluster, we are going to have one ES cluster per data center, here are the three design options we have: 1. Use snapshot restore to replicate data across clusters. 2. Use tribe node to achieve across cluster queries 3. Ship and index logs to each cluster Here are our questions, and any comments will be appreciated: 1. How complex is snapshot restore, anyone has experience on this purpose? 2. Would the performance of only one tribe node be a concern or bottleneck, is it possible to have multiple tribe nodes for scale up or load balancing? 3. Is it possible to customize Kibana so that it can go to different cluster to query data depends on the query? Thank you! Abigail -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/ msgid/elasticsearch/2d46f80b-8579-4f2b-86c0-5ad654a5bba3%40goo glegroups.com https://groups.google.com/d/msgid/elasticsearch/2d46f80b-8579-4f2b-86c0-5ad654a5bba3%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to a topic in the Google Groups elasticsearch group. To unsubscribe from this topic, visit https://groups.google.com/d/ topic/elasticsearch/NPSIdmm9NX0/unsubscribe. To unsubscribe from this group and all its topics, send an email to elasticsearc...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/ msgid/elasticsearch/b69c667f-b1c6-46ce-8122-e809a22110c0% 40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/b69c667f-b1c6-46ce-8122-e809a22110c0%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to a topic in the Google Groups elasticsearch group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/NPSIdmm9NX0/unsubscribe. To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/c125f7bf-7908-4932-b022-88df49fc5f81%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/c125f7bf-7908-4932-b022-88df49fc5f81%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAJNTK58en4z%2Bfp9%3DV%2BmCpY_KThkVCTtiP0vbnYeWJn854mYWQQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
ElasticSearch across multiple data center architecture design options
Hi all, We are planning to use ELK for our log analysis. We have multiple data centers. Since it is not recommended to have across data center cluster, we are going to have one ES cluster per data center, here are the three design options we have: 1. Use snapshot restore to replicate data across clusters. 2. Use tribe node to achieve across cluster queries 3. Ship and index logs to each cluster Here are our questions, and any comments will be appreciated: 1. How complex is snapshot restore, anyone has experience on this purpose? 2. Would the performance of only one tribe node be a concern or bottleneck, is it possible to have multiple tribe nodes for scale up or load balancing? 3. Is it possible to customize Kibana so that it can go to different cluster to query data depends on the query? Thank you! Abigail -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/2d46f80b-8579-4f2b-86c0-5ad654a5bba3%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: ElasticSearch across multiple data center architecture design options
Why not load balance multiple tribe nodes, if you need multiple. On Wednesday, March 11, 2015 at 9:41:39 AM UTC-6, Abigail wrote: Hi Mark, Thank you for your reply. Is there any existing approach for kibana to communicate with multiple tribe nodes? Or is it something we should implement by ourselves by customizing kibana? Thank you! Abigail On Tuesday, March 10, 2015 at 8:56:25 PM UTC-4, Mark Walkom wrote: 1 - It's pretty simple and has been used before. 2 - it can be yes. You can have multiple tribe nodes though. 3 - This may be possible but you'd have to hack a fair bit of code, so it's not really practical. On 10 March 2015 at 13:00, Alex nay...@gmail.com wrote: Hi all, We are planning to use ELK for our log analysis. We have multiple data centers. Since it is not recommended to have across data center cluster, we are going to have one ES cluster per data center, here are the three design options we have: 1. Use snapshot restore to replicate data across clusters. 2. Use tribe node to achieve across cluster queries 3. Ship and index logs to each cluster Here are our questions, and any comments will be appreciated: 1. How complex is snapshot restore, anyone has experience on this purpose? 2. Would the performance of only one tribe node be a concern or bottleneck, is it possible to have multiple tribe nodes for scale up or load balancing? 3. Is it possible to customize Kibana so that it can go to different cluster to query data depends on the query? Thank you! Abigail -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/2d46f80b-8579-4f2b-86c0-5ad654a5bba3%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/2d46f80b-8579-4f2b-86c0-5ad654a5bba3%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/b69c667f-b1c6-46ce-8122-e809a22110c0%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: ElasticSearch across multiple data center architecture design options
Yes, that is what I meant. Is there any reference for set up the load balance for Kibana 4? Or if it is easier for Kibana 3? On Thu, Mar 12, 2015 at 12:26 PM, aa...@definemg.com wrote: Why not load balance multiple tribe nodes, if you need multiple. On Wednesday, March 11, 2015 at 9:41:39 AM UTC-6, Abigail wrote: Hi Mark, Thank you for your reply. Is there any existing approach for kibana to communicate with multiple tribe nodes? Or is it something we should implement by ourselves by customizing kibana? Thank you! Abigail On Tuesday, March 10, 2015 at 8:56:25 PM UTC-4, Mark Walkom wrote: 1 - It's pretty simple and has been used before. 2 - it can be yes. You can have multiple tribe nodes though. 3 - This may be possible but you'd have to hack a fair bit of code, so it's not really practical. On 10 March 2015 at 13:00, Alex nay...@gmail.com wrote: Hi all, We are planning to use ELK for our log analysis. We have multiple data centers. Since it is not recommended to have across data center cluster, we are going to have one ES cluster per data center, here are the three design options we have: 1. Use snapshot restore to replicate data across clusters. 2. Use tribe node to achieve across cluster queries 3. Ship and index logs to each cluster Here are our questions, and any comments will be appreciated: 1. How complex is snapshot restore, anyone has experience on this purpose? 2. Would the performance of only one tribe node be a concern or bottleneck, is it possible to have multiple tribe nodes for scale up or load balancing? 3. Is it possible to customize Kibana so that it can go to different cluster to query data depends on the query? Thank you! Abigail -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/ msgid/elasticsearch/2d46f80b-8579-4f2b-86c0-5ad654a5bba3% 40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/2d46f80b-8579-4f2b-86c0-5ad654a5bba3%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to a topic in the Google Groups elasticsearch group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/NPSIdmm9NX0/unsubscribe. To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/b69c667f-b1c6-46ce-8122-e809a22110c0%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/b69c667f-b1c6-46ce-8122-e809a22110c0%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAJNTK58QCAoD%3D1Tx-9anADBRPzp41bWZhHFvn7hx%3DHwSme%3DLqg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: ElasticSearch across multiple data center architecture design options
Yes, that is what I meant. Is there any reference for set up the load balance for Kibana 4? Or if it is easier for Kibana 3? On Thu, Mar 12, 2015 at 12:26 PM, aa...@definemg.com wrote: Why not load balance multiple tribe nodes, if you need multiple. On Wednesday, March 11, 2015 at 9:41:39 AM UTC-6, Abigail wrote: Hi Mark, Thank you for your reply. Is there any existing approach for kibana to communicate with multiple tribe nodes? Or is it something we should implement by ourselves by customizing kibana? Thank you! Abigail On Tuesday, March 10, 2015 at 8:56:25 PM UTC-4, Mark Walkom wrote: 1 - It's pretty simple and has been used before. 2 - it can be yes. You can have multiple tribe nodes though. 3 - This may be possible but you'd have to hack a fair bit of code, so it's not really practical. On 10 March 2015 at 13:00, Alex nay...@gmail.com wrote: Hi all, We are planning to use ELK for our log analysis. We have multiple data centers. Since it is not recommended to have across data center cluster, we are going to have one ES cluster per data center, here are the three design options we have: 1. Use snapshot restore to replicate data across clusters. 2. Use tribe node to achieve across cluster queries 3. Ship and index logs to each cluster Here are our questions, and any comments will be appreciated: 1. How complex is snapshot restore, anyone has experience on this purpose? 2. Would the performance of only one tribe node be a concern or bottleneck, is it possible to have multiple tribe nodes for scale up or load balancing? 3. Is it possible to customize Kibana so that it can go to different cluster to query data depends on the query? Thank you! Abigail -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/ msgid/elasticsearch/2d46f80b-8579-4f2b-86c0-5ad654a5bba3% 40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/2d46f80b-8579-4f2b-86c0-5ad654a5bba3%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to a topic in the Google Groups elasticsearch group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/NPSIdmm9NX0/unsubscribe. To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/b69c667f-b1c6-46ce-8122-e809a22110c0%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/b69c667f-b1c6-46ce-8122-e809a22110c0%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAJNTK59DepRdN39kbrDwiSAT15rC82Pm18Vmd%2BSXcBT_vhf60w%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: ElasticSearch across multiple data center architecture design options
Perhaps you are misunderstanding me. ElasticSearch does not provide a load balancer for this purpose. You would use a typical HTTP load balancer which could be anything as simple as Nginx, to something costly and expensive like a NetScalar. Configuring such a loadbalancer I believe is outside the scope of this list. On Thursday, March 12, 2015 at 11:16:12 AM UTC-6, Abigail wrote: Yes, that is what I meant. Is there any reference for set up the load balance for Kibana 4? Or if it is easier for Kibana 3? On Thu, Mar 12, 2015 at 12:26 PM, aa...@definemg.com javascript: wrote: Why not load balance multiple tribe nodes, if you need multiple. On Wednesday, March 11, 2015 at 9:41:39 AM UTC-6, Abigail wrote: Hi Mark, Thank you for your reply. Is there any existing approach for kibana to communicate with multiple tribe nodes? Or is it something we should implement by ourselves by customizing kibana? Thank you! Abigail On Tuesday, March 10, 2015 at 8:56:25 PM UTC-4, Mark Walkom wrote: 1 - It's pretty simple and has been used before. 2 - it can be yes. You can have multiple tribe nodes though. 3 - This may be possible but you'd have to hack a fair bit of code, so it's not really practical. On 10 March 2015 at 13:00, Alex nay...@gmail.com wrote: Hi all, We are planning to use ELK for our log analysis. We have multiple data centers. Since it is not recommended to have across data center cluster, we are going to have one ES cluster per data center, here are the three design options we have: 1. Use snapshot restore to replicate data across clusters. 2. Use tribe node to achieve across cluster queries 3. Ship and index logs to each cluster Here are our questions, and any comments will be appreciated: 1. How complex is snapshot restore, anyone has experience on this purpose? 2. Would the performance of only one tribe node be a concern or bottleneck, is it possible to have multiple tribe nodes for scale up or load balancing? 3. Is it possible to customize Kibana so that it can go to different cluster to query data depends on the query? Thank you! Abigail -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/ msgid/elasticsearch/2d46f80b-8579-4f2b-86c0-5ad654a5bba3% 40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/2d46f80b-8579-4f2b-86c0-5ad654a5bba3%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to a topic in the Google Groups elasticsearch group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/NPSIdmm9NX0/unsubscribe. To unsubscribe from this group and all its topics, send an email to elasticsearc...@googlegroups.com javascript:. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/b69c667f-b1c6-46ce-8122-e809a22110c0%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/b69c667f-b1c6-46ce-8122-e809a22110c0%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/c125f7bf-7908-4932-b022-88df49fc5f81%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: ElasticSearch across multiple data center architecture design options
Hi Mark, Thank you for your reply. Is there any existing approach for kibana to communicate with multiple tribe nodes? Or is it something we should implement by ourselves by customizing kibana? Thank you! Abigail On Tuesday, March 10, 2015 at 8:56:25 PM UTC-4, Mark Walkom wrote: 1 - It's pretty simple and has been used before. 2 - it can be yes. You can have multiple tribe nodes though. 3 - This may be possible but you'd have to hack a fair bit of code, so it's not really practical. On 10 March 2015 at 13:00, Alex nay...@gmail.com javascript: wrote: Hi all, We are planning to use ELK for our log analysis. We have multiple data centers. Since it is not recommended to have across data center cluster, we are going to have one ES cluster per data center, here are the three design options we have: 1. Use snapshot restore to replicate data across clusters. 2. Use tribe node to achieve across cluster queries 3. Ship and index logs to each cluster Here are our questions, and any comments will be appreciated: 1. How complex is snapshot restore, anyone has experience on this purpose? 2. Would the performance of only one tribe node be a concern or bottleneck, is it possible to have multiple tribe nodes for scale up or load balancing? 3. Is it possible to customize Kibana so that it can go to different cluster to query data depends on the query? Thank you! Abigail -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com javascript:. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/2d46f80b-8579-4f2b-86c0-5ad654a5bba3%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/2d46f80b-8579-4f2b-86c0-5ad654a5bba3%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/34fd5886-1107-47c8-a417-cbec4387d359%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: ElasticSearch across multiple data center architecture design options
1 - It's pretty simple and has been used before. 2 - it can be yes. You can have multiple tribe nodes though. 3 - This may be possible but you'd have to hack a fair bit of code, so it's not really practical. On 10 March 2015 at 13:00, Alex naye...@gmail.com wrote: Hi all, We are planning to use ELK for our log analysis. We have multiple data centers. Since it is not recommended to have across data center cluster, we are going to have one ES cluster per data center, here are the three design options we have: 1. Use snapshot restore to replicate data across clusters. 2. Use tribe node to achieve across cluster queries 3. Ship and index logs to each cluster Here are our questions, and any comments will be appreciated: 1. How complex is snapshot restore, anyone has experience on this purpose? 2. Would the performance of only one tribe node be a concern or bottleneck, is it possible to have multiple tribe nodes for scale up or load balancing? 3. Is it possible to customize Kibana so that it can go to different cluster to query data depends on the query? Thank you! Abigail -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/2d46f80b-8579-4f2b-86c0-5ad654a5bba3%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/2d46f80b-8579-4f2b-86c0-5ad654a5bba3%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEYi1X8G5cx9CkgAQqdeaioGWZztoEsnKgCwT9dK-XxHPpCpHA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
