producer-consumer issues during deployments

2015-11-26 Thread Prabhjot Bharaj
Hi, We arrange our kafka machines in groups and deploy these phases. For kafka, we’ll have to map groups with phases. During each phase of the release, all the machines in that group can go down. When this happens, there are a couple of cases:- 1. All replicas are residing in a group of mach

Re: producer-consumer issues during deployments

2015-11-26 Thread Prabhjot Bharaj
Hi, Request your expertise on these doubts of mine Thanks, Prabhjot On Thu, Nov 26, 2015 at 4:43 PM, Prabhjot Bharaj wrote: > Hi, > > We arrange our kafka machines in groups and deploy these phases. > > For kafka, we’ll have to map groups with phases. During each phase of the > release, all th

Re: producer-consumer issues during deployments

2015-11-26 Thread Ben Stopford
Hi Prabhjot I may have slightly misunderstood your question so apologies if that’s the case. The general approach to releases is to use a rolling upgrade where you take one machine offline at a time, restart it, wait for it to come online (you can monitor this via JMX) then move onto the next.

Re: producer-consumer issues during deployments

2015-11-26 Thread Prabhjot Bharaj
Hi, Thanks for your reply We have 4 phases of deploys and in each phase, we can take down few machines These releases happen every 2 weeks, because on all machines, there are a bunch of other micro services running along with the core system - Kafka in this case My only concern is that during run