Which version of ZK are you using ?
On Fri, Jun 7, 2013 at 12:15 PM, Evan Chan <[email protected]> wrote: > Hey guys, > > I sincerely apologize if this has been covered before, I haven't quite > found a similar situation. > > We are using Kafka 0.7.2 in production, and we are using the ZK high level > Scala consumer. However, we find the ZK consumer very unstable. It would > work for one or two weeks, then suddenly it would complain about ZK nodes > disappearing, and one consumer would die, then another, then another, until > our pipeline is no longer pulling any data. There are multiple > NullPointerExceptions, and other problems. We can restart it, but it > does not stay up predictably. > > On the other hand, I have a simple app which I wrote using the simple > consumer to mirror select partitions (will blog about this later) and it > just works flawlessly. > > So we are faced with a dilemma to get back on track: > 1) Use SimpleConsumer, and write our own balancing code (but honestly our > boxes almost never go down, compared to the rate of ZK mishaps) > 2) Upgrade to Kafka 0.8 and hope that that resolves the issue. > > There seem to be so many improvements in 0.8 that that seems to be the > biggest win long-term, so I am wondering if people can comment on: > - has anyone tried using 0.8 in production? Is it stable yet? > - How much more stable is the ZK consumer in 0.8? > - will it be possible to change the offset in the 0.8 consumer? That was > the other reason why we wanted to move to SimpleConsumer. > > thanks, > Evan > > -- > -- > Evan Chan > Staff Engineer > [email protected] | > > <http://www.ooyala.com/> > <http://www.facebook.com/ooyala><http://www.linkedin.com/company/ooyala>< > http://www.twitter.com/ooyala> >
