Re: kafka + autoscaling groups fuckery

/building-elastic-clusters.html, but they all rely on reusing broker IDs. Which certainly makes it easier, but imho is the wrong way to do this for all the reasons I listed before. On Sun, Jul 3, 2016 at 11:29 AM, Charity Majors <char...@hound.sh> wrote: > Great talks, but not relevant

Re: kafka + autoscaling groups fuckery

nning Kafka on Mesos, which might be relevant. > > Kafka on Mesos > > http://www.meetup.com//http-kafka-apache-org/events/222537743/?showDescription=true > > -James > > Sent from my iPhone > > > On Jul 2, 2016, at 5:15 PM, Charity Majors <char...@hound.sh> wrot

sarama and __consumer_offsets rebalancing

Hi there, I'm trying to track down why __consumer_offsets is never balancing and what to do about it. We're using Shopify's sarama client, which explicitly states that it doesn't do rebalancing for internal topics ... ever. It seems like sarama

rate-limiting on rebalancing, or sync from non-leaders?

Hi there, I'm curious if there's anything on the Kafka roadmap for adding rate-limiting or max-throughput for rebalancing processes. Alternately, if you have RF>2, maybe a setting to instruct followers to sync from other followers? I'm super impressed with how fast and efficient the kafka data

Re: kafka + autoscaling groups fuckery

Gwen, thanks for the response. 1.1 Your life may be a bit simpler if you have a way of starting a new > broker with the same ID as the old one - this means it will > automatically pick up the old replicas and you won't need to > rebalance. Makes life slightly easier in some cases. > Yeah, this

Re: kafka + autoscaling groups fuckery

y, if you guys are in AWS for everything, why not use > Kinesis? > > On Tue, Jun 28, 2016 at 3:49 PM, Charity Majors <char...@hound.sh> wrote: > > > Hi there, > > > > I just finished implementing kafka + autoscaling groups in a way that > made > >