Problem connecting to ZooKeeper with Kafka SASL enabled

2017-01-07 Thread Christian
Hi, I'm trying to set up SASL_PLAINTEXT authentication between the producer/consumer clients and the Kafka brokers only. I am not too worried about the broker to broker SASL nor do I want Kafka Broker to ZooKeeper SASL Authentication. I would prefer the just require authentication for the clients

What to do when it won't rebalance "properly"

2017-01-07 Thread Jon Yeargers
Have been messing about with Kubernetes on google-cloud. Launched a pod with 6 consumer nodes and watched the lag using 'kafka-topics .. --new-consumer --describe'. Topic has assigned all (12 in this case) nodes to the same consumer while the other 5 are sitting idle. This has been the case for ~2

Re: Problem connecting to ZooKeeper with Kafka SASL enabled

2017-01-07 Thread Christian
I figured it out. I am using Confluent 3.1.1's docker images. There is a file in https://github.com/confluentinc/cp-docker-images/blob/master/debian/base/include/cub which looks for an environment variable of "ZOOKEEPER_SASL_ENABLED" which defaults to true if using jaas. I simply set that var to fa

Re: What to do when it won't rebalance "properly"

2017-01-07 Thread Jon Yeargers
Along these same lines - I have a topic with a single consumer. When I try to look at lag ("kafka-topics .. --group ... --describe") I get the message "Consumer group is rebalancing" This continues in perpetuity despite stop/(re)start the consumer and cycle all (3) brokers. On Sat, Jan 7, 2017 a