Hi there, Firstly, I'd recommend not running the consumers and the brokers on the same machine. Are you running multiple brokers? If not, that'd be my first recommendation (it sounds like you might not be).
Secondly, yes, consumers scale up with partitions. At most you can have the same number of consumers as partitions (but each consumer can have multiple partitions). That error message seems pretty bad - it sounds like the broker is falling over repeatedly. I'd check the broker logs and metrics (see http://docs.confluent.io/2.0.1/kafka/monitoring.html). Thanks Tom Crayford Heroku Kafka On Thu, May 19, 2016 at 7:14 PM, Yazeed Alabdulkarim < y.alabdulka...@gmail.com> wrote: > Hi, > > For Kafka consumers, is it expected that the throughput will scale linearly > as I increase the number of consumers/partitions? > > Also, I keep getting this info message: "Kafka Consumer Marking the > coordinator 2147483647 dead." What is the problem? How can I fix it? My > program continues without any problem but I am worried that it maybe > impacting the performance. I tried to set all the config timeout parameters > to a large number. Both Kafka consumers and server are running on the same > machine. > > -- > Best Regards, > Yazeed Alabdulkarim >
