Re: Kafka streaming partition assignment

2018-05-13 Thread Liam Clarke
Thank you Matthias - we're using version 1.0. I can tell my team to relax and look at upgrading :) On Mon, May 14, 2018 at 3:48 PM, Matthias J. Sax wrote: > It depends on your version. The behavior is known and we put one > improvement into 1.1 release: https://github.com/apache/kafka/pull/4410

Re: Kafka streaming partition assignment

2018-05-13 Thread Matthias J. Sax
It depends on your version. The behavior is known and we put one improvement into 1.1 release: https://github.com/apache/kafka/pull/4410 Thus, it's "by design" (for 1.0 and older) but we we want to improve it. Cf: https://issues.apache.org/jira/browse/KAFKA-4969 -Matthias On 5/13/18 7:52 PM, Lia

Kafka streaming partition assignment

2018-05-13 Thread Liam Clarke
Hi all, We are running a KStreaming app with a basic topology of consume from topic A -> transform and write through topic B (making the app a consumer of topic B also) -> finally write to topic C We are running it with two instances of the application. Topic A has 100 partitions, topics B and