Re: KStreams Partition Assignment

2016-03-21 Thread Guozhang Wang
Hi Mike, Kafka Streams is suitable for distributed environment. You can read this section of the web docs for more details: http://docs.confluent.io/2.1.0-alpha1/streams/architecture.html#parallelism-model As for the Kafka Streams' customized partition assignor under the Kafka group management p

KStreams Partition Assignment

2016-03-20 Thread Michael D. Coon
I'm evaluating whether the KafkaStreams API will be something we can use on my current project. Namely, we want to be able to distribute the consumers on a Mesos/YARN cluster. It's not entirely clear to me in the code what is deciding which partitions get assigned at runtime and whether this is