On Sun, Nov 13, 2016 at 10:14 PM, Davor Bonaci <da...@google.com.invalid> wrote:
> Luke is bringing up great questions, I think. > Yes, better handling of 'desiredNumSplits' by a runner would be very useful. I wanted to limit my proposal to what a source like KafkaIO could do on its own. > My first impression is that the current state is "possibly under-split", > and the proposal is to move us to "possibly over-split" state. Neither is > the ideal solution, > I think of it more like : shift from "strictly obey parallelism suggested by the runner" to "parallelism inherent to a Kafka stream". Raghu.
