Hi Alan, The reason we do not have per-topic parallelism spec in wildcard is two folds: 1) we use a per-topic hash-based partition algorithm, and hence having each topic with the same num. of streams may give us better load balance, 2) with the topicFilter we will not know exactly which topics to consume at the construction time, hence no way to specify per-topic specs.
1) has been lifted since we have implemented new partitioning algorithm, and for 2) we need to think about how to support it if we really want to, perhaps we can also use a regex-ed topic-count map, while ensuring that each regex in the map is precedent of the topic filter, and no overlap with each other, etc. What is your usecase that requires per-topic numStream spec? Guozhang On Sun, Nov 9, 2014 at 6:03 AM, Alan Lavintman <alan.lavint...@gmail.com> wrote: > Hi guys, i have seen that if create a message stream by using: > > createMessageStreams > > I can define a map with Topic->#Streams > > Is there a reason why createMessageStreamsByFilter us not giving the same > support? I have only a TopicFilter and numStreams interface such as: > > public List<KafkaStream<byte[], byte[]>> > createMessageStreamsByFilter(TopicFilter topicFilter, int numStreams); > > But it does not allow me to specify the parallelism per topic. Am I missing > something or my assumption is correct? > > Bests and thanks, > Alan. > -- -- Guozhang
