Hi Tao,
That is expected behavior. You can use set
partition.assignment.strategy=roundrobin in consumer config. It will take
all the partitions from all topics and do a round robin assignment,
whereas range only take partitions for each individual topic for
assignment.
Jiangjie (Becket) Qin
On 3/9/15, 4:10 AM, tao xiao xiaotao...@gmail.com wrote:
Hi,
I created a message stream in my consumer using connector
.createMessageStreamsByFilter(new Whitelist(mm-benchmark-test\\w*), 5);
I
have 5 topics in my cluster and each of the topic has only one partition.
My understanding of wildcard stream is that multiple streams are shared
between selected topics. In my case 5 streams should be shared between 5
different topics. But when I looked at the log it showed a different story
2015-03-09 19:02:36 INFO kafka.utils.Logging$class:68 -
[test12345667_LM-SHC-00950667-1425898953590-d99e2d75],
test12345667_LM-SHC-00950667-1425898953590-d99e2d75-0 successfully
owned partition 0 for topic mm-benchmark-test2
2015-03-09 19:02:36 INFO kafka.utils.Logging$class:68 -
[test12345667_LM-SHC-00950667-1425898953590-d99e2d75],
test12345667_LM-SHC-00950667-1425898953590-d99e2d75-0 successfully
owned partition 0 for topic mm-benchmark-test
2015-03-09 19:02:36 INFO kafka.utils.Logging$class:68 -
[test12345667_LM-SHC-00950667-1425898953590-d99e2d75],
test12345667_LM-SHC-00950667-1425898953590-d99e2d75-0 successfully
owned partition 0 for topic mm-benchmark-test1
2015-03-09 19:02:36 INFO kafka.utils.Logging$class:68 -
[test12345667_LM-SHC-00950667-1425898953590-d99e2d75],
test12345667_LM-SHC-00950667-1425898953590-d99e2d75-0 successfully
owned partition 0 for topic mm-benchmark-test4
2015-03-09 19:02:36 INFO kafka.utils.Logging$class:68 -
[test12345667_LM-SHC-00950667-1425898953590-d99e2d75],
test12345667_LM-SHC-00950667-1425898953590-d99e2d75-0 successfully
owned partition 0 for topic mm-benchmark-test3
As indicated from the log only one stream was assigned to all topics. I
just wanted to know if this is expected behavior? if yes how do we evenly
distribute topics across different streams? by using roundrobin assigner?
--
Regards,
Tao