Here is my environment:
1 kafka server with 5 partition,3 zookeeper server,1 producer,1 consumer
There is kafka log.
topic-0
....
topic-4
Producer:
when I produce message I just let zookeeper to decide which partition
should be chosen,Like that:
*producer.send(new ProducerData<String, String>("topic", messageStr)); *
Consumer:
In consumer,I just use one partition to consume message from kafka
,eventually I get every message from kafka
*Map<String, Integer> topicCountMap = new ConcurrentHashMap<String,
Integer>();*
* topicCountMap.put(topic, new Integer(1));*
*
*
*Map<String, List<KafkaMessageStream<Message>>> consumerMap = consumer*
* .createMessageStreams(topicCountMap);*
Then I start another same consumer to consume message from kafka,eventually
I get every message from kafka too.
Here is the new environment:
1 kafka server with 5 partition in each of them,3 zookeeper server,1
producer,2 consumer
But after I start another kafka server.
Here is the new environment:
2 kafka server with 5 partition in each of them,3 zookeeper server,1
producer,2 consumer
There is no kafka log in kafka server.
And eventually I just get about half of messages from kafka.
I think maybe kafka didn't join to cluster? but actually the config is
right:
zk.connect=184.72.143.58:2181,23.20.230.102:2181,23.22.146.46:2181
then why there is no kafka log in new kafka server
and what is the relation ship between the stream number in code and
partition number in kafka.
even if there is 5 partition in kafka,I can still just use 1 stream to
consume 5 partition's messages,why?
if I use 5 stream to consume 5 partion,what will happend.
I'm confused about those conceptions.
Please help me ,thanks.
