Hi Guozhang,Thanks for your clarification for this, it's start to making sense
now.Edwin
On Wednesday, February 4, 2015 11:04 AM, Guozhang Wang
<[email protected]> wrote:
Hi Edwin,
1. Yes.
2. Yes.
3. Yes; and there is no difference in terms of parallelism.
In the new consumer client (org.apache.kafka.clients.consumer.KafkaConsumer)
that is going to be out in the next release, each consumer instance is
single-threaded, i.e. it will only have one thread for fetching data.
On Wed, Feb 4, 2015 at 3:43 AM, Zijing Guo <[email protected]> wrote:
Hi,I have some question regarding how kafka consumers achieve parallel
consuming for one topic. Say I have 2 partitions for topic1 and I have a
consumer Group A, now:1: If no consumer under consumer Group A subscribe
topic1, then no message will be delivery to this consumer group.2: If there is
only 1 consumer under consumer Group A that subscribe topic1, then this
consumer will consume all the data from topic1 that among the 2 partitions?3:in
the High level consumer example
https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Group+Exampleit
says: "if you provide more threads than there are partitions on the topic, some
threads will never see a message
- if you have more partitions than you have threads, some threads will
receive data from multiple partitions"
so back to my example, If I create 2 threads under one consumer instance that
I created and will each thread corresponding to a specific partition under the
topic1? what is the difference that I create 2 consumers, each with only 1
thread under same consumer group and consume topic1? Is kafka parallel
consuming really down to the thread level that with 1 consumer instance or
multiple consumer instance, each with 1 thread?
ThanksEdwin
--
-- Guozhang
