I have faced same problem . https://stackoverflow.com/questions/54286486/two-kafka-consumer-in-same-group-and-one-partition
On Wed, Jan 30, 2019 at 6:11 PM Avi Levi <avi.l...@bluevoyant.com> wrote: > Ok, if you guys think it's should be like that then so be it. All I am > saying is that it is not standard behaviour from kafka consumer, at least > according to the documentation > <https://kafka.apache.org/documentation/#intro_consumers> . I understand > that flink implements things differently and all I am saying that this is > not according to kafka standard consumer group. > > > On Tue, Jan 29, 2019 at 9:47 AM Tzu-Li (Gordon) Tai <tzuli...@apache.org> > wrote: > >> Hi, >> >> Yes, Dawid is correct. >> >> The " >> <https://urldefense.proofpoint.com/v2/url?u=http-3A__group.id&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=dpWtkT5FJRWFqDA3MAnB4-dRYGDQjgfQTYAocqGkRKo&m=KOx07VP_axxlgt0eYEnVHNux-bJCoSwzMInyMaKqCR4&s=6pT15T_bD5q0Rbzla6yl5aqek0zSwEhMI41vxALQAKg&e=> >> group.id >> <https://urldefense.proofpoint.com/v2/url?u=http-3A__group.id&d=DwQFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=dpWtkT5FJRWFqDA3MAnB4-dRYGDQjgfQTYAocqGkRKo&m=KOx07VP_axxlgt0eYEnVHNux-bJCoSwzMInyMaKqCR4&s=6pT15T_bD5q0Rbzla6yl5aqek0zSwEhMI41vxALQAKg&e=>" >> setting in Flink's Kafka Consumer is only used for group offset fetching >> and committing offsets back to Kafka (only for exposure purposes, not used >> for processing guarantees). >> The Flink Kafka Consumer uses static partition assignment on the >> KafkaConsumer API, and not consumer group-based automatic partition >> assignments. >> >> Cheers, >> Gordon >> >> On Sun, Jan 27, 2019 at 12:28 AM Dawid Wysakowicz <dwysakow...@apache.org> >> wrote: >> >>> Forgot to cc Gordon :) >>> >>> On 23/01/2019 18:02, Avi Levi wrote: >>> > Hi, >>> > This quite confusing. >>> > I submitted the same stateless job twice (actually I upload it once). >>> > However when I place a message on kafka, it seems that both jobs >>> > consumes it, and publish the same result (we publish the result to >>> > other kafka topic, so I actually see the massage duplicated on kafka >>> > ). how can it be ? both jobs are using the same group id (group id is >>> > fixed and not generated ) >>> > >>> > Kind regards >>> > Avi >>> >>> -- Regards, Selvaraj C
