Re: idempotency issue and transactions on producer

2020-05-15 Thread Raffaele Esposito
to be more clear, does using transactions and providing the transactional.id, makes it possible for a producer to be idempotent even across sessions ? Thanks :) On Fri, May 15, 2020 at 9:29 PM Raffaele Esposito wrote: > Hi Boynag, > Yeah what you wrote is clear to me, that's why I asked this

Re: idempotency issue and transactions on producer

2020-05-15 Thread Raffaele Esposito
Hi Boynag, Yeah what you wrote is clear to me, that's why I asked this question. What I don't understand if in the case of a producer we build a producer sourcing the data from whatever source (MySql for example) we need to take care explicitly of the mentioned idempotency per session problem.

Re: idempotency issue and transactions on producer

2020-05-15 Thread Boyang Chen
Hey Raffaele, the producer id is getting assigned upon receiving the producer.initTransaction call at the broker side. It guarantees the uniqueness of a producer for current lifecycle, which you don't have to configure manually. Transactional API on the other hand, includes idempotent produce

idempotency issue and transactions on producer

2020-05-15 Thread Raffaele Esposito
In relation to producer idempotency Kafka documentation says: ... Since each new instance of a producer is assigned a new, unique, PID, we can only guarantee idempotent production within a single producer session. Does it mean that when we build a producer sourcing the data from whatever