If you only enable "read_committed", yes.
If you set `processing.guarantees="exactly_once"` the guarantee is
stronger, because using a read-process-write pattern allows to couple
writing result data and committing offsets into a single transaction (in
contrast to a consumer-only scenario, for whic
This is not only a configuration. Like in all messaging solutions you need to
make sure that all involved applications ensure once and only once delivery
from producer(s) over Kafka to consumer(s).
> Am 30.08.2019 um 22:00 schrieb Peter Groesbeck :
>
> For a producer that emits messages to a
Does a Kafka Streams consumer also have that same limitation of possible
duplicates?
Thanks,
Chris
On Fri, Sep 27, 2019 at 11:56 AM Matthias J. Sax
wrote:
> Enabling "read_committed" only ensures that a consumer does not return
> uncommitted data.
>
> However, on failure, a consumer might still
Enabling "read_committed" only ensures that a consumer does not return
uncommitted data.
However, on failure, a consumer might still read committed messages
multiple times (if you commit offsets after processing). If you commit
offsets before you process messages, and a failure happens before
proc
You can achieve exactly once on a consumer by enabling read committed and
manually committing the offset as soon as you receive a message. That way
you know that at next poll you won't get old message again.
On Fri, Sep 27, 2019, 6:24 AM christopher palm wrote:
> I had a similar question, and ju
I had a similar question, and just watched the video on the confluent.io
site about this.
>From what I understand idempotence and transactions are there to solve the
duplicate writes and exactly once processing, respectively.
Is what you are stating below is that this only works if we produce into
Exactly-once on the producer will only ensure that no duplicate writes
happen. If a downstream consumer fails, you might still read message
multiple times for all cases (ie, without idempotence, with idempotence
enabled, or if you use transactions).
Note, that exactly-once is designed for a read-p
For a producer that emits messages to a single topic (i.e. no single
message is sent to multiple topics), will enabling idempotency but not
transactions provide exactly once guarantees for downstream consumers of
said topic?
Ordering is not important I just want to make sure consumers only consume
