Re: Kafka Streams - offset number affected by "exactly once processing guarantee"

2019-06-21 Thread Trinity Xia
Thank you so much for the detailed explanation, Matthias! It's really helpful! Hope you have a good weekend :) On Thu, Jun 20, 2019 at 5:30 PM Matthias J. Sax wrote: > The observed behavior is expected. > > > For example, if we send 2615 events to an empty topic, we expect the end > of > >>

Re: Kafka Streams - offset number affected by "exactly once processing guarantee"

2019-06-20 Thread Matthias J. Sax
The observed behavior is expected. > For example, if we send 2615 events to an empty topic, we expect the end of >> the topic to be offset 2616. This is a wrong expectation. Even if Kafka behaves that way for non-EOS, there is no "contract" that guarantees that offsets are consecutive. Kafka

Kafka Streams - offset number affected by "exactly once processing guarantee"

2019-06-20 Thread Trinity Xia
Hi, We are using Kafka Streams 2.2.1 and Kafka 2.2.0, and we noticed that the end offset number is larger than the numbers of events sent to a topic if we set *processing guarantee* as *exactly once* in a Kafka Streams app. For example, if we send 2615 events to an empty topic, we expect the end