Hey,
with a little more effort you can try to make your stream application
idempotent.
Maybe giving you the same results. Say you want to aggregate a KStream
by some key.
Instead of keeping the aggregate, you keep a Set of raw values and then
do the aggregate calculations
with a map().
This is very much more resource intensive as you have quite some O(n^2)
statechange logs etc + the framework
is not really friendly helping you with that. With caching you can
hopefully spare quite a good chunk of the quadratic properties but it
can act as an exactly once processing.
Best Jan
On 30.01.2017 05:13, Mahendra Kariya wrote:
Hey All,
I am new to Kafka streams. From the documentation
<http://docs.confluent.io/3.1.0/streams/architecture.html#processing-guarantees>,
it is pretty much clear that streams support at least once semantics. But I
couldn't find details about how this is supported. I am interested in
knowing the finer details / design of this.
Is there some documentation around this?
Is there some documentation around what semantics are followed by the
various Kafka streams examples
<https://github.com/confluentinc/examples/tree/3.1.x/kafka-streams>
available on Github? Do all of them follow at least once?
Thanks,
Mahendra
