Re: KTable as a compacted topic, implications

e-kafka-streams/ > > On 5/19/20 1:22 PM, Raffaele Esposito wrote: > > Thanks a lot Alex and Matthias, > > From Alex answer, I understand that the record is written to the > compacted > > topic > > as part of the transaction right ? > > > > > > On Tu

Re: KTable as a compacted topic, implications

ailure occurred somewhere in that > > sub-topology, then the offset for the repartition topic would not have > been > > committed, which means it would get processed again on application > > startup. Hope that helps, > > > > Alex Craig > > > > On T

KTable as a compacted topic, implications

This is the topology of a simple word count: Topologies: Sub-topology: 0 Source: KSTREAM-SOURCE-00 (topics: [word_count_input]) --> KSTREAM-FLATMAPVALUES-01 Processor: KSTREAM-FLATMAPVALUES-01 (stores: []) --> KSTREAM-KEY-SELECT-02 <-- K

Is Zombie instance problem consumer related ?

Kafka documentation states: Finally, in distributed environments, applications will crash or—worse!—temporarily lose connectivity to the rest of the system. Typically, new instances are automatically started to replace the ones which were

Re: idempotency issue and transactions on producer

to be more clear, does using transactions and providing the transactional.id, makes it possible for a producer to be idempotent even across sessions ? Thanks :) On Fri, May 15, 2020 at 9:29 PM Raffaele Esposito wrote: > Hi Boynag, > Yeah what you wrote is clear to me, that's why I

Re: idempotency issue and transactions on producer

e high level stuff through this blog: > https://www.confluent.io/blog/transactions-apache-kafka/. > > Boyang > > Boyang > > On Fri, May 15, 2020 at 11:48 AM Raffaele Esposito > > wrote: > > > In relation to producer idempotency Kafka documentation says: > > > >

idempotency issue and transactions on producer

In relation to producer idempotency Kafka documentation says: ... Since each new instance of a producer is assigned a new, unique, PID, we can only guarantee idempotent production within a single producer session. Does it mean that when we build a producer sourcing the data from whatever source

Re: KafkaStream groupBy + count on KTable behaviour

gt; https://github.com/apache/kafka/blob/873e9446ef8426061e2f1b6cd21815b270e27f03/streams/src/main/java/org/apache/kafka/streams/kstream/internals/KTableAggregate.java#L90 > > Best, > Bruno > > On Thu, May 14, 2020 at 8:50 PM Raffaele Esposito > wrote: > > > > Hi Bruno,

Re: KafkaStream groupBy + count on KTable behaviour

? Thanks again, Raffaele On Thu, May 14, 2020 at 8:39 PM Raffaele Esposito wrote: > Hi Bruno, > Thanks, > One more thing, As I told you I was consuming the repartitioning topic > created by group by > and I just saw the old and new value, as you are telling me now they are > indeed

Re: KafkaStream groupBy + count on KTable behaviour

d Change in > Kafka Streams and it is used where updates are emitted downstream. > > Best, > Bruno > > On Thu, May 14, 2020 at 12:17 PM Raffaele Esposito > wrote: > > > > I m trying to better understand KTable and I have encountered a > behaviour I > > can

KafkaStream groupBy + count on KTable behaviour

I m trying to better understand KTable and I have encountered a behaviour I cannot wrap my mind around it. So* groupByKey()* can only be applied to KStream and not to KTable, that's because of the nature of KTable that and its UPSERT logic. What I don't understand correctly and therefore ask your