Ralph Matthias Debusmann created KAFKA-13164:
------------------------------------------------
Summary: State store is attached to wrong node in the Kafka
Streams topology
Key: KAFKA-13164
URL: https://issues.apache.org/jira/browse/KAFKA-13164
Project: Kafka
Issue Type: Bug
Components: streams
Affects Versions: 2.8.0
Environment: local development (MacOS Big Sur 11.4)
Reporter: Ralph Matthias Debusmann
Attachments: 1.jpg, 3.jpg
Hi,
mjsax and me noticed a bug where a state store is attached to the wrong node in
the Kafka Streams topology.
The issue arised when I tried to read a topic into a KTable, then continued
with a mapValues(), and then joined this KTable with a KStream, like so:
var kTable = this.streamsBuilder.table(<topic>).mapValues(<mapValues function>);
and then later:
var joinedKStream = kstream.leftJoin(kTable, <leftJoin function>);
The join didn't work, and neither did it work when I added Materialized.as() to
mapValues(), like so:
var kTable = this.streamsBuilder.table(<topic>).mapValues(<mapValues function>,
*Materialized.as(<some name>)*);
Interestingly, I could get the join to work, when I first read the topic into
a *KStream*, then continued with the mapValues(), then turned the KStream into
a KTable, and then joined the KTable with the other KStream, like so:
var kTable = this.streamsBuilder.stream(<topic>).mapValues(<mapValues
function>).toTable();
(the join worked the same as above)
When mjsax and me had a look on the topology, we could see that in the former,
not working code, the state store (required for the join) is attached to the
pre-final "KTABLE-SOURCE", and not the final "KTABLE-MAPVALUES" node (see
attachment "1.jpg"). In the working code, the state store is (correctly)
attached to the final "KSTREAM-TOTABLE" node (see attachment "3.jpg").
Best regards,
xdgrulez
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
