[
https://issues.apache.org/jira/browse/KAFKA-3705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15365365#comment-15365365
]
Guozhang Wang commented on KAFKA-3705:
--------------------------------------
[~jfilipiak] I am convinced that this combo-key is necessary to avoid out of
ordering after talking with you offline. And I have updated the design proposal
wiki accordingly:
https://cwiki.apache.org/confluence/display/KAFKA/Discussion%3A+Non-key+KTable-KTable+Joins,
feel free to take a look.
Just a random thought as for your use case specifically: are relation A, B, and
C all need to be captured as a KTable (i.e. the records as binlog / etc from
some database table)? If the one with foreign key can be captured just a stream
(i.e. KStream), then what you can do is to re-model your computation as
{{(stream Join table1) Join table2}}, where {stream Join table returns a
stream}. And in Kafka Streams DSL you can just do
{{stream.selectKey(table1.key).join(table1).selectKey(table2.key).join(table2)}}.
> Support non-key joining in KTable
> ---------------------------------
>
> Key: KAFKA-3705
> URL: https://issues.apache.org/jira/browse/KAFKA-3705
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Reporter: Guozhang Wang
> Assignee: Liquan Pei
> Labels: api
> Fix For: 0.10.1.0
>
>
> Today in Kafka Streams DSL, KTable joins are only based on keys. If users
> want to join a KTable A by key {{a}} with another KTable B by key {{b}} but
> with a "foreign key" {{a}}, and assuming they are read from two topics which
> are partitioned on {{a}} and {{b}} respectively, they need to do the
> following pattern:
> {code}
> tableB' = tableB.groupBy(/* select on field "a" */).agg(...); // now tableB'
> is partitioned on "a"
> tableA.join(tableB', joiner);
> {code}
> Even if these two tables are read from two topics which are already
> partitioned on {{a}}, users still need to do the pre-aggregation in order to
> make the two joining streams to be on the same key. This is a draw-back from
> programability and we should fix it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
