Hello,

I'm trying the following join using KTable. There are two change log tables.
Table1
  111 -> aaa
  222 -> bbb
  333 -> aaa

Table2
  aaa -> 999
  bbb -> 888
  ccc -> 777

My result table should be
  111 -> 999
  222 -> 888
  333 -> 999

Its not a case for join() as the keys don't match. Its more a lookup table.

Option1 is to use a Table1.toStream().process(ProcessSupplier(),
"storeName")
punctuate() will use regular kafka consumer that reads updates from Table2
and updates a private map.
Process() will do a key-value lookup.
This has an advantage when Table1 is much larger than Table2.
Each instance of the processor will have to hold entire Table2.

Option2 is to re-partition Table1 using through(StreamPartitioner) and
partition using value.
This will ensure co-location. Then join with Table2. This part might be
tricky??

Your comments and suggestions are welcome!

Srikanth

Reply via email to