Re: GlobalKTable limitations

2017-05-26 Thread Damian Guy
l format, which is why I was > trying to see if anyone else would think this was useful. > > -Tommy > > > From: Eno Thereska [eno.there...@gmail.com] > Sent: Thursday, May 25, 2017 12:03 PM > To: dev@kafka.apache.org > Subject: Re

RE: GlobalKTable limitations

2017-05-25 Thread Thomas Becker
l. -Tommy From: Eno Thereska [eno.there...@gmail.com] Sent: Thursday, May 25, 2017 12:03 PM To: dev@kafka.apache.org Subject: Re: GlobalKTable limitations Hi Thomas, Have you considered doing the transformations on the topic, then outputting to another topi

Re: GlobalKTable limitations

2017-05-25 Thread Eno Thereska
Hi Thomas, Have you considered doing the transformations on the topic, then outputting to another topic and then constructing the GlobalKTable from the latter? The GlobalKTable has the limitations you mention since it was primarily designed for joins only. We should consider allowing a less re

GlobalKTable limitations

2017-05-25 Thread Thomas Becker
We need to do a series of joins against a KTable that we can't co- partition with the stream, so we're looking at GlobalKTable. But the topic backing the table is not ideally keyed for the sort of lookups this particular processor needs to do. Unfortunately, GlobalKTable is very limited in that yo