Re: Connect: SourceTask poll & commit interaction

2017-01-05 Thread Shikhar Bhushan
I have created https://issues.apache.org/jira/browse/KAFKA-4598 for this. On Wed, Dec 14, 2016 at 2:58 PM Shikhar Bhushan wrote: > Hi Mathieu, > > I think you are right, there is currently no mutual exclusion between > `task.commit()` and `task.poll()`. The solution you are thinking of with > ma

Re: Connect: SourceTask poll & commit interaction

2016-12-14 Thread Shikhar Bhushan
Hi Mathieu, I think you are right, there is currently no mutual exclusion between `task.commit()` and `task.poll()`. The solution you are thinking of with maintaining the committed offset state yourself seems reasonable, though inconvenient. It probably makes sense to add a new parameterized `com

Connect: SourceTask poll & commit interaction

2016-12-10 Thread Mathieu Fenniak
Hi Kafka Users, I'm looking for a bit of clarification on the documentation for implementing a SourceTask. I'm reading a replication stream from a database in my SourceTask, and I'd like to use commit or commitRecord to advance the other system's replication stream pointer so that it knows I have