[
https://issues.apache.org/jira/browse/KAFKA-4161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15668923#comment-15668923
]
ASF GitHub Bot commented on KAFKA-4161:
---------------------------------------
GitHub user shikhar opened a pull request:
https://github.com/apache/kafka/pull/2139
KAFKA-4161: KIP-89: Allow sink connectors to decouple flush and offset
commit
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/shikhar/kafka kafka-4161-deux
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/kafka/pull/2139.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2139
----
commit 706bcc860fed939a00171ebf61fdab8639d99b06
Author: Shikhar Bhushan <shik...@confluent.io>
Date: 2016-11-15T00:29:43Z
KAFKA-4161: KIP-89: Allow sink connectors to decouple flush and offset
commit
----
> Decouple flush and offset commits
> ---------------------------------
>
> Key: KAFKA-4161
> URL: https://issues.apache.org/jira/browse/KAFKA-4161
> Project: Kafka
> Issue Type: Improvement
> Components: KafkaConnect
> Reporter: Shikhar Bhushan
> Assignee: Shikhar Bhushan
> Labels: needs-kip
>
> It is desirable to have, in addition to the time-based flush interval, volume
> or size-based commits. E.g. a sink connector which is buffering in terms of
> number of records may want to request a flush when the buffer is full, or
> when sufficient amount of data has been buffered in a file.
> Having a method like say {{requestFlush()}} on the {{SinkTaskContext}} would
> allow for connectors to have flexible policies around flushes. This would be
> in addition to the time interval based flushes that are controlled with
> {{offset.flush.interval.ms}}, for which the clock should be reset when any
> kind of flush happens.
> We should probably also support requesting flushes via the
> {{SourceTaskContext}} for consistency though a use-case doesn't come to mind
> off the bat.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
