[ https://issues.apache.org/jira/browse/KAFKA-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14910100#comment-14910100 ]
ASF GitHub Bot commented on KAFKA-2482: --------------------------------------- GitHub user ewencp opened a pull request: https://github.com/apache/kafka/pull/249 KAFKA-2482: Allow sink tasks to get their current assignment, as well as pause and resume topic partitions. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ewencp/kafka kafka-2482-sink-tasks-pause-consumption Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/249.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #249 ---- commit c97df6dc18f7068866132a9237e8242286bab7c1 Author: Ewen Cheslack-Postava <m...@ewencp.org> Date: 2015-09-28T00:59:30Z KAFKA-2482: Allow sink tasks to get their current assignment, as well as pause and resume topic partitions. ---- > Allow copycat sink tasks to pause/resume consumption of specific topic > partitions > --------------------------------------------------------------------------------- > > Key: KAFKA-2482 > URL: https://issues.apache.org/jira/browse/KAFKA-2482 > Project: Kafka > Issue Type: Sub-task > Components: copycat > Reporter: Ewen Cheslack-Postava > Assignee: Ewen Cheslack-Postava > Fix For: 0.9.0.0 > > > Consider a situation where a sink is assigned 2 topic partitions. One of them > runs into a transient issue and no more data from it can be processed. > However, the other topic partition is proceeding fine. We don't want to block > the second partition by constantly throwing exceptions due to data from the > first topic partition. > The new consumer now supports pause/resume, so we should expose these to the > task. We could expose the functionality directly, although that would also > make the task responsible for scheduling some task in the future to check > whether it can resume. Another approach might be to make the API include the > backoff time. Then the framework would automatically resume consumption of > the topic partition after that time, which would presumably prompt the task > to reevaluate the situation for the topic partition when it receives another > message for it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)