[
https://issues.apache.org/jira/browse/KAFKA-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14910100#comment-14910100
]
ASF GitHub Bot commented on KAFKA-2482:
---------------------------------------
GitHub user ewencp opened a pull request:
https://github.com/apache/kafka/pull/249
KAFKA-2482: Allow sink tasks to get their current assignment, as well as
pause and resume topic partitions.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ewencp/kafka
kafka-2482-sink-tasks-pause-consumption
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/kafka/pull/249.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #249
----
commit c97df6dc18f7068866132a9237e8242286bab7c1
Author: Ewen Cheslack-Postava <m...@ewencp.org>
Date: 2015-09-28T00:59:30Z
KAFKA-2482: Allow sink tasks to get their current assignment, as well as
pause and resume topic partitions.
----
> Allow copycat sink tasks to pause/resume consumption of specific topic
> partitions
> ---------------------------------------------------------------------------------
>
> Key: KAFKA-2482
> URL: https://issues.apache.org/jira/browse/KAFKA-2482
> Project: Kafka
> Issue Type: Sub-task
> Components: copycat
> Reporter: Ewen Cheslack-Postava
> Assignee: Ewen Cheslack-Postava
> Fix For: 0.9.0.0
>
>
> Consider a situation where a sink is assigned 2 topic partitions. One of them
> runs into a transient issue and no more data from it can be processed.
> However, the other topic partition is proceeding fine. We don't want to block
> the second partition by constantly throwing exceptions due to data from the
> first topic partition.
> The new consumer now supports pause/resume, so we should expose these to the
> task. We could expose the functionality directly, although that would also
> make the task responsible for scheduling some task in the future to check
> whether it can resume. Another approach might be to make the API include the
> backoff time. Then the framework would automatically resume consumption of
> the topic partition after that time, which would presumably prompt the task
> to reevaluate the situation for the topic partition when it receives another
> message for it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
