Randall Hauch created KAFKA-6661:
------------------------------------
Summary: Sink connectors that explicitly 'resume' topic partitions
can resume a paused task
Key: KAFKA-6661
URL: https://issues.apache.org/jira/browse/KAFKA-6661
Project: Kafka
Issue Type: Bug
Components: KafkaConnect
Affects Versions: 1.0.0, 0.11.0.0, 0.10.0.0, 0.9.0.0
Reporter: Randall Hauch
Assignee: Randall Hauch
Sink connectors are allowed to use the {{SinkTaskContext}}'s methods to
explicitly pause and resume topic partitions. This is useful when connectors
need additional time processing the records for specific topic partitions
(e.g., the external system has an outage).
However, when the sink connector has been paused via the REST API, the worker
for the sink tasks pause the consumer. When the connector is polled, the poll
request might timeout and return no records. Connect then calls the task's
{{put(...)}} method (with no records), and this allows the task to optionally
call any of the {{SinkTaskContext}}'s pause or resume methods. If it calls
resume, this will unexpectedly resume the paused consumer, causing the consumer
to return messages and the connector to process those messages -- despite the
connector still being paused.
This is reported against 1.0, but the affected code has not been changed since
at least 0.9.0.0.
A workaround is to remove rather than pause a connector. It's inconvenient, but
it works.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
