[
https://issues.apache.org/jira/browse/KAFKA-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nigel Liang updated KAFKA-9314:
-------------------------------
Description:
KIP-298 outlines the retry policy of Connect when errors are encountered. In
particular, it proposes to retry on {{RetriableException}} on put() in SinkTask
and poll() in SourceTask.
However, the code does not reflect this change. For instance, WorkerSourceTask
handles {{RetriableException}} thrown from {{poll()}} by entering into a tight
retry loop without backoff. This has led to connectors having to workaround by
simply not retrying and failing the task always. Users would need to manually
restart the task to recover from even simple network glitches.
AFAICT from reading code, the same is true for {{WorkerSinkTask}} when calling
{{put()}}.
was:
KIP-298 outlines the retry policy of Connect when errors are encountered. In
particular, it proposes to retry on {{RetriableException}} on put() in SinkTask
and poll() in SourceTask.
However, the code does not reflect this change. For instance, WorkerSourceTask
handles {{RetriableException}} thrown from {{poll()}} by entering into a tight
retry loop without backoff. This has led to connectors having to workaround by
simply not retrying and failing the task always
(https://github.com/confluentinc/kafka-connect-jms/pull/88). Users would need
to manually restart the task to recover from even simple network glitches.
AFAICT from reading code, the same is true for {{WorkerSinkTask}} when calling
{{put()}}.
> Connect put() and poll() retries not conforming to KIP-298
> ----------------------------------------------------------
>
> Key: KAFKA-9314
> URL: https://issues.apache.org/jira/browse/KAFKA-9314
> Project: Kafka
> Issue Type: Bug
> Components: KafkaConnect
> Reporter: Nigel Liang
> Assignee: Nigel Liang
> Priority: Major
>
> KIP-298 outlines the retry policy of Connect when errors are encountered. In
> particular, it proposes to retry on {{RetriableException}} on put() in
> SinkTask and poll() in SourceTask.
> However, the code does not reflect this change. For instance,
> WorkerSourceTask handles {{RetriableException}} thrown from {{poll()}} by
> entering into a tight retry loop without backoff. This has led to connectors
> having to workaround by simply not retrying and failing the task always.
> Users would need to manually restart the task to recover from even simple
> network glitches.
> AFAICT from reading code, the same is true for {{WorkerSinkTask}} when
> calling {{put()}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
