Greg Harris created KAFKA-16069: ----------------------------------- Summary: Source Tasks re-transform records after Retriable exceptions Key: KAFKA-16069 URL: https://issues.apache.org/jira/browse/KAFKA-16069 Project: Kafka Issue Type: Bug Components: KafkaConnect Reporter: Greg Harris
In the SinkTask, records which fail to be delivered to the task#put with a Retriable exception are re-delivered on the next iteration. The SourceTask follows a similar pattern, where records which fail to be delivered to Producer#send with a Retriable exception are retried. However, the SinkTask accumulates the post-transform records, and does not recompute the transformations over again after a retriable exception. The SourceTask does not accumulate ProducerRecords, and instead recomputes the transformations and converters starting from the pre-transformation AbstractWorkerSourceTask#toSend list. This means that stateful transformations and converters may see the records rewind, without any indication that the records are the same. For stateless transformations and converters, this means that redundant computation is performed that may be better allocated to other tasks. -- This message was sent by Atlassian Jira (v8.20.10#820010)