[jira] [Updated] (KAFKA-15145) AbstractWorkerSourceTask re-processes records filtered out by SMTs on retriable exceptions
[ https://issues.apache.org/jira/browse/KAFKA-15145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton updated KAFKA-15145: -- Fix Version/s: 3.5.1 > AbstractWorkerSourceTask re-processes records filtered out by SMTs on > retriable exceptions > -- > > Key: KAFKA-15145 > URL: https://issues.apache.org/jira/browse/KAFKA-15145 > Project: Kafka > Issue Type: Bug > Components: KafkaConnect >Affects Versions: 0.10.2.0, 0.10.2.1, 0.10.2.2, 0.11.0.0, 0.11.0.1, > 0.11.0.2, 0.11.0.3, 1.0.0, 1.0.1, 1.0.2, 1.1.0, 1.1.1, 2.0.0, 2.0.1, 2.1.0, > 2.2.0, 2.1.1, 2.3.0, 2.2.1, 2.2.2, 2.4.0, 2.3.1, 2.5.0, 2.4.1, 2.6.0, 2.5.1, > 2.7.0, 2.6.1, 2.8.0, 2.7.1, 2.6.2, 3.1.0, 2.6.3, 2.7.2, 2.8.1, 3.0.0, 3.0.1, > 2.8.2, 3.2.0, 3.1.1, 3.3.0, 3.0.2, 3.1.2, 3.2.1, 3.4.0, 3.2.2, 3.2.3, 3.3.1, > 3.3.2, 3.5.0, 3.4.1 >Reporter: Yash Mayya >Assignee: Yash Mayya >Priority: Minor > Fix For: 3.6.0, 3.5.1 > > > If a RetriableException is thrown from an admin client or producer client > operation in > [AbstractWorkerSourceTask::sendRecords|https://github.com/apache/kafka/blob/5c2492bca71200806ccf776ea31639a90290d43e/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/AbstractWorkerSourceTask.java#L388], > the send operation is retried for the remaining records in the batch. There > is a bug in the logic for computing the remaining records in a batch which > causes records that are filtered out by the task's transformation chain to be > re-processed. This will also result in the SourceTask::commitRecord method > being called twice for the same record, which can cause certain types of > source connectors to fail. This bug seems to exist since when SMTs were first > introduced in 0.10.2 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (KAFKA-15145) AbstractWorkerSourceTask re-processes records filtered out by SMTs on retriable exceptions
[ https://issues.apache.org/jira/browse/KAFKA-15145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yash Mayya updated KAFKA-15145: --- Description: If a RetriableException is thrown from an admin client or producer client operation in [AbstractWorkerSourceTask::sendRecords|https://github.com/apache/kafka/blob/5c2492bca71200806ccf776ea31639a90290d43e/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/AbstractWorkerSourceTask.java#L388], the send operation is retried for the remaining records in the batch. There is a bug in the logic for computing the remaining records in a batch which causes records that are filtered out by the task's transformation chain to be re-processed. This will also result in the SourceTask::commitRecord method being called twice for the same record, which can cause certain types of source connectors to fail. This bug seems to exist since when SMTs were first introduced in 0.10.2 (was: If a RetriableException is thrown from an admin client or producer client operation in [AbstractWorkerSourceTask::sendRecords|https://github.com/apache/kafka/blob/5c2492bca71200806ccf776ea31639a90290d43e/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/AbstractWorkerSourceTask.java#L388], the send operation is retried for the remaining records in the batch. There is a minor bug in the logic for computing the remaining records in a batch which causes records that are filtered out by the task's transformation chain to be re-processed. This will also result in the SourceTask::commitRecord method being called twice for the same record, which can cause certain types of source connectors to fail. This bug seems to exist since when SMTs were first introduced in 0.10.2) > AbstractWorkerSourceTask re-processes records filtered out by SMTs on > retriable exceptions > -- > > Key: KAFKA-15145 > URL: https://issues.apache.org/jira/browse/KAFKA-15145 > Project: Kafka > Issue Type: Bug > Components: KafkaConnect >Affects Versions: 0.10.2.0, 0.10.2.1, 0.10.2.2, 0.11.0.0, 0.11.0.1, > 0.11.0.2, 0.11.0.3, 1.0.0, 1.0.1, 1.0.2, 1.1.0, 1.1.1, 2.0.0, 2.0.1, 2.1.0, > 2.2.0, 2.1.1, 2.3.0, 2.2.1, 2.2.2, 2.4.0, 2.3.1, 2.5.0, 2.4.1, 2.6.0, 2.5.1, > 2.7.0, 2.6.1, 2.8.0, 2.7.1, 2.6.2, 3.1.0, 2.6.3, 2.7.2, 2.8.1, 3.0.0, 3.0.1, > 2.8.2, 3.2.0, 3.1.1, 3.3.0, 3.0.2, 3.1.2, 3.2.1, 3.4.0, 3.2.2, 3.2.3, 3.3.1, > 3.3.2, 3.5.0, 3.4.1 >Reporter: Yash Mayya >Assignee: Yash Mayya >Priority: Minor > > If a RetriableException is thrown from an admin client or producer client > operation in > [AbstractWorkerSourceTask::sendRecords|https://github.com/apache/kafka/blob/5c2492bca71200806ccf776ea31639a90290d43e/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/AbstractWorkerSourceTask.java#L388], > the send operation is retried for the remaining records in the batch. There > is a bug in the logic for computing the remaining records in a batch which > causes records that are filtered out by the task's transformation chain to be > re-processed. This will also result in the SourceTask::commitRecord method > being called twice for the same record, which can cause certain types of > source connectors to fail. This bug seems to exist since when SMTs were first > introduced in 0.10.2 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (KAFKA-15145) AbstractWorkerSourceTask re-processes records filtered out by SMTs on retriable exceptions
[ https://issues.apache.org/jira/browse/KAFKA-15145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yash Mayya updated KAFKA-15145: --- Description: If a RetriableException is thrown from an admin client or producer client operation in [AbstractWorkerSourceTask::sendRecords|https://github.com/apache/kafka/blob/5c2492bca71200806ccf776ea31639a90290d43e/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/AbstractWorkerSourceTask.java#L388], the send operation is retried for the remaining records in the batch. There is a minor bug in the logic for computing the remaining records in a batch which causes records that are filtered out by the task's transformation chain to be re-processed. This will also result in the SourceTask::commitRecord method being called twice for the same record, which can cause certain types of source connectors to fail. This bug seems to exist since when SMTs were first introduced in 0.10.2 (was: If a RetriableException is thrown from an admin client or producer client operation in [AbstractWorkerSourceTask::sendRecords|https://github.com/apache/kafka/blob/5c2492bca71200806ccf776ea31639a90290d43e/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/AbstractWorkerSourceTask.java#L388], the send operation is retried for the remaining records in the batch. There is a minor bug in the logic for computing the remaining records in a batch which causes records that are filtered out by the task's transformation chain to be re-processed. This will also result in the SourceTask::commitRecord method being called twice for the same record, which can cause certain types of source connectors to fail. This bug seems to exist ever since when SMTs were first introduced in 0.10.2) > AbstractWorkerSourceTask re-processes records filtered out by SMTs on > retriable exceptions > -- > > Key: KAFKA-15145 > URL: https://issues.apache.org/jira/browse/KAFKA-15145 > Project: Kafka > Issue Type: Bug > Components: KafkaConnect >Affects Versions: 0.10.2.0, 0.10.2.1, 0.10.2.2, 0.11.0.0, 0.11.0.1, > 0.11.0.2, 0.11.0.3, 1.0.0, 1.0.1, 1.0.2, 1.1.0, 1.1.1, 2.0.0, 2.0.1, 2.1.0, > 2.2.0, 2.1.1, 2.3.0, 2.2.1, 2.2.2, 2.4.0, 2.3.1, 2.5.0, 2.4.1, 2.6.0, 2.5.1, > 2.7.0, 2.6.1, 2.8.0, 2.7.1, 2.6.2, 3.1.0, 2.6.3, 2.7.2, 2.8.1, 3.0.0, 3.0.1, > 2.8.2, 3.2.0, 3.1.1, 3.3.0, 3.0.2, 3.1.2, 3.2.1, 3.4.0, 3.2.2, 3.2.3, 3.3.1, > 3.3.2, 3.5.0, 3.4.1 >Reporter: Yash Mayya >Assignee: Yash Mayya >Priority: Minor > > If a RetriableException is thrown from an admin client or producer client > operation in > [AbstractWorkerSourceTask::sendRecords|https://github.com/apache/kafka/blob/5c2492bca71200806ccf776ea31639a90290d43e/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/AbstractWorkerSourceTask.java#L388], > the send operation is retried for the remaining records in the batch. There > is a minor bug in the logic for computing the remaining records in a batch > which causes records that are filtered out by the task's transformation chain > to be re-processed. This will also result in the SourceTask::commitRecord > method being called twice for the same record, which can cause certain types > of source connectors to fail. This bug seems to exist since when SMTs were > first introduced in 0.10.2 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (KAFKA-15145) AbstractWorkerSourceTask re-processes records filtered out by SMTs on retriable exceptions
[ https://issues.apache.org/jira/browse/KAFKA-15145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yash Mayya updated KAFKA-15145: --- Description: If a RetriableException is thrown from an admin client or producer client operation in [AbstractWorkerSourceTask::sendRecords|https://github.com/apache/kafka/blob/5c2492bca71200806ccf776ea31639a90290d43e/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/AbstractWorkerSourceTask.java#L388], the send operation is retried for the remaining records in the batch. There is a minor bug in the logic for computing the remaining records in a batch which causes records that are filtered out by the task's transformation chain to be re-processed. This will also result in the SourceTask::commitRecord method being called twice for the same record, which can cause certain types of source connectors to fail. This bug seems to exist since when SMTs were first introduced in 0.10.2 (was: If a RetriableException is thrown from an admin client or producer client operation in [AbstractWorkerSourceTask::sendRecords|https://github.com/apache/kafka/blob/5c2492bca71200806ccf776ea31639a90290d43e/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/AbstractWorkerSourceTask.java#L388], the send operation is retried for the remaining records in the batch. There is a minor bug in the logic for computing the remaining records for a batch which causes records that are filtered out by the task's transformation chain to be re-processed. This will also result in the SourceTask::commitRecord method being called twice for the same record, which can cause certain types of source connectors to fail. This bug seems to exist since when SMTs were first introduced in 0.10.2) > AbstractWorkerSourceTask re-processes records filtered out by SMTs on > retriable exceptions > -- > > Key: KAFKA-15145 > URL: https://issues.apache.org/jira/browse/KAFKA-15145 > Project: Kafka > Issue Type: Bug > Components: KafkaConnect >Affects Versions: 0.10.2.0, 0.10.2.1, 0.10.2.2, 0.11.0.0, 0.11.0.1, > 0.11.0.2, 0.11.0.3, 1.0.0, 1.0.1, 1.0.2, 1.1.0, 1.1.1, 2.0.0, 2.0.1, 2.1.0, > 2.2.0, 2.1.1, 2.3.0, 2.2.1, 2.2.2, 2.4.0, 2.3.1, 2.5.0, 2.4.1, 2.6.0, 2.5.1, > 2.7.0, 2.6.1, 2.8.0, 2.7.1, 2.6.2, 3.1.0, 2.6.3, 2.7.2, 2.8.1, 3.0.0, 3.0.1, > 2.8.2, 3.2.0, 3.1.1, 3.3.0, 3.0.2, 3.1.2, 3.2.1, 3.4.0, 3.2.2, 3.2.3, 3.3.1, > 3.3.2, 3.5.0, 3.4.1 >Reporter: Yash Mayya >Assignee: Yash Mayya >Priority: Minor > > If a RetriableException is thrown from an admin client or producer client > operation in > [AbstractWorkerSourceTask::sendRecords|https://github.com/apache/kafka/blob/5c2492bca71200806ccf776ea31639a90290d43e/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/AbstractWorkerSourceTask.java#L388], > the send operation is retried for the remaining records in the batch. There > is a minor bug in the logic for computing the remaining records in a batch > which causes records that are filtered out by the task's transformation chain > to be re-processed. This will also result in the SourceTask::commitRecord > method being called twice for the same record, which can cause certain types > of source connectors to fail. This bug seems to exist since when SMTs were > first introduced in 0.10.2 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (KAFKA-15145) AbstractWorkerSourceTask re-processes records filtered out by SMTs on retriable exceptions
[ https://issues.apache.org/jira/browse/KAFKA-15145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yash Mayya updated KAFKA-15145: --- Description: If a RetriableException is thrown from an admin client or producer client operation in [AbstractWorkerSourceTask::sendRecords|https://github.com/apache/kafka/blob/5c2492bca71200806ccf776ea31639a90290d43e/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/AbstractWorkerSourceTask.java#L388], the send operation is retried for the remaining records in the batch. There is a minor bug in the logic for computing the remaining records in a batch which causes records that are filtered out by the task's transformation chain to be re-processed. This will also result in the SourceTask::commitRecord method being called twice for the same record, which can cause certain types of source connectors to fail. This bug seems to exist ever since when SMTs were first introduced in 0.10.2 (was: If a RetriableException is thrown from an admin client or producer client operation in [AbstractWorkerSourceTask::sendRecords|https://github.com/apache/kafka/blob/5c2492bca71200806ccf776ea31639a90290d43e/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/AbstractWorkerSourceTask.java#L388], the send operation is retried for the remaining records in the batch. There is a minor bug in the logic for computing the remaining records in a batch which causes records that are filtered out by the task's transformation chain to be re-processed. This will also result in the SourceTask::commitRecord method being called twice for the same record, which can cause certain types of source connectors to fail. This bug seems to exist since when SMTs were first introduced in 0.10.2) > AbstractWorkerSourceTask re-processes records filtered out by SMTs on > retriable exceptions > -- > > Key: KAFKA-15145 > URL: https://issues.apache.org/jira/browse/KAFKA-15145 > Project: Kafka > Issue Type: Bug > Components: KafkaConnect >Affects Versions: 0.10.2.0, 0.10.2.1, 0.10.2.2, 0.11.0.0, 0.11.0.1, > 0.11.0.2, 0.11.0.3, 1.0.0, 1.0.1, 1.0.2, 1.1.0, 1.1.1, 2.0.0, 2.0.1, 2.1.0, > 2.2.0, 2.1.1, 2.3.0, 2.2.1, 2.2.2, 2.4.0, 2.3.1, 2.5.0, 2.4.1, 2.6.0, 2.5.1, > 2.7.0, 2.6.1, 2.8.0, 2.7.1, 2.6.2, 3.1.0, 2.6.3, 2.7.2, 2.8.1, 3.0.0, 3.0.1, > 2.8.2, 3.2.0, 3.1.1, 3.3.0, 3.0.2, 3.1.2, 3.2.1, 3.4.0, 3.2.2, 3.2.3, 3.3.1, > 3.3.2, 3.5.0, 3.4.1 >Reporter: Yash Mayya >Assignee: Yash Mayya >Priority: Minor > > If a RetriableException is thrown from an admin client or producer client > operation in > [AbstractWorkerSourceTask::sendRecords|https://github.com/apache/kafka/blob/5c2492bca71200806ccf776ea31639a90290d43e/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/AbstractWorkerSourceTask.java#L388], > the send operation is retried for the remaining records in the batch. There > is a minor bug in the logic for computing the remaining records in a batch > which causes records that are filtered out by the task's transformation chain > to be re-processed. This will also result in the SourceTask::commitRecord > method being called twice for the same record, which can cause certain types > of source connectors to fail. This bug seems to exist ever since when SMTs > were first introduced in 0.10.2 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (KAFKA-15145) AbstractWorkerSourceTask re-processes records filtered out by SMTs on retriable exceptions
[ https://issues.apache.org/jira/browse/KAFKA-15145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yash Mayya updated KAFKA-15145: --- Affects Version/s: 3.4.1 3.5.0 3.3.2 3.3.1 3.2.3 3.2.2 3.4.0 3.2.1 3.1.2 3.0.2 3.3.0 3.1.1 3.2.0 2.8.2 3.0.1 3.0.0 2.8.1 2.7.2 2.6.3 3.1.0 2.6.2 2.7.1 2.8.0 2.6.1 2.7.0 2.5.1 2.6.0 2.4.1 2.5.0 2.3.1 2.4.0 2.2.2 2.2.1 2.3.0 2.1.1 2.2.0 2.1.0 2.0.1 2.0.0 1.1.1 1.1.0 1.0.2 1.0.1 1.0.0 0.11.0.3 0.11.0.2 0.11.0.1 0.11.0.0 0.10.2.2 0.10.2.1 0.10.2.0 > AbstractWorkerSourceTask re-processes records filtered out by SMTs on > retriable exceptions > -- > > Key: KAFKA-15145 > URL: https://issues.apache.org/jira/browse/KAFKA-15145 > Project: Kafka > Issue Type: Bug > Components: KafkaConnect >Affects Versions: 0.10.2.0, 0.10.2.1, 0.10.2.2, 0.11.0.0, 0.11.0.1, > 0.11.0.2, 0.11.0.3, 1.0.0, 1.0.1, 1.0.2, 1.1.0, 1.1.1, 2.0.0, 2.0.1, 2.1.0, > 2.2.0, 2.1.1, 2.3.0, 2.2.1, 2.2.2, 2.4.0, 2.3.1, 2.5.0, 2.4.1, 2.6.0, 2.5.1, > 2.7.0, 2.6.1, 2.8.0, 2.7.1, 2.6.2, 3.1.0, 2.6.3, 2.7.2, 2.8.1, 3.0.0, 3.0.1, > 2.8.2, 3.2.0, 3.1.1, 3.3.0, 3.0.2, 3.1.2, 3.2.1, 3.4.0, 3.2.2, 3.2.3, 3.3.1, > 3.3.2, 3.5.0, 3.4.1 >Reporter: Yash Mayya >Assignee: Yash Mayya >Priority: Minor > > If a RetriableException is thrown from an admin client or producer client > operation in > [AbstractWorkerSourceTask::sendRecords|https://github.com/apache/kafka/blob/5c2492bca71200806ccf776ea31639a90290d43e/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/AbstractWorkerSourceTask.java#L388], > the send operation is retried for the remaining records in the batch. There > is a minor bug in the logic for computing the remaining records for a batch > which causes records that are filtered out by the task's transformation chain > to be re-processed. This will also result in the SourceTask::commitRecord > method being called twice for the same record, which can cause certain types > of source connectors to fail. This bug seems to exist since when SMTs were > first introduced in 0.10.2 -- This message was sent by Atlassian Jira (v8.20.10#820010)
