[ https://issues.apache.org/jira/browse/KAFKA-15309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matthias J. Sax updated KAFKA-15309: ------------------------------------ Description: The producer collects multiple records into batches, and a single record specific error might fail the whole batch (eg, `RecordTooLargeException`). This ticket suggests to add a per-record error handler, that allows user to opt into skipping bad records without failing the whole batch (similar to Kafka Streams `ProductionExceptionHandler`). The fix of https://issues.apache.org/jira/browse/KAFKA-9279 caused https://issues.apache.org/jira/browse/KAFKA-15259 which inspired this ticket. Another example for which a production exception handler could be useful, if a user tries to write into a non-existing topic, which returns a retryable error code; with infinite retries the producer would hang retrying forever. A handler could help to break the infinite retry loop. was: The producer collects multiple records into batches, and a single record specific error might fail the whole batch (eg, `RecordTooLargeException`). This ticket suggests to add a per-record error handler, that allows user to opt into skipping bad records without failing the whole batch (similar to Kafka Streams `ProductionExceptionHandler`). The fix of https://issues.apache.org/jira/browse/KAFKA-9279 caused https://issues.apache.org/jira/browse/KAFKA-15259 which inspired this ticket. Another example for which a production exception handler would be useful, if a user tries to write into a non-existing topic, which returns a retryable error code; with infinite retries the producer would hang retrying forever. > Add custom error handler to Producer > ------------------------------------ > > Key: KAFKA-15309 > URL: https://issues.apache.org/jira/browse/KAFKA-15309 > Project: Kafka > Issue Type: New Feature > Components: producer > Reporter: Matthias J. Sax > Priority: Major > Labels: needs-kip > > The producer collects multiple records into batches, and a single record > specific error might fail the whole batch (eg, `RecordTooLargeException`). > This ticket suggests to add a per-record error handler, that allows user to > opt into skipping bad records without failing the whole batch (similar to > Kafka Streams `ProductionExceptionHandler`). > The fix of https://issues.apache.org/jira/browse/KAFKA-9279 caused > https://issues.apache.org/jira/browse/KAFKA-15259 which inspired this ticket. > Another example for which a production exception handler could be useful, if > a user tries to write into a non-existing topic, which returns a retryable > error code; with infinite retries the producer would hang retrying forever. A > handler could help to break the infinite retry loop. -- This message was sent by Atlassian Jira (v8.20.10#820010)