[
https://issues.apache.org/jira/browse/KAFKA-12606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Gustafson updated KAFKA-12606:
------------------------------------
Description:
KIP-467 introduces an extension to the produce response to let the producer
identify the specific records that had failed validation. While implementing
KAFKA-12548, we realized a few shortcomings in this proposal:
1. Some records may fail only because they are part of a batch which had a
record failing validation. In this case, we can return a better exception to
the user so that they know the record is safe to be retried. For example,
`RecordNotAppendedException` or something like that.
2. Records in the same batch may fail validation for different reasons. For
example, one record may fail because of an invalid timestamp; another may fail
because it uses a null key and the topic is compacted. However, the schema only
allows a single error for each partition, so we cannot distinguish these cases
in order to throw more specific exception types. We should consider allowing a
record-level error code as well.
was:
KIP-467 introduces an extension to the produce response to let the producer
identify the specific records that had failed validation. While implementing
KAFKA-12548, we realized a few shortcomings in this proposal:
1. Some records may fail only because they are part of a batch which had a
record failing validation. In this case, we can return a better exception to
the user so that they know the record is safe to be retried. For example,
`RecordNotAppendedException` or something like that.
2. Records in the same batch may fail validation for different reasons. For
example, one record may fail because of an invalid timestamp; another may fail
because it uses a null key and the topic is compacted. However, the schema only
allows a single error for each partition, so we cannot distinguish these cases
in order to throw more specific exception types.
> Some improvements for produce record validation
> -----------------------------------------------
>
> Key: KAFKA-12606
> URL: https://issues.apache.org/jira/browse/KAFKA-12606
> Project: Kafka
> Issue Type: Improvement
> Reporter: Jason Gustafson
> Priority: Major
> Labels: needs-kip
>
> KIP-467 introduces an extension to the produce response to let the producer
> identify the specific records that had failed validation. While implementing
> KAFKA-12548, we realized a few shortcomings in this proposal:
> 1. Some records may fail only because they are part of a batch which had a
> record failing validation. In this case, we can return a better exception to
> the user so that they know the record is safe to be retried. For example,
> `RecordNotAppendedException` or something like that.
> 2. Records in the same batch may fail validation for different reasons. For
> example, one record may fail because of an invalid timestamp; another may
> fail because it uses a null key and the topic is compacted. However, the
> schema only allows a single error for each partition, so we cannot
> distinguish these cases in order to throw more specific exception types. We
> should consider allowing a record-level error code as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
