[
https://issues.apache.org/jira/browse/KAFKA-21101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18115694#comment-18115694
]
Chia-Ping Tsai commented on KAFKA-21101:
----------------------------------------
{quote}
Reading out of order is problematic for some applications and they would rather
repopulate the topic from the source (if there is one). So, this bit isn't
necessarily right.
{quote}
You are right. My point was that applications can't parse either the key or the
value, but they do see the failure, and some of them rely on it. I have updated
the description from "dropping them loses nothing observable" to "dropping them
is a more targeted fix than deleteRecords, which removes everything before an
offset". Either way the option stays opt-in, and we will cover the trigger
mechanism in the KIP.
> Drop the remaining records of a batch when encountering an invalid record
> -------------------------------------------------------------------------
>
> Key: KAFKA-21101
> URL: https://issues.apache.org/jira/browse/KAFKA-21101
> Project: Kafka
> Issue Type: Improvement
> Reporter: Chia-Ping Tsai
> Assignee: majialong
> Priority: Major
> Labels: need-kip
>
> Currently, an invalid record marks the partition "uncleanable", which blocks
> the cleaner from compacting all other records in that partition. Since the
> record is already invalid regardless of the root cause, nobody can read it or
> anything after it in the same batch (consumers parse a batch sequentially and
> fail at the same point), so dropping them loses nothing observable. We should
> offer an option to drop the invalid record and the remaining records of its
> batch, and keep the partition cleanable. Records before the invalid one are
> parsed successfully and are retained as usual.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
