Ayoub Omari created KAFKA-19096:
-----------------------------------
Summary: Skipped Records in FK join should not be marked as dropped
Key: KAFKA-19096
URL: https://issues.apache.org/jira/browse/KAFKA-19096
Project: Kafka
Issue Type: Improvement
Affects Versions: 4.0.0
Reporter: Ayoub Omari
Currently, we are logging all skipped records in FK join as dropped.
This seems incorrect, as a dropped record essentially means an invalid record.
In the case of FK join, we are logging as dropped, records which are totally
valid, that simply don't affect the join result (i.e. no output record, no FK
subscription, no FK subscription removal).
Quote from @mjsax:
??That's is not really the definition of "dropped records" -- "dropped records"
is a metric for malformed records that could not be processed. -- Eg, a
{{{}null{}}}-key record for a table is not valid, because we need a valid PK,
and thus we "drop" such a record ??
Check this
[link|https://github.com/apache/kafka/pull/19303#discussion_r2029558102] for
complete discussion.
The goal of this ticket is:
- Fix the code in FK join classes by not marking skipped records as dropped
when they aren't really so.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
