[
https://issues.apache.org/jira/browse/KAFKA-19096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18038285#comment-18038285
]
Matthias J. Sax commented on KAFKA-19096:
-----------------------------------------
Oh. Nice. Let's update the test for this case – seems the test is not
"complete" as my refactoring did not change any test code, so there is clearly
a testing gap.
> Skipped Records in FK join should not be marked as dropped
> ----------------------------------------------------------
>
> Key: KAFKA-19096
> URL: https://issues.apache.org/jira/browse/KAFKA-19096
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Affects Versions: 4.0.0
> Reporter: Ayoub Omari
> Priority: Major
>
> Currently, we are logging all skipped records in FK join as dropped.
> This seems incorrect, as a dropped record essentially means an invalid record.
> In the case of FK join, we are logging as dropped, records which are totally
> valid, that simply don't affect the join result (i.e. no output record, no FK
> subscription, no FK subscription removal).
> Quote from @mjsax:
> ??That's is not really the definition of "dropped records" – "dropped
> records" is a metric for malformed records that could not be processed. – Eg,
> a {{{}null{}}}-key record for a table is not valid, because we need a valid
> PK, and thus we "drop" such a record??
> Check this
> [link|https://github.com/apache/kafka/pull/19303#discussion_r2029558102] for
> complete discussion.
> The goal of this ticket is:
> - Fix the code in FK join classes by not marking skipped records as dropped
> when they aren't really so.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
