[
https://issues.apache.org/jira/browse/KAFKA-14747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthias J. Sax updated KAFKA-14747:
------------------------------------
Description:
FK-joins are subject to a race condition: If the left-hand side record is
updated, a subscription is sent to the right-hand side (including a hash value
of the left-hand side record), and the right-hand side might send back join
responses (also including the original hash). The left-hand side only processed
the responses if the returned hash matches to current hash of the left-hand
side record, because a different hash implies that the lef- hand side record
was updated in the mean time (including sending a new subscription to the right
hand side), and thus the data is stale and the response should not be processed
(joining the response to the new record could lead to incorrect results).
A similar thing can happen on a right-hand side update that triggers a
response, that might be dropped if the left-hand side record was updated in
parallel.
While the behavior is correct, we don't record if this happens. We should
consider to record this using the existing "dropped record" sensor or maybe add
a new sensor.
was:
FK-join are subject to a race condition. If the left-hand side record is
updated, a subscription is sent to the right-hand side (including a hash value
of the left-hand side record), and the right-hand side might send back join
responses (also including the original hash). The left hand side only processed
the responses if the hash matches, because a different hash implies that the
left hand side row was updated in the mean-time (including sending a new
subscription to the right hand side), and thus the data is stale and the
response should not be processed.
A similar thing can happen on a right hand side update that triggers a
response, that might be dropped if the left hand side row was updated in
parallel.
While the behavior is correct, we don't record if this happens. We should
consider to record this using the existing "dropped record" sensor or maybe add
a new sensor.
> FK join should record discarded subscription responses
> ------------------------------------------------------
>
> Key: KAFKA-14747
> URL: https://issues.apache.org/jira/browse/KAFKA-14747
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Reporter: Matthias J. Sax
> Priority: Minor
> Labels: beginner, needs-kip, newbie
>
> FK-joins are subject to a race condition: If the left-hand side record is
> updated, a subscription is sent to the right-hand side (including a hash
> value of the left-hand side record), and the right-hand side might send back
> join responses (also including the original hash). The left-hand side only
> processed the responses if the returned hash matches to current hash of the
> left-hand side record, because a different hash implies that the lef- hand
> side record was updated in the mean time (including sending a new
> subscription to the right hand side), and thus the data is stale and the
> response should not be processed (joining the response to the new record
> could lead to incorrect results).
> A similar thing can happen on a right-hand side update that triggers a
> response, that might be dropped if the left-hand side record was updated in
> parallel.
> While the behavior is correct, we don't record if this happens. We should
> consider to record this using the existing "dropped record" sensor or maybe
> add a new sensor.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
