Matthias J. Sax created KAFKA-4325:
--------------------------------------
Summary: Improve processing of late records for window operations
Key: KAFKA-4325
URL: https://issues.apache.org/jira/browse/KAFKA-4325
Project: Kafka
Issue Type: Improvement
Components: streams
Reporter: Matthias J. Sax
Assignee: Guozhang Wang
Priority: Minor
Windows are kept until their retention time passed. If a late arriving record
is processed that is older than any window kept, a new window is created
containing this single late arriving record, the aggregation is computed and
the window is immediately discarded afterward (as it is older than retention
time).
This behavior might case problems for downstream application as the original
window aggregate might we overwritten with the late single-record- aggregate
value. Thus, we should rather not process the late arriving record for this
case.
However, data loss might not be acceptable for all use cases. In order to
enable the use to not lose any data, window operators should allow to register
a handler function that is called instead of just dropping the late arriving
record.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
