[ https://issues.apache.org/jira/browse/KAFKA-6556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matthias J. Sax updated KAFKA-6556: ----------------------------------- Fix Version/s: (was: 2.1.0) 2.2.0 > allow to attach callbacks in kafka streams, to be triggered when a window is > expired > ------------------------------------------------------------------------------------- > > Key: KAFKA-6556 > URL: https://issues.apache.org/jira/browse/KAFKA-6556 > Project: Kafka > Issue Type: New Feature > Components: streams > Reporter: igor mazor > Assignee: John Roesler > Priority: Major > Fix For: 2.2.0 > > > Allowing to attach callbacks in kafka streams, to be triggered when a window > is expired, > would help in use cases when the final state of the window is required. > It would be also useful if together with that functionally the user would be > able to control whether the callback would be triggered in addition to > emitting the normal change log down the stream, or only triggering the > callback when the window is expired. (for example in cases when only the > final window state is required, and any updates to the window state during > the window time interval are not important) > An example for use case could be left join with proper sql semantics: > A left join B currently would emit (a1,null) and (a1,b1) if b1 arrives > within the defined join time window. > what I would like is to have ONLY ONE result: > (a1,null) if no b1 arrived during the defined join time window, OR ONLY > (a1,b1) if b1 did arrived in the specified join time window. > One possible solution could be to use the current kafka streams left join > with downstream processor which would put the results in a windowed Ktable. > The window size would be same as for the left join operation, however, only > the final state of that window would be emitted down the stream once the time > window is expired. > So if the left join produces (a1, null) and after X minutes no (a1, b1) was > produced, eventually only (a1, null) would be emitted, on the other hand, if > the left join produces (a1, null) and after X-t minutes (a1, b1) was produced > by the left join operation => only (a1, b1) would be emitted eventually down > the stream after X minutes. > > Another use case is when the window state is written to another kafka topic > which is then used to persist the window states into a db, However, many > times only the final window state > is required, and functionality to get only the last window state would help > in reducing load from the db, since only the final window state would be > persisted to the db, instead of multiple db writes for each window state > update. > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)