[ 
https://issues.apache.org/jira/browse/KAFKA-6556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16609420#comment-16609420
 ] 

John Roesler commented on KAFKA-6556:
-------------------------------------

Yes, you're right. I was thinking superficially before, but this ticket 
specifically mentions joins.

It should be possible to do something similar, but it wouldn't be automatically 
fixed by KIP-328 final-result mode, since joins don't result in 
`KTable<Windowed<..>,..`.

> allow to attach callbacks in kafka streams, to be triggered when a window is 
> expired 
> -------------------------------------------------------------------------------------
>
>                 Key: KAFKA-6556
>                 URL: https://issues.apache.org/jira/browse/KAFKA-6556
>             Project: Kafka
>          Issue Type: New Feature
>          Components: streams
>            Reporter: igor mazor
>            Assignee: John Roesler
>            Priority: Major
>             Fix For: 2.1.0
>
>
> Allowing to attach callbacks in kafka streams, to be triggered when a window 
> is expired,
>  would help in use cases when the final state of the window is required.
>  It would be also useful if together with that functionally the user would be 
> able to control whether the callback would be triggered in addition to 
> emitting the normal change log down the stream, or only triggering the 
> callback when the window is expired. (for example in cases when only the 
> final window state is required, and any updates to the window state during 
> the window time interval are not important)  
> An example for use case could be left join with proper sql semantics:
>  A left join B currently would emit (a1,null) and (a1,b1) if b1 arrives 
> within the defined join time window.
>  what I would like is to have ONLY ONE result:
>  (a1,null) if no b1 arrived during the defined join time window, OR ONLY 
> (a1,b1) if b1 did arrived in the specified join time window.
>  One possible solution could be to use the current kafka streams left join 
> with downstream processor which would put the results in a windowed Ktable.
>  The window size would be same as for the left join operation, however, only 
> the final state of that window would be emitted down the stream once the time 
> window is expired.
>  So if the left join produces (a1, null) and after X minutes no (a1, b1) was 
> produced, eventually only (a1, null) would be emitted, on the other hand, if 
> the left join produces (a1, null) and after X-t minutes (a1, b1) was produced 
> by the left join operation => only (a1, b1) would be emitted eventually down 
> the stream after X minutes.
>  
> Another use case is when the window state is written to another kafka topic 
> which is then used to persist the window states into a db, However, many 
> times only the final window state
>  is required, and functionality to get only the last window state would help 
> in reducing load from the db, since only the final window state would be 
> persisted to the db, instead of multiple db writes for each window state 
> update. 
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to