Hi all, the flow in our pipeline is:
1. read event X from kafka. open fixed window of 30 sec. 2. read event subscription from kafka. open GlobalWindow and store a state of all subscriptions. 3. match X and Y using key and if there is a match send an event to another kafka topic. (we use the state as side input) if a user unsubscribes (meaning we read from a different source an event of unsubscribe) we would like to delete the relevant entry from the state. Can this be achieved only using state and not using some external cache/db? I am aware there is an option to add timers on state but the expiration logic is not time based. Any suggestions? Thanks in advance Sigalit
