Hello.

I'm joining several data streams, using ConnectedStreams. Let's say something 
like A connect B which outputs AB, and then I join AB with C, which outputs ABC.

However, the relationship between A and B, or AB and C may be of 1 to many, or 
1 to 1, depending on the case. For the 1 to 1, it's expected to produce an 
output as soon as I obtain both records that match the same key, but for the 1 
to many, it's not so simple, there's no event that guarantees that I have 
obtained all of my <many> records. For this, I've searched and found out that I 
have to implement an onTimer method, in a Process Function. I also read that an 
event timer is registered in a queue and when a new watermark arrives, the 
event timers that <passed> will be triggered.

Given this, I'm running into trouble when generating new watermarks like in the 
example: 
https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/event_timestamps_watermarks.html#with-periodic-watermarks

The solution I've reached basically processes (on the onTimer method) all the 
records that I obtained previously when the first watermark arrives, but after 
that, it stops triggering.

I read that watermarks are global, so how can I create a new watermark when 
working with several streams?

Thank you

Reply via email to