Hi Joe, could you please check (in web UI) if the watermark is advancing past the join? The window operator would not trigger if it doesn't advance.
On which Flink version are you running? On Tue, Jun 8, 2021 at 10:13 PM Joseph Lorenzini <jlorenz...@gohealth.com> wrote: > Hi all, > > > > I have observed behavior joining two keyed streams together, where events > are never emitted. The source of each stream is a different kafka topic. I > am curious to know if this expected and if there’s a way to work around it. > > > > I am using a tumbling event window. All records across the two kafka > topics occurred within the same 5 second window of time. Each kafka topic > has a single partition. For each kafka topic, I configured the flink kafka > consumer like so: > > > > consumer.assignTimestampsAndWatermarks( > > WatermarkStrategy > > > .<CustomersUnion>forBoundedOutOfOrderness(Duration.ofSeconds(10)) > > .withIdleness(Duration.ofSeconds(10)) > > ); > > > > The tumbling window has a duration of 60 seconds. Now it happens to be the > case that there is only a single event when joining on a key. If I use > Tumbling Process window then events are emitted as expected. If I actually > ensure there are multiple events for a key then the events are also > emitted. However, if it’s a single event per key in a tumbling event window > then no events are emitted. > > > > Is this expected and if it how do you handle this use case? > > > > Thanks, > > Joe > Privileged/Confidential Information may be contained in this message. If > you are not the addressee indicated in this message (or responsible for > delivery of the message to such person), you may not copy or deliver this > message to anyone. In such case, you should destroy this message and kindly > notify the sender by reply email. Please advise immediately if you or your > employer does not consent to Internet email for messages of this kind. > Opinions, conclusions and other information in this message that do not > relate to the official business of my firm shall be understood as neither > given nor endorsed by it. >
