Re: Broadcasting to multiple operators

2021-03-05 Thread David Anderson
Glad to hear it! Thanks for letting us know. David On Fri, Mar 5, 2021 at 10:22 PM Roger wrote: > Confirmed. This worked! > Thanks! > Roger > > On Fri, Mar 5, 2021 at 12:41 PM Roger wrote: > >> Hey David. >> Thank you very much for your response. This is making sense now. It was >> confusing b

Re: Broadcasting to multiple operators

2021-03-05 Thread Roger
Confirmed. This worked! Thanks! Roger On Fri, Mar 5, 2021 at 12:41 PM Roger wrote: > Hey David. > Thank you very much for your response. This is making sense now. It was > confusing because I was able to use the Broadcast stream prior to adding > the second stream. However, now I realize that th

Re: Broadcasting to multiple operators

2021-03-05 Thread Roger
Hey David. Thank you very much for your response. This is making sense now. It was confusing because I was able to use the Broadcast stream prior to adding the second stream. However, now I realize that this part of the pipeline occurs after the windowing so I'm not affected the same way. This is d

Re: Broadcasting to multiple operators

2021-03-05 Thread David Anderson
This is a watermarking issue. Whenever an operator has two or more input streams, its watermark is the minimum of watermarks of the incoming streams. In this case your broadcast stream doesn't have a watermark generator, so it is preventing the watermarks from advancing. This in turn is preventing

Broadcasting to multiple operators

2021-03-05 Thread Roger
Hello. I am having an issue with a Flink 1.8 pipeline when trying to consume broadcast state across multiple operators. I currently have a working pipeline that looks like the following: records .assignTimestampsAndWatermarks( new BoundedOutOfOrdernessGenerator( Long.parseLong(proper