Hi Sagar, have you had a look at CoProcessFunction [1]? CoProcessFunction enables you to join two streams into one and also provide context to use SideOutput [2].
Best, Matthias [1] https://github.com/apache/flink/blob/c6997c97c575d334679915c328792b8a3067cfb5/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/co/CoProcessFunction.java [2] https://ci.apache.org/projects/flink/flink-docs-master/learn-flink/event_driven.html#side-outputs On Wed, Jan 20, 2021 at 4:51 PM sagar <sagarban...@gmail.com> wrote: > Hi Team, > > > I am creating a flink job with DataStream API and batch mode. > > It is having 5 different bounded sources and I need to perform some > business operations on it like joining , aggregating etc. > > > > I am using a CoGroup operator to join two streams as it serves as a left > join. So when keys are present in both the stream, I am processing and > moving ahead. > > But when there is only one key present I need to send it as an error. > > > > Some operators like Process have side output features, but CoGroup doesn't > have that feature. > > > > In order to report missing data to different stream, I am planning to > create one common error handling stream and at each CoGroup operation I am > planning to write it to error stream by using Split operator after CoGroup > > > > Let me know if that is the correct way of handling the errors? > > -- > ---Regards--- > > Sagar Bandal > > This is confidential mail ,All Rights are Reserved.If you are not the > intended recipient please ignore this email. >
