Yes. But the filter us usually a very light weight operation. From: Mingliang Liu <[email protected]> Sent: Monday, June 3, 2024 7:16 PM To: Colletta, Edward <[email protected]> Cc: mejri houssem <[email protected]>; [email protected] Subject: Re: Implementing Multiple sink
NOTICE: This email is from an external sender - do not click on links or attachments unless you recognize the sender and know the content is safe. Colletta, I think that way, the upstream stream `streamWithMultipleConditions` will get processed twice, instead of once? Thanks, On Mon, Jun 3, 2024 at 10:28 AM Colletta, Edward <[email protected]<mailto:[email protected]>> wrote: I usually just reuse the stream, sending it to through different filters and adding different sinks to the filtered streams. Something like streamWithMultipleConditions.filter(FilterForCondition1) .addSink(SinkforCondtiton1); streamWithMultipleConditions.filter(FilterForCondition2) .addSink(SinkforCondtiton2); From: Mingliang Liu <[email protected]<mailto:[email protected]>> Sent: Monday, June 3, 2024 1:03 PM To: mejri houssem <[email protected]<mailto:[email protected]>> Cc: [email protected]<mailto:[email protected]> Subject: Re: Implementing Multiple sink NOTICE: This email is from an external sender - do not click on links or attachments unless you recognize the sender and know the content is safe. Hi Mejri, Have you checked side outputs? https://nightlies.apache.org/flink/flink-docs-release-1.19/docs/dev/datastream/side_output/ On Mon, Jun 3, 2024 at 8:28 AM mejri houssem <[email protected]<mailto:[email protected]>> wrote: Hello community, We have a use case in our Flink job that requires the implementation of multiple sinks. I need to filter messages based on certain conditions (information in the message) to determine which sink to dispatch them to. To clarify, I would like to implement logic in the operator that precedes the sink in the job workflow, similar to the following: * If the condition is met, send the message to the first sink. * Otherwise, send the message to the second sink. Is it possible to implement this in Flink? Thank you in advance for your assistance. Best regards,
