Yes. But the filter us usually a very light weight operation. From: Mingliang Liu <lium...@apache.org> Sent: Monday, June 3, 2024 7:16 PM To: Colletta, Edward <edward.colle...@fmr.com> Cc: mejri houssem <mejrihousse...@gmail.com>; user@flink.apache.org Subject: Re: Implementing Multiple sink
NOTICE: This email is from an external sender - do not click on links or attachments unless you recognize the sender and know the content is safe. Colletta, I think that way, the upstream stream `streamWithMultipleConditions` will get processed twice, instead of once? Thanks, On Mon, Jun 3, 2024 at 10:28 AM Colletta, Edward <edward.colle...@fmr.com<mailto:edward.colle...@fmr.com>> wrote: I usually just reuse the stream, sending it to through different filters and adding different sinks to the filtered streams. Something like streamWithMultipleConditions.filter(FilterForCondition1) .addSink(SinkforCondtiton1); streamWithMultipleConditions.filter(FilterForCondition2) .addSink(SinkforCondtiton2); From: Mingliang Liu <lium...@apache.org<mailto:lium...@apache.org>> Sent: Monday, June 3, 2024 1:03 PM To: mejri houssem <mejrihousse...@gmail.com<mailto:mejrihousse...@gmail.com>> Cc: user@flink.apache.org<mailto:user@flink.apache.org> Subject: Re: Implementing Multiple sink NOTICE: This email is from an external sender - do not click on links or attachments unless you recognize the sender and know the content is safe. Hi Mejri, Have you checked side outputs? https://nightlies.apache.org/flink/flink-docs-release-1.19/docs/dev/datastream/side_output/ On Mon, Jun 3, 2024 at 8:28 AM mejri houssem <mejrihousse...@gmail.com<mailto:mejrihousse...@gmail.com>> wrote: Hello community, We have a use case in our Flink job that requires the implementation of multiple sinks. I need to filter messages based on certain conditions (information in the message) to determine which sink to dispatch them to. To clarify, I would like to implement logic in the operator that precedes the sink in the job workflow, similar to the following: * If the condition is met, send the message to the first sink. * Otherwise, send the message to the second sink. Is it possible to implement this in Flink? Thank you in advance for your assistance. Best regards,
