Hi Deepak,
Coming back to your original question, you’ve got a number of option (some of
them already mentioned:
* You can connect/join 2 streams of different types at a time by means of
s1.connect(s2).<some processing function>
* (your example does not work directly as written (3 streams))
* You can connect many streams of same type by means of s1.union(s2, s3,
…).<some processing function>
* The third option is new and not yet documented (marked as @experimental):
* Connect/join many streams of different types by means of a
[Keyed]MultipleInputTransformation
* If keyed: input streams need to either
* be keyed on the same key type, or
* be non-keyed (most likely) broadcast streams
* the API is still a little elaborate, a good starting point could be
this test case: [1]
If of any interest, feel free to ask for clarifications …
Thias
PS: see also my next email in a couple of minutes …
[1]
https://github.com/apache/flink/blob/master/flink-streaming-java/src/test/java/org/apache/flink/streaming/api/graph/StreamGraphGeneratorBatchExecutionTest.java#L441
From: Shammon FY <zjur...@gmail.com>
Sent: Wednesday, September 21, 2022 6:29 AM
To: user@flink.apache.org
Cc: Deepak kumar Gunjetti <deepakg...@gmail.com>; Yaroslav Tkachenko
<yaros...@goldsky.com>
Subject: Re: Is it possible to connect multiple streams
⚠EXTERNAL MESSAGE – CAUTION: Think Before You Click ⚠
Hi
Thanks @yaroslav .
And @deepakgd79 here is the document for datastream:
https://nightlies.apache.org/flink/flink-docs-master/docs/dev/datastream/operators/overview/#datastream-transformations
You can find examples for union, connect, join and other transformations
On Wed, Sep 21, 2022 at 11:55 AM Yaroslav Tkachenko
<yaros...@goldsky.com<mailto:yaros...@goldsky.com>> wrote:
Hi Deepak,
You can use a union operator. I actually gave a talk on creating an advanced
join using the union operator and multiple streams:
-
https://www.slideshare.net/sap1ens/storing-state-forever-why-it-can-be-good-for-your-analytics
- https://www.youtube.com/watch?v=tiGxEGPyqCg
I hope this helps.
On Tue, Sep 20, 2022 at 5:22 PM Deepak kumar Gunjetti
<deepakg...@gmail.com<mailto:deepakg...@gmail.com>> wrote:
Hi,
My name is Deepak, I am a new user to apache flink. It is one of the best open
source i have used. I want to thank the community for developing such a
wonderful product.
I have one query.
Is it possible to connect multiple streams, like
stream1.connect(stream2).connect(stream3).flatmap(new
RickCoFlatMapFunctionHandler())
Can someone please let me know how I can achieve this.
Thanks,
Deepak
Diese Nachricht ist ausschliesslich für den Adressaten bestimmt und beinhaltet
unter Umständen vertrauliche Mitteilungen. Da die Vertraulichkeit von
e-Mail-Nachrichten nicht gewährleistet werden kann, übernehmen wir keine
Haftung für die Gewährung der Vertraulichkeit und Unversehrtheit dieser
Mitteilung. Bei irrtümlicher Zustellung bitten wir Sie um Benachrichtigung per
e-Mail und um Löschung dieser Nachricht sowie eventueller Anhänge. Jegliche
unberechtigte Verwendung oder Verbreitung dieser Informationen ist streng
verboten.
This message is intended only for the named recipient and may contain
confidential or privileged information. As the confidentiality of email
communication cannot be guaranteed, we do not accept any responsibility for the
confidentiality and the intactness of this message. If you have received it in
error, please advise the sender by return e-mail and delete this message and
any attachments. Any unauthorised use or dissemination of this information is
strictly prohibited.
