Re: Connect more than two streams
Hello Govindarajan, one way to merge multiple streams is to union them. You can do that with the union operator described at [1]. Is that what you are looking for? [1] https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/datastream_api.html -- Jonas Am Mo, 24. Jul 2017, um 23:18, schrieb Govindarajan Srinivasaraghavan [via Apache Flink User Mailing List archive.]: > > > Hi, > > > I have two streams reading from kafka, one for data and other for > control. > The data stream is split by type and there are around six types. Each > type > has its own processing logic and finally everything has to be merged to > get > the collective state per device. I was thinking I could connect multiple > streams, process and maintain state but connect only supports two > streams. > Is there some way to achieve my desired functionality? > > > By the way after split and some processing all of them are keyed streams. > > > > > ___ > If you reply to this email, your message will be added to the discussion > below: > http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Connect-more-than-two-streams-tp14419.html > > To unsubscribe from Apache Flink User Mailing List archive., visit > http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code=1=am9uYXNAaHVudHVuLmRlfDF8LTcyNDQ4MDc3MA== -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Connect-more-than-two-streams-tp14419p14428.html Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.
Re: Anomaly Detection with Flink-ML
Hello Jeremy, it looks like what you are looking for is map (1 in, 1 out) / flatmap (1 in, 0-n out) for preprocessing on a single element basis as well as windows for looking at related MetricDefinition elements calculating some result. I suggest you look into Windows (https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/windows.html) and basic transformations (https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/datastream_api.html#datastream-transformations). Regards, Jonas -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Anomaly-Detection-with-Flink-ML-tp14149p14151.html Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.
Re: Related datastream
Hi nragon, apparently I didn't read the P.S. since I assumed its not important. Silly me. So you are trying to join stream A and B to stream C with stream A and B being keyed. Alright. Are how often do matching elements (matched by primary key) from A and B arrive on your operator to-be-implemented? This question can't be universially answered without having some more constraints on the streams A and B. To me this sounds more like a batch job because it needs to have the whole stream A or B in memory in order to join every element. -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Related-datastream-tp13901p13904.html Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.
Re: Related datastream
Hey nragon! Do the two streams A and B have some sort of id or key or how do you plan on joining them? Do you just want to join A and B with elements a and b as they arrive (one in state and join with the next arriving one from the other stream)? >From what you are asking, this should be no problem but we need a little bit more clarification here. -- Jonas -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Related-datastream-tp13901p13903.html Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.
