Hi Michael, There isn’t an operator that takes three (or more) streams, AFAIK.
There is a CoFlatMapFunction that takes two different streams in, which could be used for some types of joins. Streaming joins are (typically) windowed (bounded), by time/count/something, so if you can maintain the required windowed state in a ProcessFunction then you can implement whatever custom logic is required for your join case. And for creating a unioned stream of multiple data types, one easy way is via (e.g.) Tuple3<POJO1, POJO2, POJO3>, where only one of the three fields is non-null for each tuple. -- Ken PS - I think the u...@flink.apache.org <mailto:u...@flink.apache.org> list is probably a better forum for this question. > On Apr 7, 2018, at 10:47 AM, TechnoMage <mla...@technomage.com> wrote: > > In my case I have more elaborate logic to select data from the streams. They > are not all the same logical type, though I may be able to represent them as > the same Java type. My main question is whether it is technically feasible > to have a single operator that takes multiple streams as input. For example > Operator(stream1, stream2, stream3) and produces an output stream. Can the > checkpointing and other logic accomodate this if I write sufficient custom > code in the operator? > > Michael > >> On Apr 7, 2018, at 10:42 AM, Ken Krugler <kkrugler_li...@transpac.com> wrote: >> >> When you say “join” are you talking about a real join (so one or more fields >> can be used as a joining key), or some other operation? >> >> For more than two streams, you can do cascading window joins via multiple >> join()s that reduce your source streams down to a single stream. >> >> If the fields are the same across these streams, then a union() followed by >> say a ProcessFunction that implements your joining logic could work. >> >> Or you can convert all the streams to a common tuple format that consists of >> a unions the fields, so you can do a union() and then follow that with >> whatever logic is needed to actually do the join. >> >> Though I’m sure there are more elegant approaches :) >> >> — Ken >> >> >> >>> On Apr 6, 2018, at 5:04 PM, Michael Latta <mla...@technomage.com> wrote: >>> >>> I would like to “join” several streams (>3) in a custom operator. Is this >>> feasible in Flink? >>> >>> >>> Michael >> >> -------------------------------------------- >> http://about.me/kkrugler >> +1 530-210-6378 >> > -------------------------------------------- http://about.me/kkrugler +1 530-210-6378
