Hi Paris! The Streaming API allows you to define iterations, where parts of the stream are fed back. Do those work for you?
In general, cyclic flows are a tricky thing, as the topological order of operators is needed for scheduling (may not be important for continuous streams) but also for a clear producer/consumer relationship, which is important for fault tolerance techniques. Currently, the JobManager topologically sorts the job graph and starts scheduling operators. I am surprised to hear that a graph with cyclic dependencies works... Stephan Stephan On Wed, Jan 21, 2015 at 2:57 AM, Paris Carbone <par...@kth.se> wrote: > Hello, > > While implementing the SAMOA adapter for Flink-Streaming we stumbled upon > the need to allow loops (or circular dependencies) in the job graph. Many > incremental machine learning tasks define loops already and there is no > trivial way of getting around it. In the streaming job graph builder there > is only a check that does not allow the user to submit graphs with loops, > however, from what Gyula told me, if the check is removed the streaming job > runs as expected. Is there (still) a major reason for having this check, at > least in the streaming component? > > Paris
