Hi Saikat, Thanks for this contribution.
Anton V., please review the following contribution. — Denis > On Jul 16, 2016, at 11:09 PM, Saikat Maitra <saikat.mai...@gmail.com> wrote: > > Hi > > I have raised a PR for the following scope. > > As a Flink source => run a continuous query against one or multiple > caches > > PR https://github.com/apache/ignite/pull/870 > Jira https://issues.apache.org/jira/browse/IGNITE-3303 > > Please review and share feedback. > > Regards > Saikat > > > On Mon, Apr 4, 2016 at 8:24 PM, Stephan Ewen <se...@apache.org> wrote: > >> Hi! >> >> - Sounds like the having Ignite for snapshots should work pretty much out >> of the box (via the IGFS) >> - The source and sink connector sounds like the next logical step. Does >> Ignite have a notion of stream partitions and offsets, to build a >> consistent replay around? This should probably have its dedicated issue and >> discussion thread. >> >> - For Ignite as an execution backend - I am not sure how relevant and >> feasible that is. Many DataStream API features make use of the specific >> Flink runtime. For streaming, the runtime is not as decoupled as for batch. >> - I think the parameter server integration would not be part of the Flink >> codebase - this is a pretty application specific thing that should be its >> own project and it is actually not tightly coupled to Flink. >> >> Greetings, >> Stephan >> >> >> On Mon, Apr 4, 2016 at 4:35 PM, Robert Metzger <rmetz...@apache.org> >> wrote: >> >>> Hi Raul, >>> >>> thanks a lot for reaching out to the Flink community. >>> I'm really excited to see a Flink connector in Ignite. If you feel that >> the >>> connector would be more suitable for our "connector library" feel free to >>> open a JIRA and open a pull request. >>> >>> Were there requests in the Ignite community to have an integration with >>> Flink? >>> >>> >>> >>> On Thu, Mar 31, 2016 at 5:20 PM, Saikat Maitra <saikat.mai...@gmail.com> >>> wrote: >>> >>>> Hi , >>>> >>>> I agree with Roman and Raul. >>>> https://issues.apache.org/jira/browse/IGNITE-813 allows injecting data >>> to >>>> into cache via Data Streamer. Integrating with Ignite FileSystem for >>> source >>>> and sink will allow for bidirectional connector. It will also allow >>> easier >>>> implementation for DataStream transformations over Ignite FileSystem. >>>> >>>> Regards >>>> Saikat >>>> >>>> On Thu, Mar 31, 2016 at 2:44 PM, Aljoscha Krettek <aljos...@apache.org >>> >>>> wrote: >>>> >>>>> Hi, >>>>> it should already be possible to use the Ignite FileSystem to store >>> state >>>>> since we just use the HDFS FileSystem interface for that. Of course, >>> one >>>>> would have to properly set up the jars and paths and everything for >>> Flink >>>>> to pick up the IGFS classes. >>>>> >>>>> Cheers, >>>>> Aljoscha >>>>> >>>>> On Wed, 30 Mar 2016 at 16:50 Raul Kripalani <ra...@apache.org> >> wrote: >>>>> >>>>>> On Wed, Mar 30, 2016 at 2:20 PM, Roman <rsht...@yahoo.com.invalid> >>>>> wrote: >>>>>> >>>>>>> Raul, >>>>>>> >>>>>>> Small comment from me. >>>>>>> >>>>>>>> * As a Flink sink => inject data directly into a cache via a >>>>>> DataStreamer. >>>>>>> After reviews, IGNITE-813 is exactly this functionality. >>>>>>> >>>>>>> >>>>>> That's cool, Roman! The idea would be to host these (richer) >> modules >>> as >>>>>> Flink connectors, like they do with others: >>>>>> >>>>>> >>> https://github.com/apache/flink/tree/master/flink-streaming-connectors >>>>>> https://github.com/apache/flink/tree/master/flink-batch-connectors >>>>>> >>>>> >>>> >>> >>
