Hi Aparup, I haven't looked in detail at Siddhi's internals especially the way it handles distributed execution. I've only seen that it uses Hazelcast for a distributed in-memory cache. If a distributed cache for the communication between instances of the CEP operator is needed, then you would have to integrate that as well. Otherwise, if the state stored in the operators is disjunct, then you should be good to go with Flink's state abstraction.
If you want to work on the integration, then it's best to open a JIRA issue and describe a little bit the way the integration could look like. Cheers, Till On Mon, Aug 29, 2016 at 10:59 AM, Stephan Ewen <se...@apache.org> wrote: > Nice idea! > > If you look at the current CEP library, it is simply a custom operator. > Often, you can even get away with a custom FlatMapFunction that uses > state: https://ci.apache.org/projects/flink/flink-docs- > master/dev/state.html#using-the-keyvalue-state-interface > > Stephan > > > On Mon, Aug 29, 2016 at 10:09 AM, Aparup Banerjee (apbanerj) < > apban...@cisco.com> wrote: > >> I think siddhi is a fairly matured CEP library. I am thinking it should >> co-exist with existing CEP library. My thinking is we should be able to use >> Siddhi QL/ Siddhi Patterns on top of flink data streams. This can co-exist >> naturally with existing Java / Scala based Flink CEP Library. I am still >> reading up on Flink internals – but at high level I am thinking about a new >> Flink operator on DataStream for this. Thoughts ? >> >> Thanks, >> Aparup >> >> From: Chesnay Schepler <ches...@apache.org> >> Reply-To: "user@flink.apache.org" <user@flink.apache.org> >> Date: Monday, August 29, 2016 at 12:35 AM >> To: "user@flink.apache.org" <user@flink.apache.org> >> Subject: Re: Apache siddhi into Flink >> >> Hello Aparup, >> >> could you provide more information about Siddhi? How mature is it; how is >> the community? How does it compare to the Flink's CEP library? >> >> How should this integration look like? Are you proposing to replace the >> current CEP library, or will they co-exist with different use-cases for >> each? >> >> If we used Siddhi in Flink, how exactly would Flink's runtime be involved >> in the processing? >> >> Regards, >> Chesnay >> >> On 28.08.2016 23:21, Aparup Banerjee (apbanerj) wrote: >> >> Sorry for the semantic difference. >> >> >> >> On Aug 28, 2016, at 12:05 PM, Trevor Grant < <trevor.d.gr...@gmail.com> >> trevor.d.gr...@gmail.com> wrote: >> >> Thank you for confirming Hao, >> >> Aparup, please don't refer to it as "Apache Siddhi", that is misleading. >> >> >> Trevor Grant >> Data Scientist >> https://github.com/rawkintrevo >> http://stackexchange.com/users/3002022/rawkintrevo >> http://trevorgrant.org >> >> *"Fortunate is he, who is able to know the causes of things." -Virgil* >> >> >> On Sun, Aug 28, 2016 at 10:50 AM, Hao Chen <h...@apache.org> wrote: >> >>> Siddhi is not apache project, but licensed under apache license v2, >>> being open sourced and maintained by wso2. >>> >>> - Hao >>> >>> On Sun, Aug 28, 2016 at 11:11 PM, Trevor Grant <trevor.d.gr...@gmail.com >>> > wrote: >>> >>>> Aparup, >>>> >>>> Was Siddhi recently added as an incubator project? I can't find it in >>>> the project directory or or on github.com/apache. The closest thing I >>>> can find is this: <https://github.com/wso2/siddhi> >>>> https://github.com/wso2/siddhi >>>> >>>> tg >>>> >>>> >>>> >>>> >>>> Trevor Grant >>>> Data Scientist >>>> <https://github.com/rawkintrevo>https://github.com/rawkintrevo >>>> <http://stackexchange.com/users/3002022/rawkintrevo> >>>> http://stackexchange.com/users/3002022/rawkintrevo >>>> <http://trevorgrant.org>http://trevorgrant.org >>>> >>>> *"Fortunate is he, who is able to know the causes of things." -Virgil* >>>> >>>> >>>> On Sat, Aug 27, 2016 at 5:36 PM, Chen Qin <qinnc...@gmail.com> wrote: >>>> >>>>> +1 >>>>> >>>>> >>>>> On Aug 26, 2016, at 11:23 PM, Aparup Banerjee (apbanerj) < >>>>> <apban...@cisco.com>apban...@cisco.com> wrote: >>>>> >>>>> Hi- >>>>> >>>>> Has anyone looked into embedding apache siddhi into Flink. >>>>> >>>>> Thanks, >>>>> Aparup >>>>> >>>>> >>>>> >>>> >>> >> >> >
