Thanks Fabian for the standard Row Pattern Recognition in SQL! I just created a FLIP for this feature.
FILP-20: https://cwiki.apache.org/confluence/display/FLINK/FLIP-20%3A+Integration+of+SQL+and+CEP Cheers, Jark Wu 2017-06-16 0:49 GMT+08:00 Fabian Hueske <fhue...@gmail.com>: > Hi everybody, > > I just stumbled over this blog post [1] which discusses new features in SQL > 2016. > Apparently the match_recognize clause is part of that. The blogpost also > contains a slide set that presents the pattern matching feature and a link > to a 90 page technical report. > > I thought this might be helpful as a references. > > Cheers, Fabian > > [1] http://modern-sql.com/blog/2017-06/whats-new-in-sql-2016 > > 2017-06-15 11:53 GMT+02:00 Till Rohrmann <trohrm...@apache.org>: > > > @Jark: You should now have the permissions to create pages in the Flink > > wiki. > > > > Cheers, > > Till > > > > On Thu, Jun 15, 2017 at 5:11 AM, Jark Wu <j...@apache.org> wrote: > > > > > Hi Till, > > > > > > Could you grant me the edit permission of Flink WIKI? My id is imjark. > > > > > > Thanks, > > > Jark Wu > > > > > > 2017-06-15 0:07 GMT+08:00 Till Rohrmann <trohrm...@apache.org>: > > > > > > > I think that the integration of SQL and CEP would make a good FLIP. > > > > > > > > Cheers, > > > > Till > > > > > > > > On Wed, Jun 14, 2017 at 2:40 PM, Jark Wu <imj...@gmail.com> wrote: > > > > > > > > > Hi, > > > > > > > > > > Do you think whether we should create a FLIP for this proposal to > > track > > > > > progress? > > > > > > > > > > Regards, > > > > > Jark > > > > > > > > > > 2017-06-13 16:59 GMT+08:00 Dian Fu <dia...@apache.org>: > > > > > > > > > > > Hi Fabian, > > > > > > > > > > > > Thanks a lot. Agree that we can start working by adding the > missing > > > > > > features of the CEP library. > > > > > > > > > > > > Best regards, > > > > > > Dian > > > > > > > > > > > > On Tue, Jun 13, 2017 at 4:26 PM, Fabian Hueske < > fhue...@gmail.com> > > > > > wrote: > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > @Dian Fu: I gave you contributor permissions. :-) > > > > > > > > > > > > > > I don't think we have to wait for Calcite 1.13 to start working > > on > > > > the > > > > > > > missing features of the CEP library and extending the > prototype. > > > > > > > We might want to wait with the integration into flink-table > until > > > > > Calcite > > > > > > > 1.13 is out and we updated the dependency though. > > > > > > > > > > > > > > Best, Fabian > > > > > > > > > > > > > > 2017-06-13 9:45 GMT+02:00 jincheng sun < > sunjincheng...@gmail.com > > >: > > > > > > > > > > > > > > > Hi Jark, Dian, > > > > > > > > > > > > > > > > Thanks for bring up this discuss and share the prototype. > > > > > > > > > > > > > > > > +1 to push this great feature forward! > > > > > > > > > > > > > > > > Cheers, > > > > > > > > SunJincheng > > > > > > > > > > > > > > > > 2017-06-13 15:34 GMT+08:00 Jark Wu <j...@apache.org>: > > > > > > > > > > > > > > > > > Thank you Yueting for pointing out the mistake in the > > > prototype. > > > > I > > > > > > > > > accidentally introduced it when merge code. > > > > > > > > > > > > > > > > > > I'm so glad to see so many people are interested in the > > > feature. > > > > > > Let's > > > > > > > > work > > > > > > > > > out together to push it forward! > > > > > > > > > > > > > > > > > > Cheers, > > > > > > > > > Jark Wu > > > > > > > > > > > > > > > > > > > > > > > > > > > 2017-06-13 15:27 GMT+08:00 Liangfei Su < > suliang...@gmail.com > > >: > > > > > > > > > > > > > > > > > > > +1 for the feature. Myself was a user of Siddhi, this is > > > pretty > > > > > > user > > > > > > > > > > friendly feature to provide to user. > > > > > > > > > > > > > > > > > > > > On Tue, Jun 13, 2017 at 3:09 PM, Dian Fu < > > > > dian0511...@gmail.com> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > Hi Yueting & Dawid Wysakowicz, > > > > > > > > > > > > > > > > > > > > > > Very glad that you're interested in this feature and > > you're > > > > > > > > definitely > > > > > > > > > > > welcome to join this work and also anyone else.:) > > > > > > > > > > > > > > > > > > > > > > Best regards, > > > > > > > > > > > Dian > > > > > > > > > > > > > > > > > > > > > > On Tue, Jun 13, 2017 at 2:35 PM, Dawid Wysakowicz < > > > > > > > > > > > wysakowicz.da...@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > > > > > > > > > Integrating SQL with CEP seems like a really nice > idea. > > > > > > > > > Unfortunately I > > > > > > > > > > > had > > > > > > > > > > > > time just for a brief look at the design doc, but it > > > looks > > > > > > really > > > > > > > > > cool > > > > > > > > > > > and > > > > > > > > > > > > thorough. Also will have a second run tomorrow and > will > > > try > > > > > to > > > > > > > > > provide > > > > > > > > > > > more > > > > > > > > > > > > comments. Anyway will be glad to help pushing the > > > > initiative > > > > > > > > forward. > > > > > > > > > > > > > > > > > > > > > > > > Z pozdrowieniami! / Cheers! > > > > > > > > > > > > > > > > > > > > > > > > Dawid Wysakowicz > > > > > > > > > > > > > > > > > > > > > > > > *Data/Software Engineer* > > > > > > > > > > > > > > > > > > > > > > > > Skype: dawid_wys | Twitter: @OneMoreCoder > > > > > > > > > > > > > > > > > > > > > > > > <http://getindata.com/> > > > > > > > > > > > > > > > > > > > > > > > > 2017-06-13 8:19 GMT+02:00 yueting chen < > > > > yestinm...@gmail.com > > > > > >: > > > > > > > > > > > > > > > > > > > > > > > > > Hi Dian & Jark, > > > > > > > > > > > > > > > > > > > > > > > > > > I checked out your prototype code, but it didn't > pass > > > the > > > > > > > > CEPITCase > > > > > > > > > > > test > > > > > > > > > > > > in > > > > > > > > > > > > > the flink-table component. > > > > > > > > > > > > > It turns out that in the `MatchCodeGenerator.scala` > > > file, > > > > > > line > > > > > > > 74 > > > > > > > > > > > should > > > > > > > > > > > > > use `${classOf[IterativeCondition. > > > > > > > Context[_]].getCanonicalName}` > > > > > > > > > > > instead > > > > > > > > > > > > > of > > > > > > > > > > > > > `${classOf[IterativeCondition.Context[_]]}`. > > > > > > > > > > > > > > > > > > > > > > > > > > I've also read your desgin document and it looks > fine > > > to > > > > > me. > > > > > > > > > > Actually, > > > > > > > > > > > I > > > > > > > > > > > > am > > > > > > > > > > > > > working on the same thing recently, I think maybe > we > > > can > > > > > work > > > > > > > > > > together > > > > > > > > > > > to > > > > > > > > > > > > > push this forward. > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > Yueting Chen > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Jun 13, 2017 at 10:44 AM, Dian Fu < > > > > > dia...@apache.org > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Fabian, > > > > > > > > > > > > > > > > > > > > > > > > > > > > We have evaluated the missing features of Flink > CEP > > > > > > roughly, > > > > > > > it > > > > > > > > > > > should > > > > > > > > > > > > > not > > > > > > > > > > > > > > be quite difficult to support them. Kostas, > Dawid, > > > > what's > > > > > > > your > > > > > > > > > > > thought? > > > > > > > > > > > > > > > > > > > > > > > > > > > > For supporting MATCH_RECOGNIZE, do you think we > > could > > > > > > create > > > > > > > > the > > > > > > > > > > > JIRAs > > > > > > > > > > > > > and > > > > > > > > > > > > > > start to work right now or we should wait until > the > > > > > release > > > > > > > of > > > > > > > > > > > calcite > > > > > > > > > > > > > > 1.13? > > > > > > > > > > > > > > > > > > > > > > > > > > > > Btw, could you help to add me(dian.fu) to the > > > > contributor > > > > > > > list, > > > > > > > > > > then > > > > > > > > > > > I > > > > > > > > > > > > > can > > > > > > > > > > > > > > assign the JIRAs to myself? Thanks a lot. > > > > > > > > > > > > > > > > > > > > > > > > > > > > Best regards, > > > > > > > > > > > > > > Dian > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Jun 13, 2017 at 3:59 AM, Fabian Hueske < > > > > > > > > > fhue...@gmail.com> > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Jark, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks for updating the design doc and sharing > > your > > > > > > > > prototype! > > > > > > > > > > > > > > > I didn't look at the code in detail, but the > fact > > > > that > > > > > it > > > > > > > is > > > > > > > > > less > > > > > > > > > > > > than > > > > > > > > > > > > > 1k > > > > > > > > > > > > > > > LOC is very promising. It seems that most of > the > > > > > complex > > > > > > > CEP > > > > > > > > > > logic > > > > > > > > > > > > can > > > > > > > > > > > > > be > > > > > > > > > > > > > > > reused :-) > > > > > > > > > > > > > > > Adding a dependency on flink-cep should be > fine, > > > IMO. > > > > > It > > > > > > > is a > > > > > > > > > > very > > > > > > > > > > > > slim > > > > > > > > > > > > > > > library with almost none external dependencies. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Regarding the missing features of Flink CEP > that > > > you > > > > > > listed > > > > > > > > in > > > > > > > > > > the > > > > > > > > > > > > > design > > > > > > > > > > > > > > > doc, it would be good to get some in put from > > > Kostas > > > > > and > > > > > > > > Dawid > > > > > > > > > > > which > > > > > > > > > > > > > are > > > > > > > > > > > > > > > the main contributors to CEP. > > > > > > > > > > > > > > > Do you have already plans regarding some of the > > > > missing > > > > > > > > > features > > > > > > > > > > or > > > > > > > > > > > > can > > > > > > > > > > > > > > you > > > > > > > > > > > > > > > assess how hard it would be to integrate them? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Cheers, Fabian > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Btw. The Calcite community started a discussion > > > about > > > > > > > > releasing > > > > > > > > > > > > Calcite > > > > > > > > > > > > > > > 1.13. So, the missing features might soon be > > > > available. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 2017-06-12 14:25 GMT+02:00 Jark Wu < > > > j...@apache.org > > > > >: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi guys, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Good news! We have made a prototype for > > > integrating > > > > > CEP > > > > > > > and > > > > > > > > > > SQL. > > > > > > > > > > > > See > > > > > > > > > > > > > > this > > > > > > > > > > > > > > > > link > > > > > > > > > > > > > > > > https://github.com/apache/ > flink/compare/master > > . > > > .. > > > > > > > > > > > > > > > > wuchong:cep-on-sql?expand=1 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > You can check CepITCase to try the simple CQL > > > > > example. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Meanwhile, we updated our design doc with > > > > additional > > > > > > > > > > > implementation > > > > > > > > > > > > > > > detail, > > > > > > > > > > > > > > > > including how > > > > > > > > > > > > > > > > to translate MATCH_RECOGNIZE into CEP API, > and > > > the > > > > > > > features > > > > > > > > > > > needed > > > > > > > > > > > > to > > > > > > > > > > > > > > add > > > > > > > > > > > > > > > > to Flink CEP, > > > > > > > > > > > > > > > > and the implementation plan. See the document > > > > > > > > > > > > > > > > https://docs.google.com/document/d/ > > > > > > > > > > > 1HaaO5eYI1VZjyhtVPZOi3jVzikU7i > > > > > > > > > > > > > > > > K15H0YbniTnN30/edit#heading=h.4oas4koy8qu3 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > In the prototype, we make flink-table > > dependency > > > on > > > > > > > > > flink-cep. > > > > > > > > > > Do > > > > > > > > > > > > you > > > > > > > > > > > > > > > think > > > > > > > > > > > > > > > > that is fine? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > What do you think about the prototype and the > > > > design > > > > > > doc > > > > > > > ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Any feedbacks are welcome! > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Cheers, > > > > > > > > > > > > > > > > Jark Wu > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 2017-06-08 17:54 GMT+08:00 Till Rohrmann < > > > > > > > > > trohrm...@apache.org > > > > > > > > > > >: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks for sharing your ideas with the > > > > community. I > > > > > > > > really > > > > > > > > > > like > > > > > > > > > > > > the > > > > > > > > > > > > > > > > design > > > > > > > > > > > > > > > > > document and think that it's a good > approach > > to > > > > > > follow > > > > > > > > > > Oracle's > > > > > > > > > > > > SQL > > > > > > > > > > > > > > > > > extension for pattern matching. Looking > > forward > > > > to > > > > > > > having > > > > > > > > > > > support > > > > > > > > > > > > > for > > > > > > > > > > > > > > > SQL > > > > > > > > > > > > > > > > > with CEP capabilities :-) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Cheers, > > > > > > > > > > > > > > > > > Till > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Jun 8, 2017 at 8:57 AM, Jark Wu < > > > > > > > j...@apache.org > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi @Kostas, @Fabian, thank you for your > > > > support. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > @Fabian, I totally agree with you that we > > > > should > > > > > > > focus > > > > > > > > on > > > > > > > > > > SQL > > > > > > > > > > > > > > first. > > > > > > > > > > > > > > > > > Let's > > > > > > > > > > > > > > > > > > keep Table API in mind and discuss that > > > later. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Regarding to the orderBy() clause, I'm > not > > > sure > > > > > > about > > > > > > > > > > that. I > > > > > > > > > > > > > think > > > > > > > > > > > > > > > it > > > > > > > > > > > > > > > > > > makes sense to make it required in > > streaming > > > > > > > > mode(either > > > > > > > > > > > order > > > > > > > > > > > > by > > > > > > > > > > > > > > > > rowtime > > > > > > > > > > > > > > > > > > or order by proctime). But CEP also works > > in > > > > > batch > > > > > > > > mode, > > > > > > > > > > and > > > > > > > > > > > > not > > > > > > > > > > > > > > > > > necessary > > > > > > > > > > > > > > > > > > to order by some column. Nevertheless, we > > can > > > > > > support > > > > > > > > CEP > > > > > > > > > > on > > > > > > > > > > > > > batch > > > > > > > > > > > > > > > SQL > > > > > > > > > > > > > > > > > > later. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > We are estimating how to implement > > > > > MATCH_RECOGNIZE > > > > > > > with > > > > > > > > > CEP > > > > > > > > > > > > > library > > > > > > > > > > > > > > > > (with > > > > > > > > > > > > > > > > > > NFA, CEP operator). And we will output a > > > > detailed > > > > > > doc > > > > > > > > > and a > > > > > > > > > > > > > > prototype > > > > > > > > > > > > > > > > in > > > > > > > > > > > > > > > > > > the next days. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Regards, > > > > > > > > > > > > > > > > > > Jark Wu > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 2017-06-07 21:40 GMT+08:00 Fabian Hueske > < > > > > > > > > > > fhue...@gmail.com > > > > > > > > > > > >: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> Thanks Dian and Jark for this proposal! > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> As you wrote, Till and I (and Kostas) > have > > > > been > > > > > > > > thinking > > > > > > > > > > > about > > > > > > > > > > > > > > this > > > > > > > > > > > > > > > > for > > > > > > > > > > > > > > > > > >> some time but haven't had time to work > on > > > this > > > > > > > > feature. > > > > > > > > > > > > > > > > > >> I think it would be a great addition and > > > value > > > > > add > > > > > > > for > > > > > > > > > > > Flink's > > > > > > > > > > > > > SQL > > > > > > > > > > > > > > > > > >> support and Table API. > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> I read the proposal and think it is very > > > good. > > > > > We > > > > > > > > might > > > > > > > > > > need > > > > > > > > > > > > to > > > > > > > > > > > > > > add > > > > > > > > > > > > > > > a > > > > > > > > > > > > > > > > > bit > > > > > > > > > > > > > > > > > >> more details, esp. when planning the > > > concrete > > > > > > steps > > > > > > > of > > > > > > > > > the > > > > > > > > > > > > > > > > > implementation. > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> A few comments to the proposal: > > > > > > > > > > > > > > > > > >> - IMO, the development should start > > focusing > > > > on > > > > > > SQL > > > > > > > > and > > > > > > > > > > its > > > > > > > > > > > > > > > semantics. > > > > > > > > > > > > > > > > > >> Pattern support for the Table API should > > be > > > > > added > > > > > > > > later. > > > > > > > > > > We > > > > > > > > > > > > > > followed > > > > > > > > > > > > > > > > > that > > > > > > > > > > > > > > > > > >> approach for the OVER windows and I > think > > it > > > > > > worked > > > > > > > > > quiet > > > > > > > > > > > > well. > > > > > > > > > > > > > > > > > >> - We probably want to reuse as much as > > > > possible > > > > > > from > > > > > > > > the > > > > > > > > > > CEP > > > > > > > > > > > > > > > library. > > > > > > > > > > > > > > > > > >> That means we need to check if the > > semantics > > > > of > > > > > > the > > > > > > > > CEP > > > > > > > > > > > > library > > > > > > > > > > > > > > and > > > > > > > > > > > > > > > > > >> Oracle's PATTERN syntax are aligned (or > > how > > > we > > > > > can > > > > > > > > > express > > > > > > > > > > > the > > > > > > > > > > > > > > > PATTERN > > > > > > > > > > > > > > > > > >> semantics with the CEP library). This > > should > > > > be > > > > > > one > > > > > > > of > > > > > > > > > the > > > > > > > > > > > > first > > > > > > > > > > > > > > > > steps, > > > > > > > > > > > > > > > > > IMO. > > > > > > > > > > > > > > > > > >> - I would make the orderBy() clause > > > required. > > > > In > > > > > > > > regular > > > > > > > > > > SQL > > > > > > > > > > > > > rows > > > > > > > > > > > > > > > have > > > > > > > > > > > > > > > > > no > > > > > > > > > > > > > > > > > >> order, so we need to make that explicit > > > (this > > > > > > would > > > > > > > > also > > > > > > > > > > be > > > > > > > > > > > > > > > consistent > > > > > > > > > > > > > > > > > with > > > > > > > > > > > > > > > > > >> the OVER windows). > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> Let me know what you think. > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> Best, Fabian > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >> 2017-06-07 11:41 GMT+02:00 Kostas > Kloudas > > < > > > > > > > > > > > > > > > > k.klou...@data-artisans.com > > > > > > > > > > > > > > > > > >: > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > >>> Thanks a lot for opening the > discussion! > > > > > > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > > > >>> This is a really interesting idea that > > has > > > > been > > > > > > in > > > > > > > > our > > > > > > > > > > > heads > > > > > > > > > > > > > > > > > >>> since the first implementation of the > CEP > > > > > > library. > > > > > > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > > > >>> A big +1 for moving forward with this. > > > > > > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > > > >>> And as for the design document, I will > > > > > definitely > > > > > > > > have > > > > > > > > > a > > > > > > > > > > > look > > > > > > > > > > > > > > > > > >>> and comment there. > > > > > > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > > > >>> Kostas > > > > > > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > > > >>> On Jun 7, 2017, at 10:05 AM, Jark Wu < > > > > > > > > j...@apache.org> > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > > > >>> Sorry, I forgot to cc you guys @Fabian, > > > > @Timo, > > > > > > > @Till, > > > > > > > > > > > @Kostas > > > > > > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > > > >>> 2017-06-07 15:42 GMT+08:00 Jark Wu < > > > > > > > j...@apache.org > > > > > > > > >: > > > > > > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > > > >>>> Hi devs, > > > > > > > > > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > >>>> Dian and me and our teammates have > > > > > investigated > > > > > > > this > > > > > > > > > > for a > > > > > > > > > > > > > long > > > > > > > > > > > > > > > > time. > > > > > > > > > > > > > > > > > >>>> We think consolidating Flink SQL and > CEP > > > is > > > > an > > > > > > > > > exciting > > > > > > > > > > > > thing > > > > > > > > > > > > > > for > > > > > > > > > > > > > > > > > Flink. > > > > > > > > > > > > > > > > > >>>> It'll make SQL more powerful and give > > > users > > > > > the > > > > > > > > > ability > > > > > > > > > > to > > > > > > > > > > > > > > easily > > > > > > > > > > > > > > > > and > > > > > > > > > > > > > > > > > >>>> quickly build CEP applications. And I > > > find > > > > > > Flink > > > > > > > > > > > community > > > > > > > > > > > > > has > > > > > > > > > > > > > > > also > > > > > > > > > > > > > > > > > talked > > > > > > > > > > > > > > > > > >>>> about this idea before, such as the > > > mailing > > > > > list > > > > > > > [1] > > > > > > > > > and > > > > > > > > > > > [2] > > > > > > > > > > > > > and > > > > > > > > > > > > > > > > > Fabian & > > > > > > > > > > > > > > > > > >>>> Till's talk in Flink Forward 2016 [3]. > > > > > > > > > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > >>>> I think THIS IS THE POINT to bring up > > this > > > > > topic > > > > > > > > > again. > > > > > > > > > > > > > Because > > > > > > > > > > > > > > we > > > > > > > > > > > > > > > > > >>>> already have pattern matching > foundation > > > in > > > > > > Flink > > > > > > > > CEP > > > > > > > > > > > > library, > > > > > > > > > > > > > > and > > > > > > > > > > > > > > > > > Stream > > > > > > > > > > > > > > > > > >>>> SQL is ready now and Calcite has > > partially > > > > > > > supported > > > > > > > > > > > pattern > > > > > > > > > > > > > > > > matching > > > > > > > > > > > > > > > > > >>>> syntax! We also drafted a design doc > > > about > > > > > how > > > > > > to > > > > > > > > > > > integrate > > > > > > > > > > > > > SQL > > > > > > > > > > > > > > > and > > > > > > > > > > > > > > > > > CEP, > > > > > > > > > > > > > > > > > >>>> and how to support CEP on Table API. > > > > > > > > > > > > > > > https://docs.google.com/docume > > > > > > > > > > > > > > > > > >>>> nt/d/1HaaO5eYI1VZjyhtVPZOi3jVzikU7i > > > > > > > > > > > K15H0YbniTnN30/edit?usp= > > > > > > > > > > > > > > > sharing > > > > > > > > > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > >>>> @Fabian, @Timo, @Till, @Kostas I > include > > > you > > > > > > into > > > > > > > > this > > > > > > > > > > > > > > discussion, > > > > > > > > > > > > > > > > it > > > > > > > > > > > > > > > > > >>>> would be great to hear your response. > > > > > > > > > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > >>>> What do others think? > > > > > > > > > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > >>>> [1] http://apache-flink-mailing- > > > > > > > > > list-archive.1008284.n3 > > > > > > > > > > . > > > > > > > > > > > > > > nabble.c > > > > > > > > > > > > > > > > > >>>> om/Add-CEP-library-to-Flink- > > > > td9743.html#a9787 > > > > > > > > > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > >>>> [2] http://apache-flink-mailing- > > > > > > > > > list-archive.1008284.n3 > > > > > > > > > > . > > > > > > > > > > > > > > nabble.c > > > > > > > > > > > > > > > > > >>>> om/Effort-to-add-SQL- > > > > > StreamSQL-to-Flink-td9727. > > > > > > > > > > html#a9790 > > > > > > > > > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > >>>> [3] https://www.slideshare.net/ > > > > > > > > > tillrohrmann/streaming- > > > > > > > > > > > > > > analytics- > > > > > > > > > > > > > > > > > >>>> cep-two-sides-of-the-same-coin > > > > > > > > > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > >>>> Regards, > > > > > > > > > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > >>>> Jark Wu > > > > > > > > > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > >>>> 2017-06-07 13:50 GMT+08:00 Dian Fu < > > > > > > > > dia...@apache.org > > > > > > > > > >: > > > > > > > > > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > >>>>> Hi everyone, > > > > > > > > > > > > > > > > > >>>>> > > > > > > > > > > > > > > > > > >>>>> Flink's CEP library is a great > library > > > for > > > > > > > complex > > > > > > > > > > event > > > > > > > > > > > > > > > > processing, > > > > > > > > > > > > > > > > > >>>>> more > > > > > > > > > > > > > > > > > >>>>> and more customers are expressing > their > > > > > > interests > > > > > > > > in > > > > > > > > > > it. > > > > > > > > > > > > But > > > > > > > > > > > > > it > > > > > > > > > > > > > > > > also > > > > > > > > > > > > > > > > > >>>>> has > > > > > > > > > > > > > > > > > >>>>> some limitations that users usually > > have > > > to > > > > > > > write a > > > > > > > > > lot > > > > > > > > > > > of > > > > > > > > > > > > > code > > > > > > > > > > > > > > > > even > > > > > > > > > > > > > > > > > >>>>> for a > > > > > > > > > > > > > > > > > >>>>> very simple pattern match use case as > > it > > > > > > > currently > > > > > > > > > only > > > > > > > > > > > > > > supports > > > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > >>>>> Java > > > > > > > > > > > > > > > > > >>>>> API. > > > > > > > > > > > > > > > > > >>>>> We have investigated some popular CEP > > > > > products > > > > > > > such > > > > > > > > > as > > > > > > > > > > > > esper > > > > > > > > > > > > > > [1] > > > > > > > > > > > > > > > > and > > > > > > > > > > > > > > > > > >>>>> siddhi > > > > > > > > > > > > > > > > > >>>>> [2] and found that most of these CEP > > > > products > > > > > > > > support > > > > > > > > > > > > > SQL-like > > > > > > > > > > > > > > > > > >>>>> expressions > > > > > > > > > > > > > > > > > >>>>> such as EPL to describe the match > > > pattern. > > > > > But > > > > > > > > these > > > > > > > > > > > > > solutions > > > > > > > > > > > > > > > also > > > > > > > > > > > > > > > > > >>>>> have > > > > > > > > > > > > > > > > > >>>>> the drawbacks that the pattern match > > > > > languages > > > > > > > are > > > > > > > > > not > > > > > > > > > > > > > standard > > > > > > > > > > > > > > > > SQL, > > > > > > > > > > > > > > > > > >>>>> the > > > > > > > > > > > > > > > > > >>>>> learn curve is steep for users and > it's > > > > > > > impossible > > > > > > > > to > > > > > > > > > > > > > integrate > > > > > > > > > > > > > > > > them > > > > > > > > > > > > > > > > > >>>>> into > > > > > > > > > > > > > > > > > >>>>> the Flink Table API & SQL. > > > > > > > > > > > > > > > > > >>>>> We find that Oracle's CEP solution > CQL > > > [3] > > > > > > > > supports a > > > > > > > > > > new > > > > > > > > > > > > > > pattern > > > > > > > > > > > > > > > > > >>>>> recognition clause match_recognize > > which > > > > is a > > > > > > > > pattern > > > > > > > > > > > > > > recognition > > > > > > > > > > > > > > > > > >>>>> clause > > > > > > > > > > > > > > > > > >>>>> proposed in this paper [4]. It > > proposes a > > > > set > > > > > > of > > > > > > > > new > > > > > > > > > > > > syntaxes > > > > > > > > > > > > > > to > > > > > > > > > > > > > > > > > define > > > > > > > > > > > > > > > > > >>>>> match pattern in sql expression. > > Calcite > > > > > > already > > > > > > > > > > supports > > > > > > > > > > > > > part > > > > > > > > > > > > > > of > > > > > > > > > > > > > > > > > this > > > > > > > > > > > > > > > > > >>>>> standard [5]. I think it will be of > > > great > > > > > > value > > > > > > > to > > > > > > > > > > > support > > > > > > > > > > > > > > > > > expressing > > > > > > > > > > > > > > > > > >>>>> pattern recognition clause with > > > > > match_recognize > > > > > > > > > clause > > > > > > > > > > by > > > > > > > > > > > > > > > > integrating > > > > > > > > > > > > > > > > > >>>>> it > > > > > > > > > > > > > > > > > >>>>> with Flink Table API & SQL and the > > Flink > > > > CEP > > > > > > > > library. > > > > > > > > > > Any > > > > > > > > > > > > > > > thoughts? > > > > > > > > > > > > > > > > > >>>>> > > > > > > > > > > > > > > > > > >>>>> [1] http://www.espertech.com > > > > > > > > > > > > > > > > > >>>>> [2] https://github.com/wso2/siddhi > > > > > > > > > > > > > > > > > >>>>> [3] > > > > > > > > > > > > > > > > > >>>>> https://docs.oracle.com/ > > middleware/1213/ > > > > > > > > > > > > eventprocessing/cql- > > > > > > > > > > > > > > > > > >>>>> reference/GUID-34D4968E-C55A- > > > > > > > > > > 4BC7-B1CE-C84B202217BD.htm# > > > > > > > > > > > > > > > CQLLR1531 > > > > > > > > > > > > > > > > > >>>>> [4] > > > > > > > > > > > > > > > > > >>>>> http://web.cs.ucla.edu/ > > > > > > > > > classes/winter17/cs240B/notes/ > > > > > > > > > > > > row-pat > > > > > > > > > > > > > > > > > >>>>> tern-recogniton-11.pdf > > > > > > > > > > > > > > > > > >>>>> [5] https://issues.apache.org/ > > > > > > > > > jira/browse/CALCITE-1570 > > > > > > > > > > > > > > > > > >>>>> > > > > > > > > > > > > > > > > > >>>>> Best Regards, > > > > > > > > > > > > > > > > > >>>>> Dian > > > > > > > > > > > > > > > > > >>>>> > > > > > > > > > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > >>>> > > > > > > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
