how will this sql support state both in streaming and batch mode 2017-04-07 4:54 GMT+08:00 Mingmin Xu <mingm...@gmail.com>:
> @Tyler, there's no big change in the previous design doc, I added some > details in chapter 'Part 2. DML( [INSERT] SELECT )' , describing steps to > process a query, feel free to leave a comment. > > Come through your doc of 'EMIT', it's awesome from my perspective. I've > some tests on GroupBy with default triggers/allowed_lateness now. EMIT > syntax can be added to fill the gap. > > On Thu, Apr 6, 2017 at 1:04 PM, Tyler Akidau <taki...@apache.org> wrote: > > > I'm very excited by this development as well, thanks for continuing to > push > > this forward, Mingmin. :-) > > > > I noticed you'd made some changes to your design doc > > <https://docs.google.com/document/d/1Uc5xYTpO9qsLXtT38OfuoqSLimH_ > > 0a1Bz5BsCROMzCU/edit>. > > Is it ready for another review? How reflective is it currently of the > work > > that going into the feature branch? > > > > In parallel, I'd also like to continue helping push forward the > definition > > of unified model semantics for SQL so we can get Calcite to a point where > > it supports the full Beam model. I added a comment > > <https://issues.apache.org/jira/browse/BEAM-301? > focusedCommentId=15959621& > > page=com.atlassian.jira.plugin.system.issuetabpanels: > > comment-tabpanel#comment-15959621> > > on the JIRA suggesting I create a doc with a specification proposal for > > EMIT (and any other necessary semantic changes) that we can then iterate > on > > in public with the Calcite folks. I already have most of the content > > written (and there's a significant amount of background needed to justify > > some aspects of the proposal), so it'll mostly be a matter of pulling it > > all together into something coherent. Does that sound reasonable to > > everyone? > > > > -Tyler > > > > > > On Thu, Apr 6, 2017 at 10:26 AM Kenneth Knowles <k...@google.com.invalid> > > wrote: > > > > > Very cool! I'm really excited about this integration. > > > > > > On Thu, Apr 6, 2017 at 9:39 AM, Jean-Baptiste Onofré <j...@nanthrax.net> > > > wrote: > > > > > > > Hi, > > > > > > > > Mingmin and I prepared a new branch to have the SQL DSL in dsls/sql > > > > location. > > > > > > > > Any help is welcome ! > > > > > > > > Thanks, > > > > Regards > > > > JB > > > > > > > > > > > > On 04/06/2017 06:36 PM, Mingmin Xu wrote: > > > > > > > >> @Tarush, you're very welcome to join the effort. > > > >> > > > >> On Thu, Apr 6, 2017 at 7:22 AM, tarush grover < > > tarushappt...@gmail.com> > > > >> wrote: > > > >> > > > >> Hi, > > > >>> > > > >>> Can I be also part of this feature development. > > > >>> > > > >>> Regards, > > > >>> Tarush Grover > > > >>> > > > >>> On Thu, Apr 6, 2017 at 3:17 AM, Ted Yu <yuzhih...@gmail.com> > wrote: > > > >>> > > > >>> I compiled BEAM-301 branch with calcite 1.12 - passed. > > > >>>> > > > >>>> Julian tries to not break existing things, but he will if there's > a > > > >>>> > > > >>> reason > > > >>> > > > >>>> to do so :-) > > > >>>> > > > >>>> On Wed, Apr 5, 2017 at 2:36 PM, Mingmin Xu <mingm...@gmail.com> > > > wrote: > > > >>>> > > > >>>> @Ted, thanks for the note. I intend to stick with one version, > Beam > > > >>>>> > > > >>>> 0.6.0 > > > >>> > > > >>>> and Calcite 1.11 so far, unless impacted by API change. Before > it's > > > >>>>> > > > >>>> merged > > > >>>> > > > >>>>> back to master, will upgrade to the latest version. > > > >>>>> > > > >>>>> On Wed, Apr 5, 2017 at 2:14 PM, Ted Yu <yuzhih...@gmail.com> > > wrote: > > > >>>>> > > > >>>>> Working in feature branch is good - you may want to periodically > > sync > > > >>>>>> > > > >>>>> up > > > >>>> > > > >>>>> with master. > > > >>>>>> > > > >>>>>> I noticed that you are using 1.11.0 of calcite. > > > >>>>>> 1.12 is out, FYI > > > >>>>>> > > > >>>>>> On Wed, Apr 5, 2017 at 2:05 PM, Mingmin Xu <mingm...@gmail.com> > > > >>>>>> > > > >>>>> wrote: > > > >>> > > > >>>> > > > >>>>>> Hi all, > > > >>>>>>> > > > >>>>>>> I'm working on https://issues.apache.org/ > > jira/browse/BEAM-301(Add > > > >>>>>>> > > > >>>>>> a > > > >>> > > > >>>> Beam > > > >>>>> > > > >>>>>> SQL DSL). The skeleton is already in > > > >>>>>>> https://github.com/XuMingmin/beam/tree/BEAM-301, using Java > SDK > > in > > > >>>>>>> > > > >>>>>> the > > > >>>> > > > >>>>> back-end. The goal is to provide a SQL interface over Beam, based > > > >>>>>>> > > > >>>>>> on > > > >>> > > > >>>> Calcite, including: > > > >>>>>>> 1). a translator to create Beam pipeline from SQL, > > > >>>>>>> (SELECT/INSERT/FILTER/GROUP-BY/JOIN/...); > > > >>>>>>> 2). an interactive client to submit queries; (All-SQL mode) > > > >>>>>>> 3). a SQL API which reduce the work to create a Pipeline; > > (Semi-SQL > > > >>>>>>> > > > >>>>>> mode) > > > >>>>> > > > >>>>>> > > > >>>>>>> As we see many folks are interested in this feature, would like > > to > > > >>>>>>> > > > >>>>>> create a > > > >>>>>> > > > >>>>>>> feature branch to have more involvement. > > > >>>>>>> Looking for comments and feedback. > > > >>>>>>> > > > >>>>>>> Thanks! > > > >>>>>>> ---- > > > >>>>>>> Mingmin > > > >>>>>>> > > > >>>>>>> > > > >>>>>> > > > >>>>> > > > >>>>> > > > >>>>> -- > > > >>>>> ---- > > > >>>>> Mingmin > > > >>>>> > > > >>>>> > > > >>>> > > > >>> > > > >> > > > >> > > > >> > > > > -- > > > > Jean-Baptiste Onofré > > > > jbono...@apache.org > > > > http://blog.nanthrax.net > > > > Talend - http://www.talend.com > > > > > > > > > > > > > -- > ---- > Mingmin > -- 陈竞，中科院计算技术研究所，高性能计算机中心 Jing Chen HPCC.ICT.AC China
