I'm very excited by this development as well, thanks for continuing to push this forward, Mingmin. :-)
I noticed you'd made some changes to your design doc <https://docs.google.com/document/d/1Uc5xYTpO9qsLXtT38OfuoqSLimH_0a1Bz5BsCROMzCU/edit>. Is it ready for another review? How reflective is it currently of the work that going into the feature branch? In parallel, I'd also like to continue helping push forward the definition of unified model semantics for SQL so we can get Calcite to a point where it supports the full Beam model. I added a comment <https://issues.apache.org/jira/browse/BEAM-301?focusedCommentId=15959621&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15959621> on the JIRA suggesting I create a doc with a specification proposal for EMIT (and any other necessary semantic changes) that we can then iterate on in public with the Calcite folks. I already have most of the content written (and there's a significant amount of background needed to justify some aspects of the proposal), so it'll mostly be a matter of pulling it all together into something coherent. Does that sound reasonable to everyone? -Tyler On Thu, Apr 6, 2017 at 10:26 AM Kenneth Knowles <k...@google.com.invalid> wrote: > Very cool! I'm really excited about this integration. > > On Thu, Apr 6, 2017 at 9:39 AM, Jean-Baptiste Onofré <j...@nanthrax.net> > wrote: > > > Hi, > > > > Mingmin and I prepared a new branch to have the SQL DSL in dsls/sql > > location. > > > > Any help is welcome ! > > > > Thanks, > > Regards > > JB > > > > > > On 04/06/2017 06:36 PM, Mingmin Xu wrote: > > > >> @Tarush, you're very welcome to join the effort. > >> > >> On Thu, Apr 6, 2017 at 7:22 AM, tarush grover <tarushappt...@gmail.com> > >> wrote: > >> > >> Hi, > >>> > >>> Can I be also part of this feature development. > >>> > >>> Regards, > >>> Tarush Grover > >>> > >>> On Thu, Apr 6, 2017 at 3:17 AM, Ted Yu <yuzhih...@gmail.com> wrote: > >>> > >>> I compiled BEAM-301 branch with calcite 1.12 - passed. > >>>> > >>>> Julian tries to not break existing things, but he will if there's a > >>>> > >>> reason > >>> > >>>> to do so :-) > >>>> > >>>> On Wed, Apr 5, 2017 at 2:36 PM, Mingmin Xu <mingm...@gmail.com> > wrote: > >>>> > >>>> @Ted, thanks for the note. I intend to stick with one version, Beam > >>>>> > >>>> 0.6.0 > >>> > >>>> and Calcite 1.11 so far, unless impacted by API change. Before it's > >>>>> > >>>> merged > >>>> > >>>>> back to master, will upgrade to the latest version. > >>>>> > >>>>> On Wed, Apr 5, 2017 at 2:14 PM, Ted Yu <yuzhih...@gmail.com> wrote: > >>>>> > >>>>> Working in feature branch is good - you may want to periodically sync > >>>>>> > >>>>> up > >>>> > >>>>> with master. > >>>>>> > >>>>>> I noticed that you are using 1.11.0 of calcite. > >>>>>> 1.12 is out, FYI > >>>>>> > >>>>>> On Wed, Apr 5, 2017 at 2:05 PM, Mingmin Xu <mingm...@gmail.com> > >>>>>> > >>>>> wrote: > >>> > >>>> > >>>>>> Hi all, > >>>>>>> > >>>>>>> I'm working on https://issues.apache.org/jira/browse/BEAM-301(Add > >>>>>>> > >>>>>> a > >>> > >>>> Beam > >>>>> > >>>>>> SQL DSL). The skeleton is already in > >>>>>>> https://github.com/XuMingmin/beam/tree/BEAM-301, using Java SDK in > >>>>>>> > >>>>>> the > >>>> > >>>>> back-end. The goal is to provide a SQL interface over Beam, based > >>>>>>> > >>>>>> on > >>> > >>>> Calcite, including: > >>>>>>> 1). a translator to create Beam pipeline from SQL, > >>>>>>> (SELECT/INSERT/FILTER/GROUP-BY/JOIN/...); > >>>>>>> 2). an interactive client to submit queries; (All-SQL mode) > >>>>>>> 3). a SQL API which reduce the work to create a Pipeline; (Semi-SQL > >>>>>>> > >>>>>> mode) > >>>>> > >>>>>> > >>>>>>> As we see many folks are interested in this feature, would like to > >>>>>>> > >>>>>> create a > >>>>>> > >>>>>>> feature branch to have more involvement. > >>>>>>> Looking for comments and feedback. > >>>>>>> > >>>>>>> Thanks! > >>>>>>> ---- > >>>>>>> Mingmin > >>>>>>> > >>>>>>> > >>>>>> > >>>>> > >>>>> > >>>>> -- > >>>>> ---- > >>>>> Mingmin > >>>>> > >>>>> > >>>> > >>> > >> > >> > >> > > -- > > Jean-Baptiste Onofré > > jbono...@apache.org > > http://blog.nanthrax.net > > Talend - http://www.talend.com > > >
