The most recent work on cross-language pipeline authoring is the design brainstorming at https://s.apache.org/beam-mixed-language-pipelines so it is still in the preliminary stages. There's no basic mystery, but there are a lot of practical considerations about what is easy to run on a pipeline author's machine.
Regarding Apache Calcite - it is a Java library. It doesn't really make sense to bind it to Python. Today we don't use most of its capabilities. We just use it as a parser mostly. It would be easy to find an existing parser in Python or write your own (with ply, the basics could be done within a day). But still I don't think it makes sense to reimplement and maintain the SQL-to-Beam translation in multiple languages. Kenn On Fri, Apr 13, 2018 at 2:43 AM Reuven Lax <re...@google.com> wrote: > If someone implemented it directly in Python then it would be supported > directly in Python. I don't know if anyone is actively working on that - > the current implementation uses Apache Calcite, and I don't know whether > they have a Python API. > > On Fri, Apr 13, 2018 at 9:40 AM Prabeesh K. <prabsma...@gmail.com> wrote: > >> What about supporting SQL in Python SDK? >> >> On 13 April 2018 at 13:32, Reuven Lax <re...@google.com> wrote: >> >>> The portability work will allow the Python and Java SDKs to be used in >>> the same pipeline, though this work is not yet complete. >>> >>> >> This is would be an interesting feature. >> >> On Fri, Apr 13, 2018 at 9:15 AM Gabor Hermann <m...@gaborhermann.com> >>> wrote: >>> >>>> Hey all, >>>> >>>> Are there any efforts towards supporting SQL from the Python SDK, not >>>> just from Java? I couldn't find any info about this in JIRA or mailing >>>> lists. >>>> >>>> How much effort do you think it would take to implement this? Are there >>>> some dependencies like supporting more features in Python? I know that >>>> the Python SDK is experimental. >>>> >>>> As an alternative, is there a way to combine Python and Java SDKs in >>>> the >>>> same pipeline? >>>> >>>> Thanks for your answers in advance! >>>> >>>> Cheers, >>>> Gabor >>>> >>>> >>
