Hi, all. Sorry for the late reply.

> Is the gateway mode planned to be supported for SQL Client in 1.17?
> Do you have anything you can already share so we can start with your work
or just play around with it.

Yes. @yzl is working on it and he will list the implementation plan later
and share the progress. I think the change is not very large and I think
it's not a big problem to finish this in the release-1.17. I will join to
develop this in the mid of November.

Best,
Shengkai




Jamie Grier <jgr...@apache.org> 于2022年11月5日周六 00:48写道:

> Hi Shengkai,
>
> We're doing more and more Flink development at Confluent these days and
> we're currently trying to bootstrap a prototype that relies on the SQL
> Client and Gateway.  We will be using the the components in some of our
> projects and would like to co-develop them with you and the rest of the
> Flink community.
>
> As of right now it's a pretty big blocker for our upcoming milestone that
> the SQL Client has not yet been modified to talk to the SQL Gateway and we
> want to help with this effort ASAP!  We would be even willing to take over
> the work if it's not yet started but I suspect it already is.
>
> Anyway, rather than start working immediately on the SQL Client and adding
> a the new Gateway mode ourselves we wanted to start a conversation with you
> and see where you're at with things and offer to help.
>
> Do you have anything you can already share so we can start with your work
> or just play around with it.  Like I said, we just want to get started and
> are very able to help in this area.  We see both the SQL Client and Gateway
> being very important for us and have a good team to help develop it.
>
> Let me know if there is a branch you can share, etc.  It would be much
> appreciated!
>
> -Jamie Grier
>
>
> On 2022/10/28 06:06:49 Shengkai Fang wrote:
> > Hi.
> >
> > > Is there a possibility for us to get engaged and at least introduce
> > initial changes to support authentication/authorization?
> >
> > Yes. You can write a FLIP about the design and change. We can discuss
> this
> > in the dev mail. If the FLIP passes, we can develop it together.
> >
> > > Another question about persistent Gateway: did you have any specific
> > thoughts about it or some draft design?
> >
> > We don't have any detailed plan about this. But I know Livy has a similar
> > feature.
> >
> > Best,
> > Shengkai
> >
> >
> > Alexey Leonov-Vendrovskiy <vendrov...@gmail.com> 于2022年10月27日周四 15:12写道:
> >
> > > Apologies from the delayed response on my side.
> > >
> > >  I think the authentication module is not part of our plan in 1.17
> because
> > >> of the busy work. I think we'll start the design at the end of the
> > >> release-1.17.
> > >
> > >
> > > Is there a possibility for us to get engaged and at least introduce
> > > initial changes to support authentication/authorization? Specifically,
> > > changes in the API and in SQL Client.
> > >
> > > We expect the following authentication flow:
> > >
> > > On the SQL gateway we want to be able to use a delegation token.
> > > SQL client should be able to supply an API key.
> > > The SQL Gateway *would not *be submitting jobs on behalf of the client.
> > >
> > > Ideally it would be nice to introduce some interfaces in the SQL
> Gateway
> > > that would allow implementing custom authentication and authorization.
> > >
> > > Another question about persistent Gateway: did you have any specific
> > > thoughts about it or some draft design?
> > >
> > > Thanks,
> > > Alexey
> > >
> > >
> > > On Fri, Oct 21, 2022 at 1:13 AM Shengkai Fang <fskm...@gmail.com>
> wrote:
> > >
> > >> Sorry for the late response.
> > >>
> > >> In the next version(Flink 1.17), we plan to support the SQL Client to
> > >> submit the statement to the Flink SQL Gateway. The FLINK-29486
> > >> <https://issues.apache.org/jira/browse/FLINK-29486> is the first
> step to
> > >> remove the usage of the `Parser` in the client side, which needs to
> read
> > >> the table schema during the converting sql node to operation. I think
> the authentication
> > >> module is not part of our plan in 1.17 because of the busy work. I
> think
> > >> we'll start the design at the end of the release-1.17.
> > >> But could you share more details about the requirements of the
> > >> authentication?
> > >> - Do you use the kerberos or delegation token or password to do the
> > >> authentication?
> > >> - After the authentication, do you need the sql gateway to submit the
> > >> job on behalf of the client?
> > >> - ...
> > >>
> > >> For detailed implementation, I think Hive and Presto are good
> examples to
> > >> dig in.  If you have some thoughts about the authentication module,
> > >> please let me know.
> > >>
> > >> Best,
> > >> Shengkai
> > >>
> > >> Alexey Leonov-Vendrovskiy <vendrov...@gmail.com> 于2022年10月19日周三
> 00:37写道:
> > >>
> > >>> Thank you for the response, Yuxia!
> > >>>
> > >>> Shengkai, I would like to learn more about nearest and a bit more
> > >>> distant plans about development of the SQL Gateway and the SQL
> Client.
> > >>> Do you have a description of the work planned or maybe can share
> general
> > >>> thoughts about the Authentication module, or Persistent Gateway.
> > >>> How can the authentication part be addressed on the SQL Client side?
> > >>>
> > >>> Regards,
> > >>> -Alexey
> > >>>
> > >>>
> > >>> On Wed, Oct 12, 2022 at 11:24 PM yuxia <luoyu...@alumni.sjtu.edu.cn>
> > >>> wrote:
> > >>>
> > >>>> > In what Flink’s release the connection from SQL Client to the
> Gateway
> > >>>> is
> > >>>> expected to be added?
> > >>>> Flink 1.17
> > >>>>
> > >>>> > “Authentication module” (2) and “Persistent Gateway” (4) as
> > >>>> possible future work. Were there any recent discussions on these
> > >>>> subjects?
> > >>>> No recent discussions on these subjects, but I think it'll come in
> > >>>> Flink 1.17
> > >>>>
> > >>>> > Another related topic: are there ideas around making SQL Gateway a
> > >>>> multi-tenant
> > >>>> component?
> > >>>> Yes.
> > >>>>
> > >>>> Shengkaiis the maintainer of SQL Client and SQL gateway, maybe he
> can
> > >>>> provide more information.
> > >>>>
> > >>>>
> > >>>>
> > >>>> Best regards,
> > >>>> Yuxia
> > >>>>
> > >>>> ----- 原始邮件 -----
> > >>>> 发件人: "Alexey Leonov-Vendrovskiy" <vendrov...@gmail.com>
> > >>>> 收件人: "dev" <dev@flink.apache.org>
> > >>>> 发送时间: 星期四, 2022年 10 月 13日 下午 12:33:08
> > >>>> 主题: SQL Gateway and SQL Client
> > >>>>
> > >>>> Hi all,
> > >>>>
> > >>>> I’m Alexey from Confluent. This is my first email in this discussion
> > >>>> list.
> > >>>> I’m rather new to Flink, and to local customs of communication. I
> want
> > >>>> to
> > >>>> dive deeper and hopefully get more involved over time.
> > >>>>
> > >>>> Currently I have a few questions around SQL Gateway and SQL Client.
> > >>>> Specifically I wanted to learn what is the vision around the nearest
> > >>>> future
> > >>>> of these two components.
> > >>>>
> > >>>> In what Flink’s release the connection from SQL Client to the
> Gateway is
> > >>>> expected to be added? I was looking at
> > >>>> https://issues.apache.org/jira/browse/FLINK-29486, and recently it
> got
> > >>>> renamed from “Enable SQL Client to Connect SQL Gateway in Remote
> Mode”
> > >>>> to
> > >>>> “Introduce Client Parser to get statement type”.  I did some
> search, but
> > >>>> didn’t find a good place where the client's work in this direction
> is
> > >>>> discussed or tracked.
> > >>>>
> > >>>> A couple questions about the SQL Gateway. The FLIP-91
> > >>>> <
> > >>>>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-91%3A+Support+SQL+Gateway#FLIP91:SupportSQLGateway-Futurework
> > >>>> >
> > >>>> mentions “Authentication module” (2) and “Persistent Gateway” (4) as
> > >>>> possible future work. Were there any recent discussions on these
> > >>>> subjects?
> > >>>> Or maybe there are some ideas how to move these directions forward?
> > >>>> Another
> > >>>> related topic: are there ideas around making SQL Gateway a
> multi-tenant
> > >>>> component?
> > >>>>
> > >>>> Thank you,
> > >>>>
> > >>>> Alexey
> > >>>>
> > >>>
> >
>

Reply via email to