Thanks Paul for driving this proposal. I found the sql driver has no config related options. If I understand correctly, the sql driver can be used to submit sql jobs in a 'job submission service' such as sql-gateway. In general, in addition to the default config for Flink cluster which includes k8s, ha and .etc, users may also specify configurations for SQL jobs, including parallelism, number of Task Managers, etc.
For example, in sql-gateway users can `set` dynamic parameters before submitting a sql statement, and for sql files users may put their configurations in a `yaml` file. Should sql driver support dynamic parameters and specify config file? Best, Shammon FY On Tue, May 30, 2023 at 9:57 PM Weihua Hu <huweihua....@gmail.com> wrote: > Thanks Paul for the proposal. > > +1 for this. It is valuable in improving ease of use. > > I have a few questions. > - Is SQLRunner the better name? We use this to run a SQL Job. (Not strong, > the SQLDriver is fine for me) > - Could we run SQL jobs using SQL in strings? Otherwise, we need to prepare > a SQL file in an image for Kubernetes application mode, which may be a bit > cumbersome. > - I noticed that we don't specify the SQLDriver jar in the > "run-application" > command. Does that mean we need to perform automatic detection in Flink? > > > Best, > Weihua > > > On Mon, May 29, 2023 at 7:24 PM Paul Lam <paullin3...@gmail.com> wrote: > > > Hi team, > > > > I’d like to start a discussion about FLIP-316 [1], which introduces a SQL > > driver as the > > default main class for Flink SQL jobs. > > > > Currently, Flink SQL could be executed out of the box either via SQL > > Client/Gateway > > or embedded in a Flink Java/Python program. > > > > However, each one has its drawback: > > > > - SQL Client/Gateway doesn’t support the application deployment mode [2] > > - Flink Java/Python program requires extra work to write a non-SQL > program > > > > Therefore, I propose adding a SQL driver to act as the default main class > > for SQL jobs. > > Please see the FLIP docs for details and feel free to comment. Thanks! > > > > [1] > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-316%3A+Introduce+SQL+Driver > > < > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-316:+Introduce+SQL+Driver > > > > > [2] https://issues.apache.org/jira/browse/FLINK-26541 < > > https://issues.apache.org/jira/browse/FLINK-26541> > > > > Best, > > Paul Lam >
