Hi Jing & Shammon, IMO, Jdbc is a Java standard and Flink-JDBC-Driver is an implementation to interact with Flink, so in theory, it would be doable to use Flink-JDBC-Driver in Flink-JDBC-Connector if we add it as a dialect.
However, I don't see a scenario which requires this yet. All in all, it would be an orthogonal topic, which does not block current FLIP discussion. Shammon FY <zjur...@gmail.com> 于2023年3月13日周一 08:47写道： > Hi Jing > > Flink-Jdbc-Driver is mainly used to facilitate users to submit flink sql > jobs. I think we will not use Flink-Jdbc-Driver in Flink-Jdbc-Connector to > read/write data in general. If users can read/write specific tables by > Flink-Jdbc-Driver, they can use these tables in their flink jobs directly. > Thanks > > Best, > Shammon > > > On Mon, Mar 13, 2023 at 6:21 AM Jing Ge <j...@ververica.com.invalid> > wrote: > > > Hi, Shammon, > > > > This FLIP is very interesting. Thanks for driving it! Just out of > > curiosity, Could it be used in the Flink JDBC connector[1] after > > doing the appropriate implementation there? > > > > Best regards, > > Jing > > > > [1] > > > > > https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/table/jdbc/ > > > > On Fri, Mar 3, 2023 at 8:56 AM Shammon FY <zjur...@gmail.com> wrote: > > > > > Hi, > > > > > > Thanks jinsong. I think implementing JDBC for batch mode first sounds > > good. > > > This will simplify the implementation and we can also remove the row > kind > > > first. We can claim this in the FLIP and docs, I will update the FLIP. > > > > > > Best, > > > Shammon > > > > > > > > > On Fri, Mar 3, 2023 at 2:36 PM Jingsong Li <jingsongl...@gmail.com> > > wrote: > > > > > > > Hi, Shammon, > > > > > > > > I took a look at JDBC `ResultSet` and `Statement`. They are > > > > complicated and have many interfaces. Some of the interfaces may not > > > > be very suitable for streaming. > > > > > > > > I think maybe we can just implement JDBC for batch/olap only. It is > > > > hard to have an integration for JDBC and streaming... > > > > > > > > Do you need to use JDBC in streaming mode? Or do we just implement > > > > JDBC for batch only first? > > > > > > > > Best, > > > > Jingsong > > > > > > > > > > > > On Thu, Mar 2, 2023 at 6:22 PM Shammon FY <zjur...@gmail.com> wrote: > > > > > > > > > > Hi > > > > > > > > > > Thanks for the feedback from Jingsong and Benchao. > > > > > > > > > > For @Jingsong > > > > > > If the user does not cast into a FlinkResultSet, will there be > > > > > serious consequences here (RowKind is ignored)? > > > > > > > > > > I agree with you that it's indeed a big deal if users ignore the > row > > > kind > > > > > when they must know it. One idea that comes to my mind is we can > add > > an > > > > > option such as `table.result.changelog-mode`, users can set it > > through > > > > > connection properties or set dynamic parameters. The option value > can > > > be > > > > > `insert-only`, `upset` or `all` and the default value is > > `insert-only`. > > > > > > > > > > If the result does not conform to the changelog mode, the jdbc > driver > > > > > throws an exception. What do you think? > > > > > > > > > > > > > > > For @Benchao > > > > > > Besides `java.sql.Driver`, have you considered also adding > support > > > for > > > > > `javax.sql.DataSource` interface? > > > > > > > > > > I missed the `javax.sql.DataSource` and I have added it to the > FLIP, > > > > thanks > > > > > Benchao > > > > > > > > > > > > > > > Best, > > > > > Shammon > > > > > > > > > > On Wed, Mar 1, 2023 at 7:57 PM Benchao Li <libenc...@apache.org> > > > wrote: > > > > > > > > > > > +1 for the FLIP, thanks Shammon for driving this. > > > > > > > > > > > > JDBC is quite useful in OLAP scenarios, supporting JDBC would > > enable > > > > Flink > > > > > > to be used with existing tools, such as Tableau. > > > > > > > > > > > > Regarding the JDBC interfaces listed in the FLIP, I think they > > looks > > > > good > > > > > > already. Besides `java.sql.Driver`, have you considered also > adding > > > > support > > > > > > for `javax.sql.DataSource` interface? > > > > > > > > > > > > Jingsong Li <jingsongl...@gmail.com> 于2023年3月1日周三 17:53写道： > > > > > > > > > > > > > Thanks Shammon for driving. > > > > > > > > > > > > > > Big +1 for this. > > > > > > > > > > > > > > I heard that many users want to use FlinkGateway + JDBC to do > > some > > > > > > > queries, but at present, only Hive JDBC can be used. It is Hive > > > > > > > dialect by default, and the experience is also different from > > > > > > > FlinkSQL. We need to have our own JDBC. > > > > > > > > > > > > > > I took a look at your `Public Interface` part, only > > > > > > > `FlinkResultSet.getRowKind` is a true new interface, others are > > > just > > > > > > > implementations. > > > > > > > > > > > > > > If the user does not cast into a FlinkResultSet, will there be > > > > serious > > > > > > > consequences here (RowKind is ignored)? > > > > > > > > > > > > > > Best, > > > > > > > Jingsong > > > > > > > > > > > > > > On Wed, Mar 1, 2023 at 4:59 PM Shammon FY <zjur...@gmail.com> > > > wrote: > > > > > > > > > > > > > > > > Hi devs, > > > > > > > > > > > > > > > > I'd like to start a discussion about FLIP-293: Introduce > Flink > > > Jdbc > > > > > > > Driver > > > > > > > > For Sql Gateway[1]. > > > > > > > > > > > > > > > > FLIP-275[2] supports remote sql client based on gateway, > users > > > can > > > > > > > interact > > > > > > > > with gateway by flink console. However, for users who create > > > > session > > > > > > > > clusters with Flink, they'd like to use Jdbc Driver to > interact > > > > with > > > > > > the > > > > > > > > gateway in their applications, such as olap queries.. > > > > > > > > > > > > > > > > I have discussed this proposal with @shengkaifang and > > > @jinsonglee. > > > > In > > > > > > > this > > > > > > > > FLIP, we'd like to introduce Jdbc Driver for gateway. Users > can > > > use > > > > > > Jdbc > > > > > > > > Driver to submit their queries and get results like a > database > > in > > > > their > > > > > > > > applications. > > > > > > > > > > > > > > > > Looking forward to your feedback, thanks. > > > > > > > > > > > > > > > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-293%3A+Introduce+Flink+Jdbc+Driver+For+Sql+Gateway > > > > > > > > [2] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-275%3A+Support+Remote+SQL+Client+Based+on+SQL+Gateway > > > > > > > > > > > > > > > > > > > > > > > > Best, > > > > > > > > Shammon > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > Best, > > > > > > Benchao Li > > > > > > > > > > > > > > > > -- Best, Benchao Li
