Re: Discussion on Vertica-Flink Connector

[Yanquan Lv]

Hi, Vikas.
Vertica can be connected and accessed through JDBC API, I'm not sure if you
connected through this API,
Flink already provided a flink-connector-jdbc repo, and Supported a
considerable number of databases through dialect, could we add a Vertica
implementation on this?

[1] https://github.com/apache/flink-connector-jdbc


Vikas Garg  于2024年7月23日周二 20:56写道：

> Hello,
>
> We have developed a connector for Vertica that will work with Flink. I
> have created a Google page with all the details to carry the discussion on
> the connector.
>
> The connector will be maintained by our organization. I have created
> Google Page with the details of the connector that will be used in FLIP
> post discussion:
> https://docs.google.com/document/d/17yRMtfW2qNMqTmuhUlUOAOaa0PQlEoy6/edit<
> https://docs.google.com/document/d/17yRMtfW2qNMqTmuhUlUOAOaa0PQlEoy6/edit?usp=sharing=109692294681382335070=true=true
> >
>
>
> Vikas Garg
> General Engineering  |  General Engineering
>
> Mobile: (+91) 9811716552
> LinkedIn:   https://www.linkedin.com/company/opentext
> Twitter:https://twitter.com/OpenText
> Website:https://www.opentext.com/
>
>
>
>
> [
> http://mimage.opentext.com/alt_content/binary/images/email-signature/ot2023-corporate-email-600x150-fy24.jpg
> ]
>
>
> This email message is confidential, may be privileged, and is intended for
> the exclusive use of the addressee. Any other person is strictly prohibited
> from disclosing or reproducing it. If the addressee cannot be reached or is
> unknown to you, please inform the sender by return email and delete this
> email message and all copies immediately.
>

Re: [DISCUSS] FLIP-XXX Amazon SQS Source Connector

[Yanquan Lv]

Hi Dhingra, thanks for driving this.  I am not very familiar with SQS, but
this should be some kind of message queue.  So could we add metric of
currentFetchEventTimeLag
and currentEmitEventTimeLag from FLIP-33[1] in SQS source? and I want to
know what metrics do we provide in SQS source.

[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-33%3A+Standardize+Connector+Metrics

Saurabh Singh  于2024年7月20日周六 02:43写道：

> Hi Fink Devs,
>
> Our team has been working on migrating various data pipelines to Flink to
> leverage the benefits of exactly-once processing, checkpointing, and
> stateful computing. We have several use cases built around the AWS SQS
> Service. For this migration, we have developed an SQS Source Connector,
> which enables us to run both stateless and stateful Flink-based jobs.
>
> We believe that this SQS Source Connector would be a valuable addition to
> the existing connector set. Therefore, we propose a FLIP to include it.
>
> For more information, please refer to the FLIP document.
>
>
> https://docs.google.com/document/d/1lreo27jNh0LkRs1Mj9B3wj3itrzMa38D4_XGryOIFks/edit?usp=sharing
>
> Thanks
> Saurabh & Abhi
>

Re: [VOTE] FLIP-460: Display source/sink I/O metrics on Flink Web UI

[Yanquan Lv]

+1 (non-binding)
Glad to see a more user-friendly display page.


Best,
Yanquan

Rui Fan <1996fan...@gmail.com> 于2024年7月17日周三 09:48写道：

> +1 (binding)
>
> Best,
> Rui
>
> On Wed, Jul 17, 2024 at 9:07 AM Xintong Song 
> wrote:
>
> > +1 (binding)
> >
> > Best,
> >
> > Xintong
> >
> >
> >
> > On Wed, Jul 17, 2024 at 5:58 AM Aleksandr Pilipenko 
> > wrote:
> >
> > > +1 (non-binding)
> > >
> > > Thanks,
> > > Aleksandr
> > >
> > > On Tue, 16 Jul 2024 at 22:56, Ahmed Hamdy 
> wrote:
> > >
> > > > +1 (non-binding)
> > > >
> > > > Best Regards
> > > > Ahmed Hamdy
> > > >
> > > >
> > > > On Tue, 16 Jul 2024 at 21:58, Becket Qin 
> wrote:
> > > >
> > > > > +1 (binding)
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Jiangjie (Becket) Qin
> > > > >
> > > > > On Tue, Jul 16, 2024 at 12:19 AM Leonard Xu 
> > wrote:
> > > > >
> > > > > > +1(binding)
> > > > > >
> > > > > > Best,
> > > > > > Leonard
> > > > > >
> > > > > > > 2024年7月16日 下午3:15，Robert Metzger  写道：
> > > > > > >
> > > > > > > +1 (binding)
> > > > > > >
> > > > > > > Nice to see this fixed ;)
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Tue, Jul 16, 2024 at 8:46 AM Yong Fang 
> > > wrote:
> > > > > > >
> > > > > > >> +1 (binding)
> > > > > > >>
> > > > > > >> Best,
> > > > > > >> FangYong
> > > > > > >>
> > > > > > >>
> > > > > > >> On Tue, Jul 16, 2024 at 1:14 PM Zhanghao Chen <
> > > > > > zhanghao.c...@outlook.com>
> > > > > > >> wrote:
> > > > > > >>
> > > > > > >>> Hi everyone,
> > > > > > >>>
> > > > > > >>>
> > > > > > >>> Thanks for all the feedback about the FLIP-460: Display
> > > source/sink
> > > > > I/O
> > > > > > >>> metrics on Flink Web UI [1]. The discussion
> > > > > > >>> thread is here [2]. I'd like to start a vote on it.
> > > > > > >>>
> > > > > > >>> The vote will be open for at least 72 hours unless there is
> an
> > > > > > objection
> > > > > > >>> or insufficient votes.
> > > > > > >>>
> > > > > > >>> [1]
> > > > > > >>>
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=309496355
> > > > > > >>> [2]
> > > > https://lists.apache.org/thread/sy271nhd2jr1r942f29xbvbgq7fsd841
> > > > > > >>>
> > > > > > >>> Best,
> > > > > > >>> Zhanghao Chen
> > > > > > >>>
> > > > > > >>
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: [VOTE] FLIP-465: Introduce DESCRIBE FUNCTION

[Yanquan Lv]

Hi Natea, thanks for driving it.
+1 (non-binding).

Jim Hughes  于2024年7月4日周四 04:41写道：

> Hi Natea,
>
> Looks good to me!
>
> +1 (non-binding).
>
> Cheers,
>
> Jim
>
> On Wed, Jul 3, 2024 at 3:16 PM Natea Eshetu Beshada
>  wrote:
>
> > Sorry I forgot to include the FLIP [1] and the mailing thread discussion
> > link [2] in my previous email.
> >
> > [1]
> >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-465%3A+Introduce+DESCRIBE+FUNCTION
> > [2] https://lists.apache.org/thread/s46ftnmz4ggmmssgyx6vfhqjttsk9lph
> >
> > Thanks,
> > Natea
> >
> > On Wed, Jul 3, 2024 at 12:06 PM Natea Eshetu Beshada <
> > nbesh...@confluent.io>
> > wrote:
> >
> > > Hello everyone,
> > >
> > > I would like to start a vote on FLIP-465 [1]. It proposes adding SQL
> > > syntax that would allow users to describe the metadata of a given
> > function.
> > >
> > > The vote will be open for at least 72 hours (Saturday, July 6th of July
> > > 2024,
> > > 12:30 PST) unless there is an objection or insufficient votes.
> > >
> > > Thanks,
> > > Natea
> > >
> > >
> >
>

Re: [DISCUSS] FLIP-456: Introduce DESCRIBE FUNCTION

[Yanquan Lv]

Hi, Natea. This FLIP looks good from my side.
I also look forward to showing the return type, as well as auxiliary
information such as comments or usage, although the current implementation
does not include this part.


Natea Eshetu Beshada  于2024年6月26日周三 03:01写道：

> Oh no haha, yes thanks for pointing that out Jim! Of course I make a typo
> on an editable format like email :)
>
> On Tue, Jun 25, 2024 at 11:20 AM Jim Hughes 
> wrote:
>
> > Hi Natea,
> >
> > Looks good.  As a note, in the title of this email, a number got
> switched!
> > FLIP-456 is about compiled plans for batch operators. :)
> >
> > The link below is correct.
> >
> > Cheers,
> >
> > Jim
> >
> > On Tue, Jun 25, 2024 at 1:29 PM Natea Eshetu Beshada
> >  wrote:
> >
> > > Hello all,
> > >
> > > I would like to kickstart the discussion of FLIP-465: Introduce
> DESCRIBE
> > > FUNCTION [1].
> > >
> > > The proposal is to add SQL syntax that would allow users to describe
> the
> > > metadata of a given function.
> > >
> > > I look forward to hearing feedback from the community.
> > >
> > > [1]
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-465%3A+Introduce+DESCRIBE+FUNCTION
> > >
> > > Thanks,
> > > Natea
> > >
> >
>

Re: [VOTE] FLIP-463: Schema Definition in CREATE TABLE AS Statement

[Yanquan Lv]

Thnaks Sergio, +1 (non-binding)

gongzhongqiang  于2024年6月23日周日 10:06写道：

> +1 (non-binding)
>
> Best,
> Zhongqiang Gong
>
> Sergio Pena  于2024年6月21日周五 22:18写道：
>
> > Hi everyone,
> >
> > Thanks for all the feedback about FLIP-463: Schema Definition in CREATE
> > TABLE AS Statement [1]. The discussion thread is here [2].
> >
> > I'd like to start a vote for it. The vote will be open for at least 72
> > hours unless there is an objection or insufficient votes. The FLIP will
> be
> > considered accepted if 3 binding votes (from active committers according
> to
> > the Flink bylaws [3]) are gathered by the community.
> >
> > [1]
> >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-463%3A+Schema+Definition+in+CREATE+TABLE+AS+Statement
> > [2] https://lists.apache.org/thread/1ryxxyyg3h9v4rbosc80zryvjk6c8k83
> > [3] [
> >
> >
> https://cwiki.apache.org/confluence/display/FLINK/Flink+Bylaws#FlinkBylaws-Approvals](https://cwiki.apache.org/confluence/display/FLINK/Flink+Bylaws#FlinkBylaws-Approvals)
> 
> > <
> https://cwiki.apache.org/confluence/display/FLINK/Flink+Bylaws#FlinkBylaws-Approvals](https://cwiki.apache.org/confluence/display/FLINK/Flink+Bylaws%23FlinkBylaws-Approvals)
> >
> > <
> >
> https://cwiki.apache.org/confluence/display/FLINK/Flink+Bylaws#FlinkBylaws-Approvals](https://cwiki.apache.org/confluence/display/FLINK/Flink+Bylaws%23FlinkBylaws-Approvals)
> > >
> >
> > Thanks,
> > Sergio Peña
> >
>

Re: [VOTE] Apache Flink CDC Release 3.1.1, release candidate #0

[Yanquan Lv]

+1 (non-binding)

- verified gpg signatures
- verified sha512 hash
- checked Github release tag
- checked release notes
- restarted job with savepoint from cdc 3.0.1

Xiqian YU  于2024年6月14日周五 18:34写道：

> Thanks Qingsheng for driving this release!
>
> +1 (non-binding)
>
>
>   *   Verified tarball checksum
>   *   Compiled CDC from source
>   *   Confirmed jars were compiled with JDK 8
>   *   Ran savepoint migration test from CDC 3.0.x
>   *   Ran Pipeline E2e tests with Flink 1.18.1 / 1.19.0
>
> Regards,
> Xiqian
>
> De : Qingsheng Ren 
> Date : vendredi, 14 juin 2024 à 15:06
> À : dev 
> Objet : [VOTE] Apache Flink CDC Release 3.1.1, release candidate #0
> Hi everyone,
>
> Please review and vote on the release candidate #0 for the version 3.1.1 of
> Apache Flink CDC, as follows:
> [ ] +1, Approve the release
> [ ] -1, Do not approve the release (please provide specific comments)
>
> **Release Overview**
>
> As an overview, the release consists of the following:
> a) Flink CDC source release to be deployed to dist.apache.org
> b) Maven artifacts to be deployed to the Maven Central Repository
>
> **Staging Areas to Review**
>
> The staging areas containing the above mentioned artifacts are as follows,
> for your review:
> * All artifacts for a) can be found in the corresponding dev repository at
> dist.apache.org [1], which are signed with the key with fingerprint
> A1BD477F79D036D2C30CA7DBCA8AEEC2F6EB040B [2]
> * All artifacts for b) can be found at the Apache Nexus Repository [3]
>
> Other links for your review:
> * JIRA release notes [4]
> * Source code tag "release-3.1.1-rc0" with commit hash
> b163b8e1589184bd7716cf6d002f3472766eb5db [5]
> * PR for release announcement blog post of Flink CDC 3.1.1 in flink-web [6]
>
> **Vote Duration**
>
> The voting time will run for at least 72 hours.
> It is adopted by majority approval, with at least 3 PMC affirmative votes.
>
> Thanks,
> Qingsheng Ren
>
> [1] https://dist.apache.org/repos/dist/dev/flink/flink-cdc-3.1.1-rc0
> [2] https://dist.apache.org/repos/dist/release/flink/KEYS
> [3] https://repository.apache.org/content/repositories/orgapacheflink-1739
> [4]
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12354763
> [5] https://github.com/apache/flink-cdc/releases/tag/release-3.1.1-rc0
> [6] https://github.com/apache/flink-web/pull/746
>

Re: [ANNOUNCE] New Apache Flink Committer - Zhongqiang Gong

[Yanquan Lv]

Congratulations, Zhongqiang!

Shawn Huang  于2024年6月17日周一 12:05写道：

> Congratulations !
>
> Best,
> Shawn Huang
>
>
> Yuepeng Pan  于2024年6月17日周一 12:03写道：
>
> > Congratulations ! Best regards Yuepeng Pan
> >
> >
> >
> >
> >
> > At 2024-06-17 11:20:30, "Leonard Xu"  wrote:
> > >Hi everyone,
> > >On behalf of the PMC, I'm happy to announce that Zhongqiang Gong has
> > become a new Flink Committer!
> > >
> > >Zhongqiang has been an active Flink community member since November
> 2021,
> > contributing numerous PRs to both the Flink and Flink CDC repositories.
> As
> > a core contributor to Flink CDC, he developed the Oracle and SQL Server
> CDC
> > Connectors and managed essential website and CI migrations during the
> > donation of Flink CDC to Apache Flink.
> > >
> > >Beyond his technical contributions, Zhongqiang actively participates in
> > discussions on the Flink dev mailing list and responds to threads on the
> > user and user-zh mailing lists. As an Apache StreamPark (incubating)
> > Committer, he promotes Flink SQL and Flink CDC technologies at meetups
> and
> > within the StreamPark community.
> > >
> > >Please join me in congratulating Zhongqiang Gong for becoming an Apache
> > Flink committer!
> > >
> > >Best,
> > >Leonard (on behalf of the Flink PMC)
> >
>

Re: [ANNOUNCE] New Apache Flink Committer - Hang Ruan

[Yanquan Lv]

Congratulations, Hang!

Samrat Deb  于2024年6月17日周一 11:32写道：

> Congratulations Hang Ruan !
>
> Bests,
> Samrat
>
> On Mon, Jun 17, 2024 at 8:47 AM Leonard Xu  wrote:
>
> > Hi everyone,
> > On behalf of the PMC, I'm happy to let you know that Hang Ruan has become
> > a new Flink Committer !
> >
> > Hang Ruan has been continuously contributing to the Flink project since
> > August 2021. Since then, he has continuously contributed to Flink, Flink
> > CDC, and various Flink connector repositories, including
> > flink-connector-kafka, flink-connector-elasticsearch,
> flink-connector-aws,
> > flink-connector-rabbitmq, flink-connector-pulsar, and
> > flink-connector-mongodb. Hang Ruan focuses on the improvements related to
> > connectors and catalogs and initiated FLIP-274. He is most recognized as
> a
> > core contributor and maintainer for the Flink CDC project, contributing
> > many features such as MySQL CDC newly table addition and the Schema
> > Evolution feature.
> >
> > Beyond his technical contributions, Hang Ruan is an active member of the
> > Flink community. He regularly engages in discussions on the Flink dev
> > mailing list and the user-zh and user mailing lists, participates in FLIP
> > discussions, assists with user Q, and consistently volunteers for
> release
> > verifications.
> >
> > Please join me in congratulating Hang Ruan for becoming an Apache Flink
> > committer!
> >
> > Best,
> > Leonard (on behalf of the Flink PMC)
>

Re: [DISCUSS] FLIP-463: Schema Definition in CREATE TABLE AS Statement

[Yanquan Lv]

Hi Sergio, thanks for driving it, +1 for this.

I have some comments:
1. If we have a source table with primary keys and partition keys defined,
what is the default behavior if PARTITIONED and DISTRIBUTED not specified
in the CTAS statement, It should not be inherited by default？
2. I suggest providing a complete syntax that includes table_properties
like FLIP-218.


Sergio Pena  于2024年6月12日周三 03:54写道：

> I just noticed the CREATE TABLE LIKE statement allows the definition of new
> columns in the CREATE part. The difference
> with this CTAS proposal is that TABLE LIKE appends the new columns at the
> end of the schema instead of adding them
> at the beginning like this proposal and Mysql do.
>
> > create table t1(id int, name string);
> > > create table s1(a int, b string) like t1;
> > > describe s1;
>
> +-+---+--++
> > | Column Name | Data Type | Nullable | Extras |
> > +-+---+--++
> > | id  | INT   | NULL ||
> > | name| STRING| NULL ||
> > | a   | INT   | NULL ||
> > | b   | STRING| NULL ||
> > +-+---+--++
>
>
>
> The CREATE TABLE LIKE also does not let the definition of existing columns
> in the CREATE part. The statement fails
> that the column already exists.
>
> > create table t1(id int, name string);
>
> > create table s1(id double) like t1;
> > A column named 'id' already exists in the base table.
> >
>
> What do you guys think of making it similar to the CREATE TABLE LIKE? Seems
> the best approach in order to
> be compatible with it.
>
> - Sergio
>
> On Tue, Jun 11, 2024 at 2:10 PM Sergio Pena  wrote:
>
> > Thanks Timo for answering Jeyhun questions.
> >
> > To add info more about your questions Jeyhun. This proposal is not
> > handling NULL/NOT_NULL types. I noticed that
> > the current CTAS impl. (as Timo said) adds this constraint as part of the
> > resulting schema. And when defining
> > a primary key in the CREATE part, if the resulting schema does not have a
> > NOT NULL in the column then the CTAS
> > will fail. This is similar to the CREATE TABLE LIKE which expects the
> LIKE
> > table to have a NOT NULL column if
> > the user defines a primary key in the CREATE part.
> >
> > > In some cases, redefining the column types might be redundant,
> especially
> > > when users dont change the column type. A user just wants to change the
> > > column name from the SELECT clause. Should we also support this
> scenario,
> > > similar to postgres?
> >
> > I looked into Postgres too. Postgres matches the columns based on the
> > order defined in the create and select part.
> > If you want to rename a column in the create part, then that column
> > position must be in the same position as the query column.
> > I didn't like the Postgres approach because it does not let us add
> columns
> > that do not exist in the query schema.
> >
> > i.e. query has schema (a int, b string), now the `a` column is renamed to
> > `id` because both are in the same position 0
> > `create table s1(id int) as select a, b from t1`;
> > results in: [id int, b string]
> >
> > I think, if users want to rename then they can use a different alias in
> > the select part. They could also do explicit casting
> > for changing the data types, which now makes it redundant (as you said)
> to
> > allow redefining the query columns again. But
> > perhaps there are cases where explicit casting does not work and just
> > defining the column would? i.e. making a nullable
> > type to not null? I couldn't make `cast(c1 as int not null)` to work for
> > instance, but it may work in the create part?
> >
> > > Could you also mention the casting rules in the FLIP for this case?
> >
> > I mentioned they're the same as insert/select when doing implicit
> casting.
> > I will search for more info about the insert/select
> > and add the casting rules in the flip..
> >
> > - Sergio
> >
> >
> > On Tue, Jun 11, 2024 at 12:59 AM Timo Walther 
> wrote:
> >
> >> Hi Sergio,
> >>
> >> thanks for proposing this FLIP for finalizing the CTAS statement.
> >> Adopting the logic from MySQL for deriving and potentially overwriting
> >> parts of the schema should be easy to understand for everyone. So +1 for
> >> the FLIP in general.
> >>
> >>  > How do you handle CTAS statements with SELECT clauses that have
> >>  > (implicit or explicit) NULLABLE or NOT NULLABLE columns?
> >>
> >> @Jeyhun: I don't think there is anything special about this. The current
> >> CTAS implementation should already cover that. It takes the nullability
> >> of the column's data type as a constraint into derived schema. Keep in
> >> mind that nullability is part of the data type in Flink, not only a
> >> constraint on the schema. This decision was made due to Calcite
> internals.
> >>
> >>  > redefining the column types might be redundant, especially
> >>  > when users dont change the 

Re: [VOTE] Release flink-connector-kafka v3.2.0, release candidate #1

[Yanquan Lv]

+1 (non-binding)

- verified gpg signatures
- verified sha512 hash
- built from source code with java 8/11/17
- checked Github release tag
- checked the CI result
- checked release notes

Danny Cranmer  于2024年4月22日周一 21:56写道：

> Hi everyone,
>
> Please review and vote on release candidate #1 for flink-connector-kafka
> v3.2.0, as follows:
> [ ] +1, Approve the release
> [ ] -1, Do not approve the release (please provide specific comments)
>
> This release supports Flink 1.18 and 1.19.
>
> The complete staging area is available for your review, which includes:
> * JIRA release notes [1],
> * the official Apache source release to be deployed to dist.apache.org
> [2],
> which are signed with the key with fingerprint 125FD8DB [3],
> * all artifacts to be deployed to the Maven Central Repository [4],
> * source code tag v3.2.0-rc1 [5],
> * website pull request listing the new release [6].
> * CI build of the tag [7].
>
> The vote will be open for at least 72 hours. It is adopted by majority
> approval, with at least 3 PMC affirmative votes.
>
> Thanks,
> Danny
>
> [1]
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12354209
> [2]
>
> https://dist.apache.org/repos/dist/dev/flink/flink-connector-kafka-3.2.0-rc1
> [3] https://dist.apache.org/repos/dist/release/flink/KEYS
> [4] https://repository.apache.org/content/repositories/orgapacheflink-1723
> [5]
> https://github.com/apache/flink-connector-kafka/releases/tag/v3.2.0-rc1
> [6] https://github.com/apache/flink-web/pull/738
> [7] https://github.com/apache/flink-connector-kafka
>

Re: [DISCUSS] Add a JDBC Sink Plugin to Flink-CDC-Pipeline

[Yanquan Lv]

Thanks jerry for driving this, JDBC sink for CDC pipeline is indeed a high
demand in the community.

I have one concern:
Some databases that use jdbc, such as mysql, may be time-consuming to
perform table structure changes, but FlinkCDC will not send DataChangeEvent
during this period, which can cause significant latency in sending cdc data
changes, You may need to consider and explain how to improve this situation.

Jerry  于2024年5月15日周三 15:07写道：

> Hi all
> My name is ZhengjunZhou, an user and developer of FlinkCDC. In my recent
> projects, I realized that we could enhance the capabilities of
> Flink-CDC-Pipeline by introducing a JDBC Sink plugin, enabling FlinkCDC to
> directly output change data capture (CDC) to various JDBC-supported
> database systems.
>
> Currently, while FlinkCDC offers support for a wide range of data sources,
> there is no direct solution for sinks, especially for common relational
> databases. I believe that adding a JDBC Sink plugin will significantly
> boost its applicability in data integration scenarios.
>
> Specifically, this plugin would allow users to configure database
> connections and stream data directly to SQL databases via the standard JDBC
> interface. This could be used for data migration tasks as well as real-time
> data synchronization.
>
> To further discuss this proposal and gather feedback from the community, I
> have prepared a preliminary design draft and hope to discuss it in detail
> in the upcoming community meeting. Please consider the potential value of
> this feature and provide your insights and guidance.
>
> Thank you for your time and consideration. I look forward to your active
> feedback and further discussion.
>
> [1] https://github.com/apache/flink-connector-jdbc
>

Re: [ANNOUNCE] Apache Flink CDC 3.1.0 released

[Yanquan Lv]

Congratulations!
Thanks for all selfless contributors.


Best,
Yanquan Lv

Ahmed Hamdy  于2024年5月17日周五 18:49写道：

> Congratulations,
> Great Work!
> Best Regards
> Ahmed Hamdy
>
>
> On Fri, 17 May 2024 at 10:56, ConradJam  wrote:
>
> > Congratulations!
> >
> > Thanks for the great work.
> >
> > Hang Ruan  于2024年5月17日周五 17:48写道：
> >
> > > Congratulations!
> > >
> > > Thanks for the great work.
> > >
> > > Best,
> > > Hang
> > >
> > > Qingsheng Ren  于2024年5月17日周五 17:33写道：
> > >
> > > > The Apache Flink community is very happy to announce the release of
> > > > Apache Flink CDC 3.1.0.
> > > >
> > > > Apache Flink CDC is a distributed data integration tool for real time
> > > > data and batch data, bringing the simplicity and elegance of data
> > > > integration via YAML to describe the data movement and transformation
> > > > in a data pipeline.
> > > >
> > > > Please check out the release blog post for an overview of the
> release:
> > > >
> > > >
> > >
> >
> https://flink.apache.org/2024/05/17/apache-flink-cdc-3.1.0-release-announcement/
> > > >
> > > > The release is available for download at:
> > > > https://flink.apache.org/downloads.html
> > > >
> > > > Maven artifacts for Flink CDC can be found at:
> > > > https://search.maven.org/search?q=g:org.apache.flink%20cdc
> > > >
> > > > The full release notes are available in Jira:
> > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12354387
> > > >
> > > > We would like to thank all contributors of the Apache Flink community
> > > > who made this release possible!
> > > >
> > > > Regards,
> > > > Qingsheng Ren
> > > >
> > >
> >
> >
> > --
> > Best
> >
> > ConradJam
> >
>

Re: [VOTE] Apache Flink CDC Release 3.1.0, release candidate #3

[Yanquan Lv]

+1 (non-binding)
- Validated checksum hash
- Build the source with Maven and jdk8
- Verified web PR
- Check that the jar is built by jdk8
- Check synchronizing from mysql to paimon
- Check synchronizing from mysql to kafka

Hang Ruan  于2024年5月13日周一 13:55写道：

> +1 (non-binding)
>
> - Validated checksum hash
> - Verified signature
> - Verified that no binaries exist in the source archive
> - Build the source with Maven and jdk8
> - Verified web PR
> - Check that the jar is built by jdk8
> - Check synchronizing schemas and data from mysql to starrocks following
> the quickstart
>
> Best,
> Hang
>
> Qingsheng Ren  于2024年5月11日周六 10:10写道：
>
> > Hi everyone,
> >
> > Please review and vote on the release candidate #3 for the version 3.1.0
> of
> > Apache Flink CDC, as follows:
> > [ ] +1, Approve the release
> > [ ] -1, Do not approve the release (please provide specific comments)
> >
> > **Release Overview**
> >
> > As an overview, the release consists of the following:
> > a) Flink CDC source release to be deployed to dist.apache.org
> > b) Maven artifacts to be deployed to the Maven Central Repository
> >
> > **Staging Areas to Review**
> >
> > The staging areas containing the above mentioned artifacts are as
> follows,
> > for your review:
> > * All artifacts for a) can be found in the corresponding dev repository
> at
> > dist.apache.org [1], which are signed with the key with fingerprint
> > A1BD477F79D036D2C30CA7DBCA8AEEC2F6EB040B [2]
> > * All artifacts for b) can be found at the Apache Nexus Repository [3]
> >
> > Other links for your review:
> > * JIRA release notes [4]
> > * Source code tag "release-3.1.0-rc3" with commit hash
> > 5452f30b704942d0ede64ff3d4c8699d39c63863 [5]
> > * PR for release announcement blog post of Flink CDC 3.1.0 in flink-web
> [6]
> >
> > **Vote Duration**
> >
> > The voting time will run for at least 72 hours, adopted by majority
> > approval with at least 3 PMC affirmative votes.
> >
> > Thanks,
> > Qingsheng Ren
> >
> > [1] https://dist.apache.org/repos/dist/dev/flink/flink-cdc-3.1.0-rc3/
> > [2] https://dist.apache.org/repos/dist/release/flink/KEYS
> > [3]
> https://repository.apache.org/content/repositories/orgapacheflink-1733
> > [4]
> >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12354387
> > [5] https://github.com/apache/flink-cdc/releases/tag/release-3.1.0-rc3
> > [6] https://github.com/apache/flink-web/pull/739
> >
>

Re: [ANNOUNCE] New Apache Flink Committer - Zakelly Lan

[Yanquan Lv]

Congratulations, Zakelly!

Best,
YanQuan

Yuan Mei  于2024年4月15日周一 10:51写道：

> Hi everyone,
>
> On behalf of the PMC, I'm happy to let you know that Zakelly Lan has become
> a new Flink Committer!
>
> Zakelly has been continuously contributing to the Flink project since 2020,
> with a focus area on Checkpointing, State as well as frocksdb (the default
> on-disk state db).
>
> He leads several FLIPs to improve checkpoints and state APIs, including
> File Merging for Checkpoints and configuration/API reorganizations. He is
> also one of the main contributors to the recent efforts of "disaggregated
> state management for Flink 2.0" and drives the entire discussion in the
> mailing thread, demonstrating outstanding technical depth and breadth of
> knowledge.
>
> Beyond his technical contributions, Zakelly is passionate about helping the
> community in numerous ways. He spent quite some time setting up the Flink
> Speed Center and rebuilding the benchmark pipeline after the original one
> was out of lease. He helps build frocksdb and tests for the upcoming
> frocksdb release (bump rocksdb from 6.20.3->8.10).
>
> Please join me in congratulating Zakelly for becoming an Apache Flink
> committer!
>
> Best,
> Yuan (on behalf of the Flink PMC)
>

Re: [VOTE] FLIP-399: Flink Connector Doris

[Yanquan Lv]

+1 (non-binding), thanks for it.

wudi <676366...@qq.com.invalid> 于2024年4月9日周二 10:48写道：

> Hi devs,
>
> I would like to start a vote about FLIP-399 [1]. The FLIP is about
> contributing the Flink Doris Connector[2] to the Flink community.
> Discussion thread [3].
>
> The vote will be open for at least 72 hours unless there is an objection or
> insufficient votes.
>
>
> Thanks,
> Di.Wu
>
>
> [1]
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-399%3A+Flink+Connector+Doris
> [2] https://github.com/apache/doris-flink-connector
> [3] https://lists.apache.org/thread/p3z4wsw3ftdyfs9p2wd7bbr2gfyl3xnh
>
>

Re: [ANNOUNCE] New Apache Flink PMC Member - Jing Ge

[Yanquan Lv]

Congratulations, Jing!

Jark Wu  于2024年4月12日周五 16:03写道：

> Hi everyone,
>
> On behalf of the PMC, I'm very happy to announce that Jing Ge has
> joined the Flink PMC!
>
> Jing has been contributing to Apache Flink for a long time. He continuously
> works on SQL, connectors, Source, and Sink APIs, test, and document
> modules while contributing lots of code and insightful discussions. He is
> one of the maintainers of Flink CI infra. He is also willing to help a lot
> in the
> community work, such as being the release manager for both 1.18 and 1.19,
> verifying releases, and answering questions on the mailing list. Besides
> that,
> he is continuously helping with the expansion of the Flink community and
> has
> given several talks about Flink at many conferences, such as Flink Forward
> 2022 and 2023.
>
> Congratulations and welcome Jing!
>
> Best,
> Jark (on behalf of the Flink PMC)
>

Re: [ANNOUNCE] New Apache Flink PMC Member - Lincoln Lee

[Yanquan Lv]

Congratulations, Lincoln.

Jark Wu  于2024年4月12日周五 15:59写道：

> Hi everyone,
>
> On behalf of the PMC, I'm very happy to announce that Lincoln Lee has
> joined the Flink PMC!
>
> Lincoln has been an active member of the Apache Flink community for
> many years. He mainly works on Flink SQL component and has driven
> /pushed many FLIPs around SQL, including FLIP-282/373/415/435 in
> the recent versions. He has a great technical vision of Flink SQL and
> participated in plenty of discussions in the dev mailing list. Besides
> that,
> he is community-minded, such as being the release manager of 1.19,
> verifying releases, managing release syncs, writing the release
> announcement etc.
>
> Congratulations and welcome Lincoln!
>
> Best,
> Jark (on behalf of the Flink PMC)
>

Re: [DISCUSS] Flink Website Menu Adjustment

[Yanquan Lv]

+1 for this proposal.

gongzhongqiang  于2024年3月25日周一 15:49写道：

> Hi everyone,
>
> I'd like to start a discussion on adjusting the Flink website [1] menu to
> improve accuracy and usability.While migrating Flink CDC documentation
> to the website, I found outdated links, need to review and update menus
> for the most relevant information for our users.
>
>
> Proposal:
>
> - Remove Paimon [2] from the "Getting Started" and "Documentation" menus:
> Paimon [2] is now an independent top project of ASF. CC： jingsong lees
>
> - Sort the projects in the subdirectory by the activity of the projects.
> Here I list the number of releases for each project in the past year.
>
> Flink Kubernetes Operator : 7
> Flink CDC : 5
> Flink ML  : 2
> Flink Stateful Functions : 1
>
>
> Expected Outcome :
>
> - Menu "Getting Started"
>
> Before:
>
> With Flink
>
> With Flink Stateful Functions
>
> With Flink ML
>
> With Flink Kubernetes Operator
>
> With Paimon(incubating) (formerly Flink Table Store)
>
> With Flink CDC
>
> Training Course
>
>
> After:
>
> With Flink
> With Flink Kubernetes Operator
>
> With Flink CDC
>
> With Flink ML
>
> With Flink Stateful Functions
>
> Training Course
>
>
> - Menu "Documentation" will same with "Getting Started"
>
>
> I look forward to hearing your thoughts and suggestions on this proposal.
>
> [1] https://flink.apache.org/
> [2] https://github.com/apache/incubator-paimon
> [3] https://github.com/apache/flink-statefun
>
>
>
> Best regards,
>
> Zhongqiang Gong
>

Re: [ANNOUNCE] Apache Flink 1.19.0 released

[Yanquan Lv]

Congratulations! Thanks to all contributors.

Best,
Yanquan

Shawn Huang  于2024年3月19日周二 15:12写道：

> Congratulations!
>
> Best,
> Shawn Huang
>
>
> Xuannan Su  于2024年3月19日周二 14:40写道：
>
> > Congratulations! Thanks for all the great work!
> >
> > Best regards,
> > Xuannan
> >
> > On Tue, Mar 19, 2024 at 1:31 PM Yu Li  wrote:
> > >
> > > Congrats and thanks all for the efforts!
> > >
> > > Best Regards,
> > > Yu
> > >
> > > On Tue, 19 Mar 2024 at 11:51, gongzhongqiang <
> gongzhongqi...@apache.org>
> > wrote:
> > > >
> > > > Congrats! Thanks to everyone involved!
> > > >
> > > > Best,
> > > > Zhongqiang Gong
> > > >
> > > > Lincoln Lee  于2024年3月18日周一 16:27写道：
> > > >>
> > > >> The Apache Flink community is very happy to announce the release of
> > Apache
> > > >> Flink 1.19.0, which is the fisrt release for the Apache Flink 1.19
> > series.
> > > >>
> > > >> Apache Flink® is an open-source stream processing framework for
> > > >> distributed, high-performing, always-available, and accurate data
> > streaming
> > > >> applications.
> > > >>
> > > >> The release is available for download at:
> > > >> https://flink.apache.org/downloads.html
> > > >>
> > > >> Please check out the release blog post for an overview of the
> > improvements
> > > >> for this bugfix release:
> > > >>
> >
> https://flink.apache.org/2024/03/18/announcing-the-release-of-apache-flink-1.19/
> > > >>
> > > >> The full release notes are available in Jira:
> > > >>
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12353282
> > > >>
> > > >> We would like to thank all contributors of the Apache Flink
> community
> > who
> > > >> made this release possible!
> > > >>
> > > >>
> > > >> Best,
> > > >> Yun, Jing, Martijn and Lincoln
> >
>

Re: [DISCUSS] FLIP Suggestion: Externalize Kudu Connector from Bahir

[Yanquan Lv]

Hi Ferenc, +1 for this FLIP.

Ferenc Csaky  于2024年3月9日周六 01:49写道：

> Thank you Jeyhun, Leonard, and Hang for your comments! Let me
> address them from earliest to latest.
>
> > How do you plan the review process in this case (e.g. incremental
> over existing codebase or cumulative all at once) ?
>
> I think incremental would be less time consuming and complex for
> reviewers so I would leaning towards that direction. I would
> imagine multiple subtasks for migrating the existing code, and
> updating the deprecated interfaces, so those should be separate PRs and
> the release can be initiated when everything is merged.
>
> > (1) About the release version, could you specify kudu connector version
> instead of flink version 1.18 as external connector version is different
> with flink?
> > (2) About the connector config options, could you enumerate these
> options so that we can review they’re reasonable or not?
>
> I added these to the FLIP, copied the current configs options as is, PTAL.
>
> > (3) Metrics is also key part of connector, could you add the supported
> connector metrics to public interface as well?
>
> The current Bahir conenctor code does not include any metrics and I did
> not plan to include them into the scope of this FLIP.
>
> > I think that how to state this code originally lived in Bahir may be in
> the
> FLIP.
>
> I might miss your point, but the FLIP contains this: "Migrating the
> current code keeping the history and noting it explicitly it was forked
> from the Bahir repository [2]." Pls. share if you meant something else.
>
> Best,
> Ferenc
>
>
>
> On Friday, March 8th, 2024 at 10:42, Hang Ruan 
> wrote:
>
> >
> >
> > Hi, Ferenc.
> >
> > Thanks for the FLIP discussion. +1 for the proposal.
> > I think that how to state this code originally lived in Bahir may be in
> the
> > FLIP.
> >
> > Best,
> > Hang
> >
> > Leonard Xu xbjt...@gmail.com 于2024年3月7日周四 14:14写道：
> >
> > > Thanks Ferenc for kicking off this discussion, I left some comments
> here:
> > >
> > > (1) About the release version, could you specify kudu connector version
> > > instead of flink version 1.18 as external connector version is
> different
> > > with flink ?
> > >
> > > (2) About the connector config options, could you enumerate these
> options
> > > so that we can review they’re reasonable or not?
> > >
> > > (3) Metrics is also key part of connector, could you add the supported
> > > connector metrics to public interface as well?
> > >
> > > Best,
> > > Leonard
> > >
> > > > 2024年3月6日 下午11:23，Ferenc Csaky ferenc.cs...@pm.me.INVALID 写道：
> > > >
> > > > Hello devs,
> > > >
> > > > Opening this thread to discuss a FLIP [1] about externalizing the
> Kudu
> > > > connector, as recently
> > > > the Apache Bahir project were moved to the attic [2]. Some details
> were
> > > > discussed already
> > > > in another thread [3]. I am proposing to externalize this connector
> and
> > > > keep it maintainable,
> > > > and up to date.
> > > >
> > > > Best regards,
> > > > Ferenc
> > > >
> > > > [1]
> > > >
> https://docs.google.com/document/d/1vHF_uVe0FTYCb6PRVStovqDeqb_C_FKjt2P5xXa7uhE
> > > > [2] https://bahir.apache.org/
> > > > [3] https://lists.apache.org/thread/2nb8dxxfznkyl4hlhdm3vkomm8rk4oyq
>

Re: [DISCUSS] FLIP-399: Flink Connector Doris

[Yanquan Lv]

Thanks for driving this.
The content is very detailed, it is recommended to add a section on Test
Plan for more completeness.

Di Wu  于2024年1月25日周四 15:40写道：

> Hi all,
>
> Previously, we had some discussions about contributing Flink Doris
> Connector to the Flink community [1]. I want to further promote this work.
> I hope everyone will help participate in this FLIP discussion and provide
> more valuable opinions and suggestions.
> Thanks.
>
> [1] https://lists.apache.org/thread/lvh8g9o6qj8bt3oh60q81z0o1cv3nn8p
>
> Brs,
> di.wu
>
>
>
> On 2023/12/07 05:02:46 wudi wrote:
> >
> > Hi all,
> >
> > As discussed in the previous email [1], about contributing the Flink
> Doris Connector to the Flink community.
> >
> >
> > Apache Doris[2] is a high-performance, real-time analytical database
> based on MPP architecture, for scenarios where Flink is used for data
> analysis, processing, or real-time writing on Doris, Flink Doris Connector
> is an effective tool.
> >
> > At the same time, Contributing Flink Doris Connector to the Flink
> community will further expand the Flink Connectors ecosystem.
> >
> > So I would like to start an official discussion FLIP-399: Flink
> Connector Doris[3].
> >
> > Looking forward to comments, feedbacks and suggestions from the
> community on the proposal.
> >
> > [1] https://lists.apache.org/thread/lvh8g9o6qj8bt3oh60q81z0o1cv3nn8p
> > [2] https://doris.apache.org/docs/dev/get-starting/what-is-apache-doris/
> > [3]
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-399%3A+Flink+Connector+Doris
> >
> >
> > Brs,
> >
> > di.wu
> >
>

Re: [VOTE] Release flink-connector-parent 1.1.0 release candidate #2

[Yanquan Lv]

+1 (non-binding)
- verified release notes
- verified checksum and signature
- build with java8/11/17
- Verified no binaries in source files
- reviewed the web PR


Etienne Chauchot  于2024年2月20日周二 01:34写道：

> Hi everyone,
> Please review and vote on the release candidate #2 for the version
> 1.1.0, as follows:
> [ ] +1, Approve the release
> [ ] -1, Do not approve the release (please provide specific comments)
>
>
> The complete staging area is available for your review, which includes:
> * JIRA release notes [1],
> * the official Apache source release to be deployed to dist.apache.org
> [2], which are signed with the key with fingerprint
> D1A76BA19D6294DD0033F6843A019F0B8DD163EA [3],
> * all artifacts to be deployed to the Maven Central Repository [4],
> * source code tag v1.1.0-rc2 [5],
> * website pull request listing the new release [6].
>
> * confluence wiki: connector parent upgrade to version 1.1.0 that will
> be validated after the artifact is released (there is no PR mechanism on
> the wiki) [7]
>
>
> The vote will be open for at least 72 hours. It is adopted by majority
> approval, with at least 3 PMC affirmative votes.
>
> Thanks,
> Etienne
>
> [1]
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12353442
> [2]
>
> https://dist.apache.org/repos/dist/dev/flink/flink-connector-parent-1.1.0-rc2
> [3] https://dist.apache.org/repos/dist/release/flink/KEYS
> [4] https://repository.apache.org/content/repositories/orgapacheflink-1707
> [5]
>
> https://github.com/apache/flink-connector-shared-utils/releases/tag/v1.1.0-rc2
>
> [6] https://github.com/apache/flink-web/pull/717
>
> [7]
>
> https://cwiki.apache.org/confluence/display/FLINK/Externalized+Connector+development
>

Re: [ANNOUNCE] New Apache Flink Committer - Jiabao Sun

[Yanquan Lv]

Congratulations, Jiabao.

He Wang  于2024年2月19日周一 19:21写道：

> Congrats, Jiabao!
>
> On Mon, Feb 19, 2024 at 7:19 PM Benchao Li  wrote:
>
> > Congrats, Jiabao!
> >
> > Zhanghao Chen  于2024年2月19日周一 18:42写道：
> > >
> > > Congrats, Jiaba!
> > >
> > > Best,
> > > Zhanghao Chen
> > > 
> > > From: Qingsheng Ren 
> > > Sent: Monday, February 19, 2024 17:53
> > > To: dev ; jiabao...@apache.org <
> > jiabao...@apache.org>
> > > Subject: [ANNOUNCE] New Apache Flink Committer - Jiabao Sun
> > >
> > > Hi everyone,
> > >
> > > On behalf of the PMC, I'm happy to announce Jiabao Sun as a new Flink
> > > Committer.
> > >
> > > Jiabao began contributing in August 2022 and has contributed 60+
> commits
> > > for Flink main repo and various connectors. His most notable
> contribution
> > > is being the core author and maintainer of MongoDB connector, which is
> > > fully functional in DataStream and Table/SQL APIs. Jiabao is also the
> > > author of FLIP-377 and the main contributor of JUnit 5 migration in
> > runtime
> > > and table planner modules.
> > >
> > > Beyond his technical contributions, Jiabao is an active member of our
> > > community, participating in the mailing list and consistently
> > volunteering
> > > for release verifications and code reviews with enthusiasm.
> > >
> > > Please join me in congratulating Jiabao for becoming an Apache Flink
> > > committer!
> > >
> > > Best,
> > > Qingsheng (on behalf of the Flink PMC)
> >
> >
> >
> > --
> >
> > Best,
> > Benchao Li
> >
>

Re: [VOTE] Release flink-connector-jdbc, release candidate #3

[Yanquan Lv]

+1 (non-binding)

- Validated checksum hash
- Verified signature
- Build the source with Maven and jdk8/11/17
- Check that the jar is built by jdk8
- Verified that no binaries exist in the source archive

Sergey Nuyanzin  于2024年2月1日周四 19:50写道：

> Hi everyone,
> Please review and vote on the release candidate #3 for the version 3.1.2,
> as follows:
> [ ] +1, Approve the release
> [ ] -1, Do not approve the release (please provide specific comments)
>
> This version is compatible with Flink 1.16.x, 1.17.x and 1.18.x.
>
> The complete staging area is available for your review, which includes:
> * JIRA release notes [1],
> * the official Apache source release to be deployed to dist.apache.org
> [2],
> which are signed with the key with fingerprint 1596BBF0726835D8 [3],
> * all artifacts to be deployed to the Maven Central Repository [4],
> * source code tag v3.1.2-rc3 [5],
> * website pull request listing the new release [6].
>
> The vote will be open for at least 72 hours. It is adopted by majority
> approval, with at least 3 PMC affirmative votes.
>
> Thanks,
> Release Manager
>
> [1]
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12354088
> [2]
> https://dist.apache.org/repos/dist/dev/flink/flink-connector-jdbc-3.1.2-rc3
> [3] https://dist.apache.org/repos/dist/release/flink/KEYS
> [4]
> https://repository.apache.org/content/repositories/orgapacheflink-1706/
> [5] https://github.com/apache/flink-connector-jdbc/releases/tag/v3.1.2-rc3
> [6] https://github.com/apache/flink-web/pull/707
>

Re: [VOTE] FLIP-372: Allow TwoPhaseCommittingSink WithPreCommitTopology to alter the type of the Committable

[Yanquan Lv]

+1 (non-binding)
Merging multiple write results into one submission is a common scenario we
encounter.
But in the process of implementing the addPreCommitTopology, I feel that
the logic of regenerating CommittableSummary objects is a bit hidden.

Best,
Yanquan Lv.

Péter Váry  于2023年12月18日周一 16:35写道：

> Hi everyone,
>
> Since there were no further comments on the discussion thread [1], I would
> like to start the vote for FLIP-372 [2].
>
> The FLIP started as a small new feature, but in the discussion thread and
> in a similar parallel thread [3] we opted for a somewhat bigger change in
> the Sink V2 API.
>
> Please read the FLIP and cast your vote.
>
> The vote will remain open for at least 72 hours and only concluded if there
> are no objections and enough (i.e. at least 3) binding votes.
>
> Thanks,
> Peter
>
> [1] - https://lists.apache.org/thread/344pzbrqbbb4w0sfj67km25msp7hxlyd
> [2] -
>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-372%3A+Allow+TwoPhaseCommittingSink+WithPreCommitTopology+to+alter+the+type+of+the+Committable
> [3] - https://lists.apache.org/thread/h6nkgth838dlh5s90sd95zd6hlsxwx57
>