Re: [VOTE] Release flink-connector-hbase v3.0.1, release candidate #2

2024-01-12 Thread Hang Ruan 
+1 (non-binding)

- Validated checksum hash
- Verified signature
- Verified that no binaries exist in the source archive
- Build the source with Maven and jdk11
- Verified web PR

Best,
Hang

Martijn Visser  于2024年1月12日周五 20:30写道：

> Hi everyone,
> Please review and vote on the release candidate #2 for the
> flink-connector-hbase version
> 3.0.1, as follows:
> [ ] +1, Approve the release
> [ ] -1, Do not approve the release (please provide specific comments)
>
> This version is compatible with Flink 1.16.x, 1.17.x and 1.18.x
>
> The complete staging area is available for your review, which includes:
> * JIRA release notes [1],
> * the official Apache source release to be deployed to dist.apache.org
> [2], which are signed with the key with fingerprint
> A5F3BCE4CBE993573EC5966A65321B8382B219AF [3],
> * all artifacts to be deployed to the Maven Central Repository [4],
> * source code tag v3.0.1-rc1 [5],
> * website pull request listing the new release [6].
>
> The vote will be open for at least 72 hours. It is adopted by majority
> approval, with at least 3 PMC affirmative votes.
>
> Thanks,
> Release Manager
>
> [1] https://issues.apache.org/jira/projects/FLINK/versions/12353603
> [2]
> https://dist.apache.org/repos/dist/dev/flink/flink-connector-hbase-3.0.1-rc2
> [3] https://dist.apache.org/repos/dist/release/flink/KEYS
> [4]
> https://repository.apache.org/content/repositories/orgapacheflink-1696/
> [5]
> https://github.com/apache/flink-connector-hbase/releases/tag/v3.0.1-rc2
> [6] https://github.com/apache/flink-web/pull/708
>

Re: Re: Re: [VOTE] Accept Flink CDC into Apache Flink

2024-01-12 Thread Rodrigo Meneses 
+1 (non binding)

On Fri, Jan 12, 2024 at 5:45 PM Dong Lin  wrote:

> +1 (binding)
>
> On Sat, Jan 13, 2024 at 6:04 AM Austin Bennett  wrote:
>
> > +1 (non-binding)
> >
> > On Fri, Jan 12, 2024 at 5:44 PM Becket Qin  wrote:
> >
> > > +1 (binding)
> > >
> > > Thanks,
> > >
> > > Jiangjie (Becket) Qin
> > >
> > > On Fri, Jan 12, 2024 at 5:58 AM Zhijiang  > > .invalid>
> > > wrote:
> > >
> > > > +1 (binding)
> > > > Best,
> > > > Zhijiang
> > > > --
> > > > From:Kurt Yang 
> > > > Send Time:2024年1月12日(星期五) 15:31
> > > > To:dev
> > > > Subject:Re: Re: Re: [VOTE] Accept Flink CDC into Apache Flink
> > > > +1 (binding)
> > > > Best,
> > > > Kurt
> > > > On Fri, Jan 12, 2024 at 2:21 PM Hequn Cheng 
> wrote:
> > > > > +1 (binding)
> > > > >
> > > > > Thanks,
> > > > > Hequn
> > > > >
> > > > > On Fri, Jan 12, 2024 at 2:19 PM godfrey he 
> > > wrote:
> > > > >
> > > > > > +1 (binding)
> > > > > >
> > > > > > Thanks,
> > > > > > Godfrey
> > > > > >
> > > > > > Zhu Zhu  于2024年1月12日周五 14:10写道：
> > > > > > >
> > > > > > > +1 (binding)
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Zhu
> > > > > > >
> > > > > > > Hangxiang Yu  于2024年1月11日周四 14:26写道：
> > > > > > >
> > > > > > > > +1 (non-binding)
> > > > > > > >
> > > > > > > > On Thu, Jan 11, 2024 at 11:19 AM Xuannan Su <
> > > suxuanna...@gmail.com
> > > > >
> > > > > > wrote:
> > > > > > > >
> > > > > > > > > +1 (non-binding)
> > > > > > > > >
> > > > > > > > > Best,
> > > > > > > > > Xuannan
> > > > > > > > >
> > > > > > > > > On Thu, Jan 11, 2024 at 10:28 AM Xuyang <
> xyzhong...@163.com>
> > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > +1 (non-binding)--
> > > > > > > > > >
> > > > > > > > > > Best！
> > > > > > > > > > Xuyang
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > 在 2024-01-11 10:00:11，"Yang Wang" <
> wangyang0...@apache.org
> > >
> > > > 写道：
> > > > > > > > > > >+1 (binding)
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >Best,
> > > > > > > > > > >Yang
> > > > > > > > > > >
> > > > > > > > > > >On Thu, Jan 11, 2024 at 9:53 AM liu ron <
> > ron9@gmail.com
> > > >
> > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > >> +1 non-binding
> > > > > > > > > > >>
> > > > > > > > > > >> Best
> > > > > > > > > > >> Ron
> > > > > > > > > > >>
> > > > > > > > > > >> Matthias Pohl 
> > > > 于2024年1月10日周三
> > > > > > > > 23:05写道：
> > > > > > > > > > >>
> > > > > > > > > > >> > +1 (binding)
> > > > > > > > > > >> >
> > > > > > > > > > >> > On Wed, Jan 10, 2024 at 3:35 PM ConradJam <
> > > > > > jam.gz...@gmail.com>
> > > > > > > > > wrote:
> > > > > > > > > > >> >
> > > > > > > > > > >> > > +1 non-binding
> > > > > > > > > > >> > >
> > > > > > > > > > >> > > Dawid Wysakowicz 
> > > 于2024年1月10日周三
> > > > > > > > 21:06写道：
> > > > > > > > > > >> > >
> > > > > > > > > > >> > > > +1 (binding)
> > > > > > > > > > >> > > > Best,
> > > > > > > > > > >> > > > Dawid
> > > > > > > > > > >> > > >
> > > > > > > > > > >> > > > On Wed, 10 Jan 2024 at 11:54, Piotr Nowojski <
> > > > > > > > > pnowoj...@apache.org>
> > > > > > > > > > >> > > wrote:
> > > > > > > > > > >> > > >
> > > > > > > > > > >> > > > > +1 (binding)
> > > > > > > > > > >> > > > >
> > > > > > > > > > >> > > > > śr., 10 sty 2024 o 11:25 Martijn Visser <
> > > > > > > > > martijnvis...@apache.org>
> > > > > > > > > > >> > > > > napisał(a):
> > > > > > > > > > >> > > > >
> > > > > > > > > > >> > > > > > +1 (binding)
> > > > > > > > > > >> > > > > >
> > > > > > > > > > >> > > > > > On Wed, Jan 10, 2024 at 4:43 AM Xingbo
> Huang <
> > > > > > > > > hxbks...@gmail.com
> > > > > > > > > > >> >
> > > > > > > > > > >> > > > wrote:
> > > > > > > > > > >> > > > > > >
> > > > > > > > > > >> > > > > > > +1 (binding)
> > > > > > > > > > >> > > > > > >
> > > > > > > > > > >> > > > > > > Best,
> > > > > > > > > > >> > > > > > > Xingbo
> > > > > > > > > > >> > > > > > >
> > > > > > > > > > >> > > > > > > Dian Fu 
> > 于2024年1月10日周三
> > > > > > 11:35写道：
> > > > > > > > > > >> > > > > > >
> > > > > > > > > > >> > > > > > > > +1 (binding)
> > > > > > > > > > >> > > > > > > >
> > > > > > > > > > >> > > > > > > > Regards,
> > > > > > > > > > >> > > > > > > > Dian
> > > > > > > > > > >> > > > > > > >
> > > > > > > > > > >> > > > > > > > On Wed, Jan 10, 2024 at 5:09 AM Sharath
> <
> > > > > > > > > > >> dsaishar...@gmail.com
> > > > > > > > > > >> > >
> > > > > > > > > > >> > > > > wrote:
> > > > > > > > > > >> > > > > > > > >
> > > > > > > > > > >> > > > > > > > > +1 (non-binding)
> > > > > > > > > > >> > > > > > > > >
> > > > > > > > > > >> > > > > > > > > Best,
> > > > > > > > > > >> > > > > > > > > Sharath
> > > > > > > > > > >> > > > > > > > >
> > > > > > > > > > >> > > > > > > > > On Tue, Jan 9, 2024 at 1:02 PM Venkata
> > > > Sanath
> > > > > > > > > Muppalla <
> > > > > > > > > > >> > > > > > > > sanath...@gmail.com>
> > > > > > > > > > >> > > > > > > > >

Re: Re: Re: [VOTE] Accept Flink CDC into Apache Flink

2024-01-12 Thread Dong Lin 
+1 (binding)

On Sat, Jan 13, 2024 at 6:04 AM Austin Bennett  wrote:

> +1 (non-binding)
>
> On Fri, Jan 12, 2024 at 5:44 PM Becket Qin  wrote:
>
> > +1 (binding)
> >
> > Thanks,
> >
> > Jiangjie (Becket) Qin
> >
> > On Fri, Jan 12, 2024 at 5:58 AM Zhijiang  > .invalid>
> > wrote:
> >
> > > +1 (binding)
> > > Best,
> > > Zhijiang
> > > --
> > > From:Kurt Yang 
> > > Send Time:2024年1月12日(星期五) 15:31
> > > To:dev
> > > Subject:Re: Re: Re: [VOTE] Accept Flink CDC into Apache Flink
> > > +1 (binding)
> > > Best,
> > > Kurt
> > > On Fri, Jan 12, 2024 at 2:21 PM Hequn Cheng  wrote:
> > > > +1 (binding)
> > > >
> > > > Thanks,
> > > > Hequn
> > > >
> > > > On Fri, Jan 12, 2024 at 2:19 PM godfrey he 
> > wrote:
> > > >
> > > > > +1 (binding)
> > > > >
> > > > > Thanks,
> > > > > Godfrey
> > > > >
> > > > > Zhu Zhu  于2024年1月12日周五 14:10写道：
> > > > > >
> > > > > > +1 (binding)
> > > > > >
> > > > > > Thanks,
> > > > > > Zhu
> > > > > >
> > > > > > Hangxiang Yu  于2024年1月11日周四 14:26写道：
> > > > > >
> > > > > > > +1 (non-binding)
> > > > > > >
> > > > > > > On Thu, Jan 11, 2024 at 11:19 AM Xuannan Su <
> > suxuanna...@gmail.com
> > > >
> > > > > wrote:
> > > > > > >
> > > > > > > > +1 (non-binding)
> > > > > > > >
> > > > > > > > Best,
> > > > > > > > Xuannan
> > > > > > > >
> > > > > > > > On Thu, Jan 11, 2024 at 10:28 AM Xuyang 
> > > > wrote:
> > > > > > > > >
> > > > > > > > > +1 (non-binding)--
> > > > > > > > >
> > > > > > > > > Best！
> > > > > > > > > Xuyang
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > 在 2024-01-11 10:00:11，"Yang Wang"  >
> > > 写道：
> > > > > > > > > >+1 (binding)
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >Best,
> > > > > > > > > >Yang
> > > > > > > > > >
> > > > > > > > > >On Thu, Jan 11, 2024 at 9:53 AM liu ron <
> ron9@gmail.com
> > >
> > > > > wrote:
> > > > > > > > > >
> > > > > > > > > >> +1 non-binding
> > > > > > > > > >>
> > > > > > > > > >> Best
> > > > > > > > > >> Ron
> > > > > > > > > >>
> > > > > > > > > >> Matthias Pohl 
> > > 于2024年1月10日周三
> > > > > > > 23:05写道：
> > > > > > > > > >>
> > > > > > > > > >> > +1 (binding)
> > > > > > > > > >> >
> > > > > > > > > >> > On Wed, Jan 10, 2024 at 3:35 PM ConradJam <
> > > > > jam.gz...@gmail.com>
> > > > > > > > wrote:
> > > > > > > > > >> >
> > > > > > > > > >> > > +1 non-binding
> > > > > > > > > >> > >
> > > > > > > > > >> > > Dawid Wysakowicz 
> > 于2024年1月10日周三
> > > > > > > 21:06写道：
> > > > > > > > > >> > >
> > > > > > > > > >> > > > +1 (binding)
> > > > > > > > > >> > > > Best,
> > > > > > > > > >> > > > Dawid
> > > > > > > > > >> > > >
> > > > > > > > > >> > > > On Wed, 10 Jan 2024 at 11:54, Piotr Nowojski <
> > > > > > > > pnowoj...@apache.org>
> > > > > > > > > >> > > wrote:
> > > > > > > > > >> > > >
> > > > > > > > > >> > > > > +1 (binding)
> > > > > > > > > >> > > > >
> > > > > > > > > >> > > > > śr., 10 sty 2024 o 11:25 Martijn Visser <
> > > > > > > > martijnvis...@apache.org>
> > > > > > > > > >> > > > > napisał(a):
> > > > > > > > > >> > > > >
> > > > > > > > > >> > > > > > +1 (binding)
> > > > > > > > > >> > > > > >
> > > > > > > > > >> > > > > > On Wed, Jan 10, 2024 at 4:43 AM Xingbo Huang <
> > > > > > > > hxbks...@gmail.com
> > > > > > > > > >> >
> > > > > > > > > >> > > > wrote:
> > > > > > > > > >> > > > > > >
> > > > > > > > > >> > > > > > > +1 (binding)
> > > > > > > > > >> > > > > > >
> > > > > > > > > >> > > > > > > Best,
> > > > > > > > > >> > > > > > > Xingbo
> > > > > > > > > >> > > > > > >
> > > > > > > > > >> > > > > > > Dian Fu 
> 于2024年1月10日周三
> > > > > 11:35写道：
> > > > > > > > > >> > > > > > >
> > > > > > > > > >> > > > > > > > +1 (binding)
> > > > > > > > > >> > > > > > > >
> > > > > > > > > >> > > > > > > > Regards,
> > > > > > > > > >> > > > > > > > Dian
> > > > > > > > > >> > > > > > > >
> > > > > > > > > >> > > > > > > > On Wed, Jan 10, 2024 at 5:09 AM Sharath <
> > > > > > > > > >> dsaishar...@gmail.com
> > > > > > > > > >> > >
> > > > > > > > > >> > > > > wrote:
> > > > > > > > > >> > > > > > > > >
> > > > > > > > > >> > > > > > > > > +1 (non-binding)
> > > > > > > > > >> > > > > > > > >
> > > > > > > > > >> > > > > > > > > Best,
> > > > > > > > > >> > > > > > > > > Sharath
> > > > > > > > > >> > > > > > > > >
> > > > > > > > > >> > > > > > > > > On Tue, Jan 9, 2024 at 1:02 PM Venkata
> > > Sanath
> > > > > > > > Muppalla <
> > > > > > > > > >> > > > > > > > sanath...@gmail.com>
> > > > > > > > > >> > > > > > > > > wrote:
> > > > > > > > > >> > > > > > > > >
> > > > > > > > > >> > > > > > > > > > +1 (non-binding)
> > > > > > > > > >> > > > > > > > > >
> > > > > > > > > >> > > > > > > > > > Thanks,
> > > > > > > > > >> > > > > > > > > > Sanath
> > > > > > > > > >> > > > > > > > > >
> > > > > > > > > >> > > > > > > > > > On Tue, Jan 9, 2024 at 11:16 AM Peter
> > > Huang
> > > > <
> > > > > > > > > >> > > > > > > > huangzhenqiu0...@gmail.com>
> > > > > > > > >

Re: Re: Re: [VOTE] Accept Flink CDC into Apache Flink

2024-01-12 Thread Austin Bennett 
+1 (non-binding)

On Fri, Jan 12, 2024 at 5:44 PM Becket Qin  wrote:

> +1 (binding)
>
> Thanks,
>
> Jiangjie (Becket) Qin
>
> On Fri, Jan 12, 2024 at 5:58 AM Zhijiang  .invalid>
> wrote:
>
> > +1 (binding)
> > Best,
> > Zhijiang
> > --
> > From:Kurt Yang 
> > Send Time:2024年1月12日(星期五) 15:31
> > To:dev
> > Subject:Re: Re: Re: [VOTE] Accept Flink CDC into Apache Flink
> > +1 (binding)
> > Best,
> > Kurt
> > On Fri, Jan 12, 2024 at 2:21 PM Hequn Cheng  wrote:
> > > +1 (binding)
> > >
> > > Thanks,
> > > Hequn
> > >
> > > On Fri, Jan 12, 2024 at 2:19 PM godfrey he 
> wrote:
> > >
> > > > +1 (binding)
> > > >
> > > > Thanks,
> > > > Godfrey
> > > >
> > > > Zhu Zhu  于2024年1月12日周五 14:10写道：
> > > > >
> > > > > +1 (binding)
> > > > >
> > > > > Thanks,
> > > > > Zhu
> > > > >
> > > > > Hangxiang Yu  于2024年1月11日周四 14:26写道：
> > > > >
> > > > > > +1 (non-binding)
> > > > > >
> > > > > > On Thu, Jan 11, 2024 at 11:19 AM Xuannan Su <
> suxuanna...@gmail.com
> > >
> > > > wrote:
> > > > > >
> > > > > > > +1 (non-binding)
> > > > > > >
> > > > > > > Best,
> > > > > > > Xuannan
> > > > > > >
> > > > > > > On Thu, Jan 11, 2024 at 10:28 AM Xuyang 
> > > wrote:
> > > > > > > >
> > > > > > > > +1 (non-binding)--
> > > > > > > >
> > > > > > > > Best！
> > > > > > > > Xuyang
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > 在 2024-01-11 10:00:11，"Yang Wang" 
> > 写道：
> > > > > > > > >+1 (binding)
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >Best,
> > > > > > > > >Yang
> > > > > > > > >
> > > > > > > > >On Thu, Jan 11, 2024 at 9:53 AM liu ron  >
> > > > wrote:
> > > > > > > > >
> > > > > > > > >> +1 non-binding
> > > > > > > > >>
> > > > > > > > >> Best
> > > > > > > > >> Ron
> > > > > > > > >>
> > > > > > > > >> Matthias Pohl 
> > 于2024年1月10日周三
> > > > > > 23:05写道：
> > > > > > > > >>
> > > > > > > > >> > +1 (binding)
> > > > > > > > >> >
> > > > > > > > >> > On Wed, Jan 10, 2024 at 3:35 PM ConradJam <
> > > > jam.gz...@gmail.com>
> > > > > > > wrote:
> > > > > > > > >> >
> > > > > > > > >> > > +1 non-binding
> > > > > > > > >> > >
> > > > > > > > >> > > Dawid Wysakowicz 
> 于2024年1月10日周三
> > > > > > 21:06写道：
> > > > > > > > >> > >
> > > > > > > > >> > > > +1 (binding)
> > > > > > > > >> > > > Best,
> > > > > > > > >> > > > Dawid
> > > > > > > > >> > > >
> > > > > > > > >> > > > On Wed, 10 Jan 2024 at 11:54, Piotr Nowojski <
> > > > > > > pnowoj...@apache.org>
> > > > > > > > >> > > wrote:
> > > > > > > > >> > > >
> > > > > > > > >> > > > > +1 (binding)
> > > > > > > > >> > > > >
> > > > > > > > >> > > > > śr., 10 sty 2024 o 11:25 Martijn Visser <
> > > > > > > martijnvis...@apache.org>
> > > > > > > > >> > > > > napisał(a):
> > > > > > > > >> > > > >
> > > > > > > > >> > > > > > +1 (binding)
> > > > > > > > >> > > > > >
> > > > > > > > >> > > > > > On Wed, Jan 10, 2024 at 4:43 AM Xingbo Huang <
> > > > > > > hxbks...@gmail.com
> > > > > > > > >> >
> > > > > > > > >> > > > wrote:
> > > > > > > > >> > > > > > >
> > > > > > > > >> > > > > > > +1 (binding)
> > > > > > > > >> > > > > > >
> > > > > > > > >> > > > > > > Best,
> > > > > > > > >> > > > > > > Xingbo
> > > > > > > > >> > > > > > >
> > > > > > > > >> > > > > > > Dian Fu  于2024年1月10日周三
> > > > 11:35写道：
> > > > > > > > >> > > > > > >
> > > > > > > > >> > > > > > > > +1 (binding)
> > > > > > > > >> > > > > > > >
> > > > > > > > >> > > > > > > > Regards,
> > > > > > > > >> > > > > > > > Dian
> > > > > > > > >> > > > > > > >
> > > > > > > > >> > > > > > > > On Wed, Jan 10, 2024 at 5:09 AM Sharath <
> > > > > > > > >> dsaishar...@gmail.com
> > > > > > > > >> > >
> > > > > > > > >> > > > > wrote:
> > > > > > > > >> > > > > > > > >
> > > > > > > > >> > > > > > > > > +1 (non-binding)
> > > > > > > > >> > > > > > > > >
> > > > > > > > >> > > > > > > > > Best,
> > > > > > > > >> > > > > > > > > Sharath
> > > > > > > > >> > > > > > > > >
> > > > > > > > >> > > > > > > > > On Tue, Jan 9, 2024 at 1:02 PM Venkata
> > Sanath
> > > > > > > Muppalla <
> > > > > > > > >> > > > > > > > sanath...@gmail.com>
> > > > > > > > >> > > > > > > > > wrote:
> > > > > > > > >> > > > > > > > >
> > > > > > > > >> > > > > > > > > > +1 (non-binding)
> > > > > > > > >> > > > > > > > > >
> > > > > > > > >> > > > > > > > > > Thanks,
> > > > > > > > >> > > > > > > > > > Sanath
> > > > > > > > >> > > > > > > > > >
> > > > > > > > >> > > > > > > > > > On Tue, Jan 9, 2024 at 11:16 AM Peter
> > Huang
> > > <
> > > > > > > > >> > > > > > > > huangzhenqiu0...@gmail.com>
> > > > > > > > >> > > > > > > > > > wrote:
> > > > > > > > >> > > > > > > > > >
> > > > > > > > >> > > > > > > > > > > +1 (non-binding)
> > > > > > > > >> > > > > > > > > > >
> > > > > > > > >> > > > > > > > > > >
> > > > > > > > >> > > > > > > > > > > Best Regards
> > > > > > > > >> > > > > > > > > > > Peter Huang
> > > > > > > > >> > > > > > > > > > >
> > > > > > > > >> > > > > > > > > > >
> > > > > > > > >> > > > > > > > > > > On Tue,

[ANNOUNCE] Apache flink-connector-gcp-pubsub 3.0.2 released

2024-01-12 Thread Danny Cranmer 
The Apache Flink community is very happy to announce the release of Apache
flink-connector-gcp-pubsub v3.0.2 with support for Flink 1.17 and 1.18.

Apache Flink® is an open-source stream processing framework for
distributed, high-performing, always-available, and accurate data streaming
applications.

The release is available for download at:
https://flink.apache.org/downloads.html

The full release notes are available in Jira:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12353144

We would like to thank all contributors of the Apache Flink community who
made this release possible!

Best Regards
Danny Cranmer

[RESULT][VOTE] flink-connector-gcp-pubsub v3.0.2, release candidate #2

2024-01-12 Thread Danny Cranmer 
I'm happy to announce that we have unanimously approved this release.

There are 4 approving votes, 3 of which are binding:
* Leonard Xu (binding)
* Martijn Visser (binding)
* Rui Fan
* Danny Cranmer (binding)

There are no disapproving votes.

Thanks everyone!
Danny

Re: [VOTE] Release flink-connector-gcp-pubsub v3.0.2, release candidate #2

2024-01-12 Thread Danny Cranmer 
Thanks all,

This vote is now closed, I will announce the results in a separate thread.

Danny.

On Fri, Jan 12, 2024 at 4:53 PM Danny Cranmer 
wrote:

> +1 (binding)
>
> - Verified signatures and checksums
> - Reviewed release notes
> - Verified contents of Maven repo
> - Verified no binaries in the source archive
> - Source builds using Maven
> - Reviewed NOTICE files (I suppose the copyright needs to be 2024 now!)
> - Verified the tag exists in Github
>
> Thanks,
> Danny
>
> On Sat, Nov 25, 2023 at 5:50 AM Rui Fan <1996fan...@gmail.com> wrote:
>
>> +1 (non-binding)
>>
>> - Validated checksum hash
>> - Verified signature
>> - Verified that no binaries exist in the source archive
>> - Build the source with Maven and jdk8
>> - Verified licenses
>> - Verified web PRs, left a minor comment
>>
>> Best,
>> Rui
>>
>> On Fri, Nov 24, 2023 at 10:26 PM Martijn Visser > >
>> wrote:
>>
>> > +1 (binding)
>> >
>> > - Validated hashes
>> > - Verified signature
>> > - Verified that no binaries exist in the source archive
>> > - Build the source with Maven
>> > - Verified licenses
>> > - Verified web PRs
>> >
>> > On Fri, Nov 24, 2023 at 12:38 PM Leonard Xu  wrote:
>> > >
>> > > Thanks Danny for driving this.
>> > >
>> > >
>> > > +1 (binding)
>> > >
>> > > - built from source code succeeded
>> > > - verified signatures
>> > > - verified hashsums
>> > > - checked the contents contains jar and pom files in apache repo
>> > > - checked the lowest supported flink version issue has been fixed
>> > > - checked Github release tag
>> > > - checked release notes
>> > > - reviewed the web PR
>> > >
>> > > Best,
>> > > Leonard
>> > >
>> > > > 2023年11月15日 下午11:48，Danny Cranmer  写道：
>> > > >
>> > > > Hi everyone,
>> > > >
>> > > > Please review and vote on the release candidate #2 for the version
>> > 3.0.2 of
>> > > > flink-connector-gcp-pubsub, as follows:
>> > > > [ ] +1, Approve the release
>> > > > [ ] -1, Do not approve the release (please provide specific
>> comments)
>> > > >
>> > > > The complete staging area is available for your review, which
>> includes:
>> > > > * JIRA release notes [1],
>> > > > * the official Apache source release to be deployed to
>> dist.apache.org
>> > [2],
>> > > > which are signed with the key with fingerprint
>> > > > 0F79F2AFB2351BC29678544591F9C1EC125FD8DB [3],
>> > > > * all artifacts to be deployed to the Maven Central Repository [4],
>> > > > * source code tag v3.0.2-rc2 [5],
>> > > > * website pull request listing the new release [6].
>> > > >
>> > > > The vote will be open for at least 72 hours. It is adopted by
>> majority
>> > > > approval, with at least 3 PMC affirmative votes.
>> > > >
>> > > > Thanks,
>> > > > Danny
>> > > >
>> > > > [1]
>> > > >
>> >
>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12353144
>> > > > [2]
>> > > >
>> >
>> https://dist.apache.org/repos/dist/dev/flink/flink-connector-gcp-pubsub-3.0.2-rc2
>> > > > [3] https://dist.apache.org/repos/dist/release/flink/KEYS
>> > > > [4]
>> > https://repository.apache.org/content/repositories/orgapacheflink-1673/
>> > > > [5]
>> > https://github.com/apache/flink-connector-gcp-pubsub/tree/v3.0.2-rc2
>> > > > [6] https://github.com/apache/flink-web/pull/695
>> > >
>> >
>>
>

Re: [VOTE] Release flink-connector-gcp-pubsub v3.0.2, release candidate #2

2024-01-12 Thread Danny Cranmer 
+1 (binding)

- Verified signatures and checksums
- Reviewed release notes
- Verified contents of Maven repo
- Verified no binaries in the source archive
- Source builds using Maven
- Reviewed NOTICE files (I suppose the copyright needs to be 2024 now!)
- Verified the tag exists in Github

Thanks,
Danny

On Sat, Nov 25, 2023 at 5:50 AM Rui Fan <1996fan...@gmail.com> wrote:

> +1 (non-binding)
>
> - Validated checksum hash
> - Verified signature
> - Verified that no binaries exist in the source archive
> - Build the source with Maven and jdk8
> - Verified licenses
> - Verified web PRs, left a minor comment
>
> Best,
> Rui
>
> On Fri, Nov 24, 2023 at 10:26 PM Martijn Visser 
> wrote:
>
> > +1 (binding)
> >
> > - Validated hashes
> > - Verified signature
> > - Verified that no binaries exist in the source archive
> > - Build the source with Maven
> > - Verified licenses
> > - Verified web PRs
> >
> > On Fri, Nov 24, 2023 at 12:38 PM Leonard Xu  wrote:
> > >
> > > Thanks Danny for driving this.
> > >
> > >
> > > +1 (binding)
> > >
> > > - built from source code succeeded
> > > - verified signatures
> > > - verified hashsums
> > > - checked the contents contains jar and pom files in apache repo
> > > - checked the lowest supported flink version issue has been fixed
> > > - checked Github release tag
> > > - checked release notes
> > > - reviewed the web PR
> > >
> > > Best,
> > > Leonard
> > >
> > > > 2023年11月15日 下午11:48，Danny Cranmer  写道：
> > > >
> > > > Hi everyone,
> > > >
> > > > Please review and vote on the release candidate #2 for the version
> > 3.0.2 of
> > > > flink-connector-gcp-pubsub, as follows:
> > > > [ ] +1, Approve the release
> > > > [ ] -1, Do not approve the release (please provide specific comments)
> > > >
> > > > The complete staging area is available for your review, which
> includes:
> > > > * JIRA release notes [1],
> > > > * the official Apache source release to be deployed to
> dist.apache.org
> > [2],
> > > > which are signed with the key with fingerprint
> > > > 0F79F2AFB2351BC29678544591F9C1EC125FD8DB [3],
> > > > * all artifacts to be deployed to the Maven Central Repository [4],
> > > > * source code tag v3.0.2-rc2 [5],
> > > > * website pull request listing the new release [6].
> > > >
> > > > The vote will be open for at least 72 hours. It is adopted by
> majority
> > > > approval, with at least 3 PMC affirmative votes.
> > > >
> > > > Thanks,
> > > > Danny
> > > >
> > > > [1]
> > > >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12353144
> > > > [2]
> > > >
> >
> https://dist.apache.org/repos/dist/dev/flink/flink-connector-gcp-pubsub-3.0.2-rc2
> > > > [3] https://dist.apache.org/repos/dist/release/flink/KEYS
> > > > [4]
> > https://repository.apache.org/content/repositories/orgapacheflink-1673/
> > > > [5]
> > https://github.com/apache/flink-connector-gcp-pubsub/tree/v3.0.2-rc2
> > > > [6] https://github.com/apache/flink-web/pull/695
> > >
> >
>

Re: Re: Re: [VOTE] Accept Flink CDC into Apache Flink

2024-01-12 Thread Becket Qin 
+1 (binding)

Thanks,

Jiangjie (Becket) Qin

On Fri, Jan 12, 2024 at 5:58 AM Zhijiang 
wrote:

> +1 (binding)
> Best,
> Zhijiang
> --
> From:Kurt Yang 
> Send Time:2024年1月12日(星期五) 15:31
> To:dev
> Subject:Re: Re: Re: [VOTE] Accept Flink CDC into Apache Flink
> +1 (binding)
> Best,
> Kurt
> On Fri, Jan 12, 2024 at 2:21 PM Hequn Cheng  wrote:
> > +1 (binding)
> >
> > Thanks,
> > Hequn
> >
> > On Fri, Jan 12, 2024 at 2:19 PM godfrey he  wrote:
> >
> > > +1 (binding)
> > >
> > > Thanks,
> > > Godfrey
> > >
> > > Zhu Zhu  于2024年1月12日周五 14:10写道：
> > > >
> > > > +1 (binding)
> > > >
> > > > Thanks,
> > > > Zhu
> > > >
> > > > Hangxiang Yu  于2024年1月11日周四 14:26写道：
> > > >
> > > > > +1 (non-binding)
> > > > >
> > > > > On Thu, Jan 11, 2024 at 11:19 AM Xuannan Su  >
> > > wrote:
> > > > >
> > > > > > +1 (non-binding)
> > > > > >
> > > > > > Best,
> > > > > > Xuannan
> > > > > >
> > > > > > On Thu, Jan 11, 2024 at 10:28 AM Xuyang 
> > wrote:
> > > > > > >
> > > > > > > +1 (non-binding)--
> > > > > > >
> > > > > > > Best！
> > > > > > > Xuyang
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > 在 2024-01-11 10:00:11，"Yang Wang" 
> 写道：
> > > > > > > >+1 (binding)
> > > > > > > >
> > > > > > > >
> > > > > > > >Best,
> > > > > > > >Yang
> > > > > > > >
> > > > > > > >On Thu, Jan 11, 2024 at 9:53 AM liu ron 
> > > wrote:
> > > > > > > >
> > > > > > > >> +1 non-binding
> > > > > > > >>
> > > > > > > >> Best
> > > > > > > >> Ron
> > > > > > > >>
> > > > > > > >> Matthias Pohl 
> 于2024年1月10日周三
> > > > > 23:05写道：
> > > > > > > >>
> > > > > > > >> > +1 (binding)
> > > > > > > >> >
> > > > > > > >> > On Wed, Jan 10, 2024 at 3:35 PM ConradJam <
> > > jam.gz...@gmail.com>
> > > > > > wrote:
> > > > > > > >> >
> > > > > > > >> > > +1 non-binding
> > > > > > > >> > >
> > > > > > > >> > > Dawid Wysakowicz  于2024年1月10日周三
> > > > > 21:06写道：
> > > > > > > >> > >
> > > > > > > >> > > > +1 (binding)
> > > > > > > >> > > > Best,
> > > > > > > >> > > > Dawid
> > > > > > > >> > > >
> > > > > > > >> > > > On Wed, 10 Jan 2024 at 11:54, Piotr Nowojski <
> > > > > > pnowoj...@apache.org>
> > > > > > > >> > > wrote:
> > > > > > > >> > > >
> > > > > > > >> > > > > +1 (binding)
> > > > > > > >> > > > >
> > > > > > > >> > > > > śr., 10 sty 2024 o 11:25 Martijn Visser <
> > > > > > martijnvis...@apache.org>
> > > > > > > >> > > > > napisał(a):
> > > > > > > >> > > > >
> > > > > > > >> > > > > > +1 (binding)
> > > > > > > >> > > > > >
> > > > > > > >> > > > > > On Wed, Jan 10, 2024 at 4:43 AM Xingbo Huang <
> > > > > > hxbks...@gmail.com
> > > > > > > >> >
> > > > > > > >> > > > wrote:
> > > > > > > >> > > > > > >
> > > > > > > >> > > > > > > +1 (binding)
> > > > > > > >> > > > > > >
> > > > > > > >> > > > > > > Best,
> > > > > > > >> > > > > > > Xingbo
> > > > > > > >> > > > > > >
> > > > > > > >> > > > > > > Dian Fu  于2024年1月10日周三
> > > 11:35写道：
> > > > > > > >> > > > > > >
> > > > > > > >> > > > > > > > +1 (binding)
> > > > > > > >> > > > > > > >
> > > > > > > >> > > > > > > > Regards,
> > > > > > > >> > > > > > > > Dian
> > > > > > > >> > > > > > > >
> > > > > > > >> > > > > > > > On Wed, Jan 10, 2024 at 5:09 AM Sharath <
> > > > > > > >> dsaishar...@gmail.com
> > > > > > > >> > >
> > > > > > > >> > > > > wrote:
> > > > > > > >> > > > > > > > >
> > > > > > > >> > > > > > > > > +1 (non-binding)
> > > > > > > >> > > > > > > > >
> > > > > > > >> > > > > > > > > Best,
> > > > > > > >> > > > > > > > > Sharath
> > > > > > > >> > > > > > > > >
> > > > > > > >> > > > > > > > > On Tue, Jan 9, 2024 at 1:02 PM Venkata
> Sanath
> > > > > > Muppalla <
> > > > > > > >> > > > > > > > sanath...@gmail.com>
> > > > > > > >> > > > > > > > > wrote:
> > > > > > > >> > > > > > > > >
> > > > > > > >> > > > > > > > > > +1 (non-binding)
> > > > > > > >> > > > > > > > > >
> > > > > > > >> > > > > > > > > > Thanks,
> > > > > > > >> > > > > > > > > > Sanath
> > > > > > > >> > > > > > > > > >
> > > > > > > >> > > > > > > > > > On Tue, Jan 9, 2024 at 11:16 AM Peter
> Huang
> > <
> > > > > > > >> > > > > > > > huangzhenqiu0...@gmail.com>
> > > > > > > >> > > > > > > > > > wrote:
> > > > > > > >> > > > > > > > > >
> > > > > > > >> > > > > > > > > > > +1 (non-binding)
> > > > > > > >> > > > > > > > > > >
> > > > > > > >> > > > > > > > > > >
> > > > > > > >> > > > > > > > > > > Best Regards
> > > > > > > >> > > > > > > > > > > Peter Huang
> > > > > > > >> > > > > > > > > > >
> > > > > > > >> > > > > > > > > > >
> > > > > > > >> > > > > > > > > > > On Tue, Jan 9, 2024 at 5:26 AM Jane
> Chan <
> > > > > > > >> > > > > qingyue@gmail.com>
> > > > > > > >> > > > > > > > wrote:
> > > > > > > >> > > > > > > > > > >
> > > > > > > >> > > > > > > > > > > > +1 (non-binding)
> > > > > > > >> > > > > > > > > > > >
> > > > > > > >> > > > > > > > > > > > Best,
> > > > > > > >> > > > > > > > > > > > Jane
> > > > > > > >> > > > > > > > > > > >
> > > > > > > >> > > > > > > > > > > > On Tue, Jan 9,

Re: [VOTE] Release flink-connector-opensearch v1.1.0, release candidate #1

2024-01-12 Thread Danny Cranmer 
Apologies I jumped the gun on this one. We only have 2 binding votes.
Reopening the thread.

On Fri, Jan 12, 2024 at 3:43 PM Danny Cranmer 
wrote:

> Thanks all, this vote is now closed, I will announce the results on a
> separate thread.
>
> Thanks,
> Danny
>
> On Fri, Jan 12, 2024 at 3:43 PM Danny Cranmer 
> wrote:
>
>> +1 (binding)
>>
>> - Verified signatures and checksums
>> - Reviewed release notes
>> - Verified no binaries in the source archive
>> - Source builds using Maven
>> - Reviewed NOTICE files (I suppose the copyright needs to be 2024 now!)
>>
>> Thanks,
>> Danny
>>
>> On Fri, Jan 12, 2024 at 12:56 PM Martijn Visser 
>> wrote:
>>
>>> One non blocking nit: the version for flink.version in the main POM is
>>> set to 1.17.1. I think this should be 1.17.0 (since that's the lowest
>>> possible Flink version that's supported).
>>>
>>> +1 (binding)
>>>
>>> - Validated hashes
>>> - Verified signature
>>> - Verified that no binaries exist in the source archive
>>> - Build the source with Maven
>>> - Verified licenses
>>> - Verified web PRs
>>>
>>> On Mon, Jan 1, 2024 at 11:57 AM Danny Cranmer 
>>> wrote:
>>> >
>>> > Hey,
>>> >
>>> > Gordon, apologies for the delay. Yes this is the correct
>>> understanding, all
>>> > connectors follow a similar pattern.
>>> >
>>> > Would appreciate some PMC eyes on this release.
>>> >
>>> > Thanks,
>>> > Danny
>>> >
>>> > On Thu, 23 Nov 2023, 23:28 Tzu-Li (Gordon) Tai, 
>>> wrote:
>>> >
>>> > > Hi Danny,
>>> > >
>>> > > Thanks for starting a RC for this.
>>> > >
>>> > > From the looks of the staged POMs for 1.1.0-1.18, the flink versions
>>> for
>>> > > Flink dependencies still point to 1.17.1.
>>> > >
>>> > > My understanding is that this is fine, as those provided scope
>>> > > dependencies (e.g. flink-streaming-java) will have their versions
>>> > > overwritten by the user POM if they do intend to compile their jobs
>>> against
>>> > > Flink 1.18.x.
>>> > > Can you clarify if this is the correct understanding of how we
>>> intend the
>>> > > externalized connector artifacts to be published? Related discussion
>>> on
>>> > > [1].
>>> > >
>>> > > Thanks,
>>> > > Gordon
>>> > >
>>> > > [1] https://lists.apache.org/thread/x1pyrrrq7o1wv1lcdovhzpo4qhd4tvb4
>>> > >
>>> > > On Thu, Nov 23, 2023 at 3:14 PM Sergey Nuyanzin >> >
>>> > > wrote:
>>> > >
>>> > > > +1 (non-binding)
>>> > > >
>>> > > > - downloaded artifacts
>>> > > > - built from source
>>> > > > - verified checksums and signatures
>>> > > > - reviewed web pr
>>> > > >
>>> > > >
>>> > > > On Mon, Nov 6, 2023 at 5:31 PM Ryan Skraba
>>> >> > > >
>>> > > > wrote:
>>> > > >
>>> > > > > Hello! +1 (non-binding) Thanks for the release!
>>> > > > >
>>> > > > > I've validated the source for the RC1:
>>> > > > > * flink-connector-opensearch-1.1.0-src.tgz at r64995
>>> > > > > * The sha512 checksum is OK.
>>> > > > > * The source file is signed correctly.
>>> > > > > * The signature 0F79F2AFB2351BC29678544591F9C1EC125FD8DB is
>>> found in
>>> > > the
>>> > > > > KEYS file, and on https://keyserver.ubuntu.com/
>>> > > > > * The source file is consistent with the GitHub tag v1.1.0-rc1,
>>> which
>>> > > > > corresponds to commit 0f659cc65131c9ff7c8c35eb91f5189e80414ea1
>>> > > > > - The files explicitly excluded by create_pristine_sources (such
>>> as
>>> > > > > .gitignore and the submodule tools/releasing/shared) are not
>>> present.
>>> > > > > * Has a LICENSE file and a NOTICE file
>>> > > > > * Does not contain any compiled binaries.
>>> > > > >
>>> > > > > * The sources can be compiled and unit tests pass with
>>> flink.version
>>> > > > 1.17.1
>>> > > > > and flink.version 1.18.0
>>> > > > >
>>> > > > > * Nexus has three staged artifact ids for 1.1.0-1.17 and
>>> 1.1.0-1.18
>>> > > > > - flink-connector-opensearch (.jar, -javadoc.jar, -sources.jar,
>>> > > > > -tests.jar and .pom)
>>> > > > > - flink-sql-connector-opensearch (.jar, -sources.jar and .pom)
>>> > > > > - flink-connector-gcp-pubsub-parent (only .pom)
>>> > > > >
>>> > > > > All my best, Ryan
>>> > > > >
>>> > > > > On Fri, Nov 3, 2023 at 10:29 AM Danny Cranmer <
>>> dannycran...@apache.org
>>> > > >
>>> > > > > wrote:
>>> > > > > >
>>> > > > > > Hi everyone,
>>> > > > > >
>>> > > > > > Please review and vote on the release candidate #1 for the
>>> version
>>> > > > 1.1.0
>>> > > > > of
>>> > > > > > flink-connector-opensearch, as follows:
>>> > > > > > [ ] +1, Approve the release
>>> > > > > > [ ] -1, Do not approve the release (please provide specific
>>> comments)
>>> > > > > >
>>> > > > > >
>>> > > > > > The complete staging area is available for your review, which
>>> > > includes:
>>> > > > > > * JIRA release notes [1],
>>> > > > > > * the official Apache source release to be deployed to
>>> > > dist.apache.org
>>> > > > > [2],
>>> > > > > > which are signed with the key with fingerprint
>>> > > > > > 0F79F2AFB2351BC29678544591F9C1EC125FD8DB [3],
>>> > > > > > * all artifacts to be deployed to the Maven Central Repository
>>> [4],
>>> > > > > > *

Re: Flink pending record metric weired after autoscaler rescaling

2024-01-12 Thread Yang LI 
Yes, https://issues.apache.org/jira/browse/FLINK-34063
does match quite well
the issues I have encountered. I'll leave a comment in that ticket then.

Thanks,
Yang

On Fri, 12 Jan 2024 at 15:39, Gyula Fóra  wrote:

> Could this be related to the issue reported here?
> https://issues.apache.org/jira/browse/FLINK-34063
>
> Gyula
>
> On Wed, Jan 10, 2024 at 4:04 PM Yang LI  wrote:
>
> > Just to give more context, my setup uses Apache Flink 1.18 with the
> > adaptive scheduler enabled, issues happen randomly particularly
> > post-restart behaviors.
> >
> > After each restart, the system logs indicate "Adding split(s) to
> reader:",
> > signifying the reassignment of partitions across different TaskManagers.
> An
> > anomaly arises with specific partitions, for example, partition-10. This
> > partition does not appear in the logs immediately post-restart. It
> remains
> > unlogged for several hours, during which no data consumption from
> > partition-10 occurs. Subsequently, the logs display "Discovered new
> > partitions:", and only then does the consumption of data from
> partition-10
> > recommence.
> >
> > Could you provide any insights or hypotheses regarding the underlying
> cause
> > of this delayed recognition and processing of certain partitions?
> >
> > Best regards,
> > Yang
> >
> > On Mon, 8 Jan 2024 at 16:24, Yang LI  wrote:
> >
> > > Dear Flink Community,
> > >
> > > I've encountered an issue during the testing of my Flink autoscaler. It
> > > appears that Flink is losing track of specific Kafka partitions,
> leading
> > to
> > > a persistent increase in lag on these partitions. The logs indicate a
> > > 'kafka connector metricGroup name collision exception.' Notably, the
> > > consumption on these Kafka partitions returns to normal after
> restarting
> > > the Kafka broker. For context, I have enabled in-place rescaling
> support
> > > with 'jobmanager.scheduler: Adaptive.'
> > >
> > > I suspect the problem may stem from:
> > >
> > > The in-place rescaling support triggering restarts of some
> taskmanagers.
> > > This process might not be restarting the metric groups registered by
> the
> > > Kafka source connector correctly, leading to a name collision exception
> > and
> > > preventing Flink from accurately reporting metrics related to Kafka
> > > consumption.
> > > A potential edge case in the metric for pending records, especially
> when
> > > different partitions exhibit varying lags. This discrepancy might be
> > > causing the pending record metric to malfunction.
> > > I would appreciate your insights on these observations.
> > >
> > > Best regards,
> > > Yang LI
> > >
> >
>

Re: [VOTE] Release flink-connector-opensearch v1.1.0, release candidate #1

2024-01-12 Thread Danny Cranmer 
+1 (binding)

- Verified signatures and checksums
- Reviewed release notes
- Verified no binaries in the source archive
- Source builds using Maven
- Reviewed NOTICE files (I suppose the copyright needs to be 2024 now!)

Thanks,
Danny

On Fri, Jan 12, 2024 at 12:56 PM Martijn Visser 
wrote:

> One non blocking nit: the version for flink.version in the main POM is
> set to 1.17.1. I think this should be 1.17.0 (since that's the lowest
> possible Flink version that's supported).
>
> +1 (binding)
>
> - Validated hashes
> - Verified signature
> - Verified that no binaries exist in the source archive
> - Build the source with Maven
> - Verified licenses
> - Verified web PRs
>
> On Mon, Jan 1, 2024 at 11:57 AM Danny Cranmer 
> wrote:
> >
> > Hey,
> >
> > Gordon, apologies for the delay. Yes this is the correct understanding,
> all
> > connectors follow a similar pattern.
> >
> > Would appreciate some PMC eyes on this release.
> >
> > Thanks,
> > Danny
> >
> > On Thu, 23 Nov 2023, 23:28 Tzu-Li (Gordon) Tai, 
> wrote:
> >
> > > Hi Danny,
> > >
> > > Thanks for starting a RC for this.
> > >
> > > From the looks of the staged POMs for 1.1.0-1.18, the flink versions
> for
> > > Flink dependencies still point to 1.17.1.
> > >
> > > My understanding is that this is fine, as those provided scope
> > > dependencies (e.g. flink-streaming-java) will have their versions
> > > overwritten by the user POM if they do intend to compile their jobs
> against
> > > Flink 1.18.x.
> > > Can you clarify if this is the correct understanding of how we intend
> the
> > > externalized connector artifacts to be published? Related discussion on
> > > [1].
> > >
> > > Thanks,
> > > Gordon
> > >
> > > [1] https://lists.apache.org/thread/x1pyrrrq7o1wv1lcdovhzpo4qhd4tvb4
> > >
> > > On Thu, Nov 23, 2023 at 3:14 PM Sergey Nuyanzin 
> > > wrote:
> > >
> > > > +1 (non-binding)
> > > >
> > > > - downloaded artifacts
> > > > - built from source
> > > > - verified checksums and signatures
> > > > - reviewed web pr
> > > >
> > > >
> > > > On Mon, Nov 6, 2023 at 5:31 PM Ryan Skraba
>  > > >
> > > > wrote:
> > > >
> > > > > Hello! +1 (non-binding) Thanks for the release!
> > > > >
> > > > > I've validated the source for the RC1:
> > > > > * flink-connector-opensearch-1.1.0-src.tgz at r64995
> > > > > * The sha512 checksum is OK.
> > > > > * The source file is signed correctly.
> > > > > * The signature 0F79F2AFB2351BC29678544591F9C1EC125FD8DB is found
> in
> > > the
> > > > > KEYS file, and on https://keyserver.ubuntu.com/
> > > > > * The source file is consistent with the GitHub tag v1.1.0-rc1,
> which
> > > > > corresponds to commit 0f659cc65131c9ff7c8c35eb91f5189e80414ea1
> > > > > - The files explicitly excluded by create_pristine_sources (such as
> > > > > .gitignore and the submodule tools/releasing/shared) are not
> present.
> > > > > * Has a LICENSE file and a NOTICE file
> > > > > * Does not contain any compiled binaries.
> > > > >
> > > > > * The sources can be compiled and unit tests pass with
> flink.version
> > > > 1.17.1
> > > > > and flink.version 1.18.0
> > > > >
> > > > > * Nexus has three staged artifact ids for 1.1.0-1.17 and 1.1.0-1.18
> > > > > - flink-connector-opensearch (.jar, -javadoc.jar, -sources.jar,
> > > > > -tests.jar and .pom)
> > > > > - flink-sql-connector-opensearch (.jar, -sources.jar and .pom)
> > > > > - flink-connector-gcp-pubsub-parent (only .pom)
> > > > >
> > > > > All my best, Ryan
> > > > >
> > > > > On Fri, Nov 3, 2023 at 10:29 AM Danny Cranmer <
> dannycran...@apache.org
> > > >
> > > > > wrote:
> > > > > >
> > > > > > Hi everyone,
> > > > > >
> > > > > > Please review and vote on the release candidate #1 for the
> version
> > > > 1.1.0
> > > > > of
> > > > > > flink-connector-opensearch, as follows:
> > > > > > [ ] +1, Approve the release
> > > > > > [ ] -1, Do not approve the release (please provide specific
> comments)
> > > > > >
> > > > > >
> > > > > > The complete staging area is available for your review, which
> > > includes:
> > > > > > * JIRA release notes [1],
> > > > > > * the official Apache source release to be deployed to
> > > dist.apache.org
> > > > > [2],
> > > > > > which are signed with the key with fingerprint
> > > > > > 0F79F2AFB2351BC29678544591F9C1EC125FD8DB [3],
> > > > > > * all artifacts to be deployed to the Maven Central Repository
> [4],
> > > > > > * source code tag v1.1.0-rc1 [5],
> > > > > > * website pull request listing the new release [6].
> > > > > >
> > > > > > The vote will be open for at least 72 hours. It is adopted by
> > > majority
> > > > > > approval, with at least 3 PMC affirmative votes.
> > > > > >
> > > > > > Thanks,
> > > > > > Danny
> > > > > >
> > > > > > [1]
> > > > > >
> > > > >
> > > >
> > >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12353141
> > > > > > [2]
> > > > > >
> > > > >
> > > >
> > >
> https://dist.apache.org/repos/dist/dev/flink/flink-connector-opensearch-1.1.0-rc1/
> > > > > > [3]

Re: [VOTE] Release flink-connector-opensearch v1.1.0, release candidate #1

2024-01-12 Thread Danny Cranmer 
Thanks all, this vote is now closed, I will announce the results on a
separate thread.

Thanks,
Danny

On Fri, Jan 12, 2024 at 3:43 PM Danny Cranmer 
wrote:

> +1 (binding)
>
> - Verified signatures and checksums
> - Reviewed release notes
> - Verified no binaries in the source archive
> - Source builds using Maven
> - Reviewed NOTICE files (I suppose the copyright needs to be 2024 now!)
>
> Thanks,
> Danny
>
> On Fri, Jan 12, 2024 at 12:56 PM Martijn Visser 
> wrote:
>
>> One non blocking nit: the version for flink.version in the main POM is
>> set to 1.17.1. I think this should be 1.17.0 (since that's the lowest
>> possible Flink version that's supported).
>>
>> +1 (binding)
>>
>> - Validated hashes
>> - Verified signature
>> - Verified that no binaries exist in the source archive
>> - Build the source with Maven
>> - Verified licenses
>> - Verified web PRs
>>
>> On Mon, Jan 1, 2024 at 11:57 AM Danny Cranmer 
>> wrote:
>> >
>> > Hey,
>> >
>> > Gordon, apologies for the delay. Yes this is the correct understanding,
>> all
>> > connectors follow a similar pattern.
>> >
>> > Would appreciate some PMC eyes on this release.
>> >
>> > Thanks,
>> > Danny
>> >
>> > On Thu, 23 Nov 2023, 23:28 Tzu-Li (Gordon) Tai, 
>> wrote:
>> >
>> > > Hi Danny,
>> > >
>> > > Thanks for starting a RC for this.
>> > >
>> > > From the looks of the staged POMs for 1.1.0-1.18, the flink versions
>> for
>> > > Flink dependencies still point to 1.17.1.
>> > >
>> > > My understanding is that this is fine, as those provided scope
>> > > dependencies (e.g. flink-streaming-java) will have their versions
>> > > overwritten by the user POM if they do intend to compile their jobs
>> against
>> > > Flink 1.18.x.
>> > > Can you clarify if this is the correct understanding of how we intend
>> the
>> > > externalized connector artifacts to be published? Related discussion
>> on
>> > > [1].
>> > >
>> > > Thanks,
>> > > Gordon
>> > >
>> > > [1] https://lists.apache.org/thread/x1pyrrrq7o1wv1lcdovhzpo4qhd4tvb4
>> > >
>> > > On Thu, Nov 23, 2023 at 3:14 PM Sergey Nuyanzin 
>> > > wrote:
>> > >
>> > > > +1 (non-binding)
>> > > >
>> > > > - downloaded artifacts
>> > > > - built from source
>> > > > - verified checksums and signatures
>> > > > - reviewed web pr
>> > > >
>> > > >
>> > > > On Mon, Nov 6, 2023 at 5:31 PM Ryan Skraba
>> > > > >
>> > > > wrote:
>> > > >
>> > > > > Hello! +1 (non-binding) Thanks for the release!
>> > > > >
>> > > > > I've validated the source for the RC1:
>> > > > > * flink-connector-opensearch-1.1.0-src.tgz at r64995
>> > > > > * The sha512 checksum is OK.
>> > > > > * The source file is signed correctly.
>> > > > > * The signature 0F79F2AFB2351BC29678544591F9C1EC125FD8DB is found
>> in
>> > > the
>> > > > > KEYS file, and on https://keyserver.ubuntu.com/
>> > > > > * The source file is consistent with the GitHub tag v1.1.0-rc1,
>> which
>> > > > > corresponds to commit 0f659cc65131c9ff7c8c35eb91f5189e80414ea1
>> > > > > - The files explicitly excluded by create_pristine_sources (such
>> as
>> > > > > .gitignore and the submodule tools/releasing/shared) are not
>> present.
>> > > > > * Has a LICENSE file and a NOTICE file
>> > > > > * Does not contain any compiled binaries.
>> > > > >
>> > > > > * The sources can be compiled and unit tests pass with
>> flink.version
>> > > > 1.17.1
>> > > > > and flink.version 1.18.0
>> > > > >
>> > > > > * Nexus has three staged artifact ids for 1.1.0-1.17 and
>> 1.1.0-1.18
>> > > > > - flink-connector-opensearch (.jar, -javadoc.jar, -sources.jar,
>> > > > > -tests.jar and .pom)
>> > > > > - flink-sql-connector-opensearch (.jar, -sources.jar and .pom)
>> > > > > - flink-connector-gcp-pubsub-parent (only .pom)
>> > > > >
>> > > > > All my best, Ryan
>> > > > >
>> > > > > On Fri, Nov 3, 2023 at 10:29 AM Danny Cranmer <
>> dannycran...@apache.org
>> > > >
>> > > > > wrote:
>> > > > > >
>> > > > > > Hi everyone,
>> > > > > >
>> > > > > > Please review and vote on the release candidate #1 for the
>> version
>> > > > 1.1.0
>> > > > > of
>> > > > > > flink-connector-opensearch, as follows:
>> > > > > > [ ] +1, Approve the release
>> > > > > > [ ] -1, Do not approve the release (please provide specific
>> comments)
>> > > > > >
>> > > > > >
>> > > > > > The complete staging area is available for your review, which
>> > > includes:
>> > > > > > * JIRA release notes [1],
>> > > > > > * the official Apache source release to be deployed to
>> > > dist.apache.org
>> > > > > [2],
>> > > > > > which are signed with the key with fingerprint
>> > > > > > 0F79F2AFB2351BC29678544591F9C1EC125FD8DB [3],
>> > > > > > * all artifacts to be deployed to the Maven Central Repository
>> [4],
>> > > > > > * source code tag v1.1.0-rc1 [5],
>> > > > > > * website pull request listing the new release [6].
>> > > > > >
>> > > > > > The vote will be open for at least 72 hours. It is adopted by
>> > > majority
>> > > > > > approval, with at least 3 PMC affirmative votes.
>> > > > > >
>> > > > > > Thanks,
>> > > > > > Danny

Re: Flink pending record metric weired after autoscaler rescaling

2024-01-12 Thread Gyula Fóra 
Could this be related to the issue reported here?
https://issues.apache.org/jira/browse/FLINK-34063

Gyula

On Wed, Jan 10, 2024 at 4:04 PM Yang LI  wrote:

> Just to give more context, my setup uses Apache Flink 1.18 with the
> adaptive scheduler enabled, issues happen randomly particularly
> post-restart behaviors.
>
> After each restart, the system logs indicate "Adding split(s) to reader:",
> signifying the reassignment of partitions across different TaskManagers. An
> anomaly arises with specific partitions, for example, partition-10. This
> partition does not appear in the logs immediately post-restart. It remains
> unlogged for several hours, during which no data consumption from
> partition-10 occurs. Subsequently, the logs display "Discovered new
> partitions:", and only then does the consumption of data from partition-10
> recommence.
>
> Could you provide any insights or hypotheses regarding the underlying cause
> of this delayed recognition and processing of certain partitions?
>
> Best regards,
> Yang
>
> On Mon, 8 Jan 2024 at 16:24, Yang LI  wrote:
>
> > Dear Flink Community,
> >
> > I've encountered an issue during the testing of my Flink autoscaler. It
> > appears that Flink is losing track of specific Kafka partitions, leading
> to
> > a persistent increase in lag on these partitions. The logs indicate a
> > 'kafka connector metricGroup name collision exception.' Notably, the
> > consumption on these Kafka partitions returns to normal after restarting
> > the Kafka broker. For context, I have enabled in-place rescaling support
> > with 'jobmanager.scheduler: Adaptive.'
> >
> > I suspect the problem may stem from:
> >
> > The in-place rescaling support triggering restarts of some taskmanagers.
> > This process might not be restarting the metric groups registered by the
> > Kafka source connector correctly, leading to a name collision exception
> and
> > preventing Flink from accurately reporting metrics related to Kafka
> > consumption.
> > A potential edge case in the metric for pending records, especially when
> > different partitions exhibit varying lags. This discrepancy might be
> > causing the pending record metric to malfunction.
> > I would appreciate your insights on these observations.
> >
> > Best regards,
> > Yang LI
> >
>

Re:Re: Re: [VOTE] Accept Flink CDC into Apache Flink

2024-01-12 Thread Zhijiang 
+1 (binding)
Best,
Zhijiang
--
From:Kurt Yang 
Send Time:2024年1月12日(星期五) 15:31
To:dev
Subject:Re: Re: Re: [VOTE] Accept Flink CDC into Apache Flink
+1 (binding)
Best,
Kurt
On Fri, Jan 12, 2024 at 2:21 PM Hequn Cheng  wrote:
> +1 (binding)
>
> Thanks,
> Hequn
>
> On Fri, Jan 12, 2024 at 2:19 PM godfrey he  wrote:
>
> > +1 (binding)
> >
> > Thanks,
> > Godfrey
> >
> > Zhu Zhu  于2024年1月12日周五 14:10写道：
> > >
> > > +1 (binding)
> > >
> > > Thanks,
> > > Zhu
> > >
> > > Hangxiang Yu  于2024年1月11日周四 14:26写道：
> > >
> > > > +1 (non-binding)
> > > >
> > > > On Thu, Jan 11, 2024 at 11:19 AM Xuannan Su 
> > wrote:
> > > >
> > > > > +1 (non-binding)
> > > > >
> > > > > Best,
> > > > > Xuannan
> > > > >
> > > > > On Thu, Jan 11, 2024 at 10:28 AM Xuyang 
> wrote:
> > > > > >
> > > > > > +1 (non-binding)--
> > > > > >
> > > > > > Best！
> > > > > > Xuyang
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > 在 2024-01-11 10:00:11，"Yang Wang"  写道：
> > > > > > >+1 (binding)
> > > > > > >
> > > > > > >
> > > > > > >Best,
> > > > > > >Yang
> > > > > > >
> > > > > > >On Thu, Jan 11, 2024 at 9:53 AM liu ron 
> > wrote:
> > > > > > >
> > > > > > >> +1 non-binding
> > > > > > >>
> > > > > > >> Best
> > > > > > >> Ron
> > > > > > >>
> > > > > > >> Matthias Pohl  于2024年1月10日周三
> > > > 23:05写道：
> > > > > > >>
> > > > > > >> > +1 (binding)
> > > > > > >> >
> > > > > > >> > On Wed, Jan 10, 2024 at 3:35 PM ConradJam <
> > jam.gz...@gmail.com>
> > > > > wrote:
> > > > > > >> >
> > > > > > >> > > +1 non-binding
> > > > > > >> > >
> > > > > > >> > > Dawid Wysakowicz  于2024年1月10日周三
> > > > 21:06写道：
> > > > > > >> > >
> > > > > > >> > > > +1 (binding)
> > > > > > >> > > > Best,
> > > > > > >> > > > Dawid
> > > > > > >> > > >
> > > > > > >> > > > On Wed, 10 Jan 2024 at 11:54, Piotr Nowojski <
> > > > > pnowoj...@apache.org>
> > > > > > >> > > wrote:
> > > > > > >> > > >
> > > > > > >> > > > > +1 (binding)
> > > > > > >> > > > >
> > > > > > >> > > > > śr., 10 sty 2024 o 11:25 Martijn Visser <
> > > > > martijnvis...@apache.org>
> > > > > > >> > > > > napisał(a):
> > > > > > >> > > > >
> > > > > > >> > > > > > +1 (binding)
> > > > > > >> > > > > >
> > > > > > >> > > > > > On Wed, Jan 10, 2024 at 4:43 AM Xingbo Huang <
> > > > > hxbks...@gmail.com
> > > > > > >> >
> > > > > > >> > > > wrote:
> > > > > > >> > > > > > >
> > > > > > >> > > > > > > +1 (binding)
> > > > > > >> > > > > > >
> > > > > > >> > > > > > > Best,
> > > > > > >> > > > > > > Xingbo
> > > > > > >> > > > > > >
> > > > > > >> > > > > > > Dian Fu  于2024年1月10日周三
> > 11:35写道：
> > > > > > >> > > > > > >
> > > > > > >> > > > > > > > +1 (binding)
> > > > > > >> > > > > > > >
> > > > > > >> > > > > > > > Regards,
> > > > > > >> > > > > > > > Dian
> > > > > > >> > > > > > > >
> > > > > > >> > > > > > > > On Wed, Jan 10, 2024 at 5:09 AM Sharath <
> > > > > > >> dsaishar...@gmail.com
> > > > > > >> > >
> > > > > > >> > > > > wrote:
> > > > > > >> > > > > > > > >
> > > > > > >> > > > > > > > > +1 (non-binding)
> > > > > > >> > > > > > > > >
> > > > > > >> > > > > > > > > Best,
> > > > > > >> > > > > > > > > Sharath
> > > > > > >> > > > > > > > >
> > > > > > >> > > > > > > > > On Tue, Jan 9, 2024 at 1:02 PM Venkata Sanath
> > > > > Muppalla <
> > > > > > >> > > > > > > > sanath...@gmail.com>
> > > > > > >> > > > > > > > > wrote:
> > > > > > >> > > > > > > > >
> > > > > > >> > > > > > > > > > +1 (non-binding)
> > > > > > >> > > > > > > > > >
> > > > > > >> > > > > > > > > > Thanks,
> > > > > > >> > > > > > > > > > Sanath
> > > > > > >> > > > > > > > > >
> > > > > > >> > > > > > > > > > On Tue, Jan 9, 2024 at 11:16 AM Peter Huang
> <
> > > > > > >> > > > > > > > huangzhenqiu0...@gmail.com>
> > > > > > >> > > > > > > > > > wrote:
> > > > > > >> > > > > > > > > >
> > > > > > >> > > > > > > > > > > +1 (non-binding)
> > > > > > >> > > > > > > > > > >
> > > > > > >> > > > > > > > > > >
> > > > > > >> > > > > > > > > > > Best Regards
> > > > > > >> > > > > > > > > > > Peter Huang
> > > > > > >> > > > > > > > > > >
> > > > > > >> > > > > > > > > > >
> > > > > > >> > > > > > > > > > > On Tue, Jan 9, 2024 at 5:26 AM Jane Chan <
> > > > > > >> > > > > qingyue@gmail.com>
> > > > > > >> > > > > > > > wrote:
> > > > > > >> > > > > > > > > > >
> > > > > > >> > > > > > > > > > > > +1 (non-binding)
> > > > > > >> > > > > > > > > > > >
> > > > > > >> > > > > > > > > > > > Best,
> > > > > > >> > > > > > > > > > > > Jane
> > > > > > >> > > > > > > > > > > >
> > > > > > >> > > > > > > > > > > > On Tue, Jan 9, 2024 at 8:41 PM Lijie
> Wang
> > <
> > > > > > >> > > > > > > > wangdachui9...@gmail.com>
> > > > > > >> > > > > > > > > > > > wrote:
> > > > > > >> > > > > > > > > > > >
> > > > > > >> > > > > > > > > > > > > +1 (non-binding)
> > > > > > >> > > > > > > > > > > > >
> > > > > > >> > > > > > > > > > > > > Best,
> > > > > > >> > > > > > > > > > > > > Lijie
> > > > > > >> > > > > > > > > > > > >
> > > > > > >> > > > > > > > > > >

Re: [VOTE] Release flink-connector-opensearch v1.1.0, release candidate #1

2024-01-12 Thread Martijn Visser 
One non blocking nit: the version for flink.version in the main POM is
set to 1.17.1. I think this should be 1.17.0 (since that's the lowest
possible Flink version that's supported).

+1 (binding)

- Validated hashes
- Verified signature
- Verified that no binaries exist in the source archive
- Build the source with Maven
- Verified licenses
- Verified web PRs

On Mon, Jan 1, 2024 at 11:57 AM Danny Cranmer  wrote:
>
> Hey,
>
> Gordon, apologies for the delay. Yes this is the correct understanding, all
> connectors follow a similar pattern.
>
> Would appreciate some PMC eyes on this release.
>
> Thanks,
> Danny
>
> On Thu, 23 Nov 2023, 23:28 Tzu-Li (Gordon) Tai,  wrote:
>
> > Hi Danny,
> >
> > Thanks for starting a RC for this.
> >
> > From the looks of the staged POMs for 1.1.0-1.18, the flink versions for
> > Flink dependencies still point to 1.17.1.
> >
> > My understanding is that this is fine, as those provided scope
> > dependencies (e.g. flink-streaming-java) will have their versions
> > overwritten by the user POM if they do intend to compile their jobs against
> > Flink 1.18.x.
> > Can you clarify if this is the correct understanding of how we intend the
> > externalized connector artifacts to be published? Related discussion on
> > [1].
> >
> > Thanks,
> > Gordon
> >
> > [1] https://lists.apache.org/thread/x1pyrrrq7o1wv1lcdovhzpo4qhd4tvb4
> >
> > On Thu, Nov 23, 2023 at 3:14 PM Sergey Nuyanzin 
> > wrote:
> >
> > > +1 (non-binding)
> > >
> > > - downloaded artifacts
> > > - built from source
> > > - verified checksums and signatures
> > > - reviewed web pr
> > >
> > >
> > > On Mon, Nov 6, 2023 at 5:31 PM Ryan Skraba  > >
> > > wrote:
> > >
> > > > Hello! +1 (non-binding) Thanks for the release!
> > > >
> > > > I've validated the source for the RC1:
> > > > * flink-connector-opensearch-1.1.0-src.tgz at r64995
> > > > * The sha512 checksum is OK.
> > > > * The source file is signed correctly.
> > > > * The signature 0F79F2AFB2351BC29678544591F9C1EC125FD8DB is found in
> > the
> > > > KEYS file, and on https://keyserver.ubuntu.com/
> > > > * The source file is consistent with the GitHub tag v1.1.0-rc1, which
> > > > corresponds to commit 0f659cc65131c9ff7c8c35eb91f5189e80414ea1
> > > > - The files explicitly excluded by create_pristine_sources (such as
> > > > .gitignore and the submodule tools/releasing/shared) are not present.
> > > > * Has a LICENSE file and a NOTICE file
> > > > * Does not contain any compiled binaries.
> > > >
> > > > * The sources can be compiled and unit tests pass with flink.version
> > > 1.17.1
> > > > and flink.version 1.18.0
> > > >
> > > > * Nexus has three staged artifact ids for 1.1.0-1.17 and 1.1.0-1.18
> > > > - flink-connector-opensearch (.jar, -javadoc.jar, -sources.jar,
> > > > -tests.jar and .pom)
> > > > - flink-sql-connector-opensearch (.jar, -sources.jar and .pom)
> > > > - flink-connector-gcp-pubsub-parent (only .pom)
> > > >
> > > > All my best, Ryan
> > > >
> > > > On Fri, Nov 3, 2023 at 10:29 AM Danny Cranmer  > >
> > > > wrote:
> > > > >
> > > > > Hi everyone,
> > > > >
> > > > > Please review and vote on the release candidate #1 for the version
> > > 1.1.0
> > > > of
> > > > > flink-connector-opensearch, as follows:
> > > > > [ ] +1, Approve the release
> > > > > [ ] -1, Do not approve the release (please provide specific comments)
> > > > >
> > > > >
> > > > > The complete staging area is available for your review, which
> > includes:
> > > > > * JIRA release notes [1],
> > > > > * the official Apache source release to be deployed to
> > dist.apache.org
> > > > [2],
> > > > > which are signed with the key with fingerprint
> > > > > 0F79F2AFB2351BC29678544591F9C1EC125FD8DB [3],
> > > > > * all artifacts to be deployed to the Maven Central Repository [4],
> > > > > * source code tag v1.1.0-rc1 [5],
> > > > > * website pull request listing the new release [6].
> > > > >
> > > > > The vote will be open for at least 72 hours. It is adopted by
> > majority
> > > > > approval, with at least 3 PMC affirmative votes.
> > > > >
> > > > > Thanks,
> > > > > Danny
> > > > >
> > > > > [1]
> > > > >
> > > >
> > >
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12353141
> > > > > [2]
> > > > >
> > > >
> > >
> > https://dist.apache.org/repos/dist/dev/flink/flink-connector-opensearch-1.1.0-rc1/
> > > > > [3] https://dist.apache.org/repos/dist/release/flink/KEYS
> > > > > [4]
> > > >
> > https://repository.apache.org/content/repositories/orgapacheflink-1666/
> > > > > [5]
> > > https://github.com/apache/flink-connector-opensearch/tree/v1.1.0-rc1
> > > > > [6] https://github.com/apache/flink-web/pull/694
> > > >
> > >
> > >
> > > --
> > > Best regards,
> > > Sergey
> > >
> >

[VOTE] Release flink-connector-rabbitmq, v3.0.2 release candidate #1

2024-01-12 Thread Martijn Visser 
Hi everyone,
Please review and vote on the release candidate #1 for the version
3.0.2, as follows:
[ ] +1, Approve the release
[ ] -1, Do not approve the release (please provide specific comments)

This version is compatible with Flink 1.16.x, 1.17.x and 1.18.x.

The complete staging area is available for your review, which includes:
* JIRA release notes [1],
* the official Apache source release to be deployed to dist.apache.org
[2], which are signed with the key with fingerprint
A5F3BCE4CBE993573EC5966A65321B8382B219AF [3],
* all artifacts to be deployed to the Maven Central Repository [4],
* source code tag v3.0.2-rc1 [5],
* website pull request listing the new release [6].

The vote will be open for at least 72 hours. It is adopted by majority
approval, with at least 3 PMC affirmative votes.

Thanks,
Release Manager

[1] 
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12353145
[2] 
https://dist.apache.org/repos/dist/dev/flink/flink-connector-rabbitmq-3.0.2-rc1
[3] https://dist.apache.org/repos/dist/release/flink/KEYS
[4] https://repository.apache.org/content/repositories/orgapacheflink-1697
[5] https://github.com/apache/flink-connector-rabbitmq/releases/tag/v3.0.2-rc1
[6] https://github.com/apache/flink-web/pull/712

[VOTE] Release flink-connector-hbase v3.0.1, release candidate #2

2024-01-12 Thread Martijn Visser 
Hi everyone,
Please review and vote on the release candidate #2 for the
flink-connector-hbase version
3.0.1, as follows:
[ ] +1, Approve the release
[ ] -1, Do not approve the release (please provide specific comments)

This version is compatible with Flink 1.16.x, 1.17.x and 1.18.x

The complete staging area is available for your review, which includes:
* JIRA release notes [1],
* the official Apache source release to be deployed to dist.apache.org
[2], which are signed with the key with fingerprint
A5F3BCE4CBE993573EC5966A65321B8382B219AF [3],
* all artifacts to be deployed to the Maven Central Repository [4],
* source code tag v3.0.1-rc1 [5],
* website pull request listing the new release [6].

The vote will be open for at least 72 hours. It is adopted by majority
approval, with at least 3 PMC affirmative votes.

Thanks,
Release Manager

[1] https://issues.apache.org/jira/projects/FLINK/versions/12353603
[2] https://dist.apache.org/repos/dist/dev/flink/flink-connector-hbase-3.0.1-rc2
[3] https://dist.apache.org/repos/dist/release/flink/KEYS
[4] https://repository.apache.org/content/repositories/orgapacheflink-1696/
[5] https://github.com/apache/flink-connector-hbase/releases/tag/v3.0.1-rc2
[6] https://github.com/apache/flink-web/pull/708

[jira] [Created] (FLINK-34070) MiniClusterITCase.testHandleStreamingJobsWhenNotEnoughSlot fails for the AdaptiveScheduler

2024-01-12 Thread Matthias Pohl (Jira) 
Matthias Pohl created FLINK-34070:
-

 Summary: 
MiniClusterITCase.testHandleStreamingJobsWhenNotEnoughSlot fails for the 
AdaptiveScheduler
 Key: FLINK-34070
 URL: https://issues.apache.org/jira/browse/FLINK-34070
 Project: Flink
  Issue Type: Bug
  Components: Runtime / Coordination
Affects Versions: 1.19.0
Reporter: Matthias Pohl


We experience test failures of 
{{MiniClusterITCase.testHandleStreamingJobsWhenNotEnoughSlot}} with the 
{{AdaptiveScheduler}} being enabled after FLINK-33414 was fixed:
{code:java}
Jan 09 02:01:16 at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
Jan 09 02:01:16 at 
java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1707)
Jan 09 02:01:16 at 
java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3313)
Jan 09 02:01:16 at 
java.util.concurrent.CompletableFuture.waitingGet(CompletableFuture.java:1742)
Jan 09 02:01:16 at 
java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1908)
Jan 09 02:01:16 at 
org.apache.flink.runtime.minicluster.MiniCluster.executeJobBlocking(MiniCluster.java:1050)
Jan 09 02:01:16 at 
org.apache.flink.runtime.minicluster.MiniClusterITCase.runHandleJobsWhenNotEnoughSlots(MiniClusterITCase.java:152)
Jan 09 02:01:16 at 
org.apache.flink.runtime.minicluster.MiniClusterITCase.lambda$testHandleStreamingJobsWhenNotEnoughSlot$0(MiniClusterITCase.java:119)
Jan 09 02:01:16 at 
org.apache.flink.runtime.minicluster.MiniClusterITCase$$Lambda$1927/1144737794.call(Unknown
 Source)
Jan 09 02:01:16 at 
org.assertj.core.api.ThrowableAssert.catchThrowable(ThrowableAssert.java:63)
Jan 09 02:01:16 at 
org.assertj.core.api.AssertionsForClassTypes.catchThrowable(AssertionsForClassTypes.java:892)
Jan 09 02:01:16 at 
org.assertj.core.api.Assertions.catchThrowable(Assertions.java:1366)
Jan 09 02:01:16 at 
org.assertj.core.api.Assertions.assertThatThrownBy(Assertions.java:1210)
Jan 09 02:01:16 at 
org.apache.flink.runtime.minicluster.MiniClusterITCase.testHandleStreamingJobsWhenNotEnoughSlot(MiniClusterITCase.java:119)
Jan 09 02:01:16 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native 
Method)
Jan 09 02:01:16 at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
Jan 09 02:01:16 at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 {code}
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=56166=logs=0e7be18f-84f2-53f0-a32d-4a5e4a174679=7c1d86e3-35bd-5fd5-3b7c-30c126a78702=10782



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (FLINK-34069) build_wheels_on_macos times out and gets cancelled

2024-01-12 Thread Matthias Pohl (Jira) 
Matthias Pohl created FLINK-34069:
-

 Summary: build_wheels_on_macos times out and gets cancelled
 Key: FLINK-34069
 URL: https://issues.apache.org/jira/browse/FLINK-34069
 Project: Flink
  Issue Type: Bug
  Components: Build System
Affects Versions: 1.17.2, 1.18.0, 1.19.0
Reporter: Matthias Pohl


[https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=56285=logs=f73b5736-8355-5390-ec71-4dfdec0ce6c5=90f7230e-bf5a-531b-8566-ad48d3e03bbb=146]

It doesn't look like the wheels step halts but rather that the step needs more 
time in general which increases the likelihood of the build running into the 
timeout here. We should investigate whether we're close to the timeout for this 
step in general.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (FLINK-34068) build_wheels_on_macos fails due to OSError

2024-01-12 Thread Matthias Pohl (Jira) 
Matthias Pohl created FLINK-34068:
-

 Summary: build_wheels_on_macos fails due to OSError
 Key: FLINK-34068
 URL: https://issues.apache.org/jira/browse/FLINK-34068
 Project: Flink
  Issue Type: Bug
  Components: Build System
Affects Versions: 1.17.2, 1.18.0, 1.19.0
Reporter: Matthias Pohl


[https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=56226=logs=f73b5736-8355-5390-ec71-4dfdec0ce6c5=90f7230e-bf5a-531b-8566-ad48d3e03bbb=187]
{code:java}
 [...]
 ERROR: Could not install packages due to an OSError: 
HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Max retries 
exceeded with url: 
/packages/a9/3e/6498bcf34a8194a2e3cc6dab0a742f4de9a28363ccd91ca37c38d3fe9cb4/apache_beam-2.48.0-cp310-cp310-macosx_10_9_x86_64.whl
 (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 8] 
nodename nor servname provided, or not known')){code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [VOTE] Release flink-connector-hbase, release candidate #1

2024-01-12 Thread Martijn Visser 
Hi Sergey,

Yep, you're right. I used the wrong JDK when building :( I'll cancel
this RC and make a new one!

Thanks,

Martijn

On Wed, Jan 10, 2024 at 10:20 PM Sergey Nuyanzin  wrote:
>
> Thanks for driving this Martijn
>
> based on the info in MANIFEST.MF of jars it seems it was built with jdk11
> shouldn't we still use jdk8 for that?
>
> On Fri, Jan 5, 2024 at 4:03 PM Martijn Visser 
> wrote:
>
> > Hi everyone,
> > Please review and vote on the release candidate #1 for the version
> > 3.0.1, as follows:
> > [ ] +1, Approve the release
> > [ ] -1, Do not approve the release (please provide specific comments)
> >
> > This version is compatible with Flink 1.16.x, 1.17.x and 1.18.x
> >
> > The complete staging area is available for your review, which includes:
> > * JIRA release notes [1],
> > * the official Apache source release to be deployed to dist.apache.org
> > [2], which are signed with the key with fingerprint
> > A5F3BCE4CBE993573EC5966A65321B8382B219AF [3],
> > * all artifacts to be deployed to the Maven Central Repository [4],
> > * source code tag v3.0.1-rc1 [5],
> > * website pull request listing the new release [6].
> >
> > The vote will be open for at least 72 hours. It is adopted by majority
> > approval, with at least 3 PMC affirmative votes.
> >
> > Thanks,
> > Release Manager
> >
> > [1] https://issues.apache.org/jira/projects/FLINK/versions/12353603
> > [2]
> > https://dist.apache.org/repos/dist/dev/flink/flink-connector-hbase-3.0.1-rc1
> > [3] https://dist.apache.org/repos/dist/release/flink/KEYS
> > [4]
> > https://repository.apache.org/content/repositories/orgapacheflink-1692/
> > [5]
> > https://github.com/apache/flink-connector-hbase/releases/tag/v3.0.1-rc1
> > [6] https://github.com/apache/flink-web/pull/708
> >
>
>
> --
> Best regards,
> Sergey

[RESULT] [VOTE] Release 1.18.1, release candidate #2

2024-01-12 Thread Jing Ge 
I'm happy to announce that we have unanimously approved this release.

There are 10 approving votes, 4 of which are binding:

- Jiabao Sun
- Hang Ruan
- Gongzhong Qiang
- Martijn Visser (binding)
- Rui Fan
- Yun Tang
- Jark Wu (binding)
- Qingsheng Ren (binding)
- Leonard Xu (binding)
- Sergey Nuyanzin


There are no disapproving votes.

I'll work on the steps to finalize the release and will send out the
announcement as soon as that has been completed.

Thanks, everyone!

Best regards,
Jing

Re: [DISCUSS] FLIP-415: Introduce a new join operator to support minibatch

2024-01-12 Thread shuai xu 
Hi all. 

The point I want to highlight is that minibatch join could potentially yield 
incomplete changelog which existing jobs are not supposed to be. For example, 
the scenario that joins two CDC sources after de-duplicating them and the 
output would be used for audit analysis could not accept incomplete changelog. 
While the minibatch processing itself would not introduce any problem. 

The internal behavior of minibatch processing is not well-defined now. I don't 
think reusing the minibatch option for minibatch join is problematic, but 
precise control is necessary to mitigate the risk of generating incomplete 
changelog within minibatch. 

Controlling the behavior on changelog within minibatch should be a global 
option. Therefore, I propose introducing a new option 
'table.exec.mini-batch.compact-changes-enabled' to precisely control changelog 
compaction within minibatch. Then we deprecate the option 
'table.exec.deduplicate.mini-batch.compact-changes-enabled' . The deduplicate 
operator would fall back to follow the newly introduced option and the 
minibatch join would follow it as well. 


> 2024年1月12日 16:30，Jane Chan  写道：
> 
> Hi shuai,
> 
> Thanks for the update! Regarding the newly introduced configuration, I hold
> the same concern with Benchao and Xuyang.
> 
> First of all, in most cases, the fact that users choose to enable
> mini-batch configuration indicates they are aware of the trade-off between
> throughput and completeness of the changelog.
> And if we finally adopt this configuration solely to avoid state
> incompatibility, does it mean that we will need to introduce a new
> configuration for every future operator's mini-batch optimization, similar
> to what we did today?
> 
> Best,
> Jane
> 
> On Fri, Jan 12, 2024 at 1:45 PM Xuyang  wrote:
> 
>> Hi, Xu Shuai. Thanks for driving this flip.
>> 
>> 
>> The CDC message amplification of cascade join has always been a problem
>> for users. Judging from the
>> nexmark results, this optimization is very meaningful. I just have the
>> same doubts as Benchao, why can't we
>> use minibatch join as the default behavior when the user turns on
>> minibatch?
>> 
>> 
>>> Although the semantic of changelog emitted by the Join operator is
>> eventual consistency, the change might
>> not be supposed for the downstream of the job which requires details of
>> changelog.
>> 
>> 
>> I think if the user adds the minibatch options to his job to enable
>> minibatch, he should know that flink will reduce
>> the amount of data sent to downstream by folding CDC messages as much as
>> possible. In scenarios where all
>> details of CDC records need to be retained, such as just synchronizing
>> data with jobs from one db to another db,
>> users have no reason to enable minibatch.
>> 
>> 
>> The only scenario I can think of that requires adding this independent
>> minibatch join option is to ensure that the state
>> is compatible between multiple versions, but we have not promised users
>> state compatibility during cross-version upgrades.
>> 
>> 
>> Maybe we need to figure it out why does the
>> 'table.exec.deduplicate.mini-batch.compact-changes-enabled' option need to
>> be added to deduplicate operator? I think this is the same reason as
>> adding a separate parameter to join to control CDC message folding.
>> 
>> 
>> 
>> 
>> --
>> 
>>Best！
>>Xuyang
>> 
>> 
>> 
>> 
>> 
>> 在 2024-01-11 16:19:30，"Benchao Li"  写道：
 the change might not be supposed for the downstream of the job which
>> requires details of changelog
>>> 
>>> Could you elaborate on this a bit? I've never met such kinds of
>>> requirements before, I'm curious what is the scenario that requires
>>> this.
>>> 
>>> shuai xu  于2024年1月11日周四 13:08写道：
 
 Thanks for your response, Benchao.
 
 Here is my thought on the newly added option.
 Users' current jobs are running on a version without minibatch join. If
>> the existing option to enable minibatch join is utilized, then when users'
>> jobs are migrated to the new version, the internal behavior of the join
>> operation within the jobs will change. Although the semantic of changelog
>> emitted by the Join operator is eventual consistency, the change might not
>> be supposed for the downstream of the job which requires details of
>> changelog. This newly added option also refers to
>> 'table.exec.deduplicate.mini-batch.compact-changes-enabled'.
 
 As for the implementation，The new operator shares the state of the
>> original operator and it merely has an additional minibatch for storing
>> records to do some optimization. The storage remains consistent, and there
>> is minor modification to the computational logic.
 
 Best,
 Xu Shuai
 
> 2024年1月10日 22:56，Benchao Li  写道：
> 
> Thanks shuai for driving this, mini-batch Join is a very useful
> optimization, +1 for the general idea.
> 
> Regarding the configuration
> "table.exec.stream.join.mini-batch-enabled", I'm not sure

Re: [ANNOUNCE] Apache Flink-shaded 18.0 released

2024-01-12 Thread Matthias Pohl 
Thanks for driving this release, Sergey. I reported it to the Apache
foundation based on the release information in Jira [1].

[1] https://issues.apache.org/jira/projects/FLINK/versions/12353081

On Fri, Jan 12, 2024 at 1:01 AM Sergey Nuyanzin  wrote:

> The Apache Flink community is very happy to announce the release of Apache
> Flink-shaded 18.0.
>
> The flink-shaded project contains a number of shaded dependencies for
> Apache Flink.
>
> Apache Flink® is an open-source stream processing framework for
> distributed, high-performing, always-available, and accurate data streaming
> applications.
>
> The release is available for download at:
> https://flink.apache.org/downloads.html
>
> The full release notes are available in Jira:
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12353081
>
> We would like to thank all contributors of the Apache Flink community who
> made this release possible!
>
> Regards,
> Release Manager
>

Re: [VOTE] Release 1.18.1, release candidate #2

2024-01-12 Thread Jing Ge 
Thank you all! I will post the vote result soon.

Best regards,
Jing

On Wed, Jan 10, 2024 at 12:34 PM Sergey Nuyanzin 
wrote:

> +1 (non-binding)
>
> -verified checksums
> -verified signatures
> -checked release tag
> -verified that there is no binary in source
> -built from sources
>
> On Wed, Jan 10, 2024 at 12:30 PM Leonard Xu  wrote:
>
> > +1 (binding)
> >
> > - verified signatures
> > - verified hashsums
> > - checked Github release tag
> > - started SQL Client, used MySQL CDC connector to capture data change
> from
> > database , the result is expected
> > - reviewed the web PR
> > - reviewed the release notes PR
> >
> > Best,
> > Leonard
> >
> > > 2024年1月10日 上午10:48，Qingsheng Ren  写道：
> > >
> > > +1 (binding)
> > >
> > > - Built from source
> > > - Verified checksum and signature
> > > - Verified that no binary exist in source
> > > - Started a standalone cluster and submit Kafka consuming and producing
> > job
> > > with SQL client
> > > - Reviewed web PR
> > >
> > > Thanks for driving this, Jing!
> > >
> > > Best,
> > > Qingsheng
> > >
> > > On Mon, Jan 8, 2024 at 8:02 PM Jark Wu  wrote:
> > >
> > >> Thanks Jing for driving this.
> > >>
> > >> +1 (binding)
> > >>
> > >> - Build and compile the source code locally: *OK*
> > >> - Verified signatures and hashes: *OK*
> > >> - Checked no missing artifacts in the staging area: *OK*
> > >> - Reviewed the website release PR: *OK*
> > >> - Went through the quick start: *OK*
> > >>  * Started a cluster and ran the examples
> > >>  * Verified web ui and log output, nothing unexpected
> > >>
> > >> Best,
> > >> Jark
> > >>
> > >> On Thu, 28 Dec 2023 at 20:59, Yun Tang  wrote:
> > >>
> > >>> Thanks Jing for driving this release.
> > >>>
> > >>> +1 (non-binding)
> > >>>
> > >>>
> > >>>  *
> > >>> Download artifacts and verify the signatures.
> > >>>  *
> > >>> Verified the web PR
> > >>>  *
> > >>> Verified the number of Python packages is 11
> > >>>  *
> > >>> Started a local cluster and verified FLIP-291 to see the rescale
> > results.
> > >>>  *
> > >>> Verified the jar packages were built with JDK8
> > >>>
> > >>> Best
> > >>> Yun Tang
> > >>>
> > >>>
> > >>> 
> > >>> From: Rui Fan <1996fan...@gmail.com>
> > >>> Sent: Thursday, December 28, 2023 10:54
> > >>> To: dev@flink.apache.org 
> > >>> Subject: Re: [VOTE] Release 1.18.1, release candidate #2
> > >>>
> > >>> Thanks Jing for driving this release!
> > >>>
> > >>> +1(non-binding)
> > >>>
> > >>> - Downloaded artifacts
> > >>> - Verified signatures and sha512
> > >>> - The source archives do not contain any binaries
> > >>> - Verified web PR
> > >>> - Build the source with Maven 3 and java8 (Checked the license as
> well)
> > >>> - bin/start-cluster.sh with java8, it works fine and no any
> unexpected
> > >> LOG-
> > >>> Ran demo, it's fine:  bin/flink
> > >>> runexamples/streaming/StateMachineExample.jar
> > >>>
> > >>> Best,
> > >>> Rui
> > >>>
> > >>> On Wed, Dec 27, 2023 at 8:45 PM Martijn Visser <
> > martijnvis...@apache.org
> > >>>
> > >>> wrote:
> > >>>
> >  Hi Jing,
> > 
> >  Thanks for driving this.
> > 
> >  +1 (binding)
> > 
> >  - Validated hashes
> >  - Verified signature
> >  - Verified that no binaries exist in the source archive
> >  - Build the source with Maven via mvn clean install
> >  -Pcheck-convergence -Dflink.version=1.18.1
> >  - Verified licenses
> >  - Verified web PR
> >  - Started a cluster and the Flink SQL client, successfully read and
> >  wrote with the Kafka connector to Confluent Cloud with AVRO and
> Schema
> >  Registry enabled
> >  - Started a cluster and submitted a job that checkpoints to GCS
> > without
> >  problems
> > 
> >  Best regards,
> > 
> >  Martijn
> > 
> >  On Thu, Dec 21, 2023 at 4:55 AM gongzhongqiang
> >   wrote:
> > >
> > > Thanks Jing Ge for driving this release.
> > >
> > > +1 (non-binding), I have checked:
> > > [✓] The checksums and signatures are validated
> > > [✓] The tag checked is fine
> > > [✓] Built from source is passed
> > > [✓] The flink-web PR is reviewed and checked
> > >
> > >
> > > Best，
> > > Zhongqiang Gong
> > 
> > >>>
> > >>
> >
> >
>
> --
> Best regards,
> Sergey
>

Re: [FLIP-412] Add the time-consuming span of each stage when starting the Flink job to TraceReporter

2024-01-12 Thread Eason Qin 
Thank you, Piotr and Rui.

Best,
Eason

Rui Fan <1996fan...@gmail.com> 于2024年1月12日周五 12:42写道：

> The permission is added by Piotr, thank you Piotr.
>
> Best,
> Rui
>
> On Thu, Jan 11, 2024 at 9:15 PM Eason Qin  wrote:
>
> > Hi all,
> >
> > Currently, I am working on the FLIP-412: Add the time-consuming span of
> > each stage when starting the Flink job to TraceReporter[1], but I have no
> > permission to update the Flink Improvement Proposals space. Can any PMC
> > help me add permissions?
> >
> > My Jira account is easonqin and my email is qinjunzh...@gmail.com.
> > My confluence account is eason.qin which is different from the Jira
> > account.
> >
> > [1]
> >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-412%3A+Add+the+time-consuming+span+of+each+stage+when+starting+the+Flink+job+to+TraceReporter
> >
> >
> > Thanks!
> >
>

Re: Re: [DISCUSS] FLIP-415: Introduce a new join operator to support minibatch

2024-01-12 Thread Jane Chan 
Hi shuai,

Thanks for the update! Regarding the newly introduced configuration, I hold
the same concern with Benchao and Xuyang.

First of all, in most cases, the fact that users choose to enable
mini-batch configuration indicates they are aware of the trade-off between
throughput and completeness of the changelog.
And if we finally adopt this configuration solely to avoid state
incompatibility, does it mean that we will need to introduce a new
configuration for every future operator's mini-batch optimization, similar
to what we did today?

Best,
Jane

On Fri, Jan 12, 2024 at 1:45 PM Xuyang  wrote:

> Hi, Xu Shuai. Thanks for driving this flip.
>
>
> The CDC message amplification of cascade join has always been a problem
> for users. Judging from the
> nexmark results, this optimization is very meaningful. I just have the
> same doubts as Benchao, why can't we
> use minibatch join as the default behavior when the user turns on
> minibatch?
>
>
> > Although the semantic of changelog emitted by the Join operator is
> eventual consistency, the change might
> not be supposed for the downstream of the job which requires details of
> changelog.
>
>
> I think if the user adds the minibatch options to his job to enable
> minibatch, he should know that flink will reduce
> the amount of data sent to downstream by folding CDC messages as much as
> possible. In scenarios where all
> details of CDC records need to be retained, such as just synchronizing
> data with jobs from one db to another db,
> users have no reason to enable minibatch.
>
>
> The only scenario I can think of that requires adding this independent
> minibatch join option is to ensure that the state
> is compatible between multiple versions, but we have not promised users
> state compatibility during cross-version upgrades.
>
>
> Maybe we need to figure it out why does the
> 'table.exec.deduplicate.mini-batch.compact-changes-enabled' option need to
> be added to deduplicate operator? I think this is the same reason as
> adding a separate parameter to join to control CDC message folding.
>
>
>
>
> --
>
> Best！
> Xuyang
>
>
>
>
>
> 在 2024-01-11 16:19:30，"Benchao Li"  写道：
> >> the change might not be supposed for the downstream of the job which
> requires details of changelog
> >
> >Could you elaborate on this a bit? I've never met such kinds of
> >requirements before, I'm curious what is the scenario that requires
> >this.
> >
> >shuai xu  于2024年1月11日周四 13:08写道：
> >>
> >> Thanks for your response, Benchao.
> >>
> >> Here is my thought on the newly added option.
> >> Users' current jobs are running on a version without minibatch join. If
> the existing option to enable minibatch join is utilized, then when users'
> jobs are migrated to the new version, the internal behavior of the join
> operation within the jobs will change. Although the semantic of changelog
> emitted by the Join operator is eventual consistency, the change might not
> be supposed for the downstream of the job which requires details of
> changelog. This newly added option also refers to
> 'table.exec.deduplicate.mini-batch.compact-changes-enabled'.
> >>
> >> As for the implementation，The new operator shares the state of the
> original operator and it merely has an additional minibatch for storing
> records to do some optimization. The storage remains consistent, and there
> is minor modification to the computational logic.
> >>
> >> Best,
> >> Xu Shuai
> >>
> >> > 2024年1月10日 22:56，Benchao Li  写道：
> >> >
> >> > Thanks shuai for driving this, mini-batch Join is a very useful
> >> > optimization, +1 for the general idea.
> >> >
> >> > Regarding the configuration
> >> > "table.exec.stream.join.mini-batch-enabled", I'm not sure it's really
> >> > necessary. The semantic of changelog emitted by the Join operator is
> >> > eventual consistency, so there is no much difference between original
> >> > Join and mini-batch Join from this aspect. Besides, introducing more
> >> > options would make it more complex for users, harder to understand and
> >> > maintain, which we should be careful about.
> >> >
> >> > One thing about the implementation, could you make the new operator
> >> > share the same state definition with the original one?
> >> >
> >> > shuai xu  于2024年1月10日周三 21:23写道：
> >> >>
> >> >> Hi devs,
> >> >>
> >> >> I’d like to start a discussion on FLIP-415: Introduce a new join
> operator to support minibatch[1].
> >> >>
> >> >> Currently, when performing cascading connections in Flink, there is
> a pain point of record amplification. Every record join operator receives
> would trigger join process. However, if records of +I and -D matches , they
> could be folded to reduce two times of join process. Besides, records of
> -U +U might output 4 records in which two records are redundant when
> encountering outer join .
> >> >>
> >> >> To address this issue, this FLIP introduces a new
> MiniBatchStreamingJoinOperator to achieve batch processing which could
> reduce number of