Re: [DISCUSS] FLIP-461: FLIP-461: Synchronize rescaling with checkpoint creation to minimize reprocessing
I have a few questions: Unaligned checkpoints Do we need to enable this feature? Whether this feature should be disabled for checkpoints that do not check it Matthias Pohl 于2024年6月4日周二 18:03写道： > Hi everyone, > I'd like to discuss FLIP-461 [1]. The FLIP proposes the synchronization of > rescaling and the completion of checkpoints. The idea is to reduce the > amount of data that needs to be processed after rescaling happened. A more > detailed motivation can be found in FLIP-461. > > I'm looking forward to feedback and suggestions. > > Best, > Matthias > > [1] > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-461%3A+Synchronize+rescaling+with+checkpoint+creation+to+minimize+reprocessing > -- Best ConradJam
Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie Guo
Congratulations！ Biao Geng 于2024年6月5日周三 10:44写道： > Congratulations, Weijie! > Best, > Biao Geng > > > Yun Tang 于2024年6月5日周三 10:42写道： > > > Congratulations, Weijie! > > > > Best > > Yun Tang > > > > From: Hangxiang Yu > > Sent: Wednesday, June 5, 2024 10:00 > > To: dev@flink.apache.org > > Subject: Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie Guo > > > > Congratulations, Weijie! > > > > On Tue, Jun 4, 2024 at 11:40 PM Zhanghao Chen > > > wrote: > > > > > Congrats, Weijie! > > > > > > Best, > > > Zhanghao Chen > > > > > > From: Hang Ruan > > > Sent: Tuesday, June 4, 2024 16:37 > > > To: dev@flink.apache.org > > > Subject: Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie Guo > > > > > > Congratulations Weijie! > > > > > > Best, > > > Hang > > > > > > Yanfei Lei 于2024年6月4日周二 16:24写道： > > > > > > > Congratulations! > > > > > > > > Best, > > > > Yanfei > > > > > > > > Leonard Xu 于2024年6月4日周二 16:20写道： > > > > > > > > > > Congratulations！ > > > > > > > > > > Best, > > > > > Leonard > > > > > > > > > > > 2024年6月4日 下午4:02，Yangze Guo 写道： > > > > > > > > > > > > Congratulations! > > > > > > > > > > > > Best, > > > > > > Yangze Guo > > > > > > > > > > > > On Tue, Jun 4, 2024 at 4:00 PM Weihua Hu > > > > > wrote: > > > > > >> > > > > > >> Congratulations, Weijie! > > > > > >> > > > > > >> Best, > > > > > >> Weihua > > > > > >> > > > > > >> > > > > > >> On Tue, Jun 4, 2024 at 3:03 PM Yuxin Tan < > tanyuxinw...@gmail.com> > > > > wrote: > > > > > >> > > > > > >>> Congratulations, Weijie! > > > > > >>> > > > > > >>> Best, > > > > > >>> Yuxin > > > > > >>> > > > > > >>> > > > > > >>> Yuepeng Pan 于2024年6月4日周二 14:57写道： > > > > > >>> > > > > > Congratulations ! > > > > > > > > > > > > > > > Best, > > > > > Yuepeng Pan > > > > > > > > > > At 2024-06-04 14:45:45, "Xintong Song" > > > > > wrote: > > > > > > Hi everyone, > > > > > > > > > > > > On behalf of the PMC, I'm very happy to announce that Weijie > > Guo > > > > has > > > > > joined > > > > > > the Flink PMC! > > > > > > > > > > > > Weijie has been an active member of the Apache Flink > community > > > for > > > > many > > > > > > years. He has made significant contributions in many > > components, > > > > > >>> including > > > > > > runtime, shuffle, sdk, connectors, etc. He has driven / > > > > participated in > > > > > > many FLIPs, authored and reviewed hundreds of PRs, been > > > > consistently > > > > > active > > > > > > on mailing lists, and also helped with release management of > > 1.20 > > > > and > > > > > > several other bugfix releases. > > > > > > > > > > > > Congratulations and welcome Weijie! > > > > > > > > > > > > Best, > > > > > > > > > > > > Xintong (on behalf of the Flink PMC) > > > > > > > > > > >>> > > > > > > > > > > > > > > > > > > -- > > Best, > > Hangxiang. > > > -- Best ConradJam
Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie Guo
Congratulations, Weijie! Best, Biao Geng Yun Tang 于2024年6月5日周三 10:42写道： > Congratulations, Weijie! > > Best > Yun Tang > > From: Hangxiang Yu > Sent: Wednesday, June 5, 2024 10:00 > To: dev@flink.apache.org > Subject: Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie Guo > > Congratulations, Weijie! > > On Tue, Jun 4, 2024 at 11:40 PM Zhanghao Chen > wrote: > > > Congrats, Weijie! > > > > Best, > > Zhanghao Chen > > > > From: Hang Ruan > > Sent: Tuesday, June 4, 2024 16:37 > > To: dev@flink.apache.org > > Subject: Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie Guo > > > > Congratulations Weijie! > > > > Best, > > Hang > > > > Yanfei Lei 于2024年6月4日周二 16:24写道： > > > > > Congratulations! > > > > > > Best, > > > Yanfei > > > > > > Leonard Xu 于2024年6月4日周二 16:20写道： > > > > > > > > Congratulations！ > > > > > > > > Best, > > > > Leonard > > > > > > > > > 2024年6月4日 下午4:02，Yangze Guo 写道： > > > > > > > > > > Congratulations! > > > > > > > > > > Best, > > > > > Yangze Guo > > > > > > > > > > On Tue, Jun 4, 2024 at 4:00 PM Weihua Hu > > > wrote: > > > > >> > > > > >> Congratulations, Weijie! > > > > >> > > > > >> Best, > > > > >> Weihua > > > > >> > > > > >> > > > > >> On Tue, Jun 4, 2024 at 3:03 PM Yuxin Tan > > > wrote: > > > > >> > > > > >>> Congratulations, Weijie! > > > > >>> > > > > >>> Best, > > > > >>> Yuxin > > > > >>> > > > > >>> > > > > >>> Yuepeng Pan 于2024年6月4日周二 14:57写道： > > > > >>> > > > > Congratulations ! > > > > > > > > > > > > Best, > > > > Yuepeng Pan > > > > > > > > At 2024-06-04 14:45:45, "Xintong Song" > > > wrote: > > > > > Hi everyone, > > > > > > > > > > On behalf of the PMC, I'm very happy to announce that Weijie > Guo > > > has > > > > joined > > > > > the Flink PMC! > > > > > > > > > > Weijie has been an active member of the Apache Flink community > > for > > > many > > > > > years. He has made significant contributions in many > components, > > > > >>> including > > > > > runtime, shuffle, sdk, connectors, etc. He has driven / > > > participated in > > > > > many FLIPs, authored and reviewed hundreds of PRs, been > > > consistently > > > > active > > > > > on mailing lists, and also helped with release management of > 1.20 > > > and > > > > > several other bugfix releases. > > > > > > > > > > Congratulations and welcome Weijie! > > > > > > > > > > Best, > > > > > > > > > > Xintong (on behalf of the Flink PMC) > > > > > > > > >>> > > > > > > > > > > > > -- > Best, > Hangxiang. >
Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie Guo
Congratulations, Weijie! Best Yun Tang From: Hangxiang Yu Sent: Wednesday, June 5, 2024 10:00 To: dev@flink.apache.org Subject: Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie Guo Congratulations, Weijie! On Tue, Jun 4, 2024 at 11:40 PM Zhanghao Chen wrote: > Congrats, Weijie! > > Best, > Zhanghao Chen > > From: Hang Ruan > Sent: Tuesday, June 4, 2024 16:37 > To: dev@flink.apache.org > Subject: Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie Guo > > Congratulations Weijie! > > Best, > Hang > > Yanfei Lei 于2024年6月4日周二 16:24写道： > > > Congratulations! > > > > Best, > > Yanfei > > > > Leonard Xu 于2024年6月4日周二 16:20写道： > > > > > > Congratulations！ > > > > > > Best, > > > Leonard > > > > > > > 2024年6月4日 下午4:02，Yangze Guo 写道： > > > > > > > > Congratulations! > > > > > > > > Best, > > > > Yangze Guo > > > > > > > > On Tue, Jun 4, 2024 at 4:00 PM Weihua Hu > > wrote: > > > >> > > > >> Congratulations, Weijie! > > > >> > > > >> Best, > > > >> Weihua > > > >> > > > >> > > > >> On Tue, Jun 4, 2024 at 3:03 PM Yuxin Tan > > wrote: > > > >> > > > >>> Congratulations, Weijie! > > > >>> > > > >>> Best, > > > >>> Yuxin > > > >>> > > > >>> > > > >>> Yuepeng Pan 于2024年6月4日周二 14:57写道： > > > >>> > > > Congratulations ! > > > > > > > > > Best, > > > Yuepeng Pan > > > > > > At 2024-06-04 14:45:45, "Xintong Song" > > wrote: > > > > Hi everyone, > > > > > > > > On behalf of the PMC, I'm very happy to announce that Weijie Guo > > has > > > joined > > > > the Flink PMC! > > > > > > > > Weijie has been an active member of the Apache Flink community > for > > many > > > > years. He has made significant contributions in many components, > > > >>> including > > > > runtime, shuffle, sdk, connectors, etc. He has driven / > > participated in > > > > many FLIPs, authored and reviewed hundreds of PRs, been > > consistently > > > active > > > > on mailing lists, and also helped with release management of 1.20 > > and > > > > several other bugfix releases. > > > > > > > > Congratulations and welcome Weijie! > > > > > > > > Best, > > > > > > > > Xintong (on behalf of the Flink PMC) > > > > > > >>> > > > > > > -- Best, Hangxiang.
Re: [DISCUSS] FLIP-459: Support Flink hybrid shuffle integration with Apache Celeborn
Thanks Yuxin for the proposal! When we first proposed Hybrid Shuffle, I wanted to support pluggable storage tier in the future. However, limited by the architecture of the legacy Hybrid Shuffle at that time, this idea has not been realized. The new architecture abstracts the tier nicely, and now it's time to introduce support for external storage. Big +1 for this one! Best regards, Weijie rexxiong 于2024年6月5日周三 00:08写道： > Thanks Yuxin for the proposal. +1, as a member of the Apache Celeborn > community, I am very excited about the integration of Flink's Hybrid > Shuffle with Apache Celeborn. The whole design of CIP-6 looks good to me. I > am looking forward to this integration. > > Thanks, > Jiashu Xiong > > Ethan Feng 于2024年6月4日周二 16:47写道： > > > +1 for this proposal. > > > > After internally reviewing the prototype of CIP-6, this would improve > > performance and stability for Flink users using Celeborn. > > > > Expect to see this feature come out to the community. > > > > As I come from the Celeborn community, I hope more users can try to > > use Celeborn when there are Flink batch jobs. > > > > Thanks, > > Ethan Feng > > > > Yuxin Tan 于2024年6月4日周二 16:34写道： > > > > > > Hi, Venkatakrishnan, > > > > > > Thanks for joining the discussion. We appreciate your interest > > > in contributing to the work. Once the FLIP and CIP proposals > > > have been approved, we will create some JIRA tickets in Flink > > > and Celeborn projects. Please feel free to take a look at the > > > tickets and select any that resonate with your interests. > > > > > > Best, > > > Yuxin > > > > > > > > > Venkatakrishnan Sowrirajan 于2024年5月31日周五 23:11写道： > > > > > > > Thanks for this FLIP. We are also interested in learning/contributing > > to > > > > the hybrid shuffle integration with celeborn for batch executions. > > > > > > > > On Tue, May 28, 2024, 7:07 PM Yuxin Tan > > wrote: > > > > > > > > > Hi, Xintong, > > > > > > > > > > > I think we can also publish the prototype codes so the > > > > > community can better understand and help with it. > > > > > > > > > > Ok, I agree on the point. I will prepare and publish the code > > > > > recently. > > > > > > > > > > Rui, > > > > > > > > > > > Kindly reminder: the image of CIP-6[1] cannot be loaded. > > > > > > > > > > Thanks for the reminder. I've updated the images. > > > > > > > > > > > > > > > Best, > > > > > Yuxin > > > > > > > > > > > > > > > Rui Fan <1996fan...@gmail.com> 于2024年5月29日周三 09:33写道： > > > > > > > > > > > Thanks Yuxin for driving this proposal! > > > > > > > > > > > > Kindly reminder: the image of CIP-6[1] cannot be loaded. > > > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > > > > > > > https://urldefense.com/v3/__https://cwiki.apache.org/confluence/display/CELEBORN/CIP-6*Support*Flink*hybrid*shuffle*integration*with*Apache*Celeborn__;KysrKysrKys!!IKRxdwAv5BmarQ!ZRTc1aUSYMDBazuIwlet1Dzk2_DD9qKTgoDLH9jSwAVLgwplcuId_8JoXkH0i7AeWxKWXkL0sxM3AeW-H9OJ6v9uGw$ > > > > > > > > > > > > Best, > > > > > > Rui > > > > > > > > > > > > On Wed, May 29, 2024 at 9:03 AM Xintong Song < > > tonysong...@gmail.com> > > > > > > wrote: > > > > > > > > > > > > > +1 for this proposal. > > > > > > > > > > > > > > We have been prototyping this feature internally at Alibaba > for a > > > > > couple > > > > > > of > > > > > > > months. Yuxin, I think we can also publish the prototype codes > > so the > > > > > > > community can better understand and help with it. > > > > > > > > > > > > > > Best, > > > > > > > > > > > > > > Xintong > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, May 28, 2024 at 8:34 PM Yuxin Tan < > > tanyuxinw...@gmail.com> > > > > > > wrote: > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > I would like to start a discussion on FLIP-459 Support Flink > > hybrid > > > > > > > shuffle > > > > > > > > integration with > > > > > > > > Apache Celeborn[1]. Flink hybrid shuffle supports transitions > > > > between > > > > > > > > memory, disk, and > > > > > > > > remote storage to improve performance and job stability. > > > > > Concurrently, > > > > > > > > Apache Celeborn > > > > > > > > provides a stable, performant, scalable remote shuffle > service. > > > > This > > > > > > > > integration proposal is to > > > > > > > > harness the benefits from both hybrid shuffle and Celeborn > > > > > > > simultaneously. > > > > > > > > > > > > > > > > Note that this proposal has two parts. > > > > > > > > 1. The Flink-side modifications are in FLIP-459[1]. > > > > > > > > 2. The Celeborn-side changes are in CIP-6[2]. > > > > > > > > > > > > > > > > Looking forward to everyone's feedback and suggestions. Thank > > you! > > > > > > > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie Guo
Congratulations, Weijie! On Tue, Jun 4, 2024 at 11:40 PM Zhanghao Chen wrote: > Congrats, Weijie! > > Best, > Zhanghao Chen > > From: Hang Ruan > Sent: Tuesday, June 4, 2024 16:37 > To: dev@flink.apache.org > Subject: Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie Guo > > Congratulations Weijie! > > Best, > Hang > > Yanfei Lei 于2024年6月4日周二 16:24写道： > > > Congratulations! > > > > Best, > > Yanfei > > > > Leonard Xu 于2024年6月4日周二 16:20写道： > > > > > > Congratulations！ > > > > > > Best, > > > Leonard > > > > > > > 2024年6月4日 下午4:02，Yangze Guo 写道： > > > > > > > > Congratulations! > > > > > > > > Best, > > > > Yangze Guo > > > > > > > > On Tue, Jun 4, 2024 at 4:00 PM Weihua Hu > > wrote: > > > >> > > > >> Congratulations, Weijie! > > > >> > > > >> Best, > > > >> Weihua > > > >> > > > >> > > > >> On Tue, Jun 4, 2024 at 3:03 PM Yuxin Tan > > wrote: > > > >> > > > >>> Congratulations, Weijie! > > > >>> > > > >>> Best, > > > >>> Yuxin > > > >>> > > > >>> > > > >>> Yuepeng Pan 于2024年6月4日周二 14:57写道： > > > >>> > > > Congratulations ! > > > > > > > > > Best, > > > Yuepeng Pan > > > > > > At 2024-06-04 14:45:45, "Xintong Song" > > wrote: > > > > Hi everyone, > > > > > > > > On behalf of the PMC, I'm very happy to announce that Weijie Guo > > has > > > joined > > > > the Flink PMC! > > > > > > > > Weijie has been an active member of the Apache Flink community > for > > many > > > > years. He has made significant contributions in many components, > > > >>> including > > > > runtime, shuffle, sdk, connectors, etc. He has driven / > > participated in > > > > many FLIPs, authored and reviewed hundreds of PRs, been > > consistently > > > active > > > > on mailing lists, and also helped with release management of 1.20 > > and > > > > several other bugfix releases. > > > > > > > > Congratulations and welcome Weijie! > > > > > > > > Best, > > > > > > > > Xintong (on behalf of the Flink PMC) > > > > > > >>> > > > > > > -- Best, Hangxiang.
[jira] [Created] (FLINK-35522) The source task may get stuck after a failover occurs in batch jobs
xingbe created FLINK-35522: -- Summary: The source task may get stuck after a failover occurs in batch jobs Key: FLINK-35522 URL: https://issues.apache.org/jira/browse/FLINK-35522 Project: Flink Issue Type: Bug Components: Runtime / Coordination Affects Versions: 1.18.1, 1.19.0, 1.17.2, 1.20.0 Reporter: xingbe Fix For: 1.20.0 If the source task does not get assigned a split because the SplitEnumerator has no more splits, and a failover occurs during the closing process, the SourceCoordinatorContext will not resend the NoMoreSplit event to the newly started source task, causing the source vertex to remain stuck indefinitely. This case may only occur in batch jobs where speculative execution has been enabled. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-35521) Flink FileSystem SQL Connector Generating SUCESS File Multiple Times
EMERSON WANG created FLINK-35521: Summary: Flink FileSystem SQL Connector Generating SUCESS File Multiple Times Key: FLINK-35521 URL: https://issues.apache.org/jira/browse/FLINK-35521 Project: Flink Issue Type: Improvement Components: Connectors / FileSystem Affects Versions: 1.18.1 Environment: Our PyFlink SQL jobs are running in AWS EKS environment. Reporter: EMERSON WANG Our Flink table SQL job received data from the Kafka streams and then sinked all partitioned data into the associated parquet files under the same S3 folder through the filesystem SQL connector. For the S3 filesystem SQL connector, sink.partition-commit.policy.kind was set to 'success-file' and sink.partition-commit.trigger was set to 'partition-time'. We found that _SUCCESS file in the S3 folder was generated multiple times after multiple partitions are committed. Because all partitioned parquet files and _SUCCESS file are in the same S3 folder and _SUCCESS file is used to trigger the downstream application, we really like the _SUCCESS file to be generated only once instead of multiple times after all partitions are committed and all parquet files are ready to be processed. Thus, one _SUCCESS file can be used to trigger the downstream application only once instead of multiple times. We knew we could set sink.partition-commit.trigger to 'process-time' to generate _SUCCESS file only once in the S3 folder; however, 'process-time' would not meet our business requirements. We'd request the FileSystem SQL connector should support to the following new user case: Even if sink.partition-commit.trigger is set to 'partition-time', _SUCCESS file will be generated only once after all partitions are committed and all output files are ready to be processed, and will be used to trigger the downstream application only once instead of multiple times. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [DISCUSS] FLIP-459: Support Flink hybrid shuffle integration with Apache Celeborn
Thanks Yuxin for the proposal. +1, as a member of the Apache Celeborn community, I am very excited about the integration of Flink's Hybrid Shuffle with Apache Celeborn. The whole design of CIP-6 looks good to me. I am looking forward to this integration. Thanks, Jiashu Xiong Ethan Feng 于2024年6月4日周二 16:47写道： > +1 for this proposal. > > After internally reviewing the prototype of CIP-6, this would improve > performance and stability for Flink users using Celeborn. > > Expect to see this feature come out to the community. > > As I come from the Celeborn community, I hope more users can try to > use Celeborn when there are Flink batch jobs. > > Thanks, > Ethan Feng > > Yuxin Tan 于2024年6月4日周二 16:34写道： > > > > Hi, Venkatakrishnan, > > > > Thanks for joining the discussion. We appreciate your interest > > in contributing to the work. Once the FLIP and CIP proposals > > have been approved, we will create some JIRA tickets in Flink > > and Celeborn projects. Please feel free to take a look at the > > tickets and select any that resonate with your interests. > > > > Best, > > Yuxin > > > > > > Venkatakrishnan Sowrirajan 于2024年5月31日周五 23:11写道： > > > > > Thanks for this FLIP. We are also interested in learning/contributing > to > > > the hybrid shuffle integration with celeborn for batch executions. > > > > > > On Tue, May 28, 2024, 7:07 PM Yuxin Tan > wrote: > > > > > > > Hi, Xintong, > > > > > > > > > I think we can also publish the prototype codes so the > > > > community can better understand and help with it. > > > > > > > > Ok, I agree on the point. I will prepare and publish the code > > > > recently. > > > > > > > > Rui, > > > > > > > > > Kindly reminder: the image of CIP-6[1] cannot be loaded. > > > > > > > > Thanks for the reminder. I've updated the images. > > > > > > > > > > > > Best, > > > > Yuxin > > > > > > > > > > > > Rui Fan <1996fan...@gmail.com> 于2024年5月29日周三 09:33写道： > > > > > > > > > Thanks Yuxin for driving this proposal! > > > > > > > > > > Kindly reminder: the image of CIP-6[1] cannot be loaded. > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > https://urldefense.com/v3/__https://cwiki.apache.org/confluence/display/CELEBORN/CIP-6*Support*Flink*hybrid*shuffle*integration*with*Apache*Celeborn__;KysrKysrKys!!IKRxdwAv5BmarQ!ZRTc1aUSYMDBazuIwlet1Dzk2_DD9qKTgoDLH9jSwAVLgwplcuId_8JoXkH0i7AeWxKWXkL0sxM3AeW-H9OJ6v9uGw$ > > > > > > > > > > Best, > > > > > Rui > > > > > > > > > > On Wed, May 29, 2024 at 9:03 AM Xintong Song < > tonysong...@gmail.com> > > > > > wrote: > > > > > > > > > > > +1 for this proposal. > > > > > > > > > > > > We have been prototyping this feature internally at Alibaba for a > > > > couple > > > > > of > > > > > > months. Yuxin, I think we can also publish the prototype codes > so the > > > > > > community can better understand and help with it. > > > > > > > > > > > > Best, > > > > > > > > > > > > Xintong > > > > > > > > > > > > > > > > > > > > > > > > On Tue, May 28, 2024 at 8:34 PM Yuxin Tan < > tanyuxinw...@gmail.com> > > > > > wrote: > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > I would like to start a discussion on FLIP-459 Support Flink > hybrid > > > > > > shuffle > > > > > > > integration with > > > > > > > Apache Celeborn[1]. Flink hybrid shuffle supports transitions > > > between > > > > > > > memory, disk, and > > > > > > > remote storage to improve performance and job stability. > > > > Concurrently, > > > > > > > Apache Celeborn > > > > > > > provides a stable, performant, scalable remote shuffle service. > > > This > > > > > > > integration proposal is to > > > > > > > harness the benefits from both hybrid shuffle and Celeborn > > > > > > simultaneously. > > > > > > > > > > > > > > Note that this proposal has two parts. > > > > > > > 1. The Flink-side modifications are in FLIP-459[1]. > > > > > > > 2. The Celeborn-side changes are in CIP-6[2]. > > > > > > > > > > > > > > Looking forward to everyone's feedback and suggestions. Thank > you! > > > > > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://urldefense.com/v3/__https://cwiki.apache.org/confluence/display/FLINK/FLIP-459*3A*Support*Flink*hybrid*shuffle*integration*with*Apache*Celeborn__;JSsrKysrKysr!!IKRxdwAv5BmarQ!ZRTc1aUSYMDBazuIwlet1Dzk2_DD9qKTgoDLH9jSwAVLgwplcuId_8JoXkH0i7AeWxKWXkL0sxM3AeW-H9MaOGE7hQ$ > > > > > > > [2] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://urldefense.com/v3/__https://cwiki.apache.org/confluence/display/CELEBORN/CIP-6*Support*Flink*hybrid*shuffle*integration*with*Apache*Celeborn__;KysrKysrKys!!IKRxdwAv5BmarQ!ZRTc1aUSYMDBazuIwlet1Dzk2_DD9qKTgoDLH9jSwAVLgwplcuId_8JoXkH0i7AeWxKWXkL0sxM3AeW-H9OJ6v9uGw$ > > > > > > > > > > > > > > Best, > > > > > > > Yuxin > > > > > > > > > > > > > > > > > > > > > > > > > >
Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie Guo
Congrats, Weijie! Best, Zhanghao Chen From: Hang Ruan Sent: Tuesday, June 4, 2024 16:37 To: dev@flink.apache.org Subject: Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie Guo Congratulations Weijie! Best, Hang Yanfei Lei 于2024年6月4日周二 16:24写道： > Congratulations! > > Best, > Yanfei > > Leonard Xu 于2024年6月4日周二 16:20写道： > > > > Congratulations！ > > > > Best, > > Leonard > > > > > 2024年6月4日 下午4:02，Yangze Guo 写道： > > > > > > Congratulations! > > > > > > Best, > > > Yangze Guo > > > > > > On Tue, Jun 4, 2024 at 4:00 PM Weihua Hu > wrote: > > >> > > >> Congratulations, Weijie! > > >> > > >> Best, > > >> Weihua > > >> > > >> > > >> On Tue, Jun 4, 2024 at 3:03 PM Yuxin Tan > wrote: > > >> > > >>> Congratulations, Weijie! > > >>> > > >>> Best, > > >>> Yuxin > > >>> > > >>> > > >>> Yuepeng Pan 于2024年6月4日周二 14:57写道： > > >>> > > Congratulations ! > > > > > > Best, > > Yuepeng Pan > > > > At 2024-06-04 14:45:45, "Xintong Song" > wrote: > > > Hi everyone, > > > > > > On behalf of the PMC, I'm very happy to announce that Weijie Guo > has > > joined > > > the Flink PMC! > > > > > > Weijie has been an active member of the Apache Flink community for > many > > > years. He has made significant contributions in many components, > > >>> including > > > runtime, shuffle, sdk, connectors, etc. He has driven / > participated in > > > many FLIPs, authored and reviewed hundreds of PRs, been > consistently > > active > > > on mailing lists, and also helped with release management of 1.20 > and > > > several other bugfix releases. > > > > > > Congratulations and welcome Weijie! > > > > > > Best, > > > > > > Xintong (on behalf of the Flink PMC) > > > > >>> > > >
Re: [DISCUSS] Add Flink CDC Channel to Apache Flink Slack Workspace
Thanks a lot Zhongqiang for the proposal (+1) and Leonard for creating the channel! Regards, Jeyhun On Tue, Jun 4, 2024 at 6:12 AM Leonard Xu wrote: > I’ve created flink-cdc channel in Apache Flink Slack Workspace via > https://issues.apache.org/jira/browse/FLINK-35514 < > https://issues.apache.org/jira/browse/FLINK-35514> > > Best, > Leonard > > > 2024年5月29日 下午9:53，Ahmed Hamdy 写道： > > > > Thanks Zhongqiang, +1 for sure. > > Best Regards > > Ahmed Hamdy > > > > > > On Wed, 29 May 2024 at 13:48, ConradJam wrote: > > > >> +1 best > >> > >> Hang Ruan 于2024年5月29日周三 11:28写道： > >> > >>> Hi, zhongqiang. > >>> > >>> Thanks for the proposal. +1 for it. > >>> > >>> Best, > >>> Hang > >>> > >>> Leonard Xu 于2024年5月28日周二 11:58写道： > >>> > > Thanks Zhongqiang for the proposal, we need the Channel and I should > >> have > been created it but not yet, +1 from my side. > > Best, > Leonard > > > 2024年5月28日 上午11:54，gongzhongqiang 写道： > > > > Hi devs, > > > > I would like to propose adding a dedicated Flink CDC channel to the > Apache > > Flink Slack workspace. > > > > Creating a channel focused on Flink CDC will help community members > easily > > find previous discussions > > and target new discussions and questions to the correct place. Flink > >>> CDC > is > > a sufficiently distinct component > > within the Apache Flink ecosystem, and having a dedicated channel > >> will > make > > it viable and useful for > > those specifically working with or interested in this technology. > > > > Looking forward to your feedback and support on this proposal. > > > > > > Best, > > Zhongqiang Gong > > > >>> > >> > >> > >> -- > >> Best > >> > >> ConradJam > >> > >
Re: Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie Guo
Congratulations Weijie! Well deserved. Regards, Jeyhun On Tue, Jun 4, 2024 at 3:50 PM Feng Jin wrote: > Congratulations, Weijie! > > > Best, > Feng Jin > > On Tue, Jun 4, 2024 at 9:03 PM Wencong Liu wrote: > > > Congratulations, Weijie! > > > > > > > > > > Best, > > > > Wencong > > > > > > > > > > At 2024-06-04 20:46:52, "Lijie Wang" wrote: > > >Congratulations, Weijie! > > > > > >Best, > > >Lijie > > > > > >Zakelly Lan 于2024年6月4日周二 20:45写道： > > > > > >> Congratulations, Weijie! > > >> > > >> Best, > > >> Zakelly > > >> > > >> On Tue, Jun 4, 2024 at 7:49 PM Sergey Nuyanzin > > >> wrote: > > >> > > >> > Congratulations Weijio Guo! > > >> > > > >> > On Tue, Jun 4, 2024, 13:45 Jark Wu wrote: > > >> > > > >> > > Congratulations, Weijie! > > >> > > > > >> > > Best, > > >> > > Jark > > >> > > > > >> > > On Tue, 4 Jun 2024 at 19:10, spoon_lz wrote: > > >> > > > > >> > > > Congratulations, Weijie! > > >> > > > > > >> > > > > > >> > > > > > >> > > > Regards, > > >> > > > Zhuo. > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > Replied Message > > >> > > > | From | Aleksandr Pilipenko | > > >> > > > | Date | 06/4/2024 18:59 | > > >> > > > | To | | > > >> > > > | Subject | Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie > > Guo | > > >> > > > Congratulations, Weijie! > > >> > > > > > >> > > > Best, > > >> > > > Aleksandr > > >> > > > > > >> > > > On Tue, 4 Jun 2024 at 11:42, Abdulquddus Babatunde Ekemode < > > >> > > > abdulqud...@aligence.io> wrote: > > >> > > > > > >> > > > Congratulations! I wish you all the best. > > >> > > > > > >> > > > Best Regards, > > >> > > > Abdulquddus > > >> > > > > > >> > > > On Tue, 4 Jun 2024 at 13:14, Ahmed Hamdy > > >> wrote: > > >> > > > > > >> > > > Congratulations Weijie > > >> > > > Best Regards > > >> > > > Ahmed Hamdy > > >> > > > > > >> > > > > > >> > > > On Tue, 4 Jun 2024 at 10:51, Matthias Pohl > > >> wrote: > > >> > > > > > >> > > > Congratulations, Weijie! > > >> > > > > > >> > > > Matthias > > >> > > > > > >> > > > On Tue, Jun 4, 2024 at 11:12 AM Guowei Ma > > > >> > > > wrote: > > >> > > > > > >> > > > Congratulations! > > >> > > > > > >> > > > Best, > > >> > > > Guowei > > >> > > > > > >> > > > > > >> > > > On Tue, Jun 4, 2024 at 4:55 PM gongzhongqiang < > > >> > > > gongzhongqi...@apache.org > > >> > > > > > >> > > > wrote: > > >> > > > > > >> > > > Congratulations Weijie! Best, > > >> > > > Zhongqiang Gong > > >> > > > > > >> > > > Xintong Song 于2024年6月4日周二 14:46写道： > > >> > > > > > >> > > > Hi everyone, > > >> > > > > > >> > > > On behalf of the PMC, I'm very happy to announce that Weijie Guo > > >> > > > has > > >> > > > joined > > >> > > > the Flink PMC! > > >> > > > > > >> > > > Weijie has been an active member of the Apache Flink community > > >> > > > for > > >> > > > many > > >> > > > years. He has made significant contributions in many components, > > >> > > > including > > >> > > > runtime, shuffle, sdk, connectors, etc. He has driven / > > >> > > > participated > > >> > > > in > > >> > > > many FLIPs, authored and reviewed hundreds of PRs, been > > >> > > > consistently > > >> > > > active > > >> > > > on mailing lists, and also helped with release management of > 1.20 > > >> > > > and > > >> > > > several other bugfix releases. > > >> > > > > > >> > > > Congratulations and welcome Weijie! > > >> > > > > > >> > > > Best, > > >> > > > > > >> > > > Xintong (on behalf of the Flink PMC) > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > >> > > > >> > > >
Re: Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie Guo
Congratulations, Weijie! Best, Feng Jin On Tue, Jun 4, 2024 at 9:03 PM Wencong Liu wrote: > Congratulations, Weijie! > > > > > Best, > > Wencong > > > > > At 2024-06-04 20:46:52, "Lijie Wang" wrote: > >Congratulations, Weijie! > > > >Best, > >Lijie > > > >Zakelly Lan 于2024年6月4日周二 20:45写道： > > > >> Congratulations, Weijie! > >> > >> Best, > >> Zakelly > >> > >> On Tue, Jun 4, 2024 at 7:49 PM Sergey Nuyanzin > >> wrote: > >> > >> > Congratulations Weijio Guo! > >> > > >> > On Tue, Jun 4, 2024, 13:45 Jark Wu wrote: > >> > > >> > > Congratulations, Weijie! > >> > > > >> > > Best, > >> > > Jark > >> > > > >> > > On Tue, 4 Jun 2024 at 19:10, spoon_lz wrote: > >> > > > >> > > > Congratulations, Weijie! > >> > > > > >> > > > > >> > > > > >> > > > Regards, > >> > > > Zhuo. > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > Replied Message > >> > > > | From | Aleksandr Pilipenko | > >> > > > | Date | 06/4/2024 18:59 | > >> > > > | To | | > >> > > > | Subject | Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie > Guo | > >> > > > Congratulations, Weijie! > >> > > > > >> > > > Best, > >> > > > Aleksandr > >> > > > > >> > > > On Tue, 4 Jun 2024 at 11:42, Abdulquddus Babatunde Ekemode < > >> > > > abdulqud...@aligence.io> wrote: > >> > > > > >> > > > Congratulations! I wish you all the best. > >> > > > > >> > > > Best Regards, > >> > > > Abdulquddus > >> > > > > >> > > > On Tue, 4 Jun 2024 at 13:14, Ahmed Hamdy > >> wrote: > >> > > > > >> > > > Congratulations Weijie > >> > > > Best Regards > >> > > > Ahmed Hamdy > >> > > > > >> > > > > >> > > > On Tue, 4 Jun 2024 at 10:51, Matthias Pohl > >> wrote: > >> > > > > >> > > > Congratulations, Weijie! > >> > > > > >> > > > Matthias > >> > > > > >> > > > On Tue, Jun 4, 2024 at 11:12 AM Guowei Ma > >> > > > wrote: > >> > > > > >> > > > Congratulations! > >> > > > > >> > > > Best, > >> > > > Guowei > >> > > > > >> > > > > >> > > > On Tue, Jun 4, 2024 at 4:55 PM gongzhongqiang < > >> > > > gongzhongqi...@apache.org > >> > > > > >> > > > wrote: > >> > > > > >> > > > Congratulations Weijie! Best, > >> > > > Zhongqiang Gong > >> > > > > >> > > > Xintong Song 于2024年6月4日周二 14:46写道： > >> > > > > >> > > > Hi everyone, > >> > > > > >> > > > On behalf of the PMC, I'm very happy to announce that Weijie Guo > >> > > > has > >> > > > joined > >> > > > the Flink PMC! > >> > > > > >> > > > Weijie has been an active member of the Apache Flink community > >> > > > for > >> > > > many > >> > > > years. He has made significant contributions in many components, > >> > > > including > >> > > > runtime, shuffle, sdk, connectors, etc. He has driven / > >> > > > participated > >> > > > in > >> > > > many FLIPs, authored and reviewed hundreds of PRs, been > >> > > > consistently > >> > > > active > >> > > > on mailing lists, and also helped with release management of 1.20 > >> > > > and > >> > > > several other bugfix releases. > >> > > > > >> > > > Congratulations and welcome Weijie! > >> > > > > >> > > > Best, > >> > > > > >> > > > Xintong (on behalf of the Flink PMC) > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> > > > >> > > >> >
Re:Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie Guo
Congratulations, Weijie! Best, Wencong At 2024-06-04 20:46:52, "Lijie Wang" wrote: >Congratulations, Weijie! > >Best, >Lijie > >Zakelly Lan 于2024年6月4日周二 20:45写道： > >> Congratulations, Weijie! >> >> Best, >> Zakelly >> >> On Tue, Jun 4, 2024 at 7:49 PM Sergey Nuyanzin >> wrote: >> >> > Congratulations Weijio Guo! >> > >> > On Tue, Jun 4, 2024, 13:45 Jark Wu wrote: >> > >> > > Congratulations, Weijie! >> > > >> > > Best, >> > > Jark >> > > >> > > On Tue, 4 Jun 2024 at 19:10, spoon_lz wrote: >> > > >> > > > Congratulations, Weijie! >> > > > >> > > > >> > > > >> > > > Regards, >> > > > Zhuo. >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > Replied Message >> > > > | From | Aleksandr Pilipenko | >> > > > | Date | 06/4/2024 18:59 | >> > > > | To | | >> > > > | Subject | Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie Guo | >> > > > Congratulations, Weijie! >> > > > >> > > > Best, >> > > > Aleksandr >> > > > >> > > > On Tue, 4 Jun 2024 at 11:42, Abdulquddus Babatunde Ekemode < >> > > > abdulqud...@aligence.io> wrote: >> > > > >> > > > Congratulations! I wish you all the best. >> > > > >> > > > Best Regards, >> > > > Abdulquddus >> > > > >> > > > On Tue, 4 Jun 2024 at 13:14, Ahmed Hamdy >> wrote: >> > > > >> > > > Congratulations Weijie >> > > > Best Regards >> > > > Ahmed Hamdy >> > > > >> > > > >> > > > On Tue, 4 Jun 2024 at 10:51, Matthias Pohl >> wrote: >> > > > >> > > > Congratulations, Weijie! >> > > > >> > > > Matthias >> > > > >> > > > On Tue, Jun 4, 2024 at 11:12 AM Guowei Ma >> > > > wrote: >> > > > >> > > > Congratulations! >> > > > >> > > > Best, >> > > > Guowei >> > > > >> > > > >> > > > On Tue, Jun 4, 2024 at 4:55 PM gongzhongqiang < >> > > > gongzhongqi...@apache.org >> > > > >> > > > wrote: >> > > > >> > > > Congratulations Weijie! Best, >> > > > Zhongqiang Gong >> > > > >> > > > Xintong Song 于2024年6月4日周二 14:46写道： >> > > > >> > > > Hi everyone, >> > > > >> > > > On behalf of the PMC, I'm very happy to announce that Weijie Guo >> > > > has >> > > > joined >> > > > the Flink PMC! >> > > > >> > > > Weijie has been an active member of the Apache Flink community >> > > > for >> > > > many >> > > > years. He has made significant contributions in many components, >> > > > including >> > > > runtime, shuffle, sdk, connectors, etc. He has driven / >> > > > participated >> > > > in >> > > > many FLIPs, authored and reviewed hundreds of PRs, been >> > > > consistently >> > > > active >> > > > on mailing lists, and also helped with release management of 1.20 >> > > > and >> > > > several other bugfix releases. >> > > > >> > > > Congratulations and welcome Weijie! >> > > > >> > > > Best, >> > > > >> > > > Xintong (on behalf of the Flink PMC) >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > >> > >>
Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie Guo
Congratulations, Weijie! Best, Lijie Zakelly Lan 于2024年6月4日周二 20:45写道： > Congratulations, Weijie! > > Best, > Zakelly > > On Tue, Jun 4, 2024 at 7:49 PM Sergey Nuyanzin > wrote: > > > Congratulations Weijio Guo! > > > > On Tue, Jun 4, 2024, 13:45 Jark Wu wrote: > > > > > Congratulations, Weijie! > > > > > > Best, > > > Jark > > > > > > On Tue, 4 Jun 2024 at 19:10, spoon_lz wrote: > > > > > > > Congratulations, Weijie! > > > > > > > > > > > > > > > > Regards, > > > > Zhuo. > > > > > > > > > > > > > > > > > > > > > > > > Replied Message > > > > | From | Aleksandr Pilipenko | > > > > | Date | 06/4/2024 18:59 | > > > > | To | | > > > > | Subject | Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie Guo | > > > > Congratulations, Weijie! > > > > > > > > Best, > > > > Aleksandr > > > > > > > > On Tue, 4 Jun 2024 at 11:42, Abdulquddus Babatunde Ekemode < > > > > abdulqud...@aligence.io> wrote: > > > > > > > > Congratulations! I wish you all the best. > > > > > > > > Best Regards, > > > > Abdulquddus > > > > > > > > On Tue, 4 Jun 2024 at 13:14, Ahmed Hamdy > wrote: > > > > > > > > Congratulations Weijie > > > > Best Regards > > > > Ahmed Hamdy > > > > > > > > > > > > On Tue, 4 Jun 2024 at 10:51, Matthias Pohl > wrote: > > > > > > > > Congratulations, Weijie! > > > > > > > > Matthias > > > > > > > > On Tue, Jun 4, 2024 at 11:12 AM Guowei Ma > > > > wrote: > > > > > > > > Congratulations! > > > > > > > > Best, > > > > Guowei > > > > > > > > > > > > On Tue, Jun 4, 2024 at 4:55 PM gongzhongqiang < > > > > gongzhongqi...@apache.org > > > > > > > > wrote: > > > > > > > > Congratulations Weijie! Best, > > > > Zhongqiang Gong > > > > > > > > Xintong Song 于2024年6月4日周二 14:46写道： > > > > > > > > Hi everyone, > > > > > > > > On behalf of the PMC, I'm very happy to announce that Weijie Guo > > > > has > > > > joined > > > > the Flink PMC! > > > > > > > > Weijie has been an active member of the Apache Flink community > > > > for > > > > many > > > > years. He has made significant contributions in many components, > > > > including > > > > runtime, shuffle, sdk, connectors, etc. He has driven / > > > > participated > > > > in > > > > many FLIPs, authored and reviewed hundreds of PRs, been > > > > consistently > > > > active > > > > on mailing lists, and also helped with release management of 1.20 > > > > and > > > > several other bugfix releases. > > > > > > > > Congratulations and welcome Weijie! > > > > > > > > Best, > > > > > > > > Xintong (on behalf of the Flink PMC) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie Guo
Congratulations, Weijie! Best, Zakelly On Tue, Jun 4, 2024 at 7:49 PM Sergey Nuyanzin wrote: > Congratulations Weijio Guo! > > On Tue, Jun 4, 2024, 13:45 Jark Wu wrote: > > > Congratulations, Weijie! > > > > Best, > > Jark > > > > On Tue, 4 Jun 2024 at 19:10, spoon_lz wrote: > > > > > Congratulations, Weijie! > > > > > > > > > > > > Regards, > > > Zhuo. > > > > > > > > > > > > > > > > > > Replied Message > > > | From | Aleksandr Pilipenko | > > > | Date | 06/4/2024 18:59 | > > > | To | | > > > | Subject | Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie Guo | > > > Congratulations, Weijie! > > > > > > Best, > > > Aleksandr > > > > > > On Tue, 4 Jun 2024 at 11:42, Abdulquddus Babatunde Ekemode < > > > abdulqud...@aligence.io> wrote: > > > > > > Congratulations! I wish you all the best. > > > > > > Best Regards, > > > Abdulquddus > > > > > > On Tue, 4 Jun 2024 at 13:14, Ahmed Hamdy wrote: > > > > > > Congratulations Weijie > > > Best Regards > > > Ahmed Hamdy > > > > > > > > > On Tue, 4 Jun 2024 at 10:51, Matthias Pohl wrote: > > > > > > Congratulations, Weijie! > > > > > > Matthias > > > > > > On Tue, Jun 4, 2024 at 11:12 AM Guowei Ma > > > wrote: > > > > > > Congratulations! > > > > > > Best, > > > Guowei > > > > > > > > > On Tue, Jun 4, 2024 at 4:55 PM gongzhongqiang < > > > gongzhongqi...@apache.org > > > > > > wrote: > > > > > > Congratulations Weijie! Best, > > > Zhongqiang Gong > > > > > > Xintong Song 于2024年6月4日周二 14:46写道： > > > > > > Hi everyone, > > > > > > On behalf of the PMC, I'm very happy to announce that Weijie Guo > > > has > > > joined > > > the Flink PMC! > > > > > > Weijie has been an active member of the Apache Flink community > > > for > > > many > > > years. He has made significant contributions in many components, > > > including > > > runtime, shuffle, sdk, connectors, etc. He has driven / > > > participated > > > in > > > many FLIPs, authored and reviewed hundreds of PRs, been > > > consistently > > > active > > > on mailing lists, and also helped with release management of 1.20 > > > and > > > several other bugfix releases. > > > > > > Congratulations and welcome Weijie! > > > > > > Best, > > > > > > Xintong (on behalf of the Flink PMC) > > > > > > > > > > > > > > > > > > > > > > > >
Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie Guo
Congratulations Weijio Guo! On Tue, Jun 4, 2024, 13:45 Jark Wu wrote: > Congratulations, Weijie! > > Best, > Jark > > On Tue, 4 Jun 2024 at 19:10, spoon_lz wrote: > > > Congratulations, Weijie! > > > > > > > > Regards, > > Zhuo. > > > > > > > > > > > > Replied Message > > | From | Aleksandr Pilipenko | > > | Date | 06/4/2024 18:59 | > > | To | | > > | Subject | Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie Guo | > > Congratulations, Weijie! > > > > Best, > > Aleksandr > > > > On Tue, 4 Jun 2024 at 11:42, Abdulquddus Babatunde Ekemode < > > abdulqud...@aligence.io> wrote: > > > > Congratulations! I wish you all the best. > > > > Best Regards, > > Abdulquddus > > > > On Tue, 4 Jun 2024 at 13:14, Ahmed Hamdy wrote: > > > > Congratulations Weijie > > Best Regards > > Ahmed Hamdy > > > > > > On Tue, 4 Jun 2024 at 10:51, Matthias Pohl wrote: > > > > Congratulations, Weijie! > > > > Matthias > > > > On Tue, Jun 4, 2024 at 11:12 AM Guowei Ma > > wrote: > > > > Congratulations! > > > > Best, > > Guowei > > > > > > On Tue, Jun 4, 2024 at 4:55 PM gongzhongqiang < > > gongzhongqi...@apache.org > > > > wrote: > > > > Congratulations Weijie! Best, > > Zhongqiang Gong > > > > Xintong Song 于2024年6月4日周二 14:46写道： > > > > Hi everyone, > > > > On behalf of the PMC, I'm very happy to announce that Weijie Guo > > has > > joined > > the Flink PMC! > > > > Weijie has been an active member of the Apache Flink community > > for > > many > > years. He has made significant contributions in many components, > > including > > runtime, shuffle, sdk, connectors, etc. He has driven / > > participated > > in > > many FLIPs, authored and reviewed hundreds of PRs, been > > consistently > > active > > on mailing lists, and also helped with release management of 1.20 > > and > > several other bugfix releases. > > > > Congratulations and welcome Weijie! > > > > Best, > > > > Xintong (on behalf of the Flink PMC) > > > > > > > > > > > > > > >
Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie Guo
Congratulations, Weijie! Best, Jark On Tue, 4 Jun 2024 at 19:10, spoon_lz wrote: > Congratulations, Weijie! > > > > Regards, > Zhuo. > > > > > > Replied Message > | From | Aleksandr Pilipenko | > | Date | 06/4/2024 18:59 | > | To | | > | Subject | Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie Guo | > Congratulations, Weijie! > > Best, > Aleksandr > > On Tue, 4 Jun 2024 at 11:42, Abdulquddus Babatunde Ekemode < > abdulqud...@aligence.io> wrote: > > Congratulations! I wish you all the best. > > Best Regards, > Abdulquddus > > On Tue, 4 Jun 2024 at 13:14, Ahmed Hamdy wrote: > > Congratulations Weijie > Best Regards > Ahmed Hamdy > > > On Tue, 4 Jun 2024 at 10:51, Matthias Pohl wrote: > > Congratulations, Weijie! > > Matthias > > On Tue, Jun 4, 2024 at 11:12 AM Guowei Ma > wrote: > > Congratulations! > > Best, > Guowei > > > On Tue, Jun 4, 2024 at 4:55 PM gongzhongqiang < > gongzhongqi...@apache.org > > wrote: > > Congratulations Weijie! Best, > Zhongqiang Gong > > Xintong Song 于2024年6月4日周二 14:46写道： > > Hi everyone, > > On behalf of the PMC, I'm very happy to announce that Weijie Guo > has > joined > the Flink PMC! > > Weijie has been an active member of the Apache Flink community > for > many > years. He has made significant contributions in many components, > including > runtime, shuffle, sdk, connectors, etc. He has driven / > participated > in > many FLIPs, authored and reviewed hundreds of PRs, been > consistently > active > on mailing lists, and also helped with release management of 1.20 > and > several other bugfix releases. > > Congratulations and welcome Weijie! > > Best, > > Xintong (on behalf of the Flink PMC) > > > > > > >
Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie Guo
Congratulations, Weijie! Regards, Zhuo. Replied Message | From | Aleksandr Pilipenko | | Date | 06/4/2024 18:59 | | To | | | Subject | Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie Guo | Congratulations, Weijie! Best, Aleksandr On Tue, 4 Jun 2024 at 11:42, Abdulquddus Babatunde Ekemode < abdulqud...@aligence.io> wrote: Congratulations! I wish you all the best. Best Regards, Abdulquddus On Tue, 4 Jun 2024 at 13:14, Ahmed Hamdy wrote: Congratulations Weijie Best Regards Ahmed Hamdy On Tue, 4 Jun 2024 at 10:51, Matthias Pohl wrote: Congratulations, Weijie! Matthias On Tue, Jun 4, 2024 at 11:12 AM Guowei Ma wrote: Congratulations! Best, Guowei On Tue, Jun 4, 2024 at 4:55 PM gongzhongqiang < gongzhongqi...@apache.org wrote: Congratulations Weijie! Best, Zhongqiang Gong Xintong Song 于2024年6月4日周二 14:46写道： Hi everyone, On behalf of the PMC, I'm very happy to announce that Weijie Guo has joined the Flink PMC! Weijie has been an active member of the Apache Flink community for many years. He has made significant contributions in many components, including runtime, shuffle, sdk, connectors, etc. He has driven / participated in many FLIPs, authored and reviewed hundreds of PRs, been consistently active on mailing lists, and also helped with release management of 1.20 and several other bugfix releases. Congratulations and welcome Weijie! Best, Xintong (on behalf of the Flink PMC)
Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie Guo
Congratulations, Weijie! Best, Aleksandr On Tue, 4 Jun 2024 at 11:42, Abdulquddus Babatunde Ekemode < abdulqud...@aligence.io> wrote: > Congratulations! I wish you all the best. > > Best Regards, > Abdulquddus > > On Tue, 4 Jun 2024 at 13:14, Ahmed Hamdy wrote: > > > Congratulations Weijie > > Best Regards > > Ahmed Hamdy > > > > > > On Tue, 4 Jun 2024 at 10:51, Matthias Pohl wrote: > > > > > Congratulations, Weijie! > > > > > > Matthias > > > > > > On Tue, Jun 4, 2024 at 11:12 AM Guowei Ma > wrote: > > > > > > > Congratulations! > > > > > > > > Best, > > > > Guowei > > > > > > > > > > > > On Tue, Jun 4, 2024 at 4:55 PM gongzhongqiang < > > gongzhongqi...@apache.org > > > > > > > > wrote: > > > > > > > > > Congratulations Weijie! Best, > > > > > Zhongqiang Gong > > > > > > > > > > Xintong Song 于2024年6月4日周二 14:46写道： > > > > > > > > > > > Hi everyone, > > > > > > > > > > > > On behalf of the PMC, I'm very happy to announce that Weijie Guo > > has > > > > > joined > > > > > > the Flink PMC! > > > > > > > > > > > > Weijie has been an active member of the Apache Flink community > for > > > many > > > > > > years. He has made significant contributions in many components, > > > > > including > > > > > > runtime, shuffle, sdk, connectors, etc. He has driven / > > participated > > > in > > > > > > many FLIPs, authored and reviewed hundreds of PRs, been > > consistently > > > > > active > > > > > > on mailing lists, and also helped with release management of 1.20 > > and > > > > > > several other bugfix releases. > > > > > > > > > > > > Congratulations and welcome Weijie! > > > > > > > > > > > > Best, > > > > > > > > > > > > Xintong (on behalf of the Flink PMC) > > > > > > > > > > > > > > > > > > > > >
Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie Guo
Congratulations! I wish you all the best. Best Regards, Abdulquddus On Tue, 4 Jun 2024 at 13:14, Ahmed Hamdy wrote: > Congratulations Weijie > Best Regards > Ahmed Hamdy > > > On Tue, 4 Jun 2024 at 10:51, Matthias Pohl wrote: > > > Congratulations, Weijie! > > > > Matthias > > > > On Tue, Jun 4, 2024 at 11:12 AM Guowei Ma wrote: > > > > > Congratulations! > > > > > > Best, > > > Guowei > > > > > > > > > On Tue, Jun 4, 2024 at 4:55 PM gongzhongqiang < > gongzhongqi...@apache.org > > > > > > wrote: > > > > > > > Congratulations Weijie! Best, > > > > Zhongqiang Gong > > > > > > > > Xintong Song 于2024年6月4日周二 14:46写道： > > > > > > > > > Hi everyone, > > > > > > > > > > On behalf of the PMC, I'm very happy to announce that Weijie Guo > has > > > > joined > > > > > the Flink PMC! > > > > > > > > > > Weijie has been an active member of the Apache Flink community for > > many > > > > > years. He has made significant contributions in many components, > > > > including > > > > > runtime, shuffle, sdk, connectors, etc. He has driven / > participated > > in > > > > > many FLIPs, authored and reviewed hundreds of PRs, been > consistently > > > > active > > > > > on mailing lists, and also helped with release management of 1.20 > and > > > > > several other bugfix releases. > > > > > > > > > > Congratulations and welcome Weijie! > > > > > > > > > > Best, > > > > > > > > > > Xintong (on behalf of the Flink PMC) > > > > > > > > > > > > > > >
Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie Guo
Congratulations Weijie Best Regards Ahmed Hamdy On Tue, 4 Jun 2024 at 10:51, Matthias Pohl wrote: > Congratulations, Weijie! > > Matthias > > On Tue, Jun 4, 2024 at 11:12 AM Guowei Ma wrote: > > > Congratulations! > > > > Best, > > Guowei > > > > > > On Tue, Jun 4, 2024 at 4:55 PM gongzhongqiang > > > wrote: > > > > > Congratulations Weijie! Best, > > > Zhongqiang Gong > > > > > > Xintong Song 于2024年6月4日周二 14:46写道： > > > > > > > Hi everyone, > > > > > > > > On behalf of the PMC, I'm very happy to announce that Weijie Guo has > > > joined > > > > the Flink PMC! > > > > > > > > Weijie has been an active member of the Apache Flink community for > many > > > > years. He has made significant contributions in many components, > > > including > > > > runtime, shuffle, sdk, connectors, etc. He has driven / participated > in > > > > many FLIPs, authored and reviewed hundreds of PRs, been consistently > > > active > > > > on mailing lists, and also helped with release management of 1.20 and > > > > several other bugfix releases. > > > > > > > > Congratulations and welcome Weijie! > > > > > > > > Best, > > > > > > > > Xintong (on behalf of the Flink PMC) > > > > > > > > > >
[DISCUSS] FLIP-461: FLIP-461: Synchronize rescaling with checkpoint creation to minimize reprocessing
Hi everyone, I'd like to discuss FLIP-461 [1]. The FLIP proposes the synchronization of rescaling and the completion of checkpoints. The idea is to reduce the amount of data that needs to be processed after rescaling happened. A more detailed motivation can be found in FLIP-461. I'm looking forward to feedback and suggestions. Best, Matthias [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-461%3A+Synchronize+rescaling+with+checkpoint+creation+to+minimize+reprocessing
Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie Guo
Congratulations, Weijie! Matthias On Tue, Jun 4, 2024 at 11:12 AM Guowei Ma wrote: > Congratulations! > > Best, > Guowei > > > On Tue, Jun 4, 2024 at 4:55 PM gongzhongqiang > wrote: > > > Congratulations Weijie! Best, > > Zhongqiang Gong > > > > Xintong Song 于2024年6月4日周二 14:46写道： > > > > > Hi everyone, > > > > > > On behalf of the PMC, I'm very happy to announce that Weijie Guo has > > joined > > > the Flink PMC! > > > > > > Weijie has been an active member of the Apache Flink community for many > > > years. He has made significant contributions in many components, > > including > > > runtime, shuffle, sdk, connectors, etc. He has driven / participated in > > > many FLIPs, authored and reviewed hundreds of PRs, been consistently > > active > > > on mailing lists, and also helped with release management of 1.20 and > > > several other bugfix releases. > > > > > > Congratulations and welcome Weijie! > > > > > > Best, > > > > > > Xintong (on behalf of the Flink PMC) > > > > > >
[jira] [Created] (FLINK-35520) Nightly build can't compile as problems were detected from NoticeFileChecker
Weijie Guo created FLINK-35520: -- Summary: Nightly build can't compile as problems were detected from NoticeFileChecker Key: FLINK-35520 URL: https://issues.apache.org/jira/browse/FLINK-35520 Project: Flink Issue Type: Bug Components: Build System / CI Affects Versions: 1.20.0 Reporter: Weijie Guo -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-35519) Flink Job fails with SingleValueAggFunction received more than one element
Dawid Wysakowicz created FLINK-35519:
Summary: Flink Job fails with SingleValueAggFunction received more
than one element
Key: FLINK-35519
URL: https://issues.apache.org/jira/browse/FLINK-35519
Project: Flink
Issue Type: Bug
Components: Table SQL / Planner
Affects Versions: 1.19.0
Reporter: Dawid Wysakowicz
When running a query:
{code}
select
(SELECT
t.id
FROM raw_pagerduty_users, UNNEST(teams) AS t(id, type, summary, self,
html_url))
from raw_pagerduty_users;
{code}
it is translated to:
{code}
Sink(table=[default_catalog.default_database.sink], fields=[EXPR$0])
+- Calc(select=[$f0 AS EXPR$0])
+- Join(joinType=[LeftOuterJoin], where=[true], select=[c, $f0],
leftInputSpec=[NoUniqueKey], rightInputSpec=[JoinKeyContainsUniqueKey])
:- Exchange(distribution=[single])
: +- Calc(select=[c])
: +- TableSourceScan(table=[[default_catalog, default_database,
raw_pagerduty_users, project=[c, teams], metadata=[]]], fields=[c,
teams])(reuse_id=[1])
+- Exchange(distribution=[single])
+- GroupAggregate(select=[SINGLE_VALUE(id) AS $f0])
+- Exchange(distribution=[single])
+- Calc(select=[id])
+- Correlate(invocation=[$UNNEST_ROWS$1($cor0.teams)],
correlate=[table($UNNEST_ROWS$1($cor0.teams))],
select=[c,teams,id,type,summary,self,html_url], rowType=[RecordType(BIGINT c,
RecordType:peek_no_expand(VARCHAR(2147483647) id, VARCHAR(2147483647) type,
VARCHAR(2147483647) summary, VARCHAR(2147483647) self, VARCHAR(2147483647)
html_url) ARRAY teams, VARCHAR(2147483647) id, VARCHAR(2147483647) type,
VARCHAR(2147483647) summary, VARCHAR(2147483647) self, VARCHAR(2147483647)
html_url)], joinType=[INNER])
+- Reused(reference_id=[1])
{code}
and it fails with:
{code}
java.lang.RuntimeException: SingleValueAggFunction received more than one
element.
at GroupAggsHandler$150.accumulate(Unknown Source)
at
org.apache.flink.table.runtime.operators.aggregate.GroupAggFunction.processElement(GroupAggFunction.java:151)
at
org.apache.flink.table.runtime.operators.aggregate.GroupAggFunction.processElement(GroupAggFunction.java:43)
at
org.apache.flink.streaming.api.operators.KeyedProcessOperator.processElement(KeyedProcessOperator.java:83)
at
org.apache.flink.streaming.runtime.io.RecordProcessorUtils.lambda$getRecordProcessor$0(RecordProcessorUtils.java:60)
at
org.apache.flink.streaming.runtime.tasks.OneInputStreamTask$StreamTaskNetworkOutput.emitRecord(OneInputStreamTask.java:237)
at
org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.processElement(AbstractStreamTaskNetworkInput.java:146)
at
org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.emitNext(AbstractStreamTaskNetworkInput.java:110)
at
org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:571)
at
org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:900)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:849)
at
org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:953)
at
org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:932)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:746)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:562)
at java.base/java.lang.Thread.run(Thread.java:829)
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[June 15 Feature Freeze][SUMMARY] Flink 1.20 Release Sync 06/04/2024
Dear devs, This is the fifth meeting for Flink 1.20 release cycle. I'd like to share the information synced in the meeting. - Feature Freeze It is worth noting that there are only 2 weeks left until the feature freeze time(June 15, 2024), and developers need to pay attention to the feature freeze time. - Features: So far we've had 13 flips/features, there are two FLIPs that unlikely to be done in 1.20, and the status of the other is good. It is encouraged to continuously updating the 1.20 wiki page[1] for contributors. - Blockers: [open] FLINK-35423 - ARRAY_EXCEPT should support set semantics - We need someone familiar with SQL part to review the pull request attached. By the way, we have closed two performance regression blockers(FLINK-35040 and FLINK-35215), thanks to everyone involved! - Notice CI pipeline triggered by pull request seems sort of unstable, sometimes it doesn't trigger properly. I have create a ticket to track it: https://issues.apache.org/jira/browse/FLINK-35517 - Sync meeting[2]: The next meeting is 06/11/2024 10am (UTC+2) and 4pm (UTC+8), please feel free to join us. Lastly, we encourage attendees to fill out the topics to be discussed at the bottom of 1.20 wiki page[1] a day in advance, to make it easier for everyone to understand the background of the topics, thanks! [1] https://cwiki.apache.org/confluence/display/FLINK/1.20+Release [2] https://meet.google.com/mtj-huez-apu Best, Robert, Rui, Ufuk, Weijie
Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie Guo
Congratulations! Best, Guowei On Tue, Jun 4, 2024 at 4:55 PM gongzhongqiang wrote: > Congratulations Weijie! Best, > Zhongqiang Gong > > Xintong Song 于2024年6月4日周二 14:46写道： > > > Hi everyone, > > > > On behalf of the PMC, I'm very happy to announce that Weijie Guo has > joined > > the Flink PMC! > > > > Weijie has been an active member of the Apache Flink community for many > > years. He has made significant contributions in many components, > including > > runtime, shuffle, sdk, connectors, etc. He has driven / participated in > > many FLIPs, authored and reviewed hundreds of PRs, been consistently > active > > on mailing lists, and also helped with release management of 1.20 and > > several other bugfix releases. > > > > Congratulations and welcome Weijie! > > > > Best, > > > > Xintong (on behalf of the Flink PMC) > > >
[jira] [Created] (FLINK-35518) CI Bot doesn't run on PRs
Piotr Nowojski created FLINK-35518: -- Summary: CI Bot doesn't run on PRs Key: FLINK-35518 URL: https://issues.apache.org/jira/browse/FLINK-35518 Project: Flink Issue Type: Bug Components: Build System / CI Affects Versions: 1.20.0 Reporter: Piotr Nowojski Doesn't want to run on my PR/branch. I was doing force-pushes, rebases, asking flink bot to run, closed and opened new PR, but nothing helped https://github.com/apache/flink/pull/24868 https://github.com/apache/flink/pull/24883 I've heard others were having similar problems recently. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [VOTE] Release flink-connector-kafka v3.2.0, release candidate #1
+1 (non-binding) - Verified signature - Verified checksum - Built release tag from source with JDK 11 - approved docs PR Best, Fabian On Wed, May 22, 2024 at 2:22 PM Leonard Xu wrote: > +1 (binding) > > - verified signatures > - verified hashsums > - built from source code with java 1.8 succeeded > - checked Github release tag > - reviewed the web PR > - checked the CI result, > minor: the link [7] you post should be [1] > - checked release notes, > minor: the issue FLINK-34961[2] should move to next version > > > Best, > Leonard > > [1] > https://github.com/apache/flink-connector-kafka/actions/runs/8785158288 > [2] https://issues.apache.org/jira/browse/FLINK-34961 > > > > 2024年4月29日 上午12:34，Aleksandr Pilipenko 写道： > > > > +1 (non-binding) > > > > - Validated checksum > > - Verified signature > > - Checked that no binaries exist in the source archive > > - Build source > > - Verified web PR > > > > Thanks, > > Aleksandr > > > > On Sun, 28 Apr 2024 at 11:35, Hang Ruan wrote: > > > >> +1 (non-binding) > >> > >> - Validated checksum hash > >> - Verified signature > >> - Verified that no binaries exist in the source archive > >> - Build the source with Maven and jdk8 > >> - Verified web PR > >> - Check that the jar is built by jdk8 > >> > >> Best, > >> Hang > >> > >> Ahmed Hamdy 于2024年4月24日周三 17:21写道： > >> > >>> Thanks Danny, > >>> +1 (non-binding) > >>> > >>> - Verified Checksums and hashes > >>> - Verified Signatures > >>> - Reviewed web PR > >>> - github tag exists > >>> - Build source > >>> > >>> > >>> Best Regards > >>> Ahmed Hamdy > >>> > >>> > >>> On Tue, 23 Apr 2024 at 03:47, Muhammet Orazov > >>> > >>> wrote: > >>> > Thanks Danny, +1 (non-binding) > > - Checked 512 hash > - Checked gpg signature > - Reviewed pr > - Built the source with JDK 11 & 8 > > Best, > Muhammet > > On 2024-04-22 13:55, Danny Cranmer wrote: > > Hi everyone, > > > > Please review and vote on release candidate #1 for > > flink-connector-kafka > > v3.2.0, as follows: > > [ ] +1, Approve the release > > [ ] -1, Do not approve the release (please provide specific comments) > > > > This release supports Flink 1.18 and 1.19. > > > > The complete staging area is available for your review, which > >> includes: > > * JIRA release notes [1], > > * the official Apache source release to be deployed to > >> dist.apache.org > > [2], > > which are signed with the key with fingerprint 125FD8DB [3], > > * all artifacts to be deployed to the Maven Central Repository [4], > > * source code tag v3.2.0-rc1 [5], > > * website pull request listing the new release [6]. > > * CI build of the tag [7]. > > > > The vote will be open for at least 72 hours. It is adopted by > >> majority > > approval, with at least 3 PMC affirmative votes. > > > > Thanks, > > Danny > > > > [1] > > > > >>> > >> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12354209 > > [2] > > > > >>> > >> > https://dist.apache.org/repos/dist/dev/flink/flink-connector-kafka-3.2.0-rc1 > > [3] https://dist.apache.org/repos/dist/release/flink/KEYS > > [4] > > > >> https://repository.apache.org/content/repositories/orgapacheflink-1723 > > [5] > > > >>> > https://github.com/apache/flink-connector-kafka/releases/tag/v3.2.0-rc1 > > [6] https://github.com/apache/flink-web/pull/738 > > [7] https://github.com/apache/flink-connector-kafka > > >>> > >> > >
Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie Guo
Congratulations Weijie! Best, Zhongqiang Gong Xintong Song 于2024年6月4日周二 14:46写道： > Hi everyone, > > On behalf of the PMC, I'm very happy to announce that Weijie Guo has joined > the Flink PMC! > > Weijie has been an active member of the Apache Flink community for many > years. He has made significant contributions in many components, including > runtime, shuffle, sdk, connectors, etc. He has driven / participated in > many FLIPs, authored and reviewed hundreds of PRs, been consistently active > on mailing lists, and also helped with release management of 1.20 and > several other bugfix releases. > > Congratulations and welcome Weijie! > > Best, > > Xintong (on behalf of the Flink PMC) >
Re: [DISCUSS] FLIP-459: Support Flink hybrid shuffle integration with Apache Celeborn
+1 for this proposal. After internally reviewing the prototype of CIP-6, this would improve performance and stability for Flink users using Celeborn. Expect to see this feature come out to the community. As I come from the Celeborn community, I hope more users can try to use Celeborn when there are Flink batch jobs. Thanks, Ethan Feng Yuxin Tan 于2024年6月4日周二 16:34写道： > > Hi, Venkatakrishnan, > > Thanks for joining the discussion. We appreciate your interest > in contributing to the work. Once the FLIP and CIP proposals > have been approved, we will create some JIRA tickets in Flink > and Celeborn projects. Please feel free to take a look at the > tickets and select any that resonate with your interests. > > Best, > Yuxin > > > Venkatakrishnan Sowrirajan 于2024年5月31日周五 23:11写道： > > > Thanks for this FLIP. We are also interested in learning/contributing to > > the hybrid shuffle integration with celeborn for batch executions. > > > > On Tue, May 28, 2024, 7:07 PM Yuxin Tan wrote: > > > > > Hi, Xintong, > > > > > > > I think we can also publish the prototype codes so the > > > community can better understand and help with it. > > > > > > Ok, I agree on the point. I will prepare and publish the code > > > recently. > > > > > > Rui, > > > > > > > Kindly reminder: the image of CIP-6[1] cannot be loaded. > > > > > > Thanks for the reminder. I've updated the images. > > > > > > > > > Best, > > > Yuxin > > > > > > > > > Rui Fan <1996fan...@gmail.com> 于2024年5月29日周三 09:33写道： > > > > > > > Thanks Yuxin for driving this proposal! > > > > > > > > Kindly reminder: the image of CIP-6[1] cannot be loaded. > > > > > > > > [1] > > > > > > > > > > > > > https://urldefense.com/v3/__https://cwiki.apache.org/confluence/display/CELEBORN/CIP-6*Support*Flink*hybrid*shuffle*integration*with*Apache*Celeborn__;KysrKysrKys!!IKRxdwAv5BmarQ!ZRTc1aUSYMDBazuIwlet1Dzk2_DD9qKTgoDLH9jSwAVLgwplcuId_8JoXkH0i7AeWxKWXkL0sxM3AeW-H9OJ6v9uGw$ > > > > > > > > Best, > > > > Rui > > > > > > > > On Wed, May 29, 2024 at 9:03 AM Xintong Song > > > > wrote: > > > > > > > > > +1 for this proposal. > > > > > > > > > > We have been prototyping this feature internally at Alibaba for a > > > couple > > > > of > > > > > months. Yuxin, I think we can also publish the prototype codes so the > > > > > community can better understand and help with it. > > > > > > > > > > Best, > > > > > > > > > > Xintong > > > > > > > > > > > > > > > > > > > > On Tue, May 28, 2024 at 8:34 PM Yuxin Tan > > > > wrote: > > > > > > > > > > > Hi all, > > > > > > > > > > > > I would like to start a discussion on FLIP-459 Support Flink hybrid > > > > > shuffle > > > > > > integration with > > > > > > Apache Celeborn[1]. Flink hybrid shuffle supports transitions > > between > > > > > > memory, disk, and > > > > > > remote storage to improve performance and job stability. > > > Concurrently, > > > > > > Apache Celeborn > > > > > > provides a stable, performant, scalable remote shuffle service. > > This > > > > > > integration proposal is to > > > > > > harness the benefits from both hybrid shuffle and Celeborn > > > > > simultaneously. > > > > > > > > > > > > Note that this proposal has two parts. > > > > > > 1. The Flink-side modifications are in FLIP-459[1]. > > > > > > 2. The Celeborn-side changes are in CIP-6[2]. > > > > > > > > > > > > Looking forward to everyone's feedback and suggestions. Thank you! > > > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > > > > > > > > > https://urldefense.com/v3/__https://cwiki.apache.org/confluence/display/FLINK/FLIP-459*3A*Support*Flink*hybrid*shuffle*integration*with*Apache*Celeborn__;JSsrKysrKysr!!IKRxdwAv5BmarQ!ZRTc1aUSYMDBazuIwlet1Dzk2_DD9qKTgoDLH9jSwAVLgwplcuId_8JoXkH0i7AeWxKWXkL0sxM3AeW-H9MaOGE7hQ$ > > > > > > [2] > > > > > > > > > > > > > > > > > > > > > > > > > > https://urldefense.com/v3/__https://cwiki.apache.org/confluence/display/CELEBORN/CIP-6*Support*Flink*hybrid*shuffle*integration*with*Apache*Celeborn__;KysrKysrKys!!IKRxdwAv5BmarQ!ZRTc1aUSYMDBazuIwlet1Dzk2_DD9qKTgoDLH9jSwAVLgwplcuId_8JoXkH0i7AeWxKWXkL0sxM3AeW-H9OJ6v9uGw$ > > > > > > > > > > > > Best, > > > > > > Yuxin > > > > > > > > > > > > > > > > > > > >
Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie Guo
Congratulations Weijie! Best, Hang Yanfei Lei 于2024年6月4日周二 16:24写道： > Congratulations! > > Best, > Yanfei > > Leonard Xu 于2024年6月4日周二 16:20写道： > > > > Congratulations！ > > > > Best, > > Leonard > > > > > 2024年6月4日 下午4:02，Yangze Guo 写道： > > > > > > Congratulations! > > > > > > Best, > > > Yangze Guo > > > > > > On Tue, Jun 4, 2024 at 4:00 PM Weihua Hu > wrote: > > >> > > >> Congratulations, Weijie! > > >> > > >> Best, > > >> Weihua > > >> > > >> > > >> On Tue, Jun 4, 2024 at 3:03 PM Yuxin Tan > wrote: > > >> > > >>> Congratulations, Weijie! > > >>> > > >>> Best, > > >>> Yuxin > > >>> > > >>> > > >>> Yuepeng Pan 于2024年6月4日周二 14:57写道： > > >>> > > Congratulations ! > > > > > > Best, > > Yuepeng Pan > > > > At 2024-06-04 14:45:45, "Xintong Song" > wrote: > > > Hi everyone, > > > > > > On behalf of the PMC, I'm very happy to announce that Weijie Guo > has > > joined > > > the Flink PMC! > > > > > > Weijie has been an active member of the Apache Flink community for > many > > > years. He has made significant contributions in many components, > > >>> including > > > runtime, shuffle, sdk, connectors, etc. He has driven / > participated in > > > many FLIPs, authored and reviewed hundreds of PRs, been > consistently > > active > > > on mailing lists, and also helped with release management of 1.20 > and > > > several other bugfix releases. > > > > > > Congratulations and welcome Weijie! > > > > > > Best, > > > > > > Xintong (on behalf of the Flink PMC) > > > > >>> > > >
Re: [DISCUSS] FLIP-459: Support Flink hybrid shuffle integration with Apache Celeborn
Hi, Venkatakrishnan, Thanks for joining the discussion. We appreciate your interest in contributing to the work. Once the FLIP and CIP proposals have been approved, we will create some JIRA tickets in Flink and Celeborn projects. Please feel free to take a look at the tickets and select any that resonate with your interests. Best, Yuxin Venkatakrishnan Sowrirajan 于2024年5月31日周五 23:11写道： > Thanks for this FLIP. We are also interested in learning/contributing to > the hybrid shuffle integration with celeborn for batch executions. > > On Tue, May 28, 2024, 7:07 PM Yuxin Tan wrote: > > > Hi, Xintong, > > > > > I think we can also publish the prototype codes so the > > community can better understand and help with it. > > > > Ok, I agree on the point. I will prepare and publish the code > > recently. > > > > Rui, > > > > > Kindly reminder: the image of CIP-6[1] cannot be loaded. > > > > Thanks for the reminder. I've updated the images. > > > > > > Best, > > Yuxin > > > > > > Rui Fan <1996fan...@gmail.com> 于2024年5月29日周三 09:33写道： > > > > > Thanks Yuxin for driving this proposal! > > > > > > Kindly reminder: the image of CIP-6[1] cannot be loaded. > > > > > > [1] > > > > > > > > > https://urldefense.com/v3/__https://cwiki.apache.org/confluence/display/CELEBORN/CIP-6*Support*Flink*hybrid*shuffle*integration*with*Apache*Celeborn__;KysrKysrKys!!IKRxdwAv5BmarQ!ZRTc1aUSYMDBazuIwlet1Dzk2_DD9qKTgoDLH9jSwAVLgwplcuId_8JoXkH0i7AeWxKWXkL0sxM3AeW-H9OJ6v9uGw$ > > > > > > Best, > > > Rui > > > > > > On Wed, May 29, 2024 at 9:03 AM Xintong Song > > > wrote: > > > > > > > +1 for this proposal. > > > > > > > > We have been prototyping this feature internally at Alibaba for a > > couple > > > of > > > > months. Yuxin, I think we can also publish the prototype codes so the > > > > community can better understand and help with it. > > > > > > > > Best, > > > > > > > > Xintong > > > > > > > > > > > > > > > > On Tue, May 28, 2024 at 8:34 PM Yuxin Tan > > > wrote: > > > > > > > > > Hi all, > > > > > > > > > > I would like to start a discussion on FLIP-459 Support Flink hybrid > > > > shuffle > > > > > integration with > > > > > Apache Celeborn[1]. Flink hybrid shuffle supports transitions > between > > > > > memory, disk, and > > > > > remote storage to improve performance and job stability. > > Concurrently, > > > > > Apache Celeborn > > > > > provides a stable, performant, scalable remote shuffle service. > This > > > > > integration proposal is to > > > > > harness the benefits from both hybrid shuffle and Celeborn > > > > simultaneously. > > > > > > > > > > Note that this proposal has two parts. > > > > > 1. The Flink-side modifications are in FLIP-459[1]. > > > > > 2. The Celeborn-side changes are in CIP-6[2]. > > > > > > > > > > Looking forward to everyone's feedback and suggestions. Thank you! > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > > > https://urldefense.com/v3/__https://cwiki.apache.org/confluence/display/FLINK/FLIP-459*3A*Support*Flink*hybrid*shuffle*integration*with*Apache*Celeborn__;JSsrKysrKysr!!IKRxdwAv5BmarQ!ZRTc1aUSYMDBazuIwlet1Dzk2_DD9qKTgoDLH9jSwAVLgwplcuId_8JoXkH0i7AeWxKWXkL0sxM3AeW-H9MaOGE7hQ$ > > > > > [2] > > > > > > > > > > > > > > > > > > > > https://urldefense.com/v3/__https://cwiki.apache.org/confluence/display/CELEBORN/CIP-6*Support*Flink*hybrid*shuffle*integration*with*Apache*Celeborn__;KysrKysrKys!!IKRxdwAv5BmarQ!ZRTc1aUSYMDBazuIwlet1Dzk2_DD9qKTgoDLH9jSwAVLgwplcuId_8JoXkH0i7AeWxKWXkL0sxM3AeW-H9OJ6v9uGw$ > > > > > > > > > > Best, > > > > > Yuxin > > > > > > > > > > > > > > >
[jira] [Created] (FLINK-35517) CI pipeline triggered by pull request seems unstable
Weijie Guo created FLINK-35517: -- Summary: CI pipeline triggered by pull request seems unstable Key: FLINK-35517 URL: https://issues.apache.org/jira/browse/FLINK-35517 Project: Flink Issue Type: Bug Components: Build System / CI Affects Versions: 1.20.0 Reporter: Weijie Guo Flink CI pipeline triggered by pull request seems sort of unstable. For example, https://github.com/apache/flink/pull/24883 was filed 15 hours ago, but CI report is UNKNOWN. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie Guo
Congratulations! Best, Yanfei Leonard Xu 于2024年6月4日周二 16:20写道： > > Congratulations！ > > Best, > Leonard > > > 2024年6月4日 下午4:02，Yangze Guo 写道： > > > > Congratulations! > > > > Best, > > Yangze Guo > > > > On Tue, Jun 4, 2024 at 4:00 PM Weihua Hu wrote: > >> > >> Congratulations, Weijie! > >> > >> Best, > >> Weihua > >> > >> > >> On Tue, Jun 4, 2024 at 3:03 PM Yuxin Tan wrote: > >> > >>> Congratulations, Weijie! > >>> > >>> Best, > >>> Yuxin > >>> > >>> > >>> Yuepeng Pan 于2024年6月4日周二 14:57写道： > >>> > Congratulations ! > > > Best, > Yuepeng Pan > > At 2024-06-04 14:45:45, "Xintong Song" wrote: > > Hi everyone, > > > > On behalf of the PMC, I'm very happy to announce that Weijie Guo has > joined > > the Flink PMC! > > > > Weijie has been an active member of the Apache Flink community for many > > years. He has made significant contributions in many components, > >>> including > > runtime, shuffle, sdk, connectors, etc. He has driven / participated in > > many FLIPs, authored and reviewed hundreds of PRs, been consistently > active > > on mailing lists, and also helped with release management of 1.20 and > > several other bugfix releases. > > > > Congratulations and welcome Weijie! > > > > Best, > > > > Xintong (on behalf of the Flink PMC) > > >>> >
Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie Guo
Congratulations！ Best, Leonard > 2024年6月4日 下午4:02，Yangze Guo 写道： > > Congratulations! > > Best, > Yangze Guo > > On Tue, Jun 4, 2024 at 4:00 PM Weihua Hu wrote: >> >> Congratulations, Weijie! >> >> Best, >> Weihua >> >> >> On Tue, Jun 4, 2024 at 3:03 PM Yuxin Tan wrote: >> >>> Congratulations, Weijie! >>> >>> Best, >>> Yuxin >>> >>> >>> Yuepeng Pan 于2024年6月4日周二 14:57写道： >>> Congratulations ! Best, Yuepeng Pan At 2024-06-04 14:45:45, "Xintong Song" wrote: > Hi everyone, > > On behalf of the PMC, I'm very happy to announce that Weijie Guo has joined > the Flink PMC! > > Weijie has been an active member of the Apache Flink community for many > years. He has made significant contributions in many components, >>> including > runtime, shuffle, sdk, connectors, etc. He has driven / participated in > many FLIPs, authored and reviewed hundreds of PRs, been consistently active > on mailing lists, and also helped with release management of 1.20 and > several other bugfix releases. > > Congratulations and welcome Weijie! > > Best, > > Xintong (on behalf of the Flink PMC) >>>
Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie Guo
Congratulations! Best, Yangze Guo On Tue, Jun 4, 2024 at 4:00 PM Weihua Hu wrote: > > Congratulations, Weijie! > > Best, > Weihua > > > On Tue, Jun 4, 2024 at 3:03 PM Yuxin Tan wrote: > > > Congratulations, Weijie! > > > > Best, > > Yuxin > > > > > > Yuepeng Pan 于2024年6月4日周二 14:57写道： > > > > > Congratulations ! > > > > > > > > > Best, > > > Yuepeng Pan > > > > > > At 2024-06-04 14:45:45, "Xintong Song" wrote: > > > >Hi everyone, > > > > > > > >On behalf of the PMC, I'm very happy to announce that Weijie Guo has > > > joined > > > >the Flink PMC! > > > > > > > >Weijie has been an active member of the Apache Flink community for many > > > >years. He has made significant contributions in many components, > > including > > > >runtime, shuffle, sdk, connectors, etc. He has driven / participated in > > > >many FLIPs, authored and reviewed hundreds of PRs, been consistently > > > active > > > >on mailing lists, and also helped with release management of 1.20 and > > > >several other bugfix releases. > > > > > > > >Congratulations and welcome Weijie! > > > > > > > >Best, > > > > > > > >Xintong (on behalf of the Flink PMC) > > > > >
Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie Guo
Congratulations, Weijie! Best, Weihua On Tue, Jun 4, 2024 at 3:03 PM Yuxin Tan wrote: > Congratulations, Weijie! > > Best, > Yuxin > > > Yuepeng Pan 于2024年6月4日周二 14:57写道： > > > Congratulations ! > > > > > > Best, > > Yuepeng Pan > > > > At 2024-06-04 14:45:45, "Xintong Song" wrote: > > >Hi everyone, > > > > > >On behalf of the PMC, I'm very happy to announce that Weijie Guo has > > joined > > >the Flink PMC! > > > > > >Weijie has been an active member of the Apache Flink community for many > > >years. He has made significant contributions in many components, > including > > >runtime, shuffle, sdk, connectors, etc. He has driven / participated in > > >many FLIPs, authored and reviewed hundreds of PRs, been consistently > > active > > >on mailing lists, and also helped with release management of 1.20 and > > >several other bugfix releases. > > > > > >Congratulations and welcome Weijie! > > > > > >Best, > > > > > >Xintong (on behalf of the Flink PMC) > > >
Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie Guo
Congratulations, Weijie! Best, Yuxin Yuepeng Pan 于2024年6月4日周二 14:57写道： > Congratulations ! > > > Best, > Yuepeng Pan > > At 2024-06-04 14:45:45, "Xintong Song" wrote: > >Hi everyone, > > > >On behalf of the PMC, I'm very happy to announce that Weijie Guo has > joined > >the Flink PMC! > > > >Weijie has been an active member of the Apache Flink community for many > >years. He has made significant contributions in many components, including > >runtime, shuffle, sdk, connectors, etc. He has driven / participated in > >many FLIPs, authored and reviewed hundreds of PRs, been consistently > active > >on mailing lists, and also helped with release management of 1.20 and > >several other bugfix releases. > > > >Congratulations and welcome Weijie! > > > >Best, > > > >Xintong (on behalf of the Flink PMC) >
Re:[ANNOUNCE] New Apache Flink PMC Member - Weijie Guo
Congratulations ! Best, Yuepeng Pan At 2024-06-04 14:45:45, "Xintong Song" wrote: >Hi everyone, > >On behalf of the PMC, I'm very happy to announce that Weijie Guo has joined >the Flink PMC! > >Weijie has been an active member of the Apache Flink community for many >years. He has made significant contributions in many components, including >runtime, shuffle, sdk, connectors, etc. He has driven / participated in >many FLIPs, authored and reviewed hundreds of PRs, been consistently active >on mailing lists, and also helped with release management of 1.20 and >several other bugfix releases. > >Congratulations and welcome Weijie! > >Best, > >Xintong (on behalf of the Flink PMC)
Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie Guo
Congratulations Weijie! Best, Junrui Xintong Song 于2024年6月4日周二 14:46写道： > Hi everyone, > > On behalf of the PMC, I'm very happy to announce that Weijie Guo has joined > the Flink PMC! > > Weijie has been an active member of the Apache Flink community for many > years. He has made significant contributions in many components, including > runtime, shuffle, sdk, connectors, etc. He has driven / participated in > many FLIPs, authored and reviewed hundreds of PRs, been consistently active > on mailing lists, and also helped with release management of 1.20 and > several other bugfix releases. > > Congratulations and welcome Weijie! > > Best, > > Xintong (on behalf of the Flink PMC) >
Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie Guo
Congratulations Weijie, well deserved! Best, Rui On Tue, Jun 4, 2024 at 2:47 PM Xintong Song wrote: > Hi everyone, > > On behalf of the PMC, I'm very happy to announce that Weijie Guo has joined > the Flink PMC! > > Weijie has been an active member of the Apache Flink community for many > years. He has made significant contributions in many components, including > runtime, shuffle, sdk, connectors, etc. He has driven / participated in > many FLIPs, authored and reviewed hundreds of PRs, been consistently active > on mailing lists, and also helped with release management of 1.20 and > several other bugfix releases. > > Congratulations and welcome Weijie! > > Best, > > Xintong (on behalf of the Flink PMC) >
[ANNOUNCE] New Apache Flink PMC Member - Weijie Guo
Hi everyone, On behalf of the PMC, I'm very happy to announce that Weijie Guo has joined the Flink PMC! Weijie has been an active member of the Apache Flink community for many years. He has made significant contributions in many components, including runtime, shuffle, sdk, connectors, etc. He has driven / participated in many FLIPs, authored and reviewed hundreds of PRs, been consistently active on mailing lists, and also helped with release management of 1.20 and several other bugfix releases. Congratulations and welcome Weijie! Best, Xintong (on behalf of the Flink PMC)
