Re: [ANNOUNCE] New Apache Flink PMC Member - Fan Rui

Congratulations, Rui! Regards, Xiangyu Feng Feng Jin 于2024年6月5日周三 20:42写道： > Congratulations, Rui! > > > Best, > Feng Jin > > On Wed, Jun 5, 2024 at 8:23 PM Yanfei Lei wrote: > > > Congratulations, Rui! > > > > Best, > > Yanfei > > >

Re: [ANNOUNCE] New Apache Flink Committer - Zakelly Lan

Congratulations, Zakelly! Regards, Xiangyu Feng yh z 于2024年4月18日周四 14:27写道： > Congratulations Zakelly! > > Best regards, > Yunhong (swuferhong) > > gongzhongqiang 于2024年4月17日周三 21:26写道： > > > Congratulations, Zakelly! > > > > > > Best, > >

Re: [ANNOUNCE] New Apache Flink PMC Member - Lincoln Lee

Congratulations, Lincoln! Best, Xiangyu Feng Feifan Wang 于2024年4月12日周五 17:19写道： > Congratulations, Lincoln! > > > —— > > Best regards, > > Feifan Wang > > > > > At 2024-04-12 15:59:00, "Jark Wu" wrote: > >Hi everyone,

Re: [DISCUSS] FLIP-414: Support Retry Mechanism in RocksDBStateDataTransfer

that. Thx again for your valuable options~ [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-414%3A+Support+Retry+Mechanism+in+RocksDBStateDataTransfer Best Regards, Xiangyu Feng Piotr Nowojski 于2024年1月12日周五 01:39写道： > Hi, > > Thanks for the proposal. I second the Hangxiang's su

[jira] [Created] (FLINK-34452) Release Testing: Verify FLINK-15959 Add min number of slots configuration to limit total number of slots

xiangyu feng created FLINK-34452: Summary: Release Testing: Verify FLINK-15959 Add min number of slots configuration to limit total number of slots Key: FLINK-34452 URL: https://issues.apache.org/jira/browse

[jira] [Created] (FLINK-34226) Add the backoff-multiplier configuration in ExponentialBackoffRetryStrategy

xiangyu feng created FLINK-34226: Summary: Add the backoff-multiplier configuration in ExponentialBackoffRetryStrategy Key: FLINK-34226 URL: https://issues.apache.org/jira/browse/FLINK-34226 Project

[RESULT][VOTE] FLIP-407: Improve Flink Client performance in interactive scenarios

Zhanghao (non-binding) There is no disapproving vote. Regards, Xiangyu Feng [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-407%3A+Improve+Flink+Client+performance+in+interactive+scenarios [2] https://lists.apache.org/thread/b1hqnlxkyqnqt1sn1dww7jw3bdrs030o

Re: [VOTE] FLIP-407: Improve Flink Client performance in interactive scenarios

Thank you all for the votes! I will close the voting thread and summarize the result in a separate email. Regards, Xiangyu Feng Zhanghao Chen 于2024年1月16日周二 10:18写道： > +1 (non-binding) > > Best, > Zhanghao Chen > ____ > From: xiangyu feng > Se

Re: [DISCUSS] FLIP-416: Deprecate and remove the RestoreMode#LEGACY

Hi Zakelly, Thanks for driving this. +1 to remove the LEGACY mode. This will make the semantic of snapshot ownership more clear. Best regards, Xiangyu Feng Yuan Mei 于2024年1月15日周一 23:16写道： > +1 > > Thanks for driving this Zakelly! > > Best > Yuan > > On Mon, Jan 15

[VOTE] FLIP-407: Improve Flink Client performance in interactive scenarios

Hi all, I would like to start the vote for FLIP-407: Improve Flink Client performance in interactive scenarios[1]. This FLIP was discussed in this thread [2]. The vote will be open for at least 72 hours unless there is an objection or insufficient votes. Regards, Xiangyu [1]

Re: [Discuss] FLIP-407: Improve Flink Client performance in interactive scenarios

Hi devs, Thanks for all the feedback. If there are no more comments, I would like to start a vote for this FLIP, thanks again! Best, Xiangyu Feng Weihua Hu 于2024年1月9日周二 14:45写道： > Thanks for proposing this FLIP. > > Experiments have shown that it significantly enhances the real-t

Re: [VOTE] Accept Flink CDC into Apache Flink

+1 (non-binding) Regards, Xiangyu Feng Danny Cranmer 于2024年1月9日周二 17:50写道： > +1 (binding) > > Thanks, > Danny > > On Tue, Jan 9, 2024 at 9:31 AM Feng Jin wrote: > > > +1 (non-binding) > > > > Best, > > Feng Jin > > > > On Tue, Jan 9, 20

Re: [Discuss] FLIP-407: Improve Flink Client performance in interactive scenarios

be consistent with the original behavior and reduce the configuration items at the same time. I've updated this FLIP accordingly, look forward to your feedback. Regards, Xiangyu Feng Rui Fan <1996fan...@gmail.com> 于2024年1月8日周一 15:29写道： > Only one strategy is fine to me. > > When

[DISCUSS] FLIP-414: Support Retry Mechanism in RocksDBStateDataTransfer

+in+RocksDBStateDataTransfer Best regards, Xiangyu Feng

Re: [Discuss] FLIP-407: Improve Flink Client performance in interactive scenarios

to me. > > Best, > Zhanghao Chen > ________ > From: xiangyu feng > Sent: Tuesday, December 26, 2023 16:51 > To: dev@flink.apache.org > Subject: [Discuss] FLIP-407: Improve Flink Client performance in > interactive scenarios > > Hi devs, > >

Re: [VOTE] FLIP-397: Add config options for administrator JVM options

+1 (non-binding) Regards, Xiangyu Feng Rui Fan <1996fan...@gmail.com> 于2024年1月4日周四 13:03写道： > +1 (binding) > > Best, > Rui > > On Thu, Jan 4, 2024 at 11:45 AM Benchao Li wrote: > > > +1 (binding) > > > > Zhanghao Chen 于2024年1月4日周四 10:30写道： > &

Re: [DISCUSS] FLIP-397: Add config options for administrator JVM options

_________ > From: xiangyu feng > Sent: Friday, December 29, 2023 20:20 > To: dev@flink.apache.org > Subject: Re: [DISCUSS] FLIP-397: Add config options for administrator JVM > options > > Hi Zhanghao, > > Thanks for driving this. +1 for the overall idea

Re: [DISCUSS] FLIP-397: Add config options for administrator JVM options

Hi Zhanghao, Thanks for driving this. +1 for the overall idea. One minor question, do we need separate administrator JVM options for both JobManager and TaskManager? Or just one administrator JVM option for all? I'm afraid of 6 jvm

Re: [DISCUSS] FLIP-403: High Availability Services for OLAP Scenarios

Thanks Yangze for restart this discussion. +1 for the overall idea. By splitting the HighAvailabilityServices into LeaderServices and PersistenceServices, we may support configuring different storage behind them in the future. We did run into real problems in production where too much job

[Discuss] FLIP-407: Improve Flink Client performance in interactive scenarios

Hi devs, I'm opening this thread to discuss FLIP-407: Improve Flink Client performance in interactive scenarios. The POC test results and design doc can be found at: FLIP-407

Re: FLIP-403: High Availability Services for OLAP Scenarios

Hi Yangze, Thanks for driving this. I like the idea to separate the current HighAvailabilityServices into LeaderServices and PersistentServices. AFAIK, there is no need to bind JobGraphStore/JobResultStore/BlobStore/CheckpointStore modules with LeaderElection/LeaderRetrieval services. The formers

Re: [ANNOUNCE] Experimental Java 21 support now available on master

Thanks Sergey for the great work! ZGC in JDK21 is now a multi-generational garbage collector. In jdk17, we tried using ZGC as the default garbage collector for streaming computations, but found that while it reduced STW, it did have a negative impact on job throughput because a single-generation

[jira] [Created] (FLINK-33702) Add IncrementalDelayRetryStrategy in AsyncRetryStrategies

xiangyu feng created FLINK-33702: Summary: Add IncrementalDelayRetryStrategy in AsyncRetryStrategies Key: FLINK-33702 URL: https://issues.apache.org/jira/browse/FLINK-33702 Project: Flink

[jira] [Created] (FLINK-33698) Fix the backoff time calculation in ExponentialBackoffDelayRetryStrategy

xiangyu feng created FLINK-33698: Summary: Fix the backoff time calculation in ExponentialBackoffDelayRetryStrategy Key: FLINK-33698 URL: https://issues.apache.org/jira/browse/FLINK-33698 Project

[jira] [Created] (FLINK-33688) Reuse Channels in RestClient to save connection establish time

xiangyu feng created FLINK-33688: Summary: Reuse Channels in RestClient to save connection establish time Key: FLINK-33688 URL: https://issues.apache.org/jira/browse/FLINK-33688 Project: Flink

[jira] [Created] (FLINK-33687) Reuse RestClusterClient in StandaloneClusterDescriptor to avoid frequent thread create/destroy

xiangyu feng created FLINK-33687: Summary: Reuse RestClusterClient in StandaloneClusterDescriptor to avoid frequent thread create/destroy Key: FLINK-33687 URL: https://issues.apache.org/jira/browse/FLINK-33687

[jira] [Created] (FLINK-33686) Reuse StandaloneClusterDescriptor in RemoteExecutor when executing jobs on the same cluster

xiangyu feng created FLINK-33686: Summary: Reuse StandaloneClusterDescriptor in RemoteExecutor when executing jobs on the same cluster Key: FLINK-33686 URL: https://issues.apache.org/jira/browse/FLINK-33686

[jira] [Created] (FLINK-33685) StandaloneClusterId need to distinguish different remote clusters

xiangyu feng created FLINK-33685: Summary: StandaloneClusterId need to distinguish different remote clusters Key: FLINK-33685 URL: https://issues.apache.org/jira/browse/FLINK-33685 Project: Flink

[jira] [Created] (FLINK-33684) Improve the retry strategy in CollectResultFetcher

xiangyu feng created FLINK-33684: Summary: Improve the retry strategy in CollectResultFetcher Key: FLINK-33684 URL: https://issues.apache.org/jira/browse/FLINK-33684 Project: Flink Issue

[jira] [Created] (FLINK-33683) Improve the performance of submitting jobs and fetching results to a running flink cluster

xiangyu feng created FLINK-33683: Summary: Improve the performance of submitting jobs and fetching results to a running flink cluster Key: FLINK-33683 URL: https://issues.apache.org/jira/browse/FLINK-33683

Re: [VOTE] FLIP-393: Make QueryOperations SQL serializable

+1 (non-binding) Thanks for driving this. Best, Xiangyu Feng Ferenc Csaky 于2023年11月21日周二 20:07写道： > +1 (non-binding) > > Lookgin forward to this! > > Best, > Ferenc > > > > > On Tuesday, November 21st, 2023 at 12:21, Martijn Visser < > martijnvis..

Re: [DISCUSS] FLIP-393: Make QueryOperations SQL serializable

>After this FLIP is done, FLINK-25015() can utilize this ability to set > job name for queries. +1 for this. Currently, when users submit sql jobs through table api, we can't see the complete SQL string on flink ui. It would be easy for us to finish this feature if we can get serialized sql from

Re: Heartbeat of TaskManager with id container

t/test/flink/log4j.properties*). > > is it possible right?. in flink didn't find it. > > Any through on the above. > > Regards, > Nagireddy Y. > > > > On Fri, Nov 3, 2023 at 12:18 PM xiangyu feng wrote: > >> Hi Nagireddy, >> >> You can configure logg

Re: Heartbeat of TaskManager with id container

nfig for the flink run command. > > Regards, > Nagireddy Y. > > > > > On Thu, Aug 3, 2023 at 7:24 AM xiangyu feng wrote: > >> Hi ynagireddy4u, >> >> We have met this exception before. Usually it is caused by following >> reasons: >> >> 1),

Re: [VOTE] FLIP-370: Support Balanced Tasks Scheduling

Thanks for driving that. +1 (non-binding) Regards, Xiangyu Yu Chen 于2023年10月23日周一 15:19写道： > +1 (non-binding) > > We deeply need this capability to balance Tasks at the Taskmanager level in > production, which helps to make a more sufficient usage of Taskmanager > resources. Thanks for driving

Re: [ANNOUNCE] New Apache Flink Committer - Jane Chan

Congratulations Jane! Best, Xiangyu Xuannan Su 于2023年10月16日周一 10:25写道： > Congratulations Jane! > > Best, > Xuannan > > On Mon, Oct 16, 2023 at 10:21 AM Yun Tang wrote: > > > > Congratulations, Jane! > > > > Best > > Yun Tang > > > > From: Rui Fan

Re: [DISCUSS] FLIP-370 : Support Balanced Tasks Scheduling

s in my statement, please correct me. > Looking forward to your reply. > > Best regards. > Yuepeng Pan > > On 2023/10/11 08:44:51 xiangyu feng wrote: > > Hi Yuepeng, > > > > Thx for ur reply. > > > > > Nice feedback. In fact, as mentioned in the

Re: [DISCUSS] FLIP-370 : Support Balanced Tasks Scheduling

c, the LoadingWeight > interface currently only includes a description of the number of tasks. So, > IIUC, If there is a need to further expand descriptions of other resource > loads, we just extend it based on the current interface and its > implementations, right? > Please correct me if I

[jira] [Created] (FLINK-33235) Quickstart guide for Flink OLAP should support building from master branch

xiangyu feng created FLINK-33235: Summary: Quickstart guide for Flink OLAP should support building from master branch Key: FLINK-33235 URL: https://issues.apache.org/jira/browse/FLINK-33235 Project

Re: [VOTE] FLIP-374: Adding a separate configuration for specifying Java Options of the SQL Gateway

+1(non-binding) Regards, Xiangyu Shammon FY 于2023年10月11日周三 10:30写道： > +1(binding), good job! > > Best, > Shammon FY > > On Wed, Oct 11, 2023 at 10:18 AM Benchao Li wrote: > > > +1 (binding) > > > > Rui Fan <1996fan...@gmail.com> 于2023年10月11日周三 10:17写道： > > > > > > +1(binding) > > > > > >

[jira] [Created] (FLINK-33228) Fix the total current resource calculation in fulfill requirements

xiangyu feng created FLINK-33228: Summary: Fix the total current resource calculation in fulfill requirements Key: FLINK-33228 URL: https://issues.apache.org/jira/browse/FLINK-33228 Project: Flink

Re: [VOTE] FLIP-367: Support Setting Parallelism for Table/SQL Sources

+1 (non-binding) Feng Jin 于2023年10月9日周一 16:00写道： > +1 (non-binding) > > Best, > Feng > > On Mon, Oct 9, 2023 at 3:12 PM Yangze Guo wrote: > > > +1 (binding) > > > > Best, > > Yangze Guo > > > > On Mon, Oct 9, 2023 at 2:46 PM Yun Tang wrote: > > > > > > +1 (binding) > > > > > > Best > > > Yun

[jira] [Created] (FLINK-33209) Translate Flink OLAP quick start guide to Chinese

xiangyu feng created FLINK-33209: Summary: Translate Flink OLAP quick start guide to Chinese Key: FLINK-33209 URL: https://issues.apache.org/jira/browse/FLINK-33209 Project: Flink Issue Type

Re: [DISCUSS] FLIP-374: Adding a separate configuration for specifying Java Options of the SQL Gateway

Thanks for initiating this discussion. Within the development towards Streaming Warehousing, SQL Gateway will become more and more important. Big +1 to specify Java Options separately for SQL Gateway. Regards, Xiangyu Yangze Guo 于2023年10月7日周六 15:24写道： > Hi, there, > > We'd like to start a

Re: [DISCUSS] FLIP-370 : Support Balanced Tasks Scheduling

Thanks Yuepeng and Rui for driving this Discussion. Internally when we try to use Flink 1.17.1 in production, we are also suffering from the unbalanced task distribution problem for jobs with high qps and complex dag. So +1 for the overall proposal. Some questions about the details: 1, About

Re: [Discuss] FLIP-362: Support minimum resource limitation

min + slots > to keep around). > > All in all, I don't have a strong opinion here, it's a significant > improvement either way. This was just the first thing that I thought about > after reading the flip. > > Best, > D. > > On Tue, Oct 3, 2023 at 2:10 PM xiangyu feng wro

Re: [Discuss] FLIP-362: Support minimum resource limitation

d to >> > > configure slotmanager.number-of-slots.max for streaming jobs, does it >> > make >> > > sense to disable it for common streaming jobs? At least disable the >> check >> > > for avoiding the oscillation? >> > > >> > >

[RESULT][VOTE] FLIP-362: Support minimum resource limitation

Hi all, Thanks for your review and the votes! I am happy to announce that FLIP-362: Support minimum resource limitation [1] has been accepted. There are 4 binding votes and 2 non-binding votes [2]: - Yangze Guo (binding) - Ahmed Hamdy (non-binding) - Chen Zhanghao (non-binding) - Jing Ge

Re: [VOTE] FLIP-362: Support minimum resource limitation

n Mon, Sep 25, 2023 at 3:48 AM Chen Zhanghao < > zhanghao.c...@outlook.com > > > > > > wrote: > > > > > > > Thanks for driving this. +1 (non-binding) > > > > > > > > Best, > > > > Zhanghao Chen > > >

[VOTE] FLIP-362: Support minimum resource limitation

Hi all, I would like to start the vote for FLIP-362: Support minimum resource limitation[1]. This FLIP was discussed in this thread [2]. The vote will be open for at least 72 hours unless there is an objection or insufficient votes. Regards, Xiangyu [1]

Re: [Discuss] FLIP-362: Support minimum resource limitation

plication mode? Since it is not recommended to > > > > configure slotmanager.number-of-slots.max for streaming jobs, does it > > > make > > > > sense to disable it for common streaming jobs? At least disable the > check > > > > for avoiding the os

Re: [Discuss] FLIP-362: Support minimum resource limitation

t; > > wrote: > > > > > Thanks for driving this, Xiangyu. We use Session clusters for quick SQL > > > debugging internally, and found cold-start job submission slow due to > lack > > > of the exact minimum resource reservation feature proposed here. This >

[Discuss] FLIP-362: Support minimum resource limitation

Hi Devs, I'm opening this thread to discuss FLIP-362: Support minimum resource limitation. The design doc can be found at: FLIP-362: Support minimum resource limitation Currently, the Flink

[REQUEST] Apply edit permissions for FLIP

Hi devs, I would like to request the edit permission for FLIP. I'm currently working on FLINK-15959 which need change the ResourceManagerOptions. My wiki username is xiangyu0xf. Thx, Xiangyu

[jira] [Created] (FLINK-33016) Invalid download url for Flink JDBC Driver page

xiangyu feng created FLINK-33016: Summary: Invalid download url for Flink JDBC Driver page Key: FLINK-33016 URL: https://issues.apache.org/jira/browse/FLINK-33016 Project: Flink Issue Type

[jira] [Created] (FLINK-32985) Support setting env.java.opts.sql-gateway to specify the jvm opts for sql gateway

xiangyu feng created FLINK-32985: Summary: Support setting env.java.opts.sql-gateway to specify the jvm opts for sql gateway Key: FLINK-32985 URL: https://issues.apache.org/jira/browse/FLINK-32985

Re: Heartbeat of TaskManager with id container

nks Xiangyu, > > I have one issue, while running flink with kafka connector. its a working > fine for couple of days. > > But suddenly kafka lag went to "Negative value" > > I am trying to find the root cause for that. Any suggestions? > > > On Sat, Aug 5, 2

[jira] [Created] (FLINK-32898) Improvement of usability for Flink OLAP

xiangyu feng created FLINK-32898: Summary: Improvement of usability for Flink OLAP Key: FLINK-32898 URL: https://issues.apache.org/jira/browse/FLINK-32898 Project: Flink Issue Type

[jira] [Created] (FLINK-32880) Redundant taskManager should be replenished in FineGrainedSlotManager

xiangyu feng created FLINK-32880: Summary: Redundant taskManager should be replenished in FineGrainedSlotManager Key: FLINK-32880 URL: https://issues.apache.org/jira/browse/FLINK-32880 Project: Flink

Re: [DISCUSS] How about adding OLAP to Flink Roadmap?

Thank you Shammon for initiating this discussion. As one of the Flink OLAP developers in ByteDance, I would also like to share a real case of our users. About two years ago we found our first OLAP user internally by integrating Flink OLAP with ByteHTAP. Users are willing to use Flink as an OLAP

[jira] [Created] (FLINK-32764) SlotManager supports pulling up all TaskManagers at initialization

xiangyu feng created FLINK-32764: Summary: SlotManager supports pulling up all TaskManagers at initialization Key: FLINK-32764 URL: https://issues.apache.org/jira/browse/FLINK-32764 Project: Flink

[jira] [Created] (FLINK-32762) Support concurrency control when submitting OLAP jobs to Dispatcher

xiangyu feng created FLINK-32762: Summary: Support concurrency control when submitting OLAP jobs to Dispatcher Key: FLINK-32762 URL: https://issues.apache.org/jira/browse/FLINK-32762 Project: Flink

Re: Heartbeat of TaskManager with id container

ile which was committed > by subsequent successful checkpoints, *Flink will refuse to resume and it > will throw an exception as it cannot locate the in-progress file*. > > Currently i am facing same issue in the PROD code. > > > > Regards, > Nagireddy Y. > > > > &

[jira] [Created] (FLINK-32756) Reues ZK CuratorFramework when submitting OLAP jobs to Flink session cluster

xiangyu feng created FLINK-32756: Summary: Reues ZK CuratorFramework when submitting OLAP jobs to Flink session cluster Key: FLINK-32756 URL: https://issues.apache.org/jira/browse/FLINK-32756 Project

[jira] [Created] (FLINK-32755) Add quick start guide for Flink OLAP

xiangyu feng created FLINK-32755: Summary: Add quick start guide for Flink OLAP Key: FLINK-32755 URL: https://issues.apache.org/jira/browse/FLINK-32755 Project: Flink Issue Type: Sub-task

Re: Heartbeat of TaskManager with id container

a:101) > at org.apache.flink.streaming.runtime.tasks.StreamTask > .notifyCheckpointComplete(StreamTask.java:930) > ... 12 more > > It would be great, if you have any workaround for that. > > Regards, > Nagireddy Y. > > > > > > > On Thu, Aug 3, 202

Re: [ANNOUNCE] New Apache Flink Committer - Weihua Hu

Congratulations Weihua! Best, Xiangyu Leonard Xu 于2023年8月4日周五 11:31写道： > Congratulations Weihua! > > > Best, > Leonard > > > On Aug 4, 2023, at 11:18 AM, Xintong Song wrote: > > > > Hi everyone, > > > > On behalf of the PMC, I'm very happy to announce Weihua Hu as a new Flink > > Committer!

[jira] [Created] (FLINK-32746) Enable ZGC in JDK17 to solve long time class unloading STW during fullgc

xiangyu feng created FLINK-32746: Summary: Enable ZGC in JDK17 to solve long time class unloading STW during fullgc Key: FLINK-32746 URL: https://issues.apache.org/jira/browse/FLINK-32746 Project

Re: Heartbeat of TaskManager with id container

Hi ynagireddy4u, We have met this exception before. Usually it is caused by following reasons: 1), TaskManager is too busy with other works to send the heartbeat to JobMaster or TaskManager process might already exited; 2), There might be a network issues between this TaskManager and JobMaster;

Re: [ANNOUNCE] New Apache Flink Committer - Yong Fang

Congratulations, Yong! Best, Xiangyu liu ron 于2023年7月24日周一 11:48写道： > Congratulations, > > Best, > Ron > > Qingsheng Ren 于2023年7月24日周一 11:18写道： > > > Congratulations and welcome aboard, Yong! > > > > Best, > > Qingsheng > > > > On Mon, Jul 24, 2023 at 11:14 AM Chen Zhanghao < >