Hi, Rui. Thank you very much for your review and reply. > I prefer the Candidate design 1 that mentioned in FLIP for 2 reasons: > 1. The main design does not support fine grain settings for flink sql, but > nowadays Flink SQL is becoming increasingly popular. > 2. As you mentioned in FLIP, the main design is complex.
> Not sure if it is possible to introduce the adaptive partitioner on job > level in this FLIP first, and improve it for both jar and sql jobs if the > fine grained is needed in the future. > > WDYT? Thanks for the comments, That's fine to me! And I adjusted the wiki content and structure for changing the original candidate design 1 as the new main design and changing the old main design as the rejected design. PTAL~ > > taskmanager.network.adaptive-partitioner.enable > > Also, one small tip: most enabled configuration option names end with > "enabled". Nice catch and thanks for the reminder. Updated. Looking forward to your next review~ Best regards, Yuepeng Pan Rui Fan <[email protected]> 于2026年1月7日周三 10:18写道： > Thanks Yuepeng for driving this FLIP, it could avoid bottlenecks when part > of downstream subtask is under pressure. > > I prefer the Candidate design 1 that mentioned in FLIP for 2 reasons: > 1. The main design does not support fine grain settings for flink sql, but > nowadays Flink SQL is becoming increasingly popular. > 2. As you mentioned in FLIP, the main design is complex. > > Not sure if it is possible to introduce the adaptive partitioner on job > level in this FLIP first, and improve it for both jar and sql jobs if the > fine grained is needed in the future. > > WDYT? > > > taskmanager.network.adaptive-partitioner.enable > > Also, one small tip: most enabled configuration option names end with > "enabled". > > Best, > Rui > > On Wed, Jan 7, 2026 at 9:34 AM Yuepeng Pan <[email protected]> wrote: > > > Bumping this thread. Thanks! > > > > Best regards, > > Yuepeng Pan > > > > > > Pan Yuepeng <[email protected]> 于2025年12月15日周一 12:40写道： > > > > > Hi devs, > > > > > > I re-sorted out and supplemented the 'FLIP-339[1] Support Adaptive > > > Partition Selection for StreamPartitioner' based on Flink JIRA[2]. > > > > > > Flink offers multiple partition strategies, some of which bind data to > > > downstream subtasks, while others do not (e.g., shuffle, rescale, > > > rebalance). > > > For [Data not bound to subtasks] scenarios, overloaded sub-task-nodes > may > > > slow down the processing of Flink jobs, leading to backpressure and > data > > > lag. Dynamically adjusting the partition of data to subtasks based on > the > > > processing load of downstream operators helps achieve a peak-shaving > and > > > valley-filling effect, thereby striving to maintain the throughput of > > Flink > > > jobs. > > > > > > The raw discussions could be found in the Flink JIRA[2]. > > > I really appreciate developers involved in the discussion for the > > valuable > > > help and suggestions in advance. > > > > > > Please refer to the FLIP[1] wiki for more details about the proposed > > > design and implementation. > > > > > > Welcome any feedback and opinions on this proposal. > > > > > > [1] https://cwiki.apache.org/confluence/x/nYyzDw > > > [2] https://issues.apache.org/jira/browse/FLINK-31655 > > > > > > Best regards, > > > Yuepeng Pan > > > > > >
