Thanks Wencong for driving this FLIP. +1 from my side. It appears to significantly improve the handling of full-window data within the DataStream API. However, I do have a small question regarding the current limitation to batch processing: does this stem from performance-related considerations? Additionally, is there any possibility that support for streaming in the future?
In addition, some format of the section `API implementation` is not right (some lines have exceeded the text box), maybe we can update and fix it. Best, Yuxin weijie guo <[email protected]> 于2023年12月12日周二 15:09写道： > Thanks Wencong for driving this! > > I believe this is a useful feature, so +1 from my side. > > I only have one minor question about the exchange mode of `xxxPartition` > method. Does this means the window operator must be connected to the > upstream operator in forward edge (otherwise the concept of mapPartition is > a bit far-fetched). > > Best regards, > > Weijie > > > Wencong Liu <[email protected]> 于2023年12月1日周五 14:04写道： > > > Hi devs, > > > > I'm excited to propose a new FLIP[1] aimed at enhancing the DataStream > API > > > > to support full window processing on non-keyed streams. This feature > > addresses > > the current limitation where non-keyed DataStreams cannot accumulate > > records > > per subtask for collective processing at the end of input. > > > > Key proposals include: > > > > > > 1. Introduction of PartitionWindowedStream allowing non-keyed DataStreams > > to > > be transformed for full window processing per subtask. > > > > 2. Addition of four new APIs - mapPartition, sortPartition, aggregate, > and > > reduce > > - to enable powerful operations on PartitionWindowedStream. > > > > This initiative seeks to fill the gap left by the deprecation of the > > DataSet API, > > marrying its partition processing strengths with the dynamic capabilities > > of the DataStream API. > > > > Looking forward to your feedback on this FLIP. > > > > [1] > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-380%3A+Support+Full+Partition+Processing+On+Non-keyed+DataStream > > > > Best regards, > > Wencong Liu >
