[ https://issues.apache.org/jira/browse/FLINK-15526 ]
luoyuxia deleted comment on FLINK-15526: ---------------------------------- was (Author: luoyuxia): [~lzljs3620320] I see DefaultRollingPolicy.shouldRollOnEvent can already avoid writing a big file by file size. Does the Jira aim to add a policy that can avoid writing a big file by record numebr? > Add RollingPolicy to PartitionWriters > ------------------------------------- > > Key: FLINK-15526 > URL: https://issues.apache.org/jira/browse/FLINK-15526 > Project: Flink > Issue Type: Sub-task > Components: Table SQL / Ecosystem > Reporter: Jingsong Lee > Priority: Major > > Now our partition writers will write a whole file in a checkpoint if there is > no partition change. > Sometimes this file is too big, it is better to add a RollingPolicy for > controlling file size. > We can just add policy controlled by file size and record number. -- This message was sent by Atlassian Jira (v8.20.7#820007)