[ https://issues.apache.org/jira/browse/FLINK-9609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
mingleizhang updated FLINK-9609: -------------------------------- Description: Currently, BucketingSink only support {{notifyCheckpointComplete}}. But users want to do some extra work when a bucket is ready. It would be nice if we can support {{BucketReady}} functionality for user. For example, One bucket is created for every 5 minutes, and then at the end of 5 minutes before creating the next bucket, the user needs to do something for the previous bucket ready, such as sending the timestamp of the bucket ready time to a server or some other stuff. Here, Bucket ready means all the part files name suffix under a bucket neither {{.pending}} nor {{.in-progress}}. Then we can think this bucket is ready for user use. was: Currently, BucketingSink only support {{notifyCheckpointComplete}}. But users want to do some extra work when a bucket is ready. It would be nice if we can support {{BucketReady}} functionality for user. For example, One bucket is created for every 5 minutes, and then at the end of 5 minutes before creating the next bucket, the user needs to do something for the previous bucket ready, such as sending the timestamp of the bucket ready time to a server or some other stuff. Here, Bucket ready means all the part files name suffix under a bucket status neither {{.pending}} nor {{.in-progress}}. Then we can think this bucket is ready for user use. > Add bucket ready mechanism for BucketingSink when checkpoint complete > --------------------------------------------------------------------- > > Key: FLINK-9609 > URL: https://issues.apache.org/jira/browse/FLINK-9609 > Project: Flink > Issue Type: New Feature > Components: filesystem-connector, State Backends, Checkpointing > Reporter: mingleizhang > Assignee: mingleizhang > Priority: Major > > Currently, BucketingSink only support {{notifyCheckpointComplete}}. But users > want to do some extra work when a bucket is ready. It would be nice if we can > support {{BucketReady}} functionality for user. For example, One bucket is > created for every 5 minutes, and then at the end of 5 minutes before creating > the next bucket, the user needs to do something for the previous bucket > ready, such as sending the timestamp of the bucket ready time to a server or > some other stuff. > Here, Bucket ready means all the part files name suffix under a bucket > neither {{.pending}} nor {{.in-progress}}. Then we can think this bucket is > ready for user use. -- This message was sent by Atlassian JIRA (v7.6.3#76005)