[
https://issues.apache.org/jira/browse/FLINK-9609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mingleizhang updated FLINK-9609:
--------------------------------
Description:
Currently, BucketingSink only support {{notifyCheckpointComplete}}. But users
want to do some extra work when a bucket is ready. It would be nice if we can
support {{BucketReady}} functionality for user. For example, One bucket is
created for every 5 minutes, and then at the end of 5 minutes before creating
the next bucket, the user needs to do something for the previous bucket ready,
such as sending the timestamp of the bucket ready time to a server or some
other stuff.
Here, Bucket ready means all the part files name suffix under a bucket status
neither {{.pending}} nor {{.in-progress}}. Then we can think this bucket is
ready.
was:
Currently, BucketingSink only support {{notifyCheckpointComplete}}. But users
want to do some extra work when a bucket is ready. It would be nice if we can
support {{BucketReady}} functionality for user. For example, One bucket is
created for every 5 minutes, and then at the end of 5 minutes before creating
the next bucket, the user needs to do something for the previous bucket ready,
such as sending the timestamp of the bucket ready time to a server or some
other stuff.
Here, Bucket ready means all the part files name under a bucket status neither
{{.pending}} nor {{.in-progress}}. Then we can think this bucket is ready.
> Add bucket ready notification mechanism for BucketingSink when checkpoint
> complete
> ----------------------------------------------------------------------------------
>
> Key: FLINK-9609
> URL: https://issues.apache.org/jira/browse/FLINK-9609
> Project: Flink
> Issue Type: Improvement
> Components: filesystem-connector, State Backends, Checkpointing
> Reporter: mingleizhang
> Assignee: mingleizhang
> Priority: Major
>
> Currently, BucketingSink only support {{notifyCheckpointComplete}}. But users
> want to do some extra work when a bucket is ready. It would be nice if we can
> support {{BucketReady}} functionality for user. For example, One bucket is
> created for every 5 minutes, and then at the end of 5 minutes before creating
> the next bucket, the user needs to do something for the previous bucket
> ready, such as sending the timestamp of the bucket ready time to a server or
> some other stuff.
> Here, Bucket ready means all the part files name suffix under a bucket status
> neither {{.pending}} nor {{.in-progress}}. Then we can think this bucket is
> ready.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
