[ https://issues.apache.org/jira/browse/FLINK-9114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Flink Jira Bot updated FLINK-9114: ---------------------------------- Labels: auto-deprioritized-major auto-unassigned (was: auto-unassigned stale-major) Priority: Minor (was: Major) This issue was labeled "stale-major" 7 ago and has not received any updates so it is being deprioritized. If this ticket is actually Major, please raise the priority and ask a committer to assign you the issue or revive the public discussion. > Enable user-provided, custom CheckpointRecoveryFactory for non-HA deployments > ----------------------------------------------------------------------------- > > Key: FLINK-9114 > URL: https://issues.apache.org/jira/browse/FLINK-9114 > Project: Flink > Issue Type: Improvement > Components: Runtime / Configuration, Runtime / State Backends > Reporter: Jacob Park > Priority: Minor > Labels: auto-deprioritized-major, auto-unassigned > > When you operate a Flink application that uses externalized checkpoints to > S3, it becomes difficult to determine which checkpoint is the latest to > recover from. Because S3 provides read-after-write consistency only for PUTS, > listing a S3 path is not guaranteed to be consistent, so we do not know what > checkpoint to recover from. > The goal of this improvement is to allow users to provide a custom > CheckpointRecoveryFactory for non-HA deployments such that we can use this > feature to fail checkpoints if we cannot guarantee we will know where a > checkpoint will be in S3, and co-publish checkpoint metadata to a strongly > consistent data store. > I propose the following changes: > # Modify AbstractNonHaServices and StandaloneHaServices to accept an > Executor for the custom CheckpointRecoveryFactory. > # Create a CheckpointRecoveryFactoryLoader to provide the custom > CheckpointRecoveryFactory from configurations. > # Add new configurations for this feature. > We considered the pluggable StateBackend and the potentially pluggable > HighAvailabilityServices. These were too convoluted to solve our problem, so > we would like to implement a custom CheckpointRecoveryFactory mechanism. -- This message was sent by Atlassian Jira (v8.3.4#803005)