[
https://issues.apache.org/jira/browse/FLINK-5823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15999804#comment-15999804
]
ASF GitHub Bot commented on FLINK-5823:
---------------------------------------
Github user StephanEwen commented on the issue:
https://github.com/apache/flink/pull/3522
I would like to merge this for 1.3 with a slightly reduced scope.
I would include the refactoring to make the State Backends also responsible
for storing the Metadata.
I would NOT merge the started work about cleanup hooks. The reason for that
is that after some digging through this, I found we actually have to treat
different file systems quite differently. For example for HDFS and posix-style
file systems, we want to dispose the checkpoint-exclusive state via a `rm -r`.
However, for S3 we explicitly do not want to do that, but issue DELETE
commands for each state object independently, because Flink has a more
consistent view on what objects exist for a checkpoint than S3 has. Also,
because a `rm -r` on the S3 file system means "list objects with path prefix;
foreach object: delete" which is even worse.
> Store Checkpoint Root Metadata in StateBackend (not in HA custom store)
> -----------------------------------------------------------------------
>
> Key: FLINK-5823
> URL: https://issues.apache.org/jira/browse/FLINK-5823
> Project: Flink
> Issue Type: Sub-task
> Components: State Backends, Checkpointing
> Reporter: Stephan Ewen
> Assignee: Stephan Ewen
> Fix For: 1.3.0
>
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
