[
https://issues.apache.org/jira/browse/FLINK-6014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15932486#comment-15932486
]
ASF GitHub Bot commented on FLINK-6014:
---------------------------------------
Github user StephanEwen commented on the issue:
https://github.com/apache/flink/pull/3524
@shixiaogang Thanks for the fast response!
Can y In the initial design document, you suggest that shared state is only
s
I think what we need is a subclass of `StateHandle` that is a
`SharedStateHandle`. I would suggest that the shared state handle has a method
`String getKey()` (or `Object getKey()`) which gives the unique identifier of
the shared state. The `SharedStateRegistry` internally could use something like
a `Map<String, SharedStateHandleAndRefCount>`. I think that would give us a bit
more flexibility in how we describe "equality of shared state": we don't need
to make sure that the state handles themselves implement `equals()` such that
it meets the semantics of the shared state registry. In the case we have
currently, the `getKey()` method could return the normalized path of the file.
> Allow the registration of state objects in checkpoints
> ------------------------------------------------------
>
> Key: FLINK-6014
> URL: https://issues.apache.org/jira/browse/FLINK-6014
> Project: Flink
> Issue Type: Sub-task
> Components: State Backends, Checkpointing
> Reporter: Xiaogang Shi
> Assignee: Xiaogang Shi
>
> This issue is the very first step towards incremental checkpointing. We
> introduce a new state handle named {{CompositeStateHandle}} to be the base of
> the snapshots taken by task components. Known implementation may include
> {{KeyedStateHandle}} (for {{KeyedStateBackend}}s), {{SubtaskState}} (for
> subtasks, splits of {{JobVertex}}) and {{TaskState}} (for {{JobVertex}}s).
> Each {{CompositeStateHandle}} is composed of a collection of {{StateObject}s.
> It should register all its state objects in {{StateRegistry}} when its
> checkpoint is added into {{CompletedCheckpointStore}} (i.e., a pending
> checkpoint completes or a complete checkpoint is reloaded in the recovery).
> When a completed checkpoint is moved out of the {{CompletedCheckpointStore}},
> we should not simply discard all state objects in the checkpoint. With the
> introduction of incremental checkpointing, a {{StateObject}} may be
> referenced by different checkpoints. We should unregister all the state
> objects contained in the {{StateRegistry}} first. Only those state objects
> that are not referenced by any checkpoint can be deleted.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
