[
https://issues.apache.org/jira/browse/FLINK-6014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15947097#comment-15947097
]
ASF GitHub Bot commented on FLINK-6014:
---------------------------------------
Github user StephanEwen commented on a diff in the pull request:
https://github.com/apache/flink/pull/3524#discussion_r108670677
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/state/SharedStateRegistry.java
---
@@ -0,0 +1,192 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.state;
+
+import org.apache.flink.annotation.VisibleForTesting;
+import org.apache.flink.api.java.tuple.Tuple2;
+
+import java.io.Serializable;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+/**
+ * A {@code SharedStateRegistry} will be deployed in the
+ * {@link org.apache.flink.runtime.checkpoint.CheckpointCoordinator} to
+ * maintain the reference count of those state objects shared among
different
+ * checkpoints. Each shared state object must be identified by a unique
key.
+ */
+public class SharedStateRegistry implements Serializable {
+
+ private static final long serialVersionUID = -8357254413007773970L;
+
+ /** All registered state objects */
+ private final Map<String, Tuple2<StateObject, Integer>>
registeredStates = new HashMap<>();
+
+ /** All state objects that are not referenced any more */
+ private transient final List<StateObject> discardedStates = new
ArrayList<>();
+
+ /**
+ * Register the state in the registry
+ *
+ * @param key The key of the state to register
+ * @param state The state to register
+ */
+ public void register(String key, StateObject state) {
+ Tuple2<StateObject, Integer> stateAndRefCnt =
registeredStates.get(key);
--- End diff --
This method misses synchronization.
> Allow the registration of state objects in checkpoints
> ------------------------------------------------------
>
> Key: FLINK-6014
> URL: https://issues.apache.org/jira/browse/FLINK-6014
> Project: Flink
> Issue Type: Sub-task
> Components: State Backends, Checkpointing
> Reporter: Xiaogang Shi
> Assignee: Xiaogang Shi
>
> This issue is the very first step towards incremental checkpointing. We
> introduce a new state handle named {{CompositeStateHandle}} to be the base of
> the snapshots taken by task components. Known implementation may include
> {{KeyedStateHandle}} (for {{KeyedStateBackend}}s), {{SubtaskState}} (for
> subtasks, splits of {{JobVertex}}) and {{TaskState}} (for {{JobVertex}}s).
> Each {{CompositeStateHandle}} is composed of a collection of {{StateObject}s.
> It should register all its state objects in {{StateRegistry}} when its
> checkpoint is added into {{CompletedCheckpointStore}} (i.e., a pending
> checkpoint completes or a complete checkpoint is reloaded in the recovery).
> When a completed checkpoint is moved out of the {{CompletedCheckpointStore}},
> we should not simply discard all state objects in the checkpoint. With the
> introduction of incremental checkpointing, a {{StateObject}} may be
> referenced by different checkpoints. We should unregister all the state
> objects contained in the {{StateRegistry}} first. Only those state objects
> that are not referenced by any checkpoint can be deleted.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
