sanpwc commented on code in PR #937:
URL: https://github.com/apache/ignite-3/pull/937#discussion_r927734597
##########
modules/table/src/main/java/org/apache/ignite/internal/table/distributed/raft/PartitionListener.java:
##########
@@ -135,6 +135,18 @@ public void onWrite(Iterator<CommandClosure<WriteCommand>>
iterator) {
return;
}
+ long lastAppliedIndex = clo.lastAppliedIndex();
+
+ long storageAppliedIndex = storage.lastAppliedIndex();
+
+ assert storageAppliedIndex < lastAppliedIndex
Review Comment:
I'm not sure whether it'll be possible to preserve given assert in case of
multiple storages for the single raft log.
##########
modules/raft-client/src/main/java/org/apache/ignite/raft/client/service/CommandClosure.java:
##########
@@ -28,6 +28,14 @@
* @see RaftGroupListener
*/
public interface CommandClosure<R extends Command> {
+ /**
+ * Corresponding log index of the command. Present for write commands only.
+ * Returns {@code 0} for read commands.
+ */
+ default long lastAppliedIndex() {
Review Comment:
Not sure whether it's a proper naming, just because corresponding command
ins't applied yet. Actually it's an apply**ing** index or just index))
##########
modules/table/src/main/java/org/apache/ignite/internal/table/distributed/raft/snapshot/PartitionSnapshotStorageFactory.java:
##########
@@ -0,0 +1,85 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.table.distributed.raft.snapshot;
+
+import java.util.List;
+import org.apache.ignite.internal.raft.storage.SnapshotStorageFactory;
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.raft.jraft.RaftMessagesFactory;
+import org.apache.ignite.raft.jraft.entity.RaftOutter.SnapshotMeta;
+import org.apache.ignite.raft.jraft.option.RaftOptions;
+import org.apache.ignite.raft.jraft.storage.SnapshotStorage;
+import org.apache.ignite.raft.jraft.storage.snapshot.SnapshotReader;
+import org.apache.ignite.raft.jraft.storage.snapshot.SnapshotWriter;
+
+/**
+ * Snapshot storage factory for {@link MvPartitionStorage}. Utilizes the fact
that every partition already stores its latest applied index
+ * and thus can inself be used as its own snapshot.
+ *
+ * <p/>Uses {@link MvPartitionStorage#persistedIndex()} and configuration,
passed into constructor, to create a {@link SnapshotMeta} object
+ * in {@link SnapshotReader#load()}.
+ *
+ * <p/>Snapshot writer doesn't allow explicit save of any actual file. {@link
SnapshotWriter#saveMeta(SnapshotMeta)} simply returns
+ * {@code true}, and {@link SnapshotWriter#addFile(String)} throws an
exception.
+ */
+public class PartitionSnapshotStorageFactory implements SnapshotStorageFactory
{
+ /** Partition storage. */
+ private final MvPartitionStorage partitionStorage;
+
+ /** List of peers. */
+ private final List<String> peers;
Review Comment:
Unfortunately that won't work. It's possible that IgniteNode will change
host/port after restart. Generally speaking it's a problem of ClusterNode ->
Peer -> ClusterNode resolution.
##########
modules/table/src/main/java/org/apache/ignite/internal/table/distributed/raft/snapshot/PartitionSnapshotStorage.java:
##########
@@ -0,0 +1,123 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.ignite.internal.table.distributed.raft.snapshot;
+
+import org.apache.ignite.internal.storage.MvPartitionStorage;
+import org.apache.ignite.raft.jraft.entity.RaftOutter.SnapshotMeta;
+import org.apache.ignite.raft.jraft.option.RaftOptions;
+import org.apache.ignite.raft.jraft.option.SnapshotCopierOptions;
+import org.apache.ignite.raft.jraft.storage.SnapshotStorage;
+import org.apache.ignite.raft.jraft.storage.SnapshotThrottle;
+import org.apache.ignite.raft.jraft.storage.snapshot.SnapshotCopier;
+import org.apache.ignite.raft.jraft.storage.snapshot.SnapshotReader;
+import org.apache.ignite.raft.jraft.storage.snapshot.SnapshotWriter;
+import org.jetbrains.annotations.Nullable;
+
+/**
+ * Snapshot storage for {@link MvPartitionStorage}.
+ *
+ * @see PartitionSnapshotStorageFactory
+ */
+class PartitionSnapshotStorage implements SnapshotStorage {
+ /** Snapshot URI. Points to a snopshot folder. Never created on physical
storage. */
Review Comment:
typo: sn**o**pshot
##########
modules/storage-page-memory/src/main/java/org/apache/ignite/internal/storage/pagememory/mv/PageMemoryMvPartitionStorage.java:
##########
@@ -71,6 +72,12 @@ public class PageMemoryMvPartitionStorage implements
MvPartitionStorage {
ScanVersionChainByTimestamp::new
);
+ /**
+ * Last applied index value.
+ */
+ //TODO Make it persistent. IGNITE-17077
+ private volatile long lastAppliedIndex = 0;
Review Comment:
It's either volatile or persistent storage (probably volatile), so it's
either necessary to make "lastAppliedIndex" persistent or implement for each.
Generally speaking storage hierarchy is a bit messy: I'd rather introduce
Volatile/PersistentPageMemoryMvParitionStorage both extending corresponding
Abstract class.
##########
modules/raft/src/test/java/org/apache/ignite/internal/raft/server/impl/RaftServerImpl.java:
##########
@@ -72,6 +73,8 @@ public class RaftServerImpl implements RaftServer {
private final BlockingQueue<CommandClosureEx<WriteCommand>> writeQueue;
+ private final AtomicLong lastAppliedIndex = new AtomicLong();
Review Comment:
Well, generally speaking RafServer manages multiple raft groups and
lastAppliedIndex (that is actually not an applied index) is an entity per raft
group. So there should be a map groupId -> index similar to listeners map.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
