timoninmaxim commented on code in PR #12602:
URL: https://github.com/apache/ignite/pull/12602#discussion_r2643372228
##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/SnapshotCheckProcess.java:
##########
@@ -406,8 +406,11 @@ private CompletableFuture<SnapshotCheckResponse>
allHandlersFuture(SnapshotCheck
AtomicInteger metasProcessed = new AtomicInteger(ctx.metas.size());
for (SnapshotMetadata meta : ctx.metas) {
- CompletableFuture<Map<String, SnapshotHandlerResult<Object>>>
metaFut = snpChecker.invokeCustomHandlers(meta,
- ctx.locFileTree.get(meta.consistentId()), ctx.req.groups(),
ctx.req.fullCheck());
+ SnapshotFileTree sft = ctx.locFileTree.get(meta.consistentId());
Review Comment:
Looks like useless change
##########
modules/core/src/main/java/org/apache/ignite/configuration/DataStorageConfiguration.java:
##########
@@ -601,6 +621,19 @@ public DataStorageConfiguration
setExtraStoragePaths(String... extraStoragePaths
return this;
}
+ /**
+ * Sets a paths to the root directories where the snapshot files stored.
+ * By default, {@link IgniteConfiguration#getSnapshotPath()} used.
+ *
+ * @param extraSnapshotPaths Extra snapshot paths where snapshot files can
be stored.
+ * @return {@code this} for chaining.
+ */
+ public DataStorageConfiguration setExtraSnapshotPaths(String...
extraSnapshotPaths) {
Review Comment:
Let's mention in docs that amount of extraSnapshotPaths must be equal amount
of extra storage paths
##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/filename/SnapshotFileTree.java:
##########
@@ -380,14 +381,41 @@ public File meta() {
* This will distribute workload to all physical device on host.
*
* @param ft Node file tree.
- * @param snpDfltPath Snapshot default path.
+ * @param cfg Ignite configuration.
*/
- private Map<String, File> snapshotExtraStorages(NodeFileTree ft, String
snpDfltPath) {
+ private Map<String, File> snapshotExtraStorages(NodeFileTree ft,
IgniteConfiguration cfg) {
+ String snpDfltPath = cfg.getSnapshotPath();
+
+ DataStorageConfiguration dsCfg = cfg.getDataStorageConfiguration();
+ boolean hasExtraSnpPaths = dsCfg != null &&
dsCfg.getExtraSnapshotPaths() != null && dsCfg.getExtraSnapshotPaths().length >
0;
+
// If path provided then create snapshot inside it, only.
- // Same rule applies if absolute path to the snapshot root dir
configured.
- if (path != null || new File(snpDfltPath).isAbsolute())
+ // Same rule applies if absolute path to the snapshot root dir
configured and no extra snapshot paths.
+ if (path != null || (new File(snpDfltPath).isAbsolute() &&
!hasExtraSnpPaths))
return Collections.emptyMap();
+ if (hasExtraSnpPaths) {
+ String[] extraSnpPaths = dsCfg.getExtraSnapshotPaths();
+ String[] extraStorages = dsCfg.getExtraStoragePaths();
+
+ assert extraSnpPaths.length == extraStorages.length;
+
+ Map<String, File> snpExtraStorages = new HashMap<>();
+
+ for (int i = 0; i < extraSnpPaths.length; i++) {
+ // Extra snapshot paths configured as "root" related.
+ File snpDir = ft.resolveDirectory(Path.of(extraSnpPaths[i],
DFLT_SNAPSHOT_DIRECTORY, name, DB_DIR).toString());
Review Comment:
Why do you insist on default snapshot directory in case of extra paths?
##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/filename/SnapshotFileTree.java:
##########
@@ -380,14 +381,41 @@ public File meta() {
* This will distribute workload to all physical device on host.
*
* @param ft Node file tree.
- * @param snpDfltPath Snapshot default path.
+ * @param cfg Ignite configuration.
*/
- private Map<String, File> snapshotExtraStorages(NodeFileTree ft, String
snpDfltPath) {
+ private Map<String, File> snapshotExtraStorages(NodeFileTree ft,
IgniteConfiguration cfg) {
+ String snpDfltPath = cfg.getSnapshotPath();
+
+ DataStorageConfiguration dsCfg = cfg.getDataStorageConfiguration();
+ boolean hasExtraSnpPaths = dsCfg != null &&
dsCfg.getExtraSnapshotPaths() != null && dsCfg.getExtraSnapshotPaths().length >
0;
+
// If path provided then create snapshot inside it, only.
- // Same rule applies if absolute path to the snapshot root dir
configured.
- if (path != null || new File(snpDfltPath).isAbsolute())
+ // Same rule applies if absolute path to the snapshot root dir
configured and no extra snapshot paths.
+ if (path != null || (new File(snpDfltPath).isAbsolute() &&
!hasExtraSnpPaths))
return Collections.emptyMap();
+ if (hasExtraSnpPaths) {
+ String[] extraSnpPaths = dsCfg.getExtraSnapshotPaths();
+ String[] extraStorages = dsCfg.getExtraStoragePaths();
+
+ assert extraSnpPaths.length == extraStorages.length;
+
+ Map<String, File> snpExtraStorages = new HashMap<>();
+
+ for (int i = 0; i < extraSnpPaths.length; i++) {
+ // Extra snapshot paths configured as "root" related.
+ File snpDir = ft.resolveDirectory(Path.of(extraSnpPaths[i],
DFLT_SNAPSHOT_DIRECTORY, name, DB_DIR).toString());
+
+ // folderName can be different from local (NodeFileTree ft)
one.
+ // In case snapshot restored on smaller topology and data from
some node copied to local.
+ snpDir = new File(snpDir.getParent(), folderName());
+
+ snpExtraStorages.put(extraStorages[i], snpDir);
+ }
+
+ return snpExtraStorages;
+ }
+
Map<String, File> snpExtraStorages = new HashMap<>();
Review Comment:
I don't like that we have 2 different logics for snapshot path depending on
snapshot root directory. Can you unify it, for example generate extra paths if
they not explicitly set.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
