sergey-chugunov-1985 commented on a change in pull request #8681:
URL: https://github.com/apache/ignite/pull/8681#discussion_r568435274
##########
File path:
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/wal/FileWriteAheadLogManager.java
##########
@@ -1521,21 +1525,129 @@ private void checkOrPrepareFiles() throws
StorageException {
}
}
- File[] allFiles = walWorkDir.listFiles(WAL_SEGMENT_FILE_FILTER);
+ FileDescriptor[] walFiles =
scan(walWorkDir.listFiles(WAL_SEGMENT_FILE_FILTER));
- if (isArchiverEnabled() && !F.isEmpty(allFiles) && allFiles.length >
dsCfg.getWalSegments()) {
- throw new StorageException("Failed to initialize wal (work
directory contains incorrect " +
- "number of segments) [cur=" + allFiles.length + ", expected="
+ dsCfg.getWalSegments() + ']');
- }
+ if (F.isEmpty(walFiles))
+ createFile(new File(walWorkDir, fileName(0)));
- // Allocate the first segment synchronously. All other segments will
be allocated by archiver in background.
- if (F.isEmpty(allFiles)) {
- File first = new File(walWorkDir, fileName(0));
+ if (isArchiverEnabled()) {
+ List<FileDescriptor> toMove = new ArrayList<>();
+ @Nullable FileDescriptor toRen = null;
+
+ if (!F.isEmpty(walFiles) && (walFiles.length >
dsCfg.getWalSegments() || walFiles[0].idx() != 0)) {
+ toMove.addAll(F.asList(walFiles));
+
+ FileDescriptor rmv = toMove.remove(toMove.size() - 1);
+ toRen = rmv.idx() != rmv.idx() % dsCfg.getWalSegments() ? rmv
: null;
+ }
+
+
toMove.addAll(F.asList(scan(walWorkDir.listFiles(WAL_SEGMENT_FILE_COMPACTED_FILTER))));
+
+ if (!toMove.isEmpty() || toRen != null) {
+ log.warning("Invalid WAL working directory contents found, for
archiving to work, will need " +
+ (toMove.isEmpty() ? "" : "move WAL segments to archive " +
(toRen == null ? "" : "and ")) +
+ (toRen == null ? "" : "rename last WAL segment") + ", this
may take some time [" +
+ (toMove.isEmpty() ? "" : "toMove=" +
F.viewReadOnly(toMove, fd -> fd.file().getName())) +
+ (toRen == null ? "" : (toMove.isEmpty() ? "" : ", ") +
"toRename=[" + toRen.file().getName() +
+ " -> " + fileName(toRen.idx() %
dsCfg.getWalSegments()) + ']') + ']');
+
+ for (FileDescriptor fd : toMove) {
+ File tmpDst = new File(walArchiveDir, fd.file().getName()
+ TMP_SUFFIX);
+ File dst = new File(walArchiveDir, fd.file().getName());
+
+ try {
+ Files.copy(fd.file().toPath(), tmpDst.toPath());
+
+ Files.move(tmpDst.toPath(), dst.toPath());
+
+ Files.delete(fd.file().toPath());
+
+ if (log.isInfoEnabled()) {
Review comment:
I can easily imagine a situation when we have dozens or even hundreds
segments to move, printing out a message for each will blow up our logs
immediately.
Suggest replacing with some batching.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
