NSAmelchev commented on code in PR #13710:
URL: https://github.com/apache/ignite/pull/13710#discussion_r4445537670


##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/SnapshotRestoreProcess.java:
##########
@@ -1797,6 +1786,52 @@ private void copyLocalAsync(
             });
     }
 
+    /**
+     * Copies a partition of a compressed cache group page by page writing 
only the compressed bytes of each page, so the
+     * rest of the page is left unallocated by the file system and the copy is 
sparse without punching holes in it.
+     */
+    private void copyCompressed(SnapshotRestoreContext opCtx, int grpId, File 
from, File to) {
+        FileIOFactory ioFactory = 
ctx.cache().context().snapshotMgr().ioFactory();
+        FilePageStoreManager storeMgr = 
(FilePageStoreManager)ctx.cache().context().pageStore();
+        int pageSize = ctx.cache().context().database().pageSize();
+
+        try (FileIO src = ioFactory.create(from, READ); FileIO dest = 
ioFactory.create(to, CREATE, WRITE)) {
+            ByteBuffer buf = locBuff.get();
+
+            buf.clear();
+            src.readFully(buf, 0);
+            buf.flip();
+
+            buf.getLong(); // Signature.
+
+            int hdrSize = storeMgr.getPageStoreFactory(grpId, 
null).headerSize(buf.getInt());
+
+            buf.rewind();
+            buf.limit(hdrSize);
+            dest.writeFully(buf, 0);

Review Comment:
   Yes, this block only finds the header size and copies the header as is. The 
size depends on the page store version stored in the file (V1 or V2). 
FileVersionCheckingFactory#createPageStore reads the version too, but it opens 
the file with CREATE, READ and WRITE, which fails on a read-only snapshot. So 
the version is read here and passed to headerSize()



##########
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/persistence/snapshot/SnapshotRestoreProcess.java:
##########
@@ -1797,6 +1786,52 @@ private void copyLocalAsync(
             });
     }
 
+    /**
+     * Copies a partition of a compressed cache group page by page writing 
only the compressed bytes of each page, so the
+     * rest of the page is left unallocated by the file system and the copy is 
sparse without punching holes in it.
+     */
+    private void copyCompressed(SnapshotRestoreContext opCtx, int grpId, File 
from, File to) {
+        FileIOFactory ioFactory = 
ctx.cache().context().snapshotMgr().ioFactory();
+        FilePageStoreManager storeMgr = 
(FilePageStoreManager)ctx.cache().context().pageStore();
+        int pageSize = ctx.cache().context().database().pageSize();
+
+        try (FileIO src = ioFactory.create(from, READ); FileIO dest = 
ioFactory.create(to, CREATE, WRITE)) {
+            ByteBuffer buf = locBuff.get();
+
+            buf.clear();
+            src.readFully(buf, 0);
+            buf.flip();
+
+            buf.getLong(); // Signature.
+
+            int hdrSize = storeMgr.getPageStoreFactory(grpId, 
null).headerSize(buf.getInt());
+
+            buf.rewind();
+            buf.limit(hdrSize);
+            dest.writeFully(buf, 0);
+
+            for (long off = hdrSize, len = src.size(); off < len; off += 
pageSize) {
+                if (opCtx.stopChecker.getAsBoolean())
+                    throw new IgniteInterruptedException("The operation has 
been stopped on copy file: " + from.getAbsolutePath());
+
+                if (Thread.interrupted())
+                    throw new IgniteInterruptedException("Thread has been 
interrupted: " + Thread.currentThread().getName());
+
+                buf.clear();
+                src.readFully(buf, off);
+                buf.flip();
+
+                if (PageIO.getCompressionType(buf) != 
CompressionProcessor.UNCOMPRESSED_PAGE)
+                    buf.limit(PageIO.getCompressedSize(buf));

Review Comment:
   You’re right, skipping bytes doesn’t guarantee holes. On XFS, extending 
writes trigger speculative preallocation, and the blocks between pages stay 
allocated: a 100 MiB partition with 50 MiB of compressed data took 100 MiB, 
even after a remount (see “speculative preallocation” in 
https://xfs.org/index.php/XFS_FAQ). ext4 isn’t affected. The file now gets its 
full size before the pages are written, so the writes no longer extend it, and 
the same file takes 50 MiB on XFS. Calling punchHole() per page gives the same 
size, but brings back one fallocate per page. Added 
SnapshotCompressionRestoreSizeTest with partitions large enough to trigger the 
preallocation: it fails on XFS without the fix.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to