Hi Filipe,
At 02/15/2017 04:35 AM, fdman...@kernel.org wrote:
From: Filipe Manana <fdman...@suse.com>
Test that both a full and incremental btrfs send operation preserves file
holes.
I found the test case fails with compress=lzo mount option.
In fact, it turns out to be 2 bugs:
[Inline-then-regular layout]
And further more, it seems that it can create inline-then-regular file
extents layout with compress=lzo.
It does not always create such inline-then-regular, but the possibility
seems quite high in my test box, near 80%.
Btrfs dump tree for snap2 (258):
item 9 key (257 INODE_REF 256) itemoff 15736 itemsize 13
inode ref index 2 namelen 3 name: foo
item 10 key (257 EXTENT_DATA 0) itemoff 15663 itemsize 73
generation 7 type 0 (inline)
inline extent data size 52 ram_bytes 4096 compression 2
(lzo)
item 11 key (257 EXTENT_DATA 4096) itemoff 15610 itemsize 53
generation 8 type 1 (regular)
extent data disk byte 0 nr 0
extent data offset 0 nr 1048576 ram 1052672
extent compression 0 (none)
item 12 key (257 EXTENT_DATA 1052672) itemoff 15557 itemsize 53
generation 8 type 1 (regular)
extent data disk byte 12582912 nr 4096
extent data offset 0 nr 4096 ram 4096
extent compression 0 (none)
[Send-stream without hole]
This bug is 100% reproducible, the send stream of snap2 contains no
hole, just the full file contents.
Thanks,
Qu
This used to fail when the filesystem had the NO_HOLES feature enabled,
that is, when the test is run with MKFS_OPTIONS="-O no-holes".
This is fixed by the following patch for the linux kernel:
"Btrfs: incremental send, fix unnecessary hole writes for sparse files"
Signed-off-by: Filipe Manana <fdman...@suse.com>
---
tests/btrfs/137 | 141 ++++++++++++++++++++++++++++++++++++++++++++++++++++
tests/btrfs/137.out | 63 +++++++++++++++++++++++
tests/btrfs/group | 1 +
3 files changed, 205 insertions(+)
create mode 100755 tests/btrfs/137
create mode 100644 tests/btrfs/137.out
diff --git a/tests/btrfs/137 b/tests/btrfs/137
new file mode 100755
index 0000000..3ff2c6b
--- /dev/null
+++ b/tests/btrfs/137
@@ -0,0 +1,141 @@
+#! /bin/bash
+# FS QA Test No. btrfs/137
+#
+# Test that both incremental and full send operations preserve file holes.
+#
+#-----------------------------------------------------------------------
+#
+# Copyright (C) 2017 SUSE Linux Products GmbH. All Rights Reserved.
+# Author: Filipe Manana <fdman...@suse.com>
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+#-----------------------------------------------------------------------
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+tmp=/tmp/$$
+status=1 # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+ cd /
+ rm -fr $send_files_dir
+ rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/punch
+
+# real QA test starts here
+_supported_fs btrfs
+_supported_os Linux
+_require_test
+_require_scratch
+_require_xfs_io_command "fiemap"
+
+send_files_dir=$TEST_DIR/btrfs-test-$seq
+
+rm -f $seqres.full
+rm -fr $send_files_dir
+mkdir $send_files_dir
+
+_scratch_mkfs >>$seqres.full 2>&1
+_scratch_mount
+
+# Create the first test file.
+$XFS_IO_PROG -f -c "pwrite -S 0xaa 0 4K" $SCRATCH_MNT/foo | _filter_xfs_io
+
+# Create a second test file with a 1Mb hole.
+$XFS_IO_PROG -f \
+ -c "pwrite -S 0xaa 0 4K" \
+ -c "pwrite -S 0xbb 1028K 4K" \
+ $SCRATCH_MNT/bar | _filter_xfs_io
+
+$BTRFS_UTIL_PROG subvolume snapshot -r $SCRATCH_MNT \
+ $SCRATCH_MNT/snap1 >/dev/null
+
+# Now add one new extent to our first test file, increasing its size and
leaving
+# a 1Mb hole between the first extent and this new extent.
+$XFS_IO_PROG -c "pwrite -S 0xbb 1028K 4K" $SCRATCH_MNT/foo | _filter_xfs_io
+
+# Now overwrite the last extent of our second test file.
+$XFS_IO_PROG -c "pwrite -S 0xcc 1028K 4K" $SCRATCH_MNT/bar | _filter_xfs_io
+
+$BTRFS_UTIL_PROG subvolume snapshot -r $SCRATCH_MNT \
+ $SCRATCH_MNT/snap2 >/dev/null
+
+echo
+echo "File digests in the original filesystem:"
+md5sum $SCRATCH_MNT/snap1/foo | _filter_scratch
+md5sum $SCRATCH_MNT/snap1/bar | _filter_scratch
+md5sum $SCRATCH_MNT/snap2/foo | _filter_scratch
+md5sum $SCRATCH_MNT/snap2/bar | _filter_scratch
+
+echo
+echo "File snap1/foo fiemap results in the original filesystem:"
+$XFS_IO_PROG -r -c "fiemap -v" $SCRATCH_MNT/snap1/foo | _filter_fiemap
+echo
+echo "File snap1/bar fiemap results in the original filesystem:"
+$XFS_IO_PROG -r -c "fiemap -v" $SCRATCH_MNT/snap1/bar | _filter_fiemap
+echo
+echo "File snap2/foo fiemap results in the original filesystem:"
+$XFS_IO_PROG -r -c "fiemap -v" $SCRATCH_MNT/snap2/foo | _filter_fiemap
+echo
+echo "File snap2/bar fiemap results in the original filesystem:"
+$XFS_IO_PROG -r -c "fiemap -v" $SCRATCH_MNT/snap2/bar | _filter_fiemap
+echo
+
+# Create the send streams to apply later on a new filesystem.
+$BTRFS_UTIL_PROG send $SCRATCH_MNT/snap1 -f $send_files_dir/1.snap 2>&1 \
+ | _filter_scratch
+$BTRFS_UTIL_PROG send -p $SCRATCH_MNT/snap1 $SCRATCH_MNT/snap2 \
+ -f $send_files_dir/2.snap 2>&1 | _filter_scratch
+
+# Create a new filesystem, receive the send streams and verify that the file
+# contents are the same as in the original filesystem and that the file holes
+# exists in both snapshots.
+_scratch_unmount
+_scratch_mkfs >>$seqres.full 2>&1
+_scratch_mount
+$BTRFS_UTIL_PROG receive $SCRATCH_MNT -f $send_files_dir/1.snap >/dev/null
+$BTRFS_UTIL_PROG receive $SCRATCH_MNT -f $send_files_dir/2.snap >/dev/null
+
+echo
+echo "File digests in the new filesystem:"
+md5sum $SCRATCH_MNT/snap1/foo | _filter_scratch
+md5sum $SCRATCH_MNT/snap1/bar | _filter_scratch
+md5sum $SCRATCH_MNT/snap2/foo | _filter_scratch
+md5sum $SCRATCH_MNT/snap2/bar | _filter_scratch
+
+echo
+echo "File snap1/foo fiemap results in the new filesystem:"
+$XFS_IO_PROG -r -c "fiemap -v" $SCRATCH_MNT/snap1/foo | _filter_fiemap
+echo
+echo "File snap1/bar fiemap results in the new filesystem:"
+$XFS_IO_PROG -r -c "fiemap -v" $SCRATCH_MNT/snap1/bar | _filter_fiemap
+echo
+echo "File snap2/foo fiemap results in the new filesystem:"
+$XFS_IO_PROG -r -c "fiemap -v" $SCRATCH_MNT/snap2/foo | _filter_fiemap
+echo
+echo "File snap2/bar fiemap results in the new filesystem:"
+$XFS_IO_PROG -r -c "fiemap -v" $SCRATCH_MNT/snap2/bar | _filter_fiemap
+
+status=0
+exit
diff --git a/tests/btrfs/137.out b/tests/btrfs/137.out
new file mode 100644
index 0000000..8554399
--- /dev/null
+++ b/tests/btrfs/137.out
@@ -0,0 +1,63 @@
+QA output created by 137
+wrote 4096/4096 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 4096/4096 bytes at offset 0
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 4096/4096 bytes at offset 1052672
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 4096/4096 bytes at offset 1052672
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 4096/4096 bytes at offset 1052672
+XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+File digests in the original filesystem:
+3e4309c7cc81f23d45e260a8f13ca860 SCRATCH_MNT/snap1/foo
+f3934f0cf164e2efa1bab71f2f164990 SCRATCH_MNT/snap1/bar
+f3934f0cf164e2efa1bab71f2f164990 SCRATCH_MNT/snap2/foo
+d3dc847171f9081bd75d7a2d3b53d322 SCRATCH_MNT/snap2/bar
+
+File snap1/foo fiemap results in the original filesystem:
+0: [0..7]: data
+
+File snap1/bar fiemap results in the original filesystem:
+0: [0..7]: data
+1: [8..2055]: hole
+2: [2056..2063]: data
+
+File snap2/foo fiemap results in the original filesystem:
+0: [0..7]: data
+1: [8..2055]: hole
+2: [2056..2063]: data
+
+File snap2/bar fiemap results in the original filesystem:
+0: [0..7]: data
+1: [8..2055]: hole
+2: [2056..2063]: data
+
+At subvol SCRATCH_MNT/snap1
+At subvol SCRATCH_MNT/snap2
+At subvol snap1
+
+File digests in the new filesystem:
+3e4309c7cc81f23d45e260a8f13ca860 SCRATCH_MNT/snap1/foo
+f3934f0cf164e2efa1bab71f2f164990 SCRATCH_MNT/snap1/bar
+f3934f0cf164e2efa1bab71f2f164990 SCRATCH_MNT/snap2/foo
+d3dc847171f9081bd75d7a2d3b53d322 SCRATCH_MNT/snap2/bar
+
+File snap1/foo fiemap results in the new filesystem:
+0: [0..7]: data
+
+File snap1/bar fiemap results in the new filesystem:
+0: [0..7]: data
+1: [8..2055]: hole
+2: [2056..2063]: data
+
+File snap2/foo fiemap results in the new filesystem:
+0: [0..7]: data
+1: [8..2055]: hole
+2: [2056..2063]: data
+
+File snap2/bar fiemap results in the new filesystem:
+0: [0..7]: data
+1: [8..2055]: hole
+2: [2056..2063]: data
diff --git a/tests/btrfs/group b/tests/btrfs/group
index ea88ba4..009dea1 100644
--- a/tests/btrfs/group
+++ b/tests/btrfs/group
@@ -139,3 +139,4 @@
134 auto quick send
135 auto quick send
136 auto convert
+137 auto quick send
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
