[PATCH v2] xfstests: test for atime-related mount options

2014-02-17 Thread Koen De Wit
Tests the noatime, relatime, strictatime and nodiratime mount options.

There is an extra check for Btrfs to ensure that the access time is
never updated on read-only subvolumes. (Regression test for bug fixed
with commit 93fd63c2f001ca6797c6b15b696a484b165b4800)

Signed-off-by: Koen De Wit koen.de@oracle.com
---

v1-v2:
- Fix typo in _cleanup()
- Explicitly passing relatime mount option
- Adding _require_relatime method to common/rc
- Adding extra test for read-only mounts 

diff --git a/common/rc b/common/rc
index e91568b..e55d09e 100644
--- a/common/rc
+++ b/common/rc
@@ -2159,6 +2159,14 @@ _verify_reflink()
|| echo $1 and $2 are not reflinks: different extents
 }
 
+_require_relatime()
+{
+_scratch_mkfs  /dev/null 21
+_mount -t $FSTYP -o relatime $SCRATCH_DEV $SCRATCH_MNT || \
+_notrun relatime not supported by the current kernel
+   _scratch_unmount
+}
+
 _create_loop_device()
 {
file=$1
diff --git a/tests/generic/323 b/tests/generic/323
new file mode 100644
index 000..54f2739
--- /dev/null
+++ b/tests/generic/323
@@ -0,0 +1,199 @@
+# Tests the noatime, relatime, strictatime and nodiratime mount options.
+# There is an extra check for Btrfs to ensure that the access time is
+# never updated on read-only subvolumes. (Regression test for bug fixed
+# with commit 93fd63c2f001ca6797c6b15b696a484b165b4800)
+#
+#---
+# Copyright (c) 2014, Oracle and/or its affiliates.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#---
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo QA output created by $seq
+
+here=`pwd`
+tmp=/tmp/$$
+status=1   # failure is the default!
+trap _cleanup; exit \$status 0 1 2 3 15
+
+_cleanup()
+{
+cd /
+rm -rf $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# real QA test starts here
+
+_supported_fs generic
+_supported_os Linux
+_require_scratch
+_require_relatime
+
+rm -f $seqres.full
+
+_stat() {
+stat --printf=%x;%y;%z $1
+}
+
+_compare_stat_times() {
+updated=$1  # 3 chars indicating if access, modify and
+# change times should be updated (Y) or not (N)
+IFS=';' read -a first_stat  $2   # Convert first stat to array
+IFS=';' read -a second_stat  $3  # Convert second stat to array
+test_step=$4# Will be printed to output stream in case of an
+# error, to make debugging easier
+types=( access modify change )
+
+for i in 0 1 2; do
+if [ ${first_stat[$i]} == ${second_stat[$i]} ]; then
+if [ ${updated:$i:1} == Y ]; then
+echo -n ERROR: ${types[$i]} time has not been updated 
+echo $test_step
+fi
+else
+if [ ${updated:$i:1} == N ]; then
+echo -n ERROR: ${types[$i]} time has changed 
+echo $test_step
+fi
+fi
+done
+}
+
+_scratch_mkfs  $seqres.full 21 || _fail mkfs failed
+_scratch_mount -o relatime
+
+if [ $FSTYP = btrfs ]; then
+TPATH=$SCRATCH_MNT/sub1
+$BTRFS_UTIL_PROG subvolume create $TPATH  $seqres.full
+else
+TPATH=$SCRATCH_MNT
+fi
+
+mkdir $TPATH/dir1
+echo aaa  $TPATH/dir1/file1
+file1_stat_before_first_access=`_stat $TPATH/dir1/file1`
+
+# Accessing file1 the first time
+cat $TPATH/dir1/file1  /dev/null
+file1_stat_after_first_access=`_stat $TPATH/dir1/file1`
+_compare_stat_times YNN $file1_stat_before_first_access \
+$file1_stat_after_first_access after accessing file1 first time
+
+# Accessing file1 a second time
+cat $TPATH/dir1/file1  /dev/null
+file1_stat_after_second_access=`_stat $TPATH/dir1/file1`
+_compare_stat_times NNN $file1_stat_after_first_access \
+$file1_stat_after_second_access after accessing file1 second time
+
+# Remounting with nodiratime option
+_scratch_unmount
+_scratch_mount -o nodiratime
+file1_stat_after_remount=`_stat $TPATH/dir1/file1`
+_compare_stat_times NNN $file1_stat_after_second_access \
+$file1_stat_after_remount for file1 after remount
+
+# Creating dir2 and file2, checking directory stats
+mkdir $TPATH/dir2
+dir2_stat_before_file_creation=`_stat $TPATH/dir2`
+echo bbb  $TPATH/dir2/file2
+dir2_stat_after_file_creation=`_stat

Re: [PATCH] xfstests: test for atime-related mount options

2014-02-17 Thread Koen De Wit

Thanks for the review, Eric!
Comments inline.

On 02/13/2014 05:42 PM, Eric Sandeen wrote:

On 2/13/14, 9:23 AM, Koen De Wit wrote:

Tests the noatime, relatime, strictatime and nodiratime mount options.

There is an extra check for Btrfs to ensure that the access time is
never updated on read-only subvolumes. (Regression test for bug fixed
with commit 93fd63c2f001ca6797c6b15b696a484b165b4800)

Signed-off-by: Koen De Wit koen.de@oracle.com
---
  tests/generic/323 |  186 +
  tests/generic/323.out |2 +
  tests/generic/group   |1 +
  3 files changed, 189 insertions(+), 0 deletions(-)
  create mode 100644 tests/generic/323
  create mode 100644 tests/generic/323.out

diff --git a/tests/generic/323 b/tests/generic/323
new file mode 100644
index 000..423b141
--- /dev/null
+++ b/tests/generic/323
@@ -0,0 +1,186 @@
+# Tests the noatime, relatime, strictatime and nodiratime mount options.
+# There is an extra check for Btrfs to ensure that the access time is
+# never updated on read-only subvolumes. (Regression test for bug fixed
+# with commit 93fd63c2f001ca6797c6b15b696a484b165b4800)
+#
+#---
+# Copyright (c) 2014, Oracle and/or its affiliates.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#---
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo QA output created by $seq
+
+here=`pwd`
+tmp=/tmp/$$
+status=1   # failure is the default!
+trap _cleanup; exit \$status 0 1 2 3 15
+
+_cleanup()
+{
+cd /
+rm -f $*
+}

is $* really what you meant?  Normally this is $tmp.*

$* is positional parameters for the script, and I don't think it takes any.


That's a typo indeed. Fixed in v2.


+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# real QA test starts here
+
+_supported_fs generic
+_supported_os Linux
+_require_scratch
+
+rm -f $seqres.full
+
+_stat() {
+stat --printf=%x;%y;%z $1
+}
+
+_compare_stat_times() {
+updated=$1  # 3 chars indicating if access, modify and
+# change times should be updated (Y) or not (N)
+IFS=';' read -a first_stat  $2   # Convert first stat to array
+IFS=';' read -a second_stat  $3  # Convert second stat to array
+test_step=$4# Will be printed to output stream in case of an
+# error, to make debugging easier
+types=( access modify change )
+
+for i in 0 1 2; do
+if [ ${first_stat[$i]} == ${second_stat[$i]} ]; then
+if [ ${updated:$i:1} == Y ]; then
+echo -n ERROR: ${types[$i]} time has not been updated 
+echo $test_step
+fi
+else
+if [ ${updated:$i:1} == N ]; then
+echo -n ERROR: ${types[$i]} time has changed 
+echo $test_step
+fi
+fi
+done
+}
+
+_scratch_mkfs  $seqres.full 21 || _fail mkfs failed
+_scratch_mount
+
+cat /proc/mounts | grep $SCRATCH_MNT | grep relatime  $seqres.full
+[ $? -ne 0 ]  echo The relatime mount option should be the default.

Ok, I guess relatime in /proc/mounts is from core vfs code and should be 
there for the foreseeable future, so seems ok.

But - relatime was added in v2.6.20, and made default in 2.6.30.  So testing older 
kernels may not go as expected; it'd probably be best to catch situations where 
relatime isn't available ( 2.6.20) or not default ( 2.6.30), by explicitly 
mounting with relatime, and skipping relatime/strictatime tests if that fails?


From the mailing list discussions in the last days, I think we can conclude 
it's best to specify the relatime mount option explicitly and include a new 
_require_relatime method as proposed by Dave Chinner. I implemented it this way 
in v2.


The rest of the test is awfully dense, but nice long understandable variable 
names, so that's good.  ;)

I wonder if in the spirit of testing a btrfs RO snapshot, you could also add a 
readonly mount test, to be sure that an RO mount doesn't update atime.  Of 
course it shouldn't, but it might be worth adding for basic sanity?


Good idea! I added a read-only mount test in v2.

Thanks,
Koen.



Thanks,
-Eric


+
+if [ $FSTYP = btrfs ]; then
+TPATH=$SCRATCH_MNT/sub1
+$BTRFS_UTIL_PROG subvolume

[PATCH] xfstests: test for atime-related mount options

2014-02-13 Thread Koen De Wit
Tests the noatime, relatime, strictatime and nodiratime mount options.

There is an extra check for Btrfs to ensure that the access time is
never updated on read-only subvolumes. (Regression test for bug fixed
with commit 93fd63c2f001ca6797c6b15b696a484b165b4800)

Signed-off-by: Koen De Wit koen.de@oracle.com
---
 tests/generic/323 |  186 +
 tests/generic/323.out |2 +
 tests/generic/group   |1 +
 3 files changed, 189 insertions(+), 0 deletions(-)
 create mode 100644 tests/generic/323
 create mode 100644 tests/generic/323.out

diff --git a/tests/generic/323 b/tests/generic/323
new file mode 100644
index 000..423b141
--- /dev/null
+++ b/tests/generic/323
@@ -0,0 +1,186 @@
+# Tests the noatime, relatime, strictatime and nodiratime mount options.
+# There is an extra check for Btrfs to ensure that the access time is
+# never updated on read-only subvolumes. (Regression test for bug fixed
+# with commit 93fd63c2f001ca6797c6b15b696a484b165b4800)
+#
+#---
+# Copyright (c) 2014, Oracle and/or its affiliates.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#---
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo QA output created by $seq
+
+here=`pwd`
+tmp=/tmp/$$
+status=1   # failure is the default!
+trap _cleanup; exit \$status 0 1 2 3 15
+
+_cleanup()
+{
+cd /
+rm -f $*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# real QA test starts here
+
+_supported_fs generic
+_supported_os Linux
+_require_scratch
+
+rm -f $seqres.full
+
+_stat() {
+stat --printf=%x;%y;%z $1
+}
+
+_compare_stat_times() {
+updated=$1  # 3 chars indicating if access, modify and
+# change times should be updated (Y) or not (N)
+IFS=';' read -a first_stat  $2   # Convert first stat to array
+IFS=';' read -a second_stat  $3  # Convert second stat to array
+test_step=$4# Will be printed to output stream in case of an
+# error, to make debugging easier
+types=( access modify change )
+
+for i in 0 1 2; do
+if [ ${first_stat[$i]} == ${second_stat[$i]} ]; then
+if [ ${updated:$i:1} == Y ]; then
+echo -n ERROR: ${types[$i]} time has not been updated 
+echo $test_step
+fi
+else
+if [ ${updated:$i:1} == N ]; then
+echo -n ERROR: ${types[$i]} time has changed 
+echo $test_step
+fi
+fi
+done
+}
+
+_scratch_mkfs  $seqres.full 21 || _fail mkfs failed
+_scratch_mount
+
+cat /proc/mounts | grep $SCRATCH_MNT | grep relatime  $seqres.full
+[ $? -ne 0 ]  echo The relatime mount option should be the default.
+
+if [ $FSTYP = btrfs ]; then
+TPATH=$SCRATCH_MNT/sub1
+$BTRFS_UTIL_PROG subvolume create $TPATH  $seqres.full
+else
+TPATH=$SCRATCH_MNT
+fi
+
+mkdir $TPATH/dir1
+echo aaa  $TPATH/dir1/file1
+file1_stat_before_first_access=`_stat $TPATH/dir1/file1`
+
+# Accessing file1 the first time
+cat $TPATH/dir1/file1  /dev/null
+file1_stat_after_first_access=`_stat $TPATH/dir1/file1`
+_compare_stat_times YNN $file1_stat_before_first_access \
+$file1_stat_after_first_access after accessing file1 first time
+
+# Accessing file1 a second time
+cat $TPATH/dir1/file1  /dev/null
+file1_stat_after_second_access=`_stat $TPATH/dir1/file1`
+_compare_stat_times NNN $file1_stat_after_first_access \
+$file1_stat_after_second_access after accessing file1 second time
+
+# Remounting with nodiratime option
+_scratch_unmount
+_scratch_mount -o nodiratime
+file1_stat_after_remount=`_stat $TPATH/dir1/file1`
+_compare_stat_times NNN $file1_stat_after_second_access \
+$file1_stat_after_remount for file1 after remount
+
+# Creating dir2 and file2, checking directory stats
+mkdir $TPATH/dir2
+dir2_stat_before_file_creation=`_stat $TPATH/dir2`
+echo bbb  $TPATH/dir2/file2
+dir2_stat_after_file_creation=`_stat $TPATH/dir2`
+_compare_stat_times NYY $dir2_stat_before_file_creation \
+$dir2_stat_after_file_creation for dir2 after file creation
+
+# Accessing file2
+file2_stat_before_first_access=`_stat $TPATH/dir2/file2`
+cat $TPATH/dir2/file2  /dev/null
+file2_stat_after_first_access=`_stat

[PATCH v3] xfstests: Btrfs: add test for large metadata blocks

2014-02-10 Thread Koen De Wit
Tests Btrfs filesystems with all possible metadata block sizes, by
setting large extended attributes on files.

Signed-off-by: Koen De Wit koen.de@oracle.com
---

v1-v2:
- Fix indentation: 8 spaces instead of 4
- Move _scratch_unmount to end of loop, add _check_scratch_fs
- Sending failure messages of mkfs.btrfs to output instead of
  $seqres.full
v2-v3:
- Sending the md5sums of the retrieved attribute values to the
  output instead of comparing them to the md5sum of the original
  value
- Always testing attribute values of 4, 8, 12, ... up to 64 KB
  regardless of the pagesize, to make the golden output independent
  of the pagesize
- Sending the output of mkfs.btrfs with illegal leafsize to
  $seqres.full and checking the return code
- Using more uniform variable names: pagesize/pagesize_kb, leafsize/
  leafsize_kb, attrsize/attrsize_kb

diff --git a/tests/btrfs/036 b/tests/btrfs/036
new file mode 100644
index 000..fb3e987
--- /dev/null
+++ b/tests/btrfs/036
@@ -0,0 +1,125 @@
+#! /bin/bash
+# FS QA Test No. 036
+#
+# Tests large metadata blocks in btrfs, which allows large extended
+# attributes.
+#
+#---
+# Copyright (c) 2014, Oracle and/or its affiliates.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#---
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo QA output created by $seq
+
+here=`pwd`
+status=1   # failure is the default!
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# real QA test starts here
+
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+_require_math
+_need_to_be_root
+
+rm -f $seqres.full
+
+pagesize=`$here/src/feature -s`
+pagesize_kb=`expr $pagesize / 1024`
+
+# Test all valid leafsizes
+for attrsize_kb in `seq 4 4 64`; do
+# The leafsize should be a multiple of the pagesize, equal to or
+# greater than the attribute size.
+leafsize_kb=$(_math ($attrsize_kb + $pagesize_kb - 1) / \
+$pagesize_kb * $pagesize_kb);
+echo Testing with attrsize ${attrsize_kb}K :
+
+_scratch_mkfs -l ${leafsize_kb}K /dev/null
+_scratch_mount
+# Calculate the size of the extended attribute value, leaving
+# 512 bytes for other metadata.
+attrsize=`expr $attrsize_kb \* 1024 - 512`
+
+touch $SCRATCH_MNT/emptyfile
+# smallfile will be inlined, bigfile not.
+$XFS_IO_PROG -f -c pwrite 0 100 $SCRATCH_MNT/smallfile \
+/dev/null
+$XFS_IO_PROG -f -c pwrite 0 9000 $SCRATCH_MNT/bigfile \
+/dev/null
+ln -s $SCRATCH_MNT/bigfile $SCRATCH_MNT/bigfile_softlink
+
+files=(emptyfile smallfile bigfile bigfile_softlink)
+chars=(a b c d)
+for i in `seq 0 1 3`; do
+char=${chars[$i]}
+file=$SCRATCH_MNT/${files[$i]}
+lnkfile=${file}_hardlink
+ln $file $lnkfile
+xattr_value=`head -c $attrsize  /dev/zero \
+| tr '\0' $char`
+
+echo -n $xattr_value | md5sum
+${ATTR_PROG} -Lq -s attr_$char -V $xattr_value $file
+${ATTR_PROG} -Lq -g attr_$char $file | md5sum
+${ATTR_PROG} -Lq -g attr_$char $lnkfile | md5sum
+done
+
+# Test attributes with a size larger than the leafsize.
+# Should result in an error.
+if [ $leafsize_kb -lt 64 ]; then
+# Bash command lines cannot be larger than 64K
+# characters, so we do not test attribute values
+# with a size 64KB.
+attrsize=`expr $attrsize_kb \* 1024 + 512`
+xattr_value=`head -c $attrsize  /dev/zero | tr '\0' x`
+${ATTR_PROG} -q -s attr_toobig -V $xattr_value \
+$SCRATCH_MNT/emptyfile 21 | _filter_scratch
+fi
+
+_scratch_unmount /dev/null 21
+_check_scratch_fs
+done
+
+_scratch_mount
+
+# Illegal attribute name (more than 256 characters)
+attr_name=`head -c 260  /dev/zero | tr '\0' n`
+${ATTR_PROG} -s $attr_name -V attribute_name_too_big \
+$SCRATCH_MNT/emptyfile 21 | head -n 1
+
+_scratch_unmount

Re: [PATCH] xfstests: Btrfs: add test for large metadata blocks

2014-02-10 Thread Koen De Wit


On 02/10/2014 12:02 AM, Dave Chinner wrote:

On Sat, Feb 08, 2014 at 09:30:51AM +0100, Koen De Wit wrote:

On 02/07/2014 11:49 PM, Dave Chinner wrote:

On Fri, Feb 07, 2014 at 06:14:45PM +0100, Koen De Wit wrote:
echo -n $xattr_value | md5sum
${ATTR_PROG} -Lq -s attr_$char -V $xattr_value $file
${ATTR_PROG} -Lq -g attr_$char $file | md5sum
${ATTR_PROG} -Lq -g attr_$char $lnkfile | md5sum

is all that neds to be done here.

The problem with this is that the length of the output will depend on the page 
size. The code above runs for every valid leafsize, which can be any multiple 
of the page size up to 64KB, as defined in the loop initialization:
 for leafsize in `seq $pagesize_kb $pagesize_kb 64`; do

That's only a limit on the mkfs leafsize parameter, yes? An the
limiation is that the leaf size can't be smaller than page size?

So really, the attribute sizes that are being tested are independent
of the mkfs parameters being tested. i.e:

for attrsize in `seq 4 4 64`; do
if [ $attrsize -lt $pagesize ]; then
leafsize=$pagesize
else
leafsize=$attrsize
fi
$BTRFS_MKFS_PROG -l $leafsize $SCRATCH_DEV

And now the test executes a fixed loop, testing the same attribute
sizes on all the filesystems under test. i.e. the attribute sizes
being tested are *independent* of the mkfs parameters being tested.
Always test the same attribute sizes, the mkfs parameters simply
vary by page size.


OK, thanks for the suggestion!  I implemented it like this in v3, I just 
changed the calculation of the leafsize because it must be a multiple of the 
pagesize. (A leafsize of 12KB is not valid for systems with 8KB pages.)


+_scratch_unmount + +# Some illegal leafsizes + +_scratch_mkfs
-l 0 2 $seqres.full +echo $?

Same again - you are dumping the error output into a different
file, then detecting the error manually. pass the output of
_scratch_mkfs through a filter, and let errors cause golden
output mismatches.

I did this to make the golden output not depend on the output of
mkfs.btrfs, inspired by
http://oss.sgi.com/cgi-bin/gitweb.cgi?p=xfs/cmds/xfstests.git;a=commit;h=fd7a8e885732475c17488e28b569ac1530c8eb59
and
http://oss.sgi.com/cgi-bin/gitweb.cgi?p=xfs/cmds/xfstests.git;a=commit;h=78d86b996c9c431542fdbac11fa08764b16ceb7d
However, in my opinion the test should simply be updated if the
output of mkfs.btrfs changes, so I agree with you and I fixed this
in v2.

While I agree with the sentiment, I'm questioning the
implementation. i.e. you've done this differently to every other
test that needs to check for failures. run_check woul dbe just
fine, as would be simply filtering the output of mkfs.


run_check will make the test fail if the return code differs from 0, and Josef brought up 
an example scenario (MKFS_OPTIONS=-O skinny-metadata) where mkfs.btrfs 
produces additional output.

In v3, I implemented the failure check similar to btrfs/022:

_scratch_mkfs -l $1 $seqres.full 21
[ $? -ne 0 ] || _fail '$1' is an illegal value for the \
leafsize option, mkfs should have failed.

Is this the right way?


Thanks,
Koen.
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] xfstests: Btrfs: add test for large metadata blocks

2014-02-08 Thread Koen De Wit

Thanks for the review, Dave!
Comments inline.

On 02/07/2014 11:49 PM, Dave Chinner wrote:

On Fri, Feb 07, 2014 at 06:14:45PM +0100, Koen De Wit wrote:

Tests Btrfs filesystems with all possible metadata block sizes, by
setting large extended attributes on files.

Signed-off-by: Koen De Wit koen.de@oracle.com

There's a few things here that need fixing.


+pagesize=`$here/src/feature -s`
+pagesize_kb=`expr $pagesize / 1024`
+
+# Test all valid leafsizes
+for leafsize in `seq $pagesize_kb $pagesize_kb 64`; do
+_scratch_unmount /dev/null 21

Indentation are tabs, and tabs are 8 spaces in size, please.


OK, I fixed this in v2.


+_scratch_mkfs -l ${leafsize}K /dev/null
+_scratch_mount

No need to use _scratch_unmount here - you should be doing a
_check_scratch_fs at the end of the loop.


Fixed in v2 too.


+# Calculate the xattr size, but leave 512 bytes for other metadata.
+xattr_size=`expr $leafsize \* 1024 - 512`
+
+touch $SCRATCH_MNT/emptyfile
+# smallfile will be inlined, bigfile not.
+$XFS_IO_PROG -f -c pwrite 0 100 $SCRATCH_MNT/smallfile /dev/null
+$XFS_IO_PROG -f -c pwrite 0 9000 $SCRATCH_MNT/bigfile /dev/null
+ln -s $SCRATCH_MNT/bigfile $SCRATCH_MNT/bigfile_softlink
+
+files=(emptyfile smallfile bigfile bigfile_softlink)
+chars=(a b c d)
+for i in `seq 0 1 3`; do
+char=${chars[$i]}
+file=$SCRATCH_MNT/${files[$i]}
+lnkfile=${file}_hardlink
+ln $file $lnkfile
+xattr_value=`head -c $xattr_size  /dev/zero | tr '\0' $char`
+
+set_md5=`echo -n $xattr_value | md5sum`

Just dump the md5sum to the output file.


+${ATTR_PROG} -Lq -s attr_$char -V $xattr_value $file
+get_md5=`${ATTR_PROG} -Lq -g attr_$char $file | md5sum`
+get_ln_md5=`${ATTR_PROG} -Lq -g attr_$char $lnkfile | md5sum`

And dump these to the output file, too. Then the golden image
matching when the test is finish will tell you if it passed or not.
i.e:

echo -n $xattr_value | md5sum
${ATTR_PROG} -Lq -s attr_$char -V $xattr_value $file
${ATTR_PROG} -Lq -g attr_$char $file | md5sum
${ATTR_PROG} -Lq -g attr_$char $lnkfile | md5sum

is all that neds to be done here.


The problem with this is that the length of the output will depend on the page 
size. The code above runs for every valid leafsize, which can be any multiple 
of the page size up to 64KB, as defined in the loop initialization:
for leafsize in `seq $pagesize_kb $pagesize_kb 64`; do


+# Test attributes with a size larger than the leafsize.
+# Should result in an error.
+if [ $leafsize -lt 64 ]; then
+# Bash command lines cannot be larger than 64K characters, so we
+# do not test attribute values with a size 64KB.
+xattr_size=`expr $leafsize \* 1024 + 512`
+xattr_value=`head -c $xattr_size  /dev/zero | tr '\0' x`
+${ATTR_PROG} -q -s attr_toobig -V $xattr_value \
+$SCRATCH_MNT/emptyfile  $seqres.full 21
+if [ $? -eq 0 ]; then
+echo Expected error, xattr_size is bigger than ${leafsize}K
+fi

What you are doing is redirecting the error to $seqres.full
so that it doesn't end up in the output file, then detecting the
absence of an error and dumping a message to the output file to make
the test fail.

IOWs, the ATTR_PROG failure message should be in the golden output
file and you don't have to do anything else to detect a pass/fail
condition.


Same here: the bigger the page size, the less this code will be executed. If 
the page size is 64KB, this code isn't executed at all.
To make sure the golden output does not depend on the page size, I chose to 
suppress all output as long as the test is successful. Is there a better way to 
accomplish this?


+_scratch_unmount
+
+# Some illegal leafsizes
+
+_scratch_mkfs -l 0 2 $seqres.full
+echo $?

Same again - you are dumping the error output into a different file,
then detecting the error manually. pass the output of _scratch_mkfs
through a filter, and let errors cause golden output mismatches.


I did this to make the golden output not depend on the output of mkfs.btrfs, 
inspired by 
http://oss.sgi.com/cgi-bin/gitweb.cgi?p=xfs/cmds/xfstests.git;a=commit;h=fd7a8e885732475c17488e28b569ac1530c8eb59
 and 
http://oss.sgi.com/cgi-bin/gitweb.cgi?p=xfs/cmds/xfstests.git;a=commit;h=78d86b996c9c431542fdbac11fa08764b16ceb7d
However, in my opinion the test should simply be updated if the output of 
mkfs.btrfs changes, so I agree with you and I fixed this in v2.

Thanks,
Koen.
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH v2] xfstests: Btrfs: add test for large metadata blocks

2014-02-08 Thread Koen De Wit
Tests Btrfs filesystems with all possible metadata block sizes, by
setting large extended attributes on files.

Signed-off-by: Koen De Wit koen.de@oracle.com
---

v1-v2: 
- Fix indentation: 8 spaces instead of 4
- Move _scratch_unmount to end of loop, add _check_scratch_fs
- Sending failure messages of mkfs.btrfs to output instead of
  $seqres.full

diff --git a/tests/btrfs/036 b/tests/btrfs/036
new file mode 100644
index 000..b14697d
--- /dev/null
+++ b/tests/btrfs/036
@@ -0,0 +1,137 @@
+#! /bin/bash
+# FS QA Test No. 036
+#
+# Tests large metadata blocks in btrfs, which allows large extended
+# attributes.
+#
+#---
+# Copyright (c) 2014, Oracle and/or its affiliates.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#---
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo QA output created by $seq
+
+here=`pwd`
+status=1   # failure is the default!
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# real QA test starts here
+
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+_need_to_be_root
+
+rm -f $seqres.full
+
+pagesize=`$here/src/feature -s`
+pagesize_kb=`expr $pagesize / 1024`
+
+# Test all valid leafsizes
+for leafsize in `seq $pagesize_kb $pagesize_kb 64`; do
+_scratch_mkfs -l ${leafsize}K /dev/null
+_scratch_mount
+# Calculate the size of the extended attribute value, leaving
+# 512 bytes for other metadata.
+xattr_size=`expr $leafsize \* 1024 - 512`
+
+touch $SCRATCH_MNT/emptyfile
+# smallfile will be inlined, bigfile not.
+$XFS_IO_PROG -f -c pwrite 0 100 $SCRATCH_MNT/smallfile \
+/dev/null
+$XFS_IO_PROG -f -c pwrite 0 9000 $SCRATCH_MNT/bigfile \
+/dev/null
+ln -s $SCRATCH_MNT/bigfile $SCRATCH_MNT/bigfile_softlink
+
+files=(emptyfile smallfile bigfile bigfile_softlink)
+chars=(a b c d)
+for i in `seq 0 1 3`; do
+char=${chars[$i]}
+file=$SCRATCH_MNT/${files[$i]}
+lnkfile=${file}_hardlink
+ln $file $lnkfile
+xattr_value=`head -c $xattr_size  /dev/zero \
+| tr '\0' $char`
+
+set_md5=`echo -n $xattr_value | md5sum`
+${ATTR_PROG} -Lq -s attr_$char -V $xattr_value $file
+get_md5=`${ATTR_PROG} -Lq -g attr_$char $file | md5sum`
+get_ln_md5=`${ATTR_PROG} -Lq -g attr_$char $lnkfile \
+| md5sum`
+
+# Using md5sums for comparison instead of the values
+# themselves because bash command lines cannot be larger
+# than 64K chars.
+if [ $set_md5 != $get_md5 ]; then
+echo -n Got unexpected xattr value for 
+echo -n attr_$char from file ${file}. 
+echo (leafsize is ${leafsize}K)
+fi
+if [ $set_md5 != $get_ln_md5 ]; then
+echo -n Value for attr_$char differs for 
+echo -n $file and ${lnkfile}. 
+echo (leafsize is ${leafsize}K)
+fi
+done
+
+# Test attributes with a size larger than the leafsize.
+# Should result in an error.
+if [ $leafsize -lt 64 ]; then
+# Bash command lines cannot be larger than 64K
+# characters, so we do not test attribute values
+# with a size 64KB.
+xattr_size=`expr $leafsize \* 1024 + 512`
+xattr_value=`head -c $xattr_size  /dev/zero | tr '\0' x`
+${ATTR_PROG} -q -s attr_toobig -V $xattr_value \
+$SCRATCH_MNT/emptyfile  $seqres.full 21
+if [ $? -eq 0 ]; then
+echo -n Expected error, xattr_size is bigger 
+echo than ${leafsize}K
+fi
+fi
+
+_scratch_unmount /dev/null 21
+_check_scratch_fs
+done
+
+_scratch_mount
+
+# Illegal attribute name (more than 256 characters)
+attr_name=`head -c 260  /dev/zero | tr '\0' n`
+${ATTR_PROG} -s $attr_name -V

[PATCH] xfstests: Btrfs: add test for large metadata blocks

2014-02-07 Thread Koen De Wit
Tests Btrfs filesystems with all possible metadata block sizes, by
setting large extended attributes on files.

Signed-off-by: Koen De Wit koen.de@oracle.com
---
 tests/btrfs/036 |  128 +++
 tests/btrfs/036.out |7 +++
 tests/btrfs/group   |1 +
 3 files changed, 136 insertions(+), 0 deletions(-)
 create mode 100644 tests/btrfs/036
 create mode 100644 tests/btrfs/036.out

diff --git a/tests/btrfs/036 b/tests/btrfs/036
new file mode 100644
index 000..0f8287a
--- /dev/null
+++ b/tests/btrfs/036
@@ -0,0 +1,128 @@
+#! /bin/bash
+# FS QA Test No. 036
+#
+# Tests large metadata blocks in btrfs, which allows large extended
+# attributes.
+#
+#---
+# Copyright (c) 2014, Oracle and/or its affiliates.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#---
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo QA output created by $seq
+
+here=`pwd`
+status=1   # failure is the default!
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# real QA test starts here
+
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+_need_to_be_root
+
+rm -f $seqres.full
+
+pagesize=`$here/src/feature -s`
+pagesize_kb=`expr $pagesize / 1024`
+
+# Test all valid leafsizes
+for leafsize in `seq $pagesize_kb $pagesize_kb 64`; do
+_scratch_unmount /dev/null 21
+_scratch_mkfs -l ${leafsize}K /dev/null
+_scratch_mount
+# Calculate the xattr size, but leave 512 bytes for other metadata.
+xattr_size=`expr $leafsize \* 1024 - 512`
+
+touch $SCRATCH_MNT/emptyfile
+# smallfile will be inlined, bigfile not.
+$XFS_IO_PROG -f -c pwrite 0 100 $SCRATCH_MNT/smallfile /dev/null
+$XFS_IO_PROG -f -c pwrite 0 9000 $SCRATCH_MNT/bigfile /dev/null
+ln -s $SCRATCH_MNT/bigfile $SCRATCH_MNT/bigfile_softlink
+
+files=(emptyfile smallfile bigfile bigfile_softlink)
+chars=(a b c d)
+for i in `seq 0 1 3`; do
+char=${chars[$i]}
+file=$SCRATCH_MNT/${files[$i]}
+lnkfile=${file}_hardlink
+ln $file $lnkfile
+xattr_value=`head -c $xattr_size  /dev/zero | tr '\0' $char`
+
+set_md5=`echo -n $xattr_value | md5sum`
+${ATTR_PROG} -Lq -s attr_$char -V $xattr_value $file
+get_md5=`${ATTR_PROG} -Lq -g attr_$char $file | md5sum`
+get_ln_md5=`${ATTR_PROG} -Lq -g attr_$char $lnkfile | md5sum`
+
+# Using md5sums for comparison instead of the values themselves
+# because bash command lines cannot be larger than 64K chars.
+if [ $set_md5 != $get_md5 ]; then
+echo Got unexpected xattr value for attr_$char
+echo from file $file . (leafsize is ${leafsize}K)
+fi
+if [ $set_md5 != $get_ln_md5 ]; then
+echo Value for attr_$char differs for $file and
+echo $lnkfile . (leafsize is ${leafsize}K)
+fi
+done
+
+# Test attributes with a size larger than the leafsize.
+# Should result in an error.
+if [ $leafsize -lt 64 ]; then
+# Bash command lines cannot be larger than 64K characters, so we
+# do not test attribute values with a size 64KB.
+xattr_size=`expr $leafsize \* 1024 + 512`
+xattr_value=`head -c $xattr_size  /dev/zero | tr '\0' x`
+${ATTR_PROG} -q -s attr_toobig -V $xattr_value \
+$SCRATCH_MNT/emptyfile  $seqres.full 21
+if [ $? -eq 0 ]; then
+echo Expected error, xattr_size is bigger than ${leafsize}K
+fi
+fi
+
+done
+
+# Illegal attribute name (more than 256 characters)
+attr_name=`head -c 260  /dev/zero | tr '\0' n`
+${ATTR_PROG} -s $attr_name -V attribute_name_too_big \
+$SCRATCH_MNT/emptyfile 21 | head -n 1
+
+_scratch_unmount
+
+# Some illegal leafsizes
+
+_scratch_mkfs -l 0 2 $seqres.full
+echo $?
+
+_scratch_mkfs -l 5678 2 $seqres.full
+echo $?
+
+_scratch_mkfs -l `expr $pagesize / 2 + $pagesize` 2 $seqres.full
+echo $?
+
+_scratch_mkfs -l 128K 2 $seqres.full
+echo $?
+
+_scratch_mkfs -l K
+
+# success, all done
+status=0
+exit
diff --git a/tests/btrfs/036.out b/tests/btrfs/036.out
new file mode 100644
index 000..1d9bdfb
--- /dev/null
+++ b/tests/btrfs/036.out
@@ -0,0 +1,7 @@
+QA output created by 036
+attr_set

[PATCH v2] xfstests: btrfs: cross-subvolume sparse copy

2014-01-22 Thread Koen De Wit
This testscript creates reflinks to files on different subvolumes,
overwrites the original files and reflinks, and moves reflinked files
between subvolumes.

Signed-off-by: Koen De Wit koen.de@oracle.com
Reviewed-by: David Sterba dste...@suse.cz
---
v1: Resend (originally submitted as test 302, btrfs/316)
v2: - use $BTRFS_UTIL_PROG instead of btrfs command
- use full subcommands
- explicitly define the always parameter to cp --reflink
- define $seqres

diff --git a/tests/btrfs/030 b/tests/btrfs/030
new file mode 100644
index 000..3a1b970
--- /dev/null
+++ b/tests/btrfs/030
@@ -0,0 +1,138 @@
+#! /bin/bash
+# FS QA Test No. 030
+#
+# Testing cross-subvolume sparse copy on btrfs
+#- Create two subvolumes, mount one of them
+#- Create a file on each (sub/root)volume,
+#  reflink them on the other volumes
+#- Change one original and two reflinked files
+#- Move reflinked files between subvolumes
+#
+#---
+# Copyright (c) 2014, Oracle and/or its affiliates.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#---
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo QA output created by $seq
+
+here=`pwd`
+tmp=/tmp/$$
+status=1# failure is the default!
+trap _cleanup; exit \$status 0 1 2 3 15
+
+_cleanup()
+{
+umount $SCRATCH_MNT
+rm -rf $TESTDIR1
+rm -rf $TESTDIR2
+$BTRFS_UTIL_PROG subvolume delete $SUBVOL1  $seqres.full
+$BTRFS_UTIL_PROG subvolume delete $SUBVOL2  $seqres.full
+cd /
+rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# real QA test starts here
+_supported_fs btrfs
+_supported_os Linux
+
+_require_scratch
+_require_cp_reflink
+
+_checksum_files() {
+for F in file1 file2 file3
+do
+echo $F:
+for D in $TESTDIR1 $SCRATCH_MNT $SUBVOL2
+do
+_md5_checksum $D/$F
+done
+done
+}
+
+TESTDIR1=$TEST_DIR/test-$seq-1
+TESTDIR2=$TEST_DIR/test-$seq-2
+SUBVOL1=$TEST_DIR/subvol-$seq-1
+SUBVOL2=$TEST_DIR/subvol-$seq-2
+
+_scratch_unmount 2/dev/null
+rm -rf $seqres.full
+rm -rf $TESTDIR1 $TESTDIR2
+$BTRFS_UTIL_PROG subvolume delete $SUBVOL1 /dev/null 21
+$BTRFS_UTIL_PROG subvolume delete $SUBVOL2 /dev/null 21
+
+mkdir $TESTDIR1
+mkdir $TESTDIR2
+$BTRFS_UTIL_PROG subvolume create $SUBVOL1  $seqres.full
+$BTRFS_UTIL_PROG subvolume create $SUBVOL2  $seqres.full
+_mount -t btrfs -o subvol=subvol-$seq-1 $TEST_DEV $SCRATCH_MNT
+
+echo Create initial files
+# TESTDIR1/file1 is very small and will be inlined
+$XFS_IO_PROG -f -c 'pwrite -S 0x61 0 10' $TESTDIR1/file1 \
+ $seqres.full
+$XFS_IO_PROG -f -c 'pwrite -S 0x62 0 13000' $SCRATCH_MNT/file2 \
+ $seqres.full
+$XFS_IO_PROG -f -c 'pwrite -S 0x63 0 17000' $SUBVOL2/file3 \
+ $seqres.full
+
+echo Create reflinks to the initial files on other subvolumes
+cp --reflink=always $TESTDIR1/file1 $SUBVOL1
+cp --reflink=always $TESTDIR1/file1 $SUBVOL2
+cp --reflink=always $SUBVOL1/file2 $TESTDIR1/
+cp --reflink=always $SUBVOL1/file2 $SUBVOL2
+cp --reflink=always $SUBVOL2/file3 $TESTDIR1/
+cp --reflink=always $SUBVOL2/file3 $SUBVOL1
+
+echo Verify the reflinks
+_verify_reflink $SCRATCH_MNT/file2 $TESTDIR1/file2
+_verify_reflink $SCRATCH_MNT/file2 $SUBVOL2/file2
+_verify_reflink $SUBVOL2/file3 $TESTDIR1/file3
+_verify_reflink $SUBVOL2/file3 $SCRATCH_MNT/file3
+echo Verify the file contents:
+_checksum_files
+
+echo -e ---\nOverwrite some files with new content
+$XFS_IO_PROG -c 'pwrite -S 0x64 0 20' $TESTDIR1/file1  $seqres.full
+$XFS_IO_PROG -c 'pwrite -S 0x66 0 21000' $SUBVOL2/file2  $seqres.full
+$XFS_IO_PROG -c 'pwrite -S 0x65 5000 5000' $SCRATCH_MNT/file3 \
+ $seqres.full
+
+echo -n Verify that non-overwritten reflinks 
+echo still have the same data blocks
+_verify_reflink $TESTDIR1/file2 $SCRATCH_MNT/file2
+_verify_reflink $TESTDIR1/file3 $SUBVOL2/file3
+echo Verify the file contents:
+_checksum_files
+
+echo -e ---\nShuffle files between directories
+mv $TESTDIR1/file* $TESTDIR2
+mv $SCRATCH_MNT/file* $TESTDIR1/
+mv $SUBVOL2/file* $SCRATCH_MNT/
+mv $TESTDIR2/file* $SUBVOL2/
+
+# No _verify_reflink here as data is copied when moving files
+# between subvols
+echo Verify the file contents:
+_checksum_files
+
+# success, all done
+status=0
+exit
diff --git

[PATCH RESEND] xfstests: btrfs: cross-subvolume sparse copy

2014-01-21 Thread Koen De Wit
This testscript creates reflinks to files on different subvolumes,
overwrites the original files and reflinks, and moves reflinked files
between subvolumes.

Signed-off-by: Koen De Wit koen.de@oracle.com
---

Originally submitted as test 302, btrfs/316

diff --git a/tests/btrfs/030 b/tests/btrfs/030
new file mode 100644
index 000..5ebc555
--- /dev/null
+++ b/tests/btrfs/030
@@ -0,0 +1,137 @@
+#! /bin/bash
+# FS QA Test No. 030
+#
+# Testing cross-subvolume sparse copy on btrfs
+#- Create two subvolumes, mount one of them
+#- Create a file on each (sub/root)volume,
+#  reflink them on the other volumes
+#- Change one original and two reflinked files
+#- Move reflinked files between subvolumes
+#
+#---
+# Copyright (c) 2014, Oracle and/or its affiliates.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#---
+
+seq=`basename $0`
+echo QA output created by $seq
+
+here=`pwd`
+tmp=/tmp/$$
+status=1# failure is the default!
+trap _cleanup; exit \$status 0 1 2 3 15
+
+_cleanup()
+{
+umount $SCRATCH_MNT
+rm -rf $TESTDIR1
+rm -rf $TESTDIR2
+btrfs subvolume delete $SUBVOL1  $seqres.full
+btrfs subvolume delete $SUBVOL2  $seqres.full
+cd /
+rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# real QA test starts here
+_supported_fs btrfs
+_supported_os Linux
+
+_require_scratch
+_require_cp_reflink
+
+_checksum_files() {
+for F in file1 file2 file3
+do
+echo $F:
+for D in $TESTDIR1 $SCRATCH_MNT $SUBVOL2
+do
+_md5_checksum $D/$F
+done
+done
+}
+
+TESTDIR1=$TEST_DIR/test-$seq-1
+TESTDIR2=$TEST_DIR/test-$seq-2
+SUBVOL1=$TEST_DIR/subvol-$seq-1
+SUBVOL2=$TEST_DIR/subvol-$seq-2
+
+_scratch_unmount 2/dev/null
+rm -rf $seqres.full
+rm -rf $TESTDIR1 $TESTDIR2
+btrfs subvol delete $SUBVOL1 /dev/null 21
+btrfs subvol delete $SUBVOL2 /dev/null 21
+
+mkdir $TESTDIR1
+mkdir $TESTDIR2
+btrfs subvolume create $SUBVOL1  $seqres.full
+btrfs subvolume create $SUBVOL2  $seqres.full
+_mount -t btrfs -o subvol=subvol-$seq-1 $TEST_DEV $SCRATCH_MNT
+
+echo Create initial files
+# TESTDIR1/file1 is very small and will be inlined
+$XFS_IO_PROG -f -c 'pwrite -S 0x61 0 10' $TESTDIR1/file1 \
+ $seqres.full
+$XFS_IO_PROG -f -c 'pwrite -S 0x62 0 13000' $SCRATCH_MNT/file2 \
+ $seqres.full
+$XFS_IO_PROG -f -c 'pwrite -S 0x63 0 17000' $SUBVOL2/file3 \
+ $seqres.full
+
+echo Create reflinks to the initial files on other subvolumes
+cp --reflink $TESTDIR1/file1 $SUBVOL1
+cp --reflink $TESTDIR1/file1 $SUBVOL2
+cp --reflink $SUBVOL1/file2 $TESTDIR1/
+cp --reflink $SUBVOL1/file2 $SUBVOL2
+cp --reflink $SUBVOL2/file3 $TESTDIR1/
+cp --reflink $SUBVOL2/file3 $SUBVOL1
+
+echo Verify the reflinks
+_verify_reflink $SCRATCH_MNT/file2 $TESTDIR1/file2
+_verify_reflink $SCRATCH_MNT/file2 $SUBVOL2/file2
+_verify_reflink $SUBVOL2/file3 $TESTDIR1/file3
+_verify_reflink $SUBVOL2/file3 $SCRATCH_MNT/file3
+echo Verify the file contents:
+_checksum_files
+
+echo -e ---\nOverwrite some files with new content
+$XFS_IO_PROG -c 'pwrite -S 0x64 0 20' $TESTDIR1/file1  $seqres.full
+$XFS_IO_PROG -c 'pwrite -S 0x66 0 21000' $SUBVOL2/file2  $seqres.full
+$XFS_IO_PROG -c 'pwrite -S 0x65 5000 5000' $SCRATCH_MNT/file3 \
+ $seqres.full
+
+echo -n Verify that non-overwritten reflinks 
+echo still have the same data blocks
+_verify_reflink $TESTDIR1/file2 $SCRATCH_MNT/file2
+_verify_reflink $TESTDIR1/file3 $SUBVOL2/file3
+echo Verify the file contents:
+_checksum_files
+
+echo -e ---\nShuffle files between directories
+mv $TESTDIR1/file* $TESTDIR2
+mv $SCRATCH_MNT/file* $TESTDIR1/
+mv $SUBVOL2/file* $SCRATCH_MNT/
+mv $TESTDIR2/file* $SUBVOL2/
+
+# No _verify_reflink here as data is copied when moving files
+# between subvols
+echo Verify the file contents:
+_checksum_files
+
+# success, all done
+status=0
+exit
diff --git a/tests/btrfs/030.out b/tests/btrfs/030.out
new file mode 100644
index 000..050ae0b
--- /dev/null
+++ b/tests/btrfs/030.out
@@ -0,0 +1,48 @@
+QA output created by 030
+Create initial files
+Create reflinks to the initial files on other subvolumes
+Verify the reflinks
+Verify the file contents:
+file1:
+e09c80c42fda55f9d992e59ca6b3307d

[PATCH RESEND] xfstests: btrfs/027: simple sparse copy testcase for btrfs

2014-01-17 Thread Koen De Wit
Tests file clone functionality of btrfs (reflinks):
   - Reflink a file
   - Reflink the reflinked file
   - Modify the original file
   - Modify the reflinked file

[sandeen: add helpers, make several mostly-cosmetic
 changes to the original testcase]

Signed-off-by: Koen De Wit koen.de@oracle.com
Signed-off-by: Eric Sandeen sand...@redhat.com
---

Originally submitted as test 297, btrfs/308

diff --git a/common/rc b/common/rc
index a585eb5..4157cc4 100644
--- a/common/rc
+++ b/common/rc
@@ -2078,6 +2078,27 @@ _require_ugid_map()
fi
 }
 
+_require_cp_reflink()
+{
+   cp --help | grep -q reflink || \
+   _notrun This test requires a cp with --reflink support.
+}
+
+# Given 2 files, verify that they have the same mapping but different
+# inodes - i.e. an undisturbed reflink
+# Silent if so, make noise if not
+_verify_reflink()
+{
+   # not a hard link or symlink?
+   cmp -s  (stat -c '%i' $1) (stat -c '%i' $2) \
+echo $1 and $2 are not reflinks: same inode number
+
+   # same mapping?
+   diff -u ($XFS_IO_PROG -c fiemap $1 | grep -v $1) \
+   ($XFS_IO_PROG -c fiemap $2 | grep -v $2) \
+   || echo $1 and $2 are not reflinks: different extents
+}
+
 _create_loop_device()
 {
file=$1
diff --git a/tests/btrfs/027 b/tests/btrfs/027
new file mode 100644
index 000..ec0d5b4
--- /dev/null
+++ b/tests/btrfs/027
@@ -0,0 +1,90 @@
+#! /bin/bash
+# FS QA Test No. 027
+#
+# Tests file clone functionality of btrfs (reflinks):
+#   - Reflink a file
+#   - Reflink the reflinked file
+#   - Modify the original file
+#   - Modify the reflinked file
+#
+#---
+# Copyright (c) 2014, Oracle and/or its affiliates.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#---
+#
+
+seq=`basename $0`
+echo QA output created by $seq
+
+here=`pwd`
+tmp=/tmp/$$
+status=1# failure is the default!
+trap _cleanup; exit \$status 0 1 2 3 15
+
+_cleanup()
+{
+cd /
+rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. common/rc
+. common/filter
+
+# real QA test starts here
+_supported_fs btrfs
+_supported_os Linux
+
+_require_xfs_io_fiemap
+_require_cp_reflink
+
+TESTDIR1=$TEST_DIR/test-$seq
+rm -rf $TESTDIR1
+mkdir $TESTDIR1
+
+_checksum_files() {
+for F in original copy1 copy2
+do
+md5sum $TESTDIR1/$F | _filter_test_dir
+done
+}
+
+rm -f $seqres.full
+
+echo Create the original file and reflink to copy1, copy2
+$XFS_IO_PROG -f -c 'pwrite -S 0x61 0 9000' $TESTDIR1/original \
+ $seqres.full 21
+cp --reflink $TESTDIR1/original $TESTDIR1/copy1
+cp --reflink $TESTDIR1/copy1 $TESTDIR1/copy2
+_verify_reflink $TESTDIR1/original $TESTDIR1/copy1
+_verify_reflink $TESTDIR1/original $TESTDIR1/copy2
+echo Original md5sums:
+_checksum_files
+
+echo Overwrite original file with new data
+$XFS_IO_PROG -c 'pwrite -S 0x62 0 9000' $TESTDIR1/original \
+ $seqres.full 21
+echo md5sums after overwriting original:
+_checksum_files
+
+echo Overwrite copy1 with different new data
+$XFS_IO_PROG -c 'pwrite -S 0x63 0 9000' $TESTDIR1/copy1 \
+ $seqres.full 21
+echo md5sums after overwriting copy1:
+_checksum_files
+
+# success, all done
+status=0
+exit
diff --git a/tests/btrfs/027.out b/tests/btrfs/027.out
new file mode 100644
index 000..13023bf
--- /dev/null
+++ b/tests/btrfs/027.out
@@ -0,0 +1,16 @@
+QA output created by 027
+Create the original file and reflink to copy1, copy2
+Original md5sums:
+42d69d1a6d333a7ebdf64792a555e392  TEST_DIR/test-027/original
+42d69d1a6d333a7ebdf64792a555e392  TEST_DIR/test-027/copy1
+42d69d1a6d333a7ebdf64792a555e392  TEST_DIR/test-027/copy2
+Overwrite original file with new data
+md5sums after overwriting original:
+4a847a25439532bf48b68c9e9536ed5b  TEST_DIR/test-027/original
+42d69d1a6d333a7ebdf64792a555e392  TEST_DIR/test-027/copy1
+42d69d1a6d333a7ebdf64792a555e392  TEST_DIR/test-027/copy2
+Overwrite copy1 with different new data
+md5sums after overwriting copy1:
+4a847a25439532bf48b68c9e9536ed5b  TEST_DIR/test-027/original
+e271cd47d9f62ebc96cb4e67ae4d16db  TEST_DIR/test-027/copy1
+42d69d1a6d333a7ebdf64792a555e392  TEST_DIR/test-027/copy2
diff --git a/tests/btrfs/group b/tests/btrfs/group
index 87e7bca..379f95b 100644
--- a/tests/btrfs/group

[PATCH RESEND] xfstests: btrfs/028: sparse copy of a directory tree on btrfs

2014-01-17 Thread Koen De Wit
Tests file clone functionality of btrfs (reflinks) on directory trees.
   - Create directory and subdirectory, each having one file
   - Create 2 recursive reflinked copies of the tree
   - Modify the original files
   - Modify one of the copies

[sandeen: mostly cosmetic changes]

Signed-off-by: Koen De Wit koen.de@oracle.com
Signed-off-by: Eric Sandeen sand...@redhat.com
---

Originally submitted as test 298, btrfs/309

The functions _require_cp_reflink() and _verify_reflink() in common/rc
have been submitted with btrfs/027

diff --git a/tests/btrfs/028 b/tests/btrfs/028
new file mode 100644
index 000..a034008
--- /dev/null
+++ b/tests/btrfs/028
@@ -0,0 +1,107 @@
+#! /bin/bash
+# FS QA Test No. 028
+#
+# Tests file clone functionality of btrfs (reflinks) on directory
+# trees.
+#   - Create directory and subdirectory, each having one file
+#   - Create 2 recursive reflinked copies of the tree
+#   - Modify the original files
+#   - Modify one of the copies
+#
+#---
+# Copyright (c) 2014, Oracle and/or its affiliates.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#---
+
+seq=`basename $0`
+echo QA output created by $seq
+
+here=`pwd`
+tmp=/tmp/$$
+status=1# failure is the default!
+trap _cleanup; exit \$status 0 1 2 3 15
+
+_cleanup()
+{
+cd /
+rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. common/rc
+. common/filter
+
+# real QA test starts here
+_supported_fs btrfs
+_supported_os Linux
+
+_require_xfs_io_fiemap
+_require_cp_reflink
+
+TESTDIR1=$TEST_DIR/test-$seq
+rm -rf $TESTDIR1
+mkdir $TESTDIR1
+
+_checksum_files() {
+for F in original/file1 original/subdir/file2 \
+ copy1/file1 copy1/subdir/file2 \
+ copy2/file1 copy2/subdir/file2
+do
+md5sum $TESTDIR1/$F | _filter_test_dir
+done
+}
+
+rm -f $seqres.full
+
+mkdir $TESTDIR1/original
+mkdir $TESTDIR1/original/subdir
+
+echo Create the original files and reflink dirs
+$XFS_IO_PROG -f -c 'pwrite -S 0x61 0 9000' $TESTDIR1/original/file1 \
+ $seqres.full 21
+$XFS_IO_PROG -f -c 'pwrite -S 0x62 0 11000' \
+$TESTDIR1/original/subdir/file2  $seqres.full 21
+cp --recursive --reflink $TESTDIR1/original $TESTDIR1/copy1
+cp --recursive --reflink $TESTDIR1/copy1 $TESTDIR1/copy2
+
+_verify_reflink $TESTDIR1/original/file1 $TESTDIR1/copy1/file1
+_verify_reflink $TESTDIR1/original/subdir/file2 \
+$TESTDIR1/copy1/subdir/file2
+_verify_reflink $TESTDIR1/original/file1 $TESTDIR1/copy2/file1
+_verify_reflink $TESTDIR1/original/subdir/file2 \
+$TESTDIR1/copy2/subdir/file2
+
+echo Original md5sums:
+_checksum_files
+
+echo Overwrite original/file1 and original/subdir/file2 with new data
+$XFS_IO_PROG -c 'pwrite -S 0x63 0 13000' $TESTDIR1/original/file1 \
+ $seqres.full 21
+$XFS_IO_PROG -c 'pwrite -S 0x64 5000 1000' \
+$TESTDIR1/original/subdir/file2  $seqres.full 21
+echo md5sums now:
+_checksum_files
+
+echo Overwrite copy1/file1 and copy1/subdir/file2 with new data
+$XFS_IO_PROG -c 'pwrite -S 0x65 0 9000' $TESTDIR1/copy1/file1 \
+ $seqres.full 21
+$XFS_IO_PROG -c 'pwrite -S 0x66 5000 25000' \
+$TESTDIR1/copy1/subdir/file2  $seqres.full 21
+echo md5sums now:
+_checksum_files
+
+# success, all done
+status=0
+exit
diff --git a/tests/btrfs/028.out b/tests/btrfs/028.out
new file mode 100644
index 000..bd078db
--- /dev/null
+++ b/tests/btrfs/028.out
@@ -0,0 +1,25 @@
+QA output created by 028
+Create the original files and reflink dirs
+Original md5sums:
+42d69d1a6d333a7ebdf64792a555e392  TEST_DIR/test-028/original/file1
+ca390545f0aedb54b808d6128c56a7dd  TEST_DIR/test-028/original/subdir/file2
+42d69d1a6d333a7ebdf64792a555e392  TEST_DIR/test-028/copy1/file1
+ca390545f0aedb54b808d6128c56a7dd  TEST_DIR/test-028/copy1/subdir/file2
+42d69d1a6d333a7ebdf64792a555e392  TEST_DIR/test-028/copy2/file1
+ca390545f0aedb54b808d6128c56a7dd  TEST_DIR/test-028/copy2/subdir/file2
+Overwrite original/file1 and original/subdir/file2 with new data
+md5sums now:
+260f6785c0537fd12582dcae28a3c1a9  TEST_DIR/test-028/original/file1
+b8d91fb600f6f2191f2ba5374860  TEST_DIR/test-028/original/subdir/file2
+42d69d1a6d333a7ebdf64792a555e392  TEST_DIR/test-028/copy1/file1
+ca390545f0aedb54b808d6128c56a7dd  TEST_DIR/test-028

[PATCH RESEND] xfstests: btrfs/029: moving and deleting sparse copies on btrfs

2014-01-17 Thread Koen De Wit
Moving and deleting cloned (reflinked) files on btrfs:
   - Create a file and a reflink
   - Move both to a directory
   - Delete the original (moved) file, check that the copy still exists.

[sandeen: mostly cosmetic changes]

Signed-off-by: Koen De Wit koen.de@oracle.com
Signed-off-by: Eric Sandeen sand...@redhat.com
---

Originally submitted as test 299, btrfs/310

The functions _require_cp_reflink() and _verify_reflink() in common/rc
have been submitted with btrfs/027

diff --git a/tests/btrfs/029 b/tests/btrfs/029
new file mode 100644
index 000..7740d8f
--- /dev/null
+++ b/tests/btrfs/029
@@ -0,0 +1,81 @@
+#! /bin/bash
+# FS QA Test No. 029
+#
+# Moving and deleting cloned (reflinked) files on btrfs:
+#   - Create a file and a reflink
+#   - Move both to a directory
+#   - Delete the original (moved) file, check that the copy still exists.
+#
+#---
+# Copyright (c) 2014, Oracle and/or its affiliates.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#---
+
+seq=`basename $0`
+echo QA output created by $seq
+
+here=`pwd`
+tmp=/tmp/$$
+status=1# failure is the default!
+trap _cleanup; exit \$status 0 1 2 3 15
+
+_cleanup()
+{
+cd /
+rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# real QA test starts here
+_supported_fs btrfs
+_supported_os Linux
+
+_require_xfs_io_fiemap
+_require_cp_reflink
+
+rm -f $seqres.full
+
+TESTDIR1=$TEST_DIR/test-$seq
+rm -rf $TESTDIR1
+mkdir $TESTDIR1
+
+echo Create the original files and reflink dirs
+$XFS_IO_PROG -f -c 'pwrite -S 0x61 0 9000' $TESTDIR1/original \
+ $seqres.full
+cp --reflink $TESTDIR1/original $TESTDIR1/copy
+
+_verify_reflink $TESTDIR1/original $TESTDIR1/copy
+
+echo Move orig  reflink copy to subdir and md5sum:
+mkdir $TESTDIR1/subdir
+mv $TESTDIR1/original $TESTDIR1/subdir/original_moved
+mv $TESTDIR1/copy $TESTDIR1/subdir/copy_moved
+_verify_reflink $TESTDIR1/subdir/original_moved \
+$TESTDIR1/subdir/copy_moved
+
+md5sum $TESTDIR1/subdir/original_moved | _filter_test_dir
+md5sum $TESTDIR1/subdir/copy_moved | _filter_test_dir
+
+echo remove orig from subdir and md5sum reflink copy:
+rm $TESTDIR1/subdir/original_moved
+md5sum $TESTDIR1/subdir/copy_moved | _filter_test_dir
+rm -rf $TESTDIR1/subdir
+
+# success, all done
+status=0
+exit
diff --git a/tests/btrfs/029.out b/tests/btrfs/029.out
new file mode 100644
index 000..94e5808
--- /dev/null
+++ b/tests/btrfs/029.out
@@ -0,0 +1,7 @@
+QA output created by 029
+Create the original files and reflink dirs
+Move orig  reflink copy to subdir and md5sum:
+42d69d1a6d333a7ebdf64792a555e392  TEST_DIR/test-029/subdir/original_moved
+42d69d1a6d333a7ebdf64792a555e392  TEST_DIR/test-029/subdir/copy_moved
+remove orig from subdir and md5sum reflink copy:
+42d69d1a6d333a7ebdf64792a555e392  TEST_DIR/test-029/subdir/copy_moved
diff --git a/tests/btrfs/group b/tests/btrfs/group
index 372585d..9fcd15d 100644
--- a/tests/btrfs/group
+++ b/tests/btrfs/group
@@ -28,3 +28,4 @@
 023 auto
 024 auto quick
 028 auto quick
+029 auto quick
-- 
1.7.1

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH RESEND] xfstests: btrfs/030: sparse copy between different filesystems/mountpoints

2014-01-17 Thread Koen De Wit
Check if creating a sparse copy (reflink) of a file on btrfs
expectedly fails when it's done between different filesystems or
different mount points of the same filesystem.

For both situations, these actions are executed:
   - Copy a file with the reflink=auto option.
 A normal copy should be created.
   - Copy a file with the reflink=always option. Should result in
 error.

[sandeen: mostly cosmetic changes]

Signed-off-by: Koen De Wit koen.de@oracle.com
Signed-off-by: Eric Sandeen sand...@redhat.com
---

Originally submitted as test 301, btrfs/311

The functions _require_cp_reflink() and _verify_reflink() in common/rc
have been submitted with btrfs/027

diff --git a/tests/btrfs/030 b/tests/btrfs/030
new file mode 100644
index 000..7bbc89a
--- /dev/null
+++ b/tests/btrfs/030
@@ -0,0 +1,108 @@
+#! /bin/bash
+# FS QA Test No. 030
+#
+# Check if creating a sparse copy (reflink) of a file on btrfs
+# expectedly fails when it's done between different filesystems or
+# different mount points of the same filesystem.
+#
+# For both situations, these actions are executed:
+#- Copy a file with the reflink=auto option.
+#  A normal copy should be created.
+#- Copy a file with the reflink=always option. Should result in
+#  error.
+#
+#---
+# Copyright (c) 2014, Oracle and/or its affiliates.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#---
+
+seq=`basename $0`
+echo QA output created by $seq
+
+here=`pwd`
+tmp=/tmp/$$
+status=1# failure is the default!
+trap _cleanup; exit \$status 0 1 2 3 15
+
+_cleanup()
+{
+umount $SCRATCH_MNT /dev/null
+cd /
+rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# real QA test starts here
+_supported_fs btrfs
+_supported_os Linux
+
+_require_scratch
+_require_cp_reflink
+
+SOURCE_DIR=$TEST_DIR/test-$seq
+CROSS_DEV_DIR=$SCRATCH_MNT/test-$seq
+# mount point  target for twice-mounted device
+TEST_DIR2=$TEST_DIR/mount2
+DUAL_MOUNT_DIR=$SCRATCH_MNT/test-bis-$seq
+
+rm -rf $SOURCE_DIR
+mkdir $SOURCE_DIR
+
+rm -f $seqres.full
+
+_scratch_mkfs
+$XFS_IO_PROG -f -c 'pwrite -S 0x61 0 9000' $SOURCE_DIR/original \
+  $seqres.full
+
+_filter_testdirs()
+{
+_filter_test_dir | _filter_scratch
+}
+
+_create_reflinks_to()
+{
+# auto reflink, should fall back to non-reflink
+rm -rf $1; mkdir $1
+echo reflink=auto:
+cp --reflink=auto $SOURCE_DIR/original $1/copy
+md5sum $SOURCE_DIR/original | _filter_testdirs
+md5sum $1/copy | _filter_testdirs
+
+# always reflink, should fail outright
+rm -rf $1; mkdir $1
+echo reflink=always:
+cp --reflink=always $SOURCE_DIR/original $1/copyfail \
+21 | _filter_testdirs
+
+# The failed target actually gets created by cp:
+ls $1/copyfail | _filter_testdirs
+}
+
+echo test reflinks across different devices
+_scratch_mount
+_create_reflinks_to $CROSS_DEV_DIR
+_scratch_unmount
+
+echo test reflinks across different mountpoints of same device
+mount $TEST_DEV $SCRATCH_MNT || _fail Couldn't double-mount $TEST_DEV
+_create_reflinks_to $DUAL_MOUNT_DIR
+umount $SCRATCH_MNT
+
+# success, all done
+status=0
+exit
diff --git a/tests/btrfs/030.out b/tests/btrfs/030.out
new file mode 100644
index 000..923a6e1
--- /dev/null
+++ b/tests/btrfs/030.out
@@ -0,0 +1,15 @@
+QA output created by 030
+test reflinks across different devices
+reflink=auto:
+42d69d1a6d333a7ebdf64792a555e392  TEST_DIR/test-030/original
+42d69d1a6d333a7ebdf64792a555e392  SCRATCH_MNT/test-030/copy
+reflink=always:
+cp: failed to clone 'SCRATCH_MNT/test-030/copyfail': Invalid cross-device link
+SCRATCH_MNT/test-030/copyfail
+test reflinks across different mountpoints of same device
+reflink=auto:
+42d69d1a6d333a7ebdf64792a555e392  TEST_DIR/test-030/original
+42d69d1a6d333a7ebdf64792a555e392  SCRATCH_MNT/test-bis-030/copy
+reflink=always:
+cp: failed to clone 'SCRATCH_MNT/test-bis-030/copyfail': Invalid cross-device 
link
+SCRATCH_MNT/test-bis-030/copyfail
diff --git a/tests/btrfs/group b/tests/btrfs/group
index 87e7bca..9231f7b 100644
--- a/tests/btrfs/group
+++ b/tests/btrfs/group
@@ -27,3 +27,4 @@
 022 auto
 023 auto
 024 auto quick
+030 auto quick
-- 
1.7.1

--
To unsubscribe from this list: send the line

Re: [PATCH v3] xfstests: btrfs/316: cross-subvolume sparse copy

2013-07-03 Thread Koen De Wit
On 07/03/2013 08:37 AM, Dave Chinner wrote:

 On Tue, Jul 02, 2013 at 11:51:21AM -0400, Eric Sandeen wrote:
 On Jul 2, 2013, at 10:28 AM, Koen De Wit koen.de@oracle.com
 wrote:

 Dave,

 Thanks for the review. I will clean up the commit message and do
 a full mail-to-myself-and-test-patch round trip to avoid errors
 like the wrong test numbers in the golden output. I'm sorry for
 this.

 About cutting out file names from the output. I did this in the
 first version of the patch:

   md5sum $TESTDIR1/$F | $AWK_PROG 'END {print $1}'

 but Eric Sandeen suggested to include them in order to provide
 more context in the output. (See
 http://oss.sgi.com/archives/xfs/2013-03/msg00231.html and
 http://oss.sgi.com/archives/xfs/2013-03/msg00220.html) That
 sounds like a good idea to me, it makes debugging failures
 easier. Whose opinion should I follow?

 Heh sorry.  IMHO maybe a middle ground; not bare md5sum but show
 only the base name?  In the end up to you; it seems Dave and I
 have different opinions on this.  :)
 
 I was just going by current xfstests convention. i.e, in common/rc:
 
 # Prints the md5 checksum of a given file
 _md5_checksum()
 {
 md5sum $1 | cut -d ' ' -f1
 }
 
 Which is used by all the hole punch tests and generic/311.


That's true, but these tests generate other context information in the
output. They don't just print a bunch of checksums.

For example, the output of generic/255 looks like:

1. into a hole
   daa100df6e6711906b61c9ab5aa16032
2. into allocated space
   0: [0..7]: extent
   1: [8..23]: hole
   2: [24..39]: extent
   cc58a7417c2d7763adc45b6fcd3fa024
3. into unwritten space
   0: [0..7]: extent
   1: [8..23]: hole
   2: [24..39]: extent
   daa100df6e6711906b61c9ab5aa16032
   (...)

The output of generic/311 looks like:

   Running test 1 buffered, normal suspend
   Random seed is 1
   ee6103415276cde95544b11b2675f132
   ee6103415276cde95544b11b2675f132
   Running test 1 direct, normal suspend
   Random seed is 1
   ee6103415276cde95544b11b2675f132
   ee6103415276cde95544b11b2675f132
   (...)

 Make of that what you will, but I'd prefer to see consistency of
 implementation across tests... ;)


Do I agree if I change _checksum_files() to:

   _checksum_files() {
   for F in file1 file2 file3
   do
   echo $F:
   for D in $TESTDIR1 $SCRATCH_MNT $SUBVOL2
   do
   _md5_checksum $D/$F
   done
   done
   }

It produces this in the output:

   (...)
   file1:
   00d620f69f30327f0f8946b95c12de44
   e09c80c42fda55f9d992e59ca6b3307d
   e09c80c42fda55f9d992e59ca6b3307d
   file2:
   d7402b46310fbbfbc5e466b1dccb043b
   d7402b46310fbbfbc5e466b1dccb043b
   917619ae44b38bb9968af261c3c45440
   file3:
   5a95800e4c04b7aa4e4de057721f
   b9f275cd638cb784c9e61def94c622a8
   5a95800e4c04b7aa4e4de057721f
   (...)


Thanks,
Koen.
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH v4] xfstests: btrfs/316: cross-subvolume sparse copy

2013-07-03 Thread Koen De Wit
This testscript creates reflinks to files on different subvolumes,
overwrites the original files and reflinks, and moves reflinked files
between subvolumes.

Signed-off-by: Koen De Wit koen.de@oracle.com
---
Two new common/rc functions used in this script (_require_cp_reflink and
_verify_reflink) have been submitted recently:
http://oss.sgi.com/archives/xfs/2013-05/msg00745.html

Changelog:
v3 - v4: Cleaning up commit message, cutting out filenames from output
  when printing md5sums, adding to group reflink
v2 - v3: Fixing wrapped patch
v1 - v2: Renamed 302 - 316, changes made based on comments from Eric
  Sandeen: http://oss.sgi.com/archives/xfs/2013-03/msg00231.html

 tests/btrfs/316 |  131 
 tests/btrfs/316.out |   48 +++
 tests/btrfs/group   |1 +
 3 files changed, 180 insertions(+), 0 deletions(-)
 create mode 100755 tests/btrfs/316
 create mode 100644 tests/btrfs/316.out

diff --git a/tests/btrfs/316 b/tests/btrfs/316
new file mode 100755
index 000..321832a
--- /dev/null
+++ b/tests/btrfs/316
@@ -0,0 +1,131 @@
+#! /bin/bash
+# FS QA Test No. btrfs/316
+#
+# Testing cross-subvolume sparse copy on btrfs
+#- Create two subvolumes, mount one of them
+#- Create a file on each (sub/root)volume,
+#  reflink them on the other volumes
+#- Change one original and two reflinked files
+#- Move reflinked files between subvolumes
+#
+#---
+# Copyright (c) 2013, Oracle and/or its affiliates.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#---
+
+seq=`basename $0`
+echo QA output created by $seq
+
+here=`pwd`
+tmp=/tmp/$$
+status=1# failure is the default!
+trap _cleanup; exit \$status 0 1 2 3 15
+
+_cleanup()
+{
+umount $SCRATCH_MNT
+rm -rf $TESTDIR1
+rm -rf $TESTDIR2
+btrfs subvolume delete $SUBVOL1  $seqres.full
+btrfs subvolume delete $SUBVOL2  $seqres.full
+cd /
+rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# real QA test starts here
+_supported_fs btrfs
+_supported_os Linux
+
+_require_scratch
+_require_cp_reflink
+
+_checksum_files() {
+for F in file1 file2 file3
+do
+echo $F:
+for D in $TESTDIR1 $SCRATCH_MNT $SUBVOL2
+do
+_md5_checksum $D/$F
+done
+done
+}
+
+TESTDIR1=$TEST_DIR/test-$seq-1
+TESTDIR2=$TEST_DIR/test-$seq-2
+SUBVOL1=$TEST_DIR/subvol-$seq-1
+SUBVOL2=$TEST_DIR/subvol-$seq-2
+
+_scratch_unmount 2/dev/null
+rm -rf $seqres.full
+rm -rf $TESTDIR1 $TESTDIR2
+btrfs subvol delete $SUBVOL1 /dev/null 21
+btrfs subvol delete $SUBVOL2 /dev/null 21
+
+mkdir $TESTDIR1
+mkdir $TESTDIR2
+btrfs subvolume create $SUBVOL1  $seqres.full
+btrfs subvolume create $SUBVOL2  $seqres.full
+_mount -t btrfs -o subvol=subvol-$seq-1 $TEST_DEV $SCRATCH_MNT
+
+echo Create initial files
+# TESTDIR1/file1 is very small and will be inlined
+$XFS_IO_PROG -f -c 'pwrite -S 0x61 0 10' $TESTDIR1/file1  $seqres.full
+$XFS_IO_PROG -f -c 'pwrite -S 0x62 0 13000' $SCRATCH_MNT/file2  $seqres.full
+$XFS_IO_PROG -f -c 'pwrite -S 0x63 0 17000' $SUBVOL2/file3  $seqres.full
+
+echo Create reflinks to the initial files on other subvolumes
+cp --reflink $TESTDIR1/file1 $SUBVOL1
+cp --reflink $TESTDIR1/file1 $SUBVOL2
+cp --reflink $SUBVOL1/file2 $TESTDIR1/
+cp --reflink $SUBVOL1/file2 $SUBVOL2
+cp --reflink $SUBVOL2/file3 $TESTDIR1/
+cp --reflink $SUBVOL2/file3 $SUBVOL1
+
+echo Verify the reflinks
+_verify_reflink $SCRATCH_MNT/file2 $TESTDIR1/file2
+_verify_reflink $SCRATCH_MNT/file2 $SUBVOL2/file2
+_verify_reflink $SUBVOL2/file3 $TESTDIR1/file3
+_verify_reflink $SUBVOL2/file3 $SCRATCH_MNT/file3
+echo Verify the file contents:
+_checksum_files
+
+echo -e ---\nOverwrite some files with new content
+$XFS_IO_PROG -c 'pwrite -S 0x64 0 20' $TESTDIR1/file1  $seqres.full
+$XFS_IO_PROG -c 'pwrite -S 0x66 0 21000' $SUBVOL2/file2  $seqres.full
+$XFS_IO_PROG -c 'pwrite -S 0x65 5000 5000' $SCRATCH_MNT/file3  $seqres.full
+
+echo Verify that non-overwritten reflinks still have the same data blocks
+_verify_reflink $TESTDIR1/file2 $SCRATCH_MNT/file2
+_verify_reflink $TESTDIR1/file3 $SUBVOL2/file3
+echo Verify the file contents:
+_checksum_files
+
+echo -e

[PATCH v4] xfstests: btrfs/311: sparse copy between different filesystems/mountpoints

2013-07-03 Thread Koen De Wit
Check if creating a sparse copy (reflink) of a file on btrfs
expectedly fails when it's done between different filesystems or
different mount points of the same filesystem.

For both situations, these actions are executed:
   - Copy a file with the reflink=auto option.
 A normal copy should be created.
   - Copy a file with the reflink=always option. Should result in error,
 no file should be created.

[sandeen: mostly cosmetic changes]

Signed-off-by: Koen De Wit koen.de.wit@xx
Signed-off-by: Eric Sandeen sandeen@xx
---
Changelog:
v3 - v4: Cleaning up commit message
v2 - v3: replacing $seqres.fll by $seqres.full
v1 - v2: (sandeen) mostly cosmetic changes, renamed 301 - 311

 tests/btrfs/311 |  107 +++
 tests/btrfs/311.out |   15 +++
 tests/btrfs/group   |4 ++
 3 files changed, 126 insertions(+), 0 deletions(-)
 create mode 100755 tests/btrfs/311
 create mode 100644 tests/btrfs/311.out

diff --git a/tests/btrfs/311 b/tests/btrfs/311
new file mode 100755
index 000..56ebd58
--- /dev/null
+++ b/tests/btrfs/311
@@ -0,0 +1,107 @@
+#! /bin/bash
+# FS QA Test No. 311
+#
+# Check if creating a sparse copy (reflink) of a file on btrfs
+# expectedly fails when it's done between different filesystems or
+# different mount points of the same filesystem.
+#
+# For both situations, these actions are executed:
+#- Copy a file with the reflink=auto option.
+#  A normal copy should be created.
+#- Copy a file with the reflink=always option. Should result in error,
+#  no file should be created.
+#
+#---
+# Copyright (c) 2013, Oracle and/or its affiliates.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#---
+
+seq=`basename $0`
+echo QA output created by $seq
+
+here=`pwd`
+tmp=/tmp/$$
+status=1# failure is the default!
+trap _cleanup; exit \$status 0 1 2 3 15
+
+_cleanup()
+{
+umount $SCRATCH_MNT /dev/null
+cd /
+rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# real QA test starts here
+_supported_fs btrfs
+_supported_os Linux
+
+_require_scratch
+_require_cp_reflink
+
+SOURCE_DIR=$TEST_DIR/test-$seq
+CROSS_DEV_DIR=$SCRATCH_MNT/test-$seq
+# mount point  target for twice-mounted device
+TEST_DIR2=$TEST_DIR/mount2
+DUAL_MOUNT_DIR=$SCRATCH_MNT/test-bis-$seq
+
+rm -rf $SOURCE_DIR
+mkdir $SOURCE_DIR
+
+rm -f $seqres.full
+
+_scratch_mkfs
+$XFS_IO_PROG -f -c 'pwrite -S 0x61 0 9000' $SOURCE_DIR/original \
+  $seqres.full
+
+_filter_testdirs()
+{
+_filter_test_dir | _filter_scratch
+}
+
+_create_reflinks_to()
+{
+# auto reflink, should fall back to non-reflink
+rm -rf $1; mkdir $1
+echo reflink=auto:
+cp --reflink=auto $SOURCE_DIR/original $1/copy
+md5sum $SOURCE_DIR/original | _filter_testdirs
+md5sum $1/copy | _filter_testdirs
+
+# always reflink, should fail outright
+rm -rf $1; mkdir $1
+echo reflink=always:
+cp --reflink=always $SOURCE_DIR/original $1/copyfail 21 | 
_filter_testdirs
+
+# The failed target actually gets created by cp:
+ls $1/copyfail | _filter_testdirs
+}
+
+echo test reflinks across different devices
+_scratch_mount
+_create_reflinks_to $CROSS_DEV_DIR
+_scratch_unmount
+
+echo test reflinks across different mountpoints of same device
+mount $TEST_DEV $SCRATCH_MNT || _fail Couldn't double-mount $TEST_DEV
+_create_reflinks_to $DUAL_MOUNT_DIR
+umount $SCRATCH_MNT
+
+# success, all done
+status=0
+exit
diff --git a/tests/btrfs/311.out b/tests/btrfs/311.out
new file mode 100644
index 000..210727b
--- /dev/null
+++ b/tests/btrfs/311.out
@@ -0,0 +1,15 @@
+QA output created by 311
+test reflinks across different devices
+reflink=auto:
+42d69d1a6d333a7ebdf64792a555e392  TEST_DIR/test-311/original
+42d69d1a6d333a7ebdf64792a555e392  SCRATCH_MNT/test-311/copy
+reflink=always:
+cp: failed to clone `SCRATCH_MNT/test-311/copyfail': Invalid cross-device link
+SCRATCH_MNT/test-311/copyfail
+test reflinks across different mountpoints of same device
+reflink=auto:
+42d69d1a6d333a7ebdf64792a555e392  TEST_DIR/test-311/original
+42d69d1a6d333a7ebdf64792a555e392  SCRATCH_MNT/test-bis-311/copy
+reflink=always:
+cp: failed to clone `SCRATCH_MNT/test-bis-311/copyfail

[PATCH v3] xfstests: btrfs/316: cross-subvolume sparse copy

2013-07-02 Thread Koen De Wit
This testscript creates reflinks to files on different subvolumes, overwrites 
the original files and reflinks, and moves reflinked files between subvolumes.

Originally submitted as testcase 302, changes are made based on comments from 
Eric: http://oss.sgi.com/archives/xfs/2013-03/msg00231.html
Two new common/rc functions used in this script (_require_cp_reflink and 
_verify_reflink) have been submitted recently: 
http://oss.sgi.com/archives/xfs/2013-05/msg00745.html
Thanks to Eric Sandeen and Dave Chinner for the reviews.

Version 3: fixing wrapped patch.

Signed-off-by: Koen De Wit koen.de@oracle.com
---
 tests/btrfs/316 |  130 +++
 tests/btrfs/316.out |   39 +++
 tests/btrfs/group   |1 +
 3 files changed, 170 insertions(+), 0 deletions(-)
 create mode 100755 tests/btrfs/316
 create mode 100644 tests/btrfs/316.out

diff --git a/tests/btrfs/316 b/tests/btrfs/316
new file mode 100755
index 000..7c9c368
--- /dev/null
+++ b/tests/btrfs/316
@@ -0,0 +1,130 @@
+#! /bin/bash
+# FS QA Test No. btrfs/316
+#
+# Testing cross-subvolume sparse copy on btrfs
+#- Create two subvolumes, mount one of them
+#- Create a file on each (sub/root)volume,
+#  reflink them on the other volumes
+#- Change one original and two reflinked files
+#- Move reflinked files between subvolumes
+#
+#---
+# Copyright (c) 2013, Oracle and/or its affiliates.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#---
+
+seq=`basename $0`
+echo QA output created by $seq
+
+here=`pwd`
+tmp=/tmp/$$
+status=1# failure is the default!
+trap _cleanup; exit \$status 0 1 2 3 15
+
+_cleanup()
+{
+umount $SCRATCH_MNT
+rm -rf $TESTDIR1
+rm -rf $TESTDIR2
+btrfs subvolume delete $SUBVOL1  $seqres.full
+btrfs subvolume delete $SUBVOL2  $seqres.full
+cd /
+rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# real QA test starts here
+_supported_fs btrfs
+_supported_os Linux
+
+_require_scratch
+_require_cp_reflink
+
+_checksum_files() {
+for F in file1 file2 file3
+do
+for D in $TESTDIR1 $SCRATCH_MNT $SUBVOL2
+do
+md5sum $D/$F | _filter_test_dir | _filter_scratch
+done
+done
+}
+
+TESTDIR1=$TEST_DIR/test-$seq-1
+TESTDIR2=$TEST_DIR/test-$seq-2
+SUBVOL1=$TEST_DIR/subvol-$seq-1
+SUBVOL2=$TEST_DIR/subvol-$seq-2
+
+_scratch_unmount 2/dev/null
+rm -rf $seqres.full
+rm -rf $TESTDIR1 $TESTDIR2
+btrfs subvol delete $SUBVOL1 /dev/null 21
+btrfs subvol delete $SUBVOL2 /dev/null 21
+
+mkdir $TESTDIR1
+mkdir $TESTDIR2
+btrfs subvolume create $SUBVOL1  $seqres.full
+btrfs subvolume create $SUBVOL2  $seqres.full
+_mount -t btrfs -o subvol=subvol-$seq-1 $TEST_DEV $SCRATCH_MNT
+
+echo Create initial files
+# TESTDIR1/file1 is very small and will be inlined
+$XFS_IO_PROG -f -c 'pwrite -S 0x61 0 10' $TESTDIR1/file1  $seqres.full
+$XFS_IO_PROG -f -c 'pwrite -S 0x62 0 13000' $SCRATCH_MNT/file2  $seqres.full
+$XFS_IO_PROG -f -c 'pwrite -S 0x63 0 17000' $SUBVOL2/file3  $seqres.full
+
+echo Create reflinks to the initial files on other subvolumes
+cp --reflink $TESTDIR1/file1 $SUBVOL1
+cp --reflink $TESTDIR1/file1 $SUBVOL2
+cp --reflink $SUBVOL1/file2 $TESTDIR1/
+cp --reflink $SUBVOL1/file2 $SUBVOL2
+cp --reflink $SUBVOL2/file3 $TESTDIR1/
+cp --reflink $SUBVOL2/file3 $SUBVOL1
+
+echo Verify the reflinks
+_verify_reflink $SCRATCH_MNT/file2 $TESTDIR1/file2
+_verify_reflink $SCRATCH_MNT/file2 $SUBVOL2/file2
+_verify_reflink $SUBVOL2/file3 $TESTDIR1/file3
+_verify_reflink $SUBVOL2/file3 $SCRATCH_MNT/file3
+echo Verify the file contents:
+_checksum_files
+
+echo -e ---\nOverwrite some files with new content
+$XFS_IO_PROG -c 'pwrite -S 0x64 0 20' $TESTDIR1/file1  $seqres.full
+$XFS_IO_PROG -c 'pwrite -S 0x66 0 21000' $SUBVOL2/file2  $seqres.full
+$XFS_IO_PROG -c 'pwrite -S 0x65 5000 5000' $SCRATCH_MNT/file3  $seqres.full
+
+echo Verify that non-overwritten reflinks still have the same data blocks
+_verify_reflink $TESTDIR1/file2 $SCRATCH_MNT/file2
+_verify_reflink $TESTDIR1/file3 $SUBVOL2/file3
+echo Verify the file contents:
+_checksum_files
+
+echo -e ---\nShuffle files between directories
+mv $TESTDIR1/file* $TESTDIR2
+mv

[PATCH v3] xfstests: btrfs/311: sparse copy between different filesystems/mountpoints

2013-07-02 Thread Koen De Wit
Thanks Eric for reviewing and improving testcases btrfs/306, 309, 310 and 311 !

I had just one comment: on line 70 the output was redirected to $seqres.fll 
instead of $seqres.full. Corrected patch below.

# Check if creating a sparse copy (reflink) of a file on btrfs
# expectedly fails when it's done between different filesystems or
# different mount points of the same filesystem.
#
# For both situations, these actions are executed:
#- Copy a file with the reflink=auto option.
#  A normal copy should be created.
#- Copy a file with the reflink=always option. Should result in error,
#  no file should be created.

[sandeen: mostly cosmetic changes]

Originally submitted as:
xfstests: 301: sparse copy between different filesystems/mountpoints on btrfs

Version 3: fixing wrapped patch.

Signed-off-by: Koen De Wit koen.de.wit@xx
Signed-off-by: Eric Sandeen sandeen@xx
---
 tests/btrfs/311 |  107 +++
 tests/btrfs/311.out |   15 +++
 tests/btrfs/group   |4 ++
 3 files changed, 126 insertions(+), 0 deletions(-)
 create mode 100755 tests/btrfs/311
 create mode 100644 tests/btrfs/311.out

diff --git a/tests/btrfs/311 b/tests/btrfs/311
new file mode 100755
index 000..56ebd58
--- /dev/null
+++ b/tests/btrfs/311
@@ -0,0 +1,107 @@
+#! /bin/bash
+# FS QA Test No. 311
+#
+# Check if creating a sparse copy (reflink) of a file on btrfs
+# expectedly fails when it's done between different filesystems or
+# different mount points of the same filesystem.
+#
+# For both situations, these actions are executed:
+#- Copy a file with the reflink=auto option.
+#  A normal copy should be created.
+#- Copy a file with the reflink=always option. Should result in error,
+#  no file should be created.
+#
+#---
+# Copyright (c) 2013, Oracle and/or its affiliates.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#---
+
+seq=`basename $0`
+echo QA output created by $seq
+
+here=`pwd`
+tmp=/tmp/$$
+status=1# failure is the default!
+trap _cleanup; exit \$status 0 1 2 3 15
+
+_cleanup()
+{
+umount $SCRATCH_MNT /dev/null
+cd /
+rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# real QA test starts here
+_supported_fs btrfs
+_supported_os Linux
+
+_require_scratch
+_require_cp_reflink
+
+SOURCE_DIR=$TEST_DIR/test-$seq
+CROSS_DEV_DIR=$SCRATCH_MNT/test-$seq
+# mount point  target for twice-mounted device
+TEST_DIR2=$TEST_DIR/mount2
+DUAL_MOUNT_DIR=$SCRATCH_MNT/test-bis-$seq
+
+rm -rf $SOURCE_DIR
+mkdir $SOURCE_DIR
+
+rm -f $seqres.full
+
+_scratch_mkfs
+$XFS_IO_PROG -f -c 'pwrite -S 0x61 0 9000' $SOURCE_DIR/original \
+  $seqres.full
+
+_filter_testdirs()
+{
+_filter_test_dir | _filter_scratch
+}
+
+_create_reflinks_to()
+{
+# auto reflink, should fall back to non-reflink
+rm -rf $1; mkdir $1
+echo reflink=auto:
+cp --reflink=auto $SOURCE_DIR/original $1/copy
+md5sum $SOURCE_DIR/original | _filter_testdirs
+md5sum $1/copy | _filter_testdirs
+
+# always reflink, should fail outright
+rm -rf $1; mkdir $1
+echo reflink=always:
+cp --reflink=always $SOURCE_DIR/original $1/copyfail 21 | 
_filter_testdirs
+
+# The failed target actually gets created by cp:
+ls $1/copyfail | _filter_testdirs
+}
+
+echo test reflinks across different devices
+_scratch_mount
+_create_reflinks_to $CROSS_DEV_DIR
+_scratch_unmount
+
+echo test reflinks across different mountpoints of same device
+mount $TEST_DEV $SCRATCH_MNT || _fail Couldn't double-mount $TEST_DEV
+_create_reflinks_to $DUAL_MOUNT_DIR
+umount $SCRATCH_MNT
+
+# success, all done
+status=0
+exit
diff --git a/tests/btrfs/311.out b/tests/btrfs/311.out
new file mode 100644
index 000..210727b
--- /dev/null
+++ b/tests/btrfs/311.out
@@ -0,0 +1,15 @@
+QA output created by 311
+test reflinks across different devices
+reflink=auto:
+42d69d1a6d333a7ebdf64792a555e392  TEST_DIR/test-311/original
+42d69d1a6d333a7ebdf64792a555e392  SCRATCH_MNT/test-311/copy
+reflink=always:
+cp: failed to clone `SCRATCH_MNT/test-311/copyfail': Invalid cross-device link
+SCRATCH_MNT/test-311/copyfail
+test reflinks across different mountpoints of same device
+reflink

Re: [PATCH v3] xfstests: btrfs/316: cross-subvolume sparse copy

2013-07-02 Thread Koen De Wit
Dave,

Thanks for the review. I will clean up the commit message and do a full 
mail-to-myself-and-test-patch round trip to avoid errors like the wrong test 
numbers in the golden output. I'm sorry for this.

About cutting out file names from the output. I did this in the first version 
of the patch:

   md5sum $TESTDIR1/$F | $AWK_PROG 'END {print $1}'

but Eric Sandeen suggested to include them in order to provide more context in 
the output. (See http://oss.sgi.com/archives/xfs/2013-03/msg00231.html and 
http://oss.sgi.com/archives/xfs/2013-03/msg00220.html) That sounds like a good 
idea to me, it makes debugging failures easier. Whose opinion should I follow?

Koen.


On 07/02/2013 12:15 PM, Dave Chinner wrote:

 On Tue, Jul 02, 2013 at 11:27:51AM +0200, Koen De Wit wrote:
 This testscript creates reflinks to files on different subvolumes, 
 overwrites the original files and reflinks, and moves reflinked files 
 between subvolumes.

 Originally submitted as testcase 302, changes are made based on comments 
 from Eric: http://oss.sgi.com/archives/xfs/2013-03/msg00231.html
 Two new common/rc functions used in this script (_require_cp_reflink and 
 _verify_reflink) have been submitted recently: 
 http://oss.sgi.com/archives/xfs/2013-05/msg00745.html
 Thanks to Eric Sandeen and Dave Chinner for the reviews.

 Version 3: fixing wrapped patch.
 
 Needs a proper commit message - wrap it at 72 columns,  change log
 should be placed below the --- devider, not be part of hte commit
 message.
 
 Test numbers do not need to be unique across all test directories,
 just unique within the tests/btrfs directory.
 
 +
 +_checksum_files() {
 +for F in file1 file2 file3
 +do
 +for D in $TESTDIR1 $SCRATCH_MNT $SUBVOL2
 +do
 +md5sum $D/$F | _filter_test_dir | _filter_scratch
 
 Just cut the file name out. No need for filtering at that point...
 
 +done
 +done
 +}
 +
 +TESTDIR1=$TEST_DIR/test-$seq-1
 +TESTDIR2=$TEST_DIR/test-$seq-2
 +SUBVOL1=$TEST_DIR/subvol-$seq-1
 +SUBVOL2=$TEST_DIR/subvol-$seq-2
 
 -316-.
 
 +Verify the file contents:
 +e09c80c42fda55f9d992e59ca6b3307d  TEST_DIR/test-302-1/file1
 
 which means the golden output is broken, despite the filtering
 
 You should test your patches before posting ;)
 
 FWIW, this is why you should simply cut the filename completely out
 like we do elsewhere
 
 Cheers,
 
 Dave.


--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH v2] xfstests: btrfs/316: cross-subvolume sparse copy

2013-07-01 Thread Koen De Wit
This testscript creates reflinks to files on different subvolumes, 
overwrites the original files and reflinks, and moves reflinked files 
between subvolumes.


Originally submitted as testcase 302, changes are made based on comments 
from Eric: http://oss.sgi.com/archives/xfs/2013-03/msg00231.html
Two new common/rc functions used in this script (_require_cp_reflink and 
_verify_reflink) have been submitted recently: 
http://oss.sgi.com/archives/xfs/2013-05/msg00745.html

Thanks to Eric Sandeen and Dave Chinner for the reviews.

Signed-off-by: Koen De Wit koen.de@oracle.com
---
 tests/btrfs/316 |  130 
+++

 tests/btrfs/316.out |   39 +++
 tests/btrfs/group   |1 +
 3 files changed, 170 insertions(+), 0 deletions(-)
 create mode 100755 tests/btrfs/316
 create mode 100644 tests/btrfs/316.out

diff --git a/tests/btrfs/316 b/tests/btrfs/316
new file mode 100755
index 000..7c9c368
--- /dev/null
+++ b/tests/btrfs/316
@@ -0,0 +1,130 @@
+#! /bin/bash
+# FS QA Test No. btrfs/316
+#
+# Testing cross-subvolume sparse copy on btrfs
+#- Create two subvolumes, mount one of them
+#- Create a file on each (sub/root)volume,
+#  reflink them on the other volumes
+#- Change one original and two reflinked files
+#- Move reflinked files between subvolumes
+#
+#---
+# Copyright (c) 2013, Oracle and/or its affiliates.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#---
+
+seq=`basename $0`
+echo QA output created by $seq
+
+here=`pwd`
+tmp=/tmp/$$
+status=1# failure is the default!
+trap _cleanup; exit \$status 0 1 2 3 15
+
+_cleanup()
+{
+umount $SCRATCH_MNT
+rm -rf $TESTDIR1
+rm -rf $TESTDIR2
+btrfs subvolume delete $SUBVOL1  $seqres.full
+btrfs subvolume delete $SUBVOL2  $seqres.full
+cd /
+rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# real QA test starts here
+_supported_fs btrfs
+_supported_os Linux
+
+_require_scratch
+_require_cp_reflink
+
+_checksum_files() {
+for F in file1 file2 file3
+do
+for D in $TESTDIR1 $SCRATCH_MNT $SUBVOL2
+do
+md5sum $D/$F | _filter_test_dir | _filter_scratch
+done
+done
+}
+
+TESTDIR1=$TEST_DIR/test-$seq-1
+TESTDIR2=$TEST_DIR/test-$seq-2
+SUBVOL1=$TEST_DIR/subvol-$seq-1
+SUBVOL2=$TEST_DIR/subvol-$seq-2
+
+_scratch_unmount 2/dev/null
+rm -rf $seqres.full
+rm -rf $TESTDIR1 $TESTDIR2
+btrfs subvol delete $SUBVOL1 /dev/null 21
+btrfs subvol delete $SUBVOL2 /dev/null 21
+
+mkdir $TESTDIR1
+mkdir $TESTDIR2
+btrfs subvolume create $SUBVOL1  $seqres.full
+btrfs subvolume create $SUBVOL2  $seqres.full
+_mount -t btrfs -o subvol=subvol-$seq-1 $TEST_DEV $SCRATCH_MNT
+
+echo Create initial files
+# TESTDIR1/file1 is very small and will be inlined
+$XFS_IO_PROG -f -c 'pwrite -S 0x61 0 10' $TESTDIR1/file1  $seqres.full
+$XFS_IO_PROG -f -c 'pwrite -S 0x62 0 13000' $SCRATCH_MNT/file2  
$seqres.full

+$XFS_IO_PROG -f -c 'pwrite -S 0x63 0 17000' $SUBVOL2/file3  $seqres.full
+
+echo Create reflinks to the initial files on other subvolumes
+cp --reflink $TESTDIR1/file1 $SUBVOL1
+cp --reflink $TESTDIR1/file1 $SUBVOL2
+cp --reflink $SUBVOL1/file2 $TESTDIR1/
+cp --reflink $SUBVOL1/file2 $SUBVOL2
+cp --reflink $SUBVOL2/file3 $TESTDIR1/
+cp --reflink $SUBVOL2/file3 $SUBVOL1
+
+echo Verify the reflinks
+_verify_reflink $SCRATCH_MNT/file2 $TESTDIR1/file2
+_verify_reflink $SCRATCH_MNT/file2 $SUBVOL2/file2
+_verify_reflink $SUBVOL2/file3 $TESTDIR1/file3
+_verify_reflink $SUBVOL2/file3 $SCRATCH_MNT/file3
+echo Verify the file contents:
+_checksum_files
+
+echo -e ---\nOverwrite some files with new content
+$XFS_IO_PROG -c 'pwrite -S 0x64 0 20' $TESTDIR1/file1  $seqres.full
+$XFS_IO_PROG -c 'pwrite -S 0x66 0 21000' $SUBVOL2/file2  $seqres.full
+$XFS_IO_PROG -c 'pwrite -S 0x65 5000 5000' $SCRATCH_MNT/file3  
$seqres.full

+
+echo Verify that non-overwritten reflinks still have the same data blocks
+_verify_reflink $TESTDIR1/file2 $SCRATCH_MNT/file2
+_verify_reflink $TESTDIR1/file3 $SUBVOL2/file3
+echo Verify the file contents:
+_checksum_files
+
+echo -e ---\nShuffle files between directories
+mv $TESTDIR1/file* $TESTDIR2
+mv $SCRATCH_MNT/file* $TESTDIR1

[PATCH v2] xfstests: btrfs/311: sparse copy between different filesystems/mountpoints

2013-07-01 Thread Koen De Wit
Thanks Eric for reviewing and improving testcases btrfs/306, 309, 310 
and 311 !


I had just one comment: on line 70 the output was redirected to 
$seqres.fll instead of $seqres.full. Corrected patch below.


# Check if creating a sparse copy (reflink) of a file on btrfs
# expectedly fails when it's done between different filesystems or
# different mount points of the same filesystem.
#
# For both situations, these actions are executed:
#- Copy a file with the reflink=auto option.
#  A normal copy should be created.
#- Copy a file with the reflink=always option. Should result in error,
#  no file should be created.

[sandeen: mostly cosmetic changes]

Originally submitted as:
xfstests: 301: sparse copy between different filesystems/mountpoints on 
btrfs


Signed-off-by: Koen De Wit koen.de.wit@xx
Signed-off-by: Eric Sandeen sandeen@xx
---
 tests/btrfs/311 |  107 
+++

 tests/btrfs/311.out |   15 +++
 tests/btrfs/group   |4 ++
 3 files changed, 126 insertions(+), 0 deletions(-)
 create mode 100755 tests/btrfs/311
 create mode 100644 tests/btrfs/311.out

diff --git a/tests/btrfs/311 b/tests/btrfs/311
new file mode 100755
index 000..56ebd58
--- /dev/null
+++ b/tests/btrfs/311
@@ -0,0 +1,107 @@
+#! /bin/bash
+# FS QA Test No. 311
+#
+# Check if creating a sparse copy (reflink) of a file on btrfs
+# expectedly fails when it's done between different filesystems or
+# different mount points of the same filesystem.
+#
+# For both situations, these actions are executed:
+#- Copy a file with the reflink=auto option.
+#  A normal copy should be created.
+#- Copy a file with the reflink=always option. Should result in error,
+#  no file should be created.
+#
+#---
+# Copyright (c) 2013, Oracle and/or its affiliates.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#---
+
+seq=`basename $0`
+echo QA output created by $seq
+
+here=`pwd`
+tmp=/tmp/$$
+status=1# failure is the default!
+trap _cleanup; exit \$status 0 1 2 3 15
+
+_cleanup()
+{
+umount $SCRATCH_MNT /dev/null
+cd /
+rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# real QA test starts here
+_supported_fs btrfs
+_supported_os Linux
+
+_require_scratch
+_require_cp_reflink
+
+SOURCE_DIR=$TEST_DIR/test-$seq
+CROSS_DEV_DIR=$SCRATCH_MNT/test-$seq
+# mount point  target for twice-mounted device
+TEST_DIR2=$TEST_DIR/mount2
+DUAL_MOUNT_DIR=$SCRATCH_MNT/test-bis-$seq
+
+rm -rf $SOURCE_DIR
+mkdir $SOURCE_DIR
+
+rm -f $seqres.full
+
+_scratch_mkfs
+$XFS_IO_PROG -f -c 'pwrite -S 0x61 0 9000' $SOURCE_DIR/original \
+  $seqres.full
+
+_filter_testdirs()
+{
+_filter_test_dir | _filter_scratch
+}
+
+_create_reflinks_to()
+{
+# auto reflink, should fall back to non-reflink
+rm -rf $1; mkdir $1
+echo reflink=auto:
+cp --reflink=auto $SOURCE_DIR/original $1/copy
+md5sum $SOURCE_DIR/original | _filter_testdirs
+md5sum $1/copy | _filter_testdirs
+
+# always reflink, should fail outright
+rm -rf $1; mkdir $1
+echo reflink=always:
+cp --reflink=always $SOURCE_DIR/original $1/copyfail 21 | 
_filter_testdirs

+
+# The failed target actually gets created by cp:
+ls $1/copyfail | _filter_testdirs
+}
+
+echo test reflinks across different devices
+_scratch_mount
+_create_reflinks_to $CROSS_DEV_DIR
+_scratch_unmount
+
+echo test reflinks across different mountpoints of same device
+mount $TEST_DEV $SCRATCH_MNT || _fail Couldn't double-mount $TEST_DEV
+_create_reflinks_to $DUAL_MOUNT_DIR
+umount $SCRATCH_MNT
+
+# success, all done
+status=0
+exit
diff --git a/tests/btrfs/311.out b/tests/btrfs/311.out
new file mode 100644
index 000..210727b
--- /dev/null
+++ b/tests/btrfs/311.out
@@ -0,0 +1,15 @@
+QA output created by 311
+test reflinks across different devices
+reflink=auto:
+42d69d1a6d333a7ebdf64792a555e392  TEST_DIR/test-311/original
+42d69d1a6d333a7ebdf64792a555e392  SCRATCH_MNT/test-311/copy
+reflink=always:
+cp: failed to clone `SCRATCH_MNT/test-311/copyfail': Invalid 
cross-device link

+SCRATCH_MNT/test-311/copyfail
+test reflinks across different mountpoints of same device
+reflink=auto

[BUG?] Btrfs quota: overwritten space is counted twice

2013-04-05 Thread Koen De Wit
When data in a file is overwritten, starting somewhere in the middle of 
the file, the overwritten space is counted twice against the space usage 
numbers. Is this a bug, or did I something wrong?


This is what I did:

I create a subvolume and limit it to 4 MB, and create a 1000 KB file in 
the subvol:


# btrfs subvol create s
  Create subvolume './s'
# btrfs qgroup limit 4m s
# btrfs qgroup show ./ | grep 260
  0/260 4096 4096
# dd if=/dev/zero of=s/file bs=1024 count=1000; sync
# ls -lah s/file
  -rw-r--r--. 1 root root 1000K Apr  6 00:13 s/file
# btrfs qgroup show ./ | grep 260
  0/260 1028096 1028096

Then I overwrite the last 900 KB of the file, and add 100 KB of data, 
resulting in a 1.1 MB file. The space usage numbers shows 2 MB however:


# dd if=/dev/zero of=s/file bs=1024 count=1000 seek=100; sync
# ls -lah s/file
  -rw-r--r--. 1 root root 1.1M Apr  6 00:13 s/file
# btrfs qgroup show ./ | grep 260
  0/260 2052096 2052096

I repeat this twice, the file becomes 1.3 MB but the usage number goes 
to almost 4 MB:


# dd if=/dev/zero of=s/file bs=1024 count=1000 seek=200; sync
# dd if=/dev/zero of=s/file bs=1024 count=1000 seek=300; sync
# ls -lah s/file
  -rw-r--r--. 1 root root 1.3M Apr  6 00:14 s/file
# btrfs qgroup show ./ | grep 260
  0/260 4100096 4100096

Doing the same again results in quota exceeded errors:

# dd if=/dev/zero of=s/file bs=1024 count=1000 seek=400; sync
  dd: writing `s/file': Disk quota exceeded
  78+0 records in
  77+0 records out
  78848 bytes (79 kB) copied, 0.00138135 s, 57.1 MB/s
# ls -lah s/file
  -rw-r--r--. 1 root root 477K Apr  6 00:15 s/file
# btrfs qgroup show ./ | grep 260
  0/260 4182016 4182016
# touch s/emptyfile
  touch: cannot touch `s/emptyfile': Disk quota exceeded

Koen.
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH] btrfs-progs: add quota-related info to usage messages

2013-03-27 Thread Koen De Wit

Extending usage messages with some info on the quota functionality:
- The -i option of subvol create and subvol snapshot was not 
documented

- The -c option of qgroup limit is the default option
- The qouta rescan command is not yet implemented, while it should be
  executed after enabling quota on a non-empty filesystem.

Signed-off-by: Koen De Wit koen.de@oracle.com
---
 cmds-qgroup.c|3 ++-
 cmds-quota.c |4 
 cmds-subvolume.c |   11 ---
 3 files changed, 14 insertions(+), 4 deletions(-)

diff --git a/cmds-qgroup.c b/cmds-qgroup.c
index 275f00f..95aca9b 100644
--- a/cmds-qgroup.c
+++ b/cmds-qgroup.c
@@ -326,7 +326,8 @@ static const char * const cmd_qgroup_limit_usage[] = {
 btrfs qgroup limit [options] size|none [qgroupid] path,
 Limit the size of a subvolume quota group.,
 ,
--c   limit amount of data after compression,
+-c   limit amount of data after compression. This is the default,,
+ it is currently not possible to turn off this option.,
 -e   limit space exclusively assigned to this qgroup,
 NULL
 };
diff --git a/cmds-quota.c b/cmds-quota.c
index 8481514..71cd9f1 100644
--- a/cmds-quota.c
+++ b/cmds-quota.c
@@ -64,6 +64,9 @@ int quota_ctl(int cmd, int argc, char **argv)
 static const char * const cmd_quota_enable_usage[] = {
 btrfs quota enable path,
 Enable subvolume quota support for a filesystem.,
+Any data already present on the filesystem will not count towards,
+the space usage numbers. It is recommended to enable quota for a,
+filesystem before writing any data to it.,
 NULL
 };

@@ -92,6 +95,7 @@ static int cmd_quota_disable(int argc, char **argv)
 static const char * const cmd_quota_rescan_usage[] = {
 btrfs quota rescan path,
 Rescan the subvolume for a changed quota setting.,
+Not yet implemented.,
 NULL
 };

diff --git a/cmds-subvolume.c b/cmds-subvolume.c
index 74e2130..b762470 100644
--- a/cmds-subvolume.c
+++ b/cmds-subvolume.c
@@ -61,10 +61,13 @@ static int test_isdir(char *path)
 }

 static const char * const cmd_subvol_create_usage[] = {
-btrfs subvolume create [dest/]name,
+btrfs subvolume create [-i qgroupid] [dest/]name,
 Create a subvolume,
 Create a subvolume name in dest.  If dest is not given,
 subvolume name will be created in the current directory.,
+,
+-i qgroupid  add the newly created subvolume to a qgroup. This,
+   option can be given multiple times.,
 NULL
 };

@@ -480,12 +483,14 @@ out:
 }

 static const char * const cmd_snapshot_usage[] = {
-btrfs subvolume snapshot [-r] source [dest/]name,
+btrfs subvolume snapshot [-r] [-i qgroupid] source 
[dest/]name,

 Create a snapshot of the subvolume,
 Create a writable/readonly snapshot of the subvolume source with,
 the name name in the dest directory,
 ,
--r create a readonly snapshot,
+-r create a readonly snapshot,
+-i qgroupid  add the newly created snapshot to a qgroup. This,
+   option can be given multiple times.,
 NULL
 };

--
1.7.2.5

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


The -c option of btrfs qgroup limit

2013-03-26 Thread Koen De Wit

All,

The btrfs qgroup limit command has an option -c which means limit amount
of data after compression. Whether this option is specified or not, I seem
to be able to write more well-compressible data than the specified limit. I
was expecting that omitting the -c option would never allow me to write
more data than specified by the limit. Am I missing something? Which
difference in behavior should I expect from the -c option?

Koen.

---

This is what I did:

Mount a btrfs filesystem with the compress option and turn on quota:
  # mkfs.btrfs -f /dev/sdg2
  # mount -o compress /dev/sdg2 /mnt
  # cd /mnt
  # btrfs quota enable ./

Create a subvol and limit the amount of data after compression. Write some
well-compressible files:
  # btrfs subvol create subvol1
  # btrfs qgroup limit -c 3m ./subvol1
  # for i in `seq 1 5`; do
dd if=/dev/zero of=subvol1/file$i bs=1024 count=1000;
sync;
done
  (no errors)
  # du -s subvol1
  5000 subvol1
We can write 5 MB of data to a file that is limited to 3 MB, which is 
normal

in this case because data from /dev/zero is very good compressible.

Create a subvol and limit the amount of data after compression. Write some
poorly compressible files:
  # btrfs subvol create subvol2
  # btrfs qgroup limit -c 3m ./subvol2
  # for i in `seq 1 5`; do
dd if=/dev/urandom of=subvol2/file$i bs=1024 count=1000;
sync;
done
  dd: writing `subvol2/file4': Disk quota exceeded
  dd: opening `subvol2/file5': Disk quota exceeded
  # du -s subvol2
  3056 subvol2
Here we get quota violations, because data from /dev/urandom is poorly
compressible.

Now write some well-compressible data to a subvol that is limited without
the -c option:
  # btrfs subvol create subvol3
  # btrfs qgroup limit 3m ./subvol3
  # for i in `seq 1 5`; do
dd if=/dev/zero of=subvol3/file$i bs=1024 count=1000;
sync;
  done
  (no errors)
  # du -s subvol3
  5000 subvol3
We're still able to write 5 MB of data to a subvol that is limited to 3 MB.
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Adding a non-empty subvol to a qgroup

2013-03-22 Thread Koen De Wit

All,

When adding a subvolume to a qgroup, pre-existing files in that 
subvolume are not counted in the referenced/exclusive space of the 
qgroup. Is this intended behavior ?


I create a subvol with one file:

  # mkfs.btrfs /dev/sdg
  # mount /dev/sdg /mnt/fulldisk
  # cd /mnt/fulldisk
  # btrfs quota enable ./
  # btrfs sub create sub1
  # dd if=/dev/zero of=sub1/file1 bs=10 count=1
  # sync
  # btrfs qgroup show ./
  0/257 106496 106496

Now I create a new qgroup on level 1 and add the qgroup of sub1 to it :

  # btrfs qgroup create 1/0 ./
  # btrfs qgroup assign 0/257 1/0 ./
  # sync
  # btrfs fi sync ./
  # btrfs quota rescan ./
  # btrfs quota rescan ./sub1
  # btrfs qgroup show ./
  0/257 106496 106496
  1/0 0 0

The pre-existing file does not contribute to the space numbers.

Let's create a new file:

  # dd if=/dev/zero of=sub1/file2 bs=5 count=1
  # sync
  # btrfs qgroup show ./
  0/257 159744 159744
  1/0 53248 53248

We see that only the new file is included in the space numbers.

Now I remove the first file:

  # rm -f sub1/file1
  # sync
  # btrfs qgroup show ./
  0/257 57344 57344
  1/0 -49152 -49152

The space numbers go below zero. Even if the behavior above is intended, 
the removal of the pre-existing file should not result in negative space 
numbers.


Koen.
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Bug: trying to create reflink on different device results in empty file

2012-12-11 Thread Koen De Wit
When trying to create a clone (reflink) of a file on a different device, 
you'll get this error:

Invalid cross-device link

However, an empty file is created on the target location. An invalid 
clone operation should not result in the creation of a file.



Example:

# mount
(...)
/dev/sdb1 on /mnt/diskb type btrfs (rw)
/dev/sdf1 on /mnt/diskf type btrfs (rw)

# ls -l diskb
total 4
-rw-r--r-- 1 root root 11 Dec 11 22:45 testfile.txt

# ls -l diskf
total 0

# cp --reflink diskb/testfile.txt diskf/testcopy.txt
cp: failed to clone `diskf/testcopy.txt' from `diskb/testfile.txt': 
Invalid cross-device link


# ls -l diskf
total 0
-rw-r--r-- 1 root root 0 Dec 11 22:56 testcopy.txt

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html