Re: [PATCH 1/2] btrfs: add small program for clone testing

2014-02-06 Thread David Disseldorp
Hi Dave, On Thu, 6 Feb 2014 10:09:36 +1100, Dave Chinner wrote: On Wed, Feb 05, 2014 at 12:16:48PM +0100, David Disseldorp wrote: The cloner program is capable of cloning files using the BTRFS_IOC_CLONE and BTRFS_IOC_CLONE_RANGE ioctls. Signed-off-by: David Disseldorp dd...@suse.de

[PATCH v2 1/2 ] Btrfs: switch to btrfs_previous_extent_item()

2014-02-06 Thread Wang Shilong
From: Wang Shilong wangsl.f...@cn.fujitsu.com Since we have introduced btrfs_previous_extent_item() to search previous extent item, just switch into it. Signed-off-by: Wang Shilong wangsl.f...@cn.fujitsu.com --- Changelog: v1-v2: set @found_key properly and fix return value(thanks to

Re: [PATCH v2 1/2 ] Btrfs: switch to btrfs_previous_extent_item()

2014-02-06 Thread Filipe David Manana
On Thu, Feb 6, 2014 at 12:02 PM, Wang Shilong wangshilong1...@gmail.com wrote: From: Wang Shilong wangsl.f...@cn.fujitsu.com Since we have introduced btrfs_previous_extent_item() to search previous extent item, just switch into it. Signed-off-by: Wang Shilong wangsl.f...@cn.fujitsu.com

Re: Provide a better free space estimate on RAID1

2014-02-06 Thread Roman Mamedov
On Thu, 06 Feb 2014 09:38:15 +0200 Brendan Hide bren...@swiftspirit.co.za wrote: This is a known issue: https://btrfs.wiki.kernel.org/index.php/FAQ#Why_does_df_show_incorrect_free_space_for_my_RAID_volume.3F Btrfs is still considered experimental It's long overdue to start tackling these

Re: [PATCH] Btrfs: add regression test for running snapshot and send concurrently

2014-02-06 Thread Wang Shilong
Hi Dave, On Mon, Feb 03, 2014 at 11:22:36PM +0800, Wang Shilong wrote: From: Wang Shilong wangsl.f...@cn.fujitsu.com Btrfs would fail to send if snapshot run concurrently, this test is to make sure we have fixed the bug. Couple of comments below. +_scratch_mkfs /dev/null 21

[PATCH v2 0/3] __btrfs_drop_extents() BUG_ON reproducer

2014-02-06 Thread David Disseldorp
This patch-set provides a reproducer for hitting the 3.14.0-rc1 BUG_ON() at: 692 int __btrfs_drop_extents(struct btrfs_trans_handle *trans, ... 839 /* 840 * | range to drop - | 841 * | extent | 842

[PATCH v2 2/3] src/cloner: use btrfs/ioctl.h header if present

2014-02-06 Thread David Disseldorp
Check for the btrfsprogs-devel ioctl.h header at configure time. Use it in src/cloner if present, otherwise fall back to using the copied clone ioctl definitions. Signed-off-by: David Disseldorp dd...@suse.de --- configure.ac | 1 + src/cloner.c | 4 2 files changed, 5 insertions(+) diff

[PATCH v2 1/3] btrfs: add small program for clone testing

2014-02-06 Thread David Disseldorp
The cloner program is capable of cloning files using the BTRFS_IOC_CLONE and BTRFS_IOC_CLONE_RANGE ioctls. Signed-off-by: David Disseldorp dd...@suse.de --- .gitignore | 1 + src/Makefile | 2 +- src/cloner.c | 188 +++ 3 files

[PATCH v2 3/3] btrfs/035: add new clone overwrite regression test

2014-02-06 Thread David Disseldorp
This test uses the newly added cloner binary to dispatch full file and range specific clone (reflink) requests. Signed-off-by: David Disseldorp dd...@suse.de --- tests/btrfs/035 | 77 + tests/btrfs/035.out | 3 +++ tests/btrfs/group | 1

[PATCH] ioctl: add note regarding CLONE_RANGE(len=0) behaviour

2014-02-06 Thread David Disseldorp
A BTRFS_IOC_CLONE_RANGE request with a src_length value of zero has the effect of cloning all data from src_offset through to end-of-file. Document this behaviour in the header file for those who (like me) incorrectly assume that no data is cloned in such a case. Signed-off-by: David Disseldorp

[PATCH v2] Btrfs: add regression test for running snapshot and send concurrently

2014-02-06 Thread Wang Shilong
From: Wang Shilong wangsl.f...@cn.fujitsu.com Btrfs would fail to send if snapshot run concurrently, this test is to make sure we have fixed the bug. Signed-off-by: Wang Shilong wangsl.f...@cn.fujitsu.com --- Changelog v1-v2: Avoid race codes and a code style update(Thanks to Dave

Re: [PATCH] Btrfs: throttle delayed refs better

2014-02-06 Thread Josef Bacik
On 02/05/2014 05:57 PM, Johannes Hirte wrote: On Wed, 5 Feb 2014 16:46:57 -0500 Josef Bacik jba...@fb.com wrote: On 02/05/2014 04:42 PM, Johannes Hirte wrote: On Wed, 5 Feb 2014 14:36:39 -0500 Josef Bacik jba...@fb.com wrote: On 02/05/2014 02:30 PM, Johannes Hirte wrote: On Wed, 5 Feb

[PATCH v3] Btrfs: add regression test for running snapshot and send concurrently

2014-02-06 Thread Wang Shilong
From: Wang Shilong wangsl.f...@cn.fujitsu.com Btrfs would fail to send if snapshot run concurrently, this test is to make sure we have fixed the bug. Signed-off-by: Wang Shilong wangsl.f...@cn.fujitsu.com --- Changelog v1-v2: Avoid race codes and a code style update(Thanks to Dave

[PATCH] btrfs-progs: Convert BUG() to BUG_ON(1)

2014-02-06 Thread Mitch Harder
Convert the instances of BUG() to BUG_ON(1) to provide information about the location of the abort. Signed-off-by: Mitch Harder mitch.har...@sabayonlinux.org --- btrfs-debug-tree.c | 4 ++-- ctree.c| 20 ++-- ctree.h| 2 +- disk-io.c | 4 ++--

Re: [PATCH] btrfs: introduce BTRFS_IOC_GET_DEVS

2014-02-06 Thread David Sterba
On Mon, Jan 27, 2014 at 08:47:09AM +, Hugo Mills wrote: On Mon, Jan 27, 2014 at 04:52:50PM +0800, Anand Jain wrote: The user land progs needs a simple way to read the raw list of disks and its parameters as btrfs kernel understands it. This patch will introduce BTRFS_IOC_GET_DEVS

Re: Provide a better free space estimate on RAID1

2014-02-06 Thread Goffredo Baroncelli
Hi Roman On 02/06/2014 01:45 PM, Roman Mamedov wrote: On Thu, 06 Feb 2014 09:38:15 +0200 [...] There's not a lot of code to include (as my 3-line patch demonstrates), it could just as easily be removed when it's obsolete. But I did not have any high hopes of defeating the broken by design

Re: [PATCH] btrfs: introduce BTRFS_IOC_GET_DEVS

2014-02-06 Thread Goffredo Baroncelli
On 02/06/2014 08:49 PM, David Sterba wrote: On Mon, Jan 27, 2014 at 08:47:09AM +, Hugo Mills wrote: On Mon, Jan 27, 2014 at 04:52:50PM +0800, Anand Jain wrote: The user land progs needs a simple way to read the raw list of disks and its parameters as btrfs kernel understands it. This

Re: [PATCH] btrfs-progs: Convert BUG() to BUG_ON(1)

2014-02-06 Thread Josef Bacik
On 02/06/2014 01:34 PM, Mitch Harder wrote: Convert the instances of BUG() to BUG_ON(1) to provide information about the location of the abort. If we're going to screw with our BUG()'s in any sort of big way like this I'd rather it be to swap us over to ASSERT() in the cases that it makes

Re: Provide a better free space estimate on RAID1

2014-02-06 Thread Josef Bacik
On 02/05/2014 03:15 PM, Roman Mamedov wrote: Hello, On a freshly-created RAID1 filesystem of two 1TB disks: # df -h /mnt/p2/ Filesystem Size Used Avail Use% Mounted on /dev/sda2 1.8T 1.1M 1.8T 1% /mnt/p2 I cannot write 2TB of user data to that RAID1, so this estimate is

[PATCH] Btrfs: balance delayed inode updates

2014-02-06 Thread Josef Bacik
While trying to reproduce a delayed ref problem I noticed the box kept falling over using all 80gb of my ram with btrfs_inode's and btrfs_delayed_node's. Turns out this is because we only throttle delayed inode updates in btrfs_dirty_inode, which doesn't actually get called that often, especially

Re: [PATCH] Btrfs: throttle delayed refs better

2014-02-06 Thread Josef Bacik
On 02/05/2014 05:57 PM, Johannes Hirte wrote: On Wed, 5 Feb 2014 16:46:57 -0500 Josef Bacik jba...@fb.com wrote: On 02/05/2014 04:42 PM, Johannes Hirte wrote: On Wed, 5 Feb 2014 14:36:39 -0500 Josef Bacik jba...@fb.com wrote: On 02/05/2014 02:30 PM, Johannes Hirte wrote: On Wed, 5 Feb

Re: [PATCH] btrfs-progs: Convert BUG() to BUG_ON(1)

2014-02-06 Thread David Sterba
On Thu, Feb 06, 2014 at 12:34:08PM -0600, Mitch Harder wrote: Convert the instances of BUG() to BUG_ON(1) to provide information about the location of the abort. kerncompat.h: #define BUG() abort() #define BUG_ON(c) assert(!(c)) I'd rather fix the definition to do the same thing, that way no

Re: [PATCH] btrfs-progs: Convert BUG() to BUG_ON(1)

2014-02-06 Thread Josef Bacik
On 02/06/2014 01:34 PM, Mitch Harder wrote: Convert the instances of BUG() to BUG_ON(1) to provide information about the location of the abort. This is where I realize this was against btrfs-progs, ignore me. Thanks, Josef -- To unsubscribe from this list: send the line unsubscribe

Re: [PATCH] Btrfs: balance delayed inode updates

2014-02-06 Thread Josef Bacik
On 02/06/2014 04:07 PM, Josef Bacik wrote: While trying to reproduce a delayed ref problem I noticed the box kept falling over using all 80gb of my ram with btrfs_inode's and btrfs_delayed_node's. Turns out this is because we only throttle delayed inode updates in btrfs_dirty_inode, which

[PATCH] Btrfs: balance delayed inode updates V2

2014-02-06 Thread Josef Bacik
While trying to reproduce a delayed ref problem I noticed the box kept falling over using all 80gb of my ram with btrfs_inode's and btrfs_delayed_node's. Turns out this is because we only throttle delayed inode updates in btrfs_dirty_inode, which doesn't actually get called that often, especially

Re: [PATCH] btrfs: introduce BTRFS_IOC_GET_DEVS

2014-02-06 Thread David Sterba
On Thu, Feb 06, 2014 at 09:09:57PM +0100, Goffredo Baroncelli wrote: With 3.14 the sysfs interface is available, but the devices under /sys/fs/btrfs/fs-uuid/devices/... are symlinks to the sysfs devices, so this btrfs-specific device information has to be located in a separate directory.

Re: [PATCH v3] Btrfs: add regression test for running snapshot and send concurrently

2014-02-06 Thread Dave Chinner
On Fri, Feb 07, 2014 at 12:10:08AM +0800, Wang Shilong wrote: +$BTRFS_UTIL_PROG subvolume snapshot -r $SCRATCH_MNT \ + $SCRATCH_MNT/snap_1 $seqres.full 21 + +do_snapshots +snapshots_pid=$! + +$BTRFS_UTIL_PROG send $SCRATCH_MNT/snap_1 /dev/null 21 || echo btrfs send failed Let's

Re: [PATCH v2 3/3] btrfs/035: add new clone overwrite regression test

2014-02-06 Thread Dave Chinner
On Thu, Feb 06, 2014 at 02:59:14PM +0100, David Disseldorp wrote: This test uses the newly added cloner binary to dispatch full file and range specific clone (reflink) requests. + +echo -n $src_str $SCRATCH_MNT/src || echo failed to create src Not exactly what I intended. If echo

Re: [PATCH] btrfs-progs: Convert BUG() to BUG_ON(1)

2014-02-06 Thread Mitch Harder
On Thu, Feb 6, 2014 at 3:22 PM, David Sterba dste...@suse.cz wrote: On Thu, Feb 06, 2014 at 12:34:08PM -0600, Mitch Harder wrote: Convert the instances of BUG() to BUG_ON(1) to provide information about the location of the abort. kerncompat.h: #define BUG() abort() #define BUG_ON(c)

Re: [PATCH] Btrfs: add regression test for running snapshot and send concurrently

2014-02-06 Thread Dave Chinner
On Thu, Feb 06, 2014 at 09:12:51PM +0800, Wang Shilong wrote: Hi Dave, On Mon, Feb 03, 2014 at 11:22:36PM +0800, Wang Shilong wrote: From: Wang Shilong wangsl.f...@cn.fujitsu.com Btrfs would fail to send if snapshot run concurrently, this test is to make sure we have fixed the bug.

Re: device delete missing panic

2014-02-06 Thread Thermionix
~ # uname -r 3.13.1-12.gfc9498b-pae ~ # btrfs --version Btrfs v3.12+20131125 ~ # mount -o degraded,recovery /pool Killed Feb 07 09:53:44 store03 kernel: btrfs: device label pool devid 4 transid

INFO: possible irq lock inversion dependency detected, btrfs

2014-02-06 Thread Chris Murphy
User has reported a Fedora Rawhide bug that includes a lot of Btrfs messages. Kernel is 3.14.0-0.rc1.git1.1.fc21.x86_64. https://bugzilla.redhat.com/show_bug.cgi?id=1062439 A more full dmesg isn't included in the bug report. I haven't hit this particular bug running the same kernel, but I am

integration-20140206 build failure

2014-02-06 Thread WorMzy Tykashi
Building the latest integration snapshot, I get the following failure: ... [CC] cmds-filesystem.o cmds-filesystem.c: In function 'cmd_show': cmds-filesystem.c:740:12: error: invalid storage class for function 'cmd_sync' static int cmd_sync(int argc, char **argv) ^

Re: Are nocow files snapshot-aware

2014-02-06 Thread Kai Krakow
Duncan 1i5t5.dun...@cox.net schrieb: Ah okay, that makes it clear. So, actually, in the snapshot the file is still nocow - just for the exception that blocks being written to become unshared and relocated. This may introduce a lot of fragmentation but it won't become worse when rewriting the

Re: Are nocow files snapshot-aware

2014-02-06 Thread cwillu
On Thu, Feb 6, 2014 at 6:32 PM, Kai Krakow hurikhan77+bt...@gmail.com wrote: Duncan 1i5t5.dun...@cox.net schrieb: Ah okay, that makes it clear. So, actually, in the snapshot the file is still nocow - just for the exception that blocks being written to become unshared and relocated. This may

Re: Are nocow files snapshot-aware

2014-02-06 Thread Chris Murphy
On Feb 6, 2014, at 6:01 PM, cwillu cwi...@cwillu.com wrote: On Thu, Feb 6, 2014 at 6:32 PM, Kai Krakow hurikhan77+bt...@gmail.com wrote: Duncan 1i5t5.dun...@cox.net schrieb: Ah okay, that makes it clear. So, actually, in the snapshot the file is still nocow - just for the exception that

Re: [PATCH v3] Btrfs: add regression test for running snapshot and send concurrently

2014-02-06 Thread Wang Shilong
Hi Dave, On Fri, Feb 07, 2014 at 12:10:08AM +0800, Wang Shilong wrote: +$BTRFS_UTIL_PROG subvolume snapshot -r $SCRATCH_MNT \ +$SCRATCH_MNT/snap_1 $seqres.full 21 + +do_snapshots +snapshots_pid=$! + +$BTRFS_UTIL_PROG send $SCRATCH_MNT/snap_1 /dev/null 21 || echo btrfs send

Re: Provide a better free space estimate on RAID1

2014-02-06 Thread Roman Mamedov
On Thu, 06 Feb 2014 20:54:19 +0100 Goffredo Baroncelli kreij...@libero.it wrote: I agree with you about the needing of a solution. However your patch to me seems even worse than the actual code. For example you cannot take in account the mix of data/linear and metadata/dup (with the

Re: [PATCH v3] Btrfs: add regression test for running snapshot and send concurrently

2014-02-06 Thread Dave Chinner
On Fri, Feb 07, 2014 at 12:18:31PM +0800, Wang Shilong wrote: Hi Dave, On Fri, Feb 07, 2014 at 12:10:08AM +0800, Wang Shilong wrote: +$BTRFS_UTIL_PROG subvolume snapshot -r $SCRATCH_MNT \ + $SCRATCH_MNT/snap_1 $seqres.full 21 + +do_snapshots +snapshots_pid=$! +

user creation/deletion of snapshots permissions bug

2014-02-06 Thread Russell Coker
$ /sbin/btrfs subvol create /tmp/test Create subvolume '/tmp/test' $ /sbin/btrfs subvol delete /tmp/test Delete subvolume '/tmp/test' ERROR: cannot delete '/tmp/test' - Operation not permitted The above is when running Debian kernel 3.12 based on Linux upstream 3.12.8. I believe that the BTRFS

Re: device delete missing panic

2014-02-06 Thread Anand Jain
~ # mount -o degraded,recovery /pool Killed Feb 07 09:53:44 store03 kernel: BUG: unable to handle kernel NULL pointer dereference at 0050 Feb 07 09:53:44 store03 kernel: IP: [f834b7b7] btrfs_sysfs_add_one+0x1a7/0x280 [btrfs] Mostly the above Bug is fixed in the patch below..

Re: Provide a better free space estimate on RAID1

2014-02-06 Thread Chris Murphy
On Feb 6, 2014, at 9:40 PM, Roman Mamedov r...@romanrm.net wrote: On Thu, 06 Feb 2014 20:54:19 +0100 Goffredo Baroncelli kreij...@libero.it wrote: I agree with you about the needing of a solution. However your patch to me seems even worse than the actual code. For example you cannot

Re: user creation/deletion of snapshots permissions bug

2014-02-06 Thread Wang Shilong
Hello, $ /sbin/btrfs subvol create /tmp/test Create subvolume '/tmp/test' $ /sbin/btrfs subvol delete /tmp/test Delete subvolume '/tmp/test' ERROR: cannot delete '/tmp/test' - Operation not permitted That is a little strange anyway. Looking into codes, i found For sub-volume creation, we

Re: Provide a better free space estimate on RAID1

2014-02-06 Thread Roman Mamedov
On Thu, 6 Feb 2014 22:30:46 -0700 Chris Murphy li...@colorremedies.com wrote: From the original post, context is a 2x 1TB raid volume: Filesystem Size Used Avail Use% Mounted on /dev/sda2 1.8T 1.1M 1.8T 1% /mnt/p2 Earlier conventions would have stated Size ~900GB, and

[PATCH 2/2] btrfs-progs: Add -p/--print-missing options for btrfs fi show

2014-02-06 Thread Qu Wenruo
Since a mounted btrfs filesystem contains all the devices info even a device is removed after mount(like btrfs/003 in xfstests), we can use the info to print the known missing device if possible. So -p/--print-missing options are added to print possible missing devices. Signed-off-by: Qu Wenruo

[PATCH 1/2] btrfs-progs: Add missing devices check for mounted btrfs.

2014-02-06 Thread Qu Wenruo
In btrfs/003 of xfstest, it will check whether btrfs fi show can find missing devices. But before the patch, btrfs-progs will not check whether device missing if given a mounted btrfs mountpoint/block device. This patch fixes the bug and will pass btrfs/003. Signed-off-by: Qu Wenruo

[PATCH v2 1/2] btrfs-progs: Add missing devices check for mounted btrfs.

2014-02-06 Thread Qu Wenruo
In btrfs/003 of xfstest, it will check whether btrfs fi show can find missing devices. But before the patch, btrfs-progs will not check whether device missing if given a mounted btrfs mountpoint/block device. This patch fixes the bug and will pass btrfs/003. Signed-off-by: Qu Wenruo

Re: Are nocow files snapshot-aware

2014-02-06 Thread Duncan
Kai Krakow posted on Fri, 07 Feb 2014 01:32:27 +0100 as excerpted: Duncan 1i5t5.dun...@cox.net schrieb: That also explains the report of a NOCOW VM-image still triggering the snapshot-aware-defrag-related pathology. It was a _heavily_ auto- snapshotted btrfs (thousands of snapshots,