Re: [RFC, crash][PATCH] btrfs: allow cross-subvolume file clone
On Tue, Aug 9, 2011 at 1:50 PM, David Sterba wrote: > On Thu, Aug 04, 2011 at 09:19:26AM +0800, Miao Xie wrote: >> > the patch has been applied on top of current linus which contains patches >> > from >> > both pull requests (ed8f37370d83). >> >> I think it is because the caller didn't reserve enough space.Could you try to >> apply the following patch? It might fix this bug. >> >> [PATCH v2] Btrfs: reserve enough space for file clone >> http://marc.info/?l=linux-btrfs&m=131192686626576&w=2 > > Thanks! Yes, it does not crash anymore. Trees reflinked succesfully, > md5sums verified. This isn't a cross-subvolume problem, I hit the same bug trying to reflink a pile of files within the same subvolume. I applied the above patch and retried and it worked correctly. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC, crash][PATCH] btrfs: allow cross-subvolume file clone
On Thu, Aug 04, 2011 at 09:19:26AM +0800, Miao Xie wrote:
> > the patch has been applied on top of current linus which contains patches
> > from
> > both pull requests (ed8f37370d83).
>
> I think it is because the caller didn't reserve enough space.Could you try to
> apply the following patch? It might fix this bug.
>
> [PATCH v2] Btrfs: reserve enough space for file clone
> http://marc.info/?l=linux-btrfs&m=131192686626576&w=2
Thanks! Yes, it does not crash anymore. Trees reflinked succesfully,
md5sums verified.
david
>
> Thanks
> Miao
>
> >
> > The filesystem consists of 5 devices 23G each, about 100G of usable space,
> > mkfs.btrfs with defaults. The kernel tree has about 6G:
> >
> > $ btrfs fi df .
> > Data, RAID0: total=10.00GB, used=5.55GB
> > Data: total=8.00MB, used=0.00
> > System, RAID1: total=8.00MB, used=4.00KB
> > System: total=4.00MB, used=0.00
> > Metadata, RAID1: total=1.50GB, used=121.75MB
> > Metadata: total=8.00MB, used=0.00
> >
> > $ df -h .
> > FilesystemSize Used Avail Use% Mounted on
> > /dev/sda5 110G 5.8G 82G 7% /mnt/sda5
> >
> > ie. plenty of free space.
> >
> > It's possible that I've omitted some important bits in the patch itself, or
> > this exposes a bug of ENOSPC or delayed-inode.
> >
> > david
> > ---
> >
> > From: David Sterba
> >
> > Lift the EXDEV condition and allow different root trees for files being
> > cloned, then pass source inode's root when searching for extents.
> >
> > Signed-off-by: David Sterba
> > ---
> > fs/btrfs/ioctl.c |7 ---
> > 1 files changed, 4 insertions(+), 3 deletions(-)
> >
> > diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c
> > index 0b980af..58eb0ef 100644
> > --- a/fs/btrfs/ioctl.c
> > +++ b/fs/btrfs/ioctl.c
> > @@ -2183,7 +2183,7 @@ static noinline long btrfs_ioctl_clone(struct file
> > *file, unsigned long srcfd,
> > goto out_fput;
> >
> > ret = -EXDEV;
> > - if (src->i_sb != inode->i_sb || BTRFS_I(src)->root != root)
> > + if (src->i_sb != inode->i_sb)
> > goto out_fput;
> >
> > ret = -ENOMEM;
> > @@ -2247,13 +2247,14 @@ static noinline long btrfs_ioctl_clone(struct file
> > *file, unsigned long srcfd,
> > * note the key will change type as we walk through the
> > * tree.
> > */
> > - ret = btrfs_search_slot(NULL, root, &key, path, 0, 0);
> > + ret = btrfs_search_slot(NULL, BTRFS_I(src)->root, &key, path,
> > + 0, 0);
> > if (ret < 0)
> > goto out;
> >
> > nritems = btrfs_header_nritems(path->nodes[0]);
> > if (path->slots[0] >= nritems) {
> > - ret = btrfs_next_leaf(root, path);
> > + ret = btrfs_next_leaf(BTRFS_I(src)->root, path);
> > if (ret < 0)
> > goto out;
> > if (ret > 0)
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC, crash][PATCH] btrfs: allow cross-subvolume file clone
David Sterba wrote: > On Wed, Aug 03, 2011 at 08:07:42PM +0200, David Sterba wrote: >> I'm working on a patch to fix cross-volume cloning, worked for simple cases >> like cloning a single file. When I cloned a full linux-2.6 tree there was a >> immediate BUG_ON (after third cloned file) in btrfs_delayed_update_inode >> with -ENOSPC : > > oh, a similar issue was already reported on 5 Jul 2011: > > "[BUG] delayed inodes and reflinks" > http://permalink.gmane.org/gmane.comp.file-systems.btrfs/11763 > We've got four reports on this bug. The cause is we didn't reserve enough space when starting a transaction. We need space for: 1. btrfs_insert_empty_item() 2. btrfs_update_inode() 3. btrfs_drop_extents() The first 2 are easy, but drop_extents is not, we have to calc the space needed for drop_extents in worst case. -- Li Zefan -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC, crash][PATCH] btrfs: allow cross-subvolume file clone
On Wed, 3 Aug 2011 20:07:42 +0200, David Sterba wrote: > I'm working on a patch to fix cross-volume cloning, worked for simple cases > like cloning a single file. When I cloned a full linux-2.6 tree there was a > immediate BUG_ON (after third cloned file) in btrfs_delayed_update_inode > with -ENOSPC : > > [ 925.546266] [ cut here ] > [ 925.549921] kernel BUG at fs/btrfs/delayed-inode.c:1693! > [ 925.549921] invalid opcode: [#1] SMP > [ 925.549921] CPU 0 > [ 925.549921] Modules linked in: btrfs > [ 925.549921] > [ 925.549921] Pid: 31167, comm: clone-file Not tainted 3.0.0-default+ #98 > Intel Corporation Santa Rosa platform/Matanzas > [ 925.549921] RIP: 0010:[] [] > btrfs_delayed_update_inode+0x2e0/0x2f0 [btrfs] > [ 925.549921] RSP: 0018:88004f229be8 EFLAGS: 00010286 > [ 925.549921] RAX: ffe4 RBX: 880048392c70 RCX: > 00018000 > [ 925.549921] RDX: 1b1a RSI: 0001 RDI: > 88007a6f8420 > [ 925.549921] RBP: 88004f229c28 R08: 0004 R09: > > [ 925.549921] R10: R11: R12: > 880048393bf8 > [ 925.549921] R13: 880048392cb8 R14: 880050ff3540 R15: > 88005294 > [ 925.549921] FS: 7fbf18b23700() GS:88007dc0() > knlGS: > [ 925.549921] CS: 0010 DS: ES: CR0: 8005003b > [ 925.549921] CR2: 7fbcc68ba000 CR3: 4b4a8000 CR4: > 06f0 > [ 925.549921] DR0: DR1: DR2: > > [ 925.549921] DR3: DR6: 0ff0 DR7: > 0400 > [ 925.549921] Process clone-file (pid: 31167, threadinfo 88004f228000, > task 88004b4e5140) > [ 925.549921] Stack: > [ 925.549921] 880048f7ddc0 00018000 88004f229c38 > 880048393bf8 > [ 925.549921] 880050ff3540 880048393bf8 880051a900a0 > 88005294 > [ 925.549921] 88004f229c78 a0034633 88004f229c58 > a005f08b > [ 925.549921] Call Trace: > [ 925.549921] [] btrfs_update_inode+0x53/0x160 [btrfs] > [ 925.549921] [] ? btrfs_tree_unlock+0x6b/0xa0 [btrfs] > [ 925.549921] [] btrfs_ioctl_clone+0xa0a/0xcc0 [btrfs] > [ 925.549921] [] ? __do_fault+0x4a1/0x590 > [ 925.549921] [] ? lock_release_holdtime+0x3d/0x1c0 > [ 925.549921] [] ? do_page_fault+0x2d0/0x580 > [ 925.549921] [] btrfs_ioctl+0x2db/0xda0 [btrfs] > [ 925.549921] [] ? do_page_fault+0x2d0/0x580 > [ 925.549921] [] ? debug_check_no_locks_freed+0x177/0x180 > [ 925.549921] [] ? kmem_cache_free+0xb5/0x1b0 > [ 925.549921] [] do_vfs_ioctl+0x98/0x570 > [ 925.549921] [] ? fget_light+0x2fd/0x3c0 > [ 925.549921] [] sys_ioctl+0x4f/0x80 > [ 925.549921] [] system_call_fastpath+0x16/0x1b > [ 925.549921] Code: e8 06 00 00 8d 0c 49 48 89 ca 48 89 4d c8 e8 c8 0f fa ff > 85 c0 48 8b 4d c8 75 10 48 89 4b 08 e9 8e fd ff ff 0f 1f 80 00 00 00 00 <0f> > 0b 66 66 66 66 66 2e 0f 1f 84 00 00 00 00 00 55 48 89 e5 53 > [ 925.549921] RIP [] > btrfs_delayed_update_inode+0x2e0/0x2f0 [btrfs] > [ 925.549921] RSP > [ 925.876182] ---[ end trace 8b4c2031e1394913 ]--- > > the patch has been applied on top of current linus which contains patches from > both pull requests (ed8f37370d83). I think it is because the caller didn't reserve enough space.Could you try to apply the following patch? It might fix this bug. [PATCH v2] Btrfs: reserve enough space for file clone http://marc.info/?l=linux-btrfs&m=131192686626576&w=2 Thanks Miao > > The filesystem consists of 5 devices 23G each, about 100G of usable space, > mkfs.btrfs with defaults. The kernel tree has about 6G: > > $ btrfs fi df . > Data, RAID0: total=10.00GB, used=5.55GB > Data: total=8.00MB, used=0.00 > System, RAID1: total=8.00MB, used=4.00KB > System: total=4.00MB, used=0.00 > Metadata, RAID1: total=1.50GB, used=121.75MB > Metadata: total=8.00MB, used=0.00 > > $ df -h . > FilesystemSize Used Avail Use% Mounted on > /dev/sda5 110G 5.8G 82G 7% /mnt/sda5 > > ie. plenty of free space. > > It's possible that I've omitted some important bits in the patch itself, or > this exposes a bug of ENOSPC or delayed-inode. > > david > --- > > From: David Sterba > > Lift the EXDEV condition and allow different root trees for files being > cloned, then pass source inode's root when searching for extents. > > Signed-off-by: David Sterba > --- > fs/btrfs/ioctl.c |7 --- > 1 files changed, 4 insertions(+), 3 deletions(-) > > diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c > index 0b980af..58eb0ef 100644 > --- a/fs/btrfs/ioctl.c > +++ b/fs/btrfs/ioctl.c > @@ -2183,7 +2183,7 @@ static noinline long btrfs_ioctl_clone(struct file > *file, unsigned long srcfd, > goto out_fput; > > ret = -EXDEV; > - if (src->i_sb != inode->i_sb || BTRFS_I(src)->root != root) > + if (src->i_sb != inode->i_sb) > goto out
Re: [RFC, crash][PATCH] btrfs: allow cross-subvolume file clone
On Wed, Aug 03, 2011 at 08:07:42PM +0200, David Sterba wrote:
> I'm working on a patch to fix cross-volume cloning, worked for simple cases
> like cloning a single file. When I cloned a full linux-2.6 tree there was a
> immediate BUG_ON (after third cloned file) in btrfs_delayed_update_inode
> with -ENOSPC :
oh, a similar issue was already reported on 5 Jul 2011:
"[BUG] delayed inodes and reflinks"
http://permalink.gmane.org/gmane.comp.file-systems.btrfs/11763
Jan Schmidt wrote:
> If I get back to a situation where I can reproduce the bug, I'll send
> a follow up.
I do have a reproducer:
$ mkfs.btrfs
$ mount ...
$ btrfs subvol create subvol1
$ btrfs subvol create subvol2
$ cp linux-2.6 subvol1
$ (in subvol1) find linux-2.6 -type d -exec mkdir -p ../subvol2/'{}' \;
$ (in subvol1) find linux-2.6 -type f -exec ./clone-file '{}' ../subvol2/'{}' \;
and this backtrace follows ...
david
> [ 925.546266] [ cut here ]
> [ 925.549921] kernel BUG at fs/btrfs/delayed-inode.c:1693!
> [ 925.549921] invalid opcode: [#1] SMP
> [ 925.549921] CPU 0
> [ 925.549921] Modules linked in: btrfs
> [ 925.549921]
> [ 925.549921] Pid: 31167, comm: clone-file Not tainted 3.0.0-default+ #98
> Intel Corporation Santa Rosa platform/Matanzas
> [ 925.549921] RIP: 0010:[] []
> btrfs_delayed_update_inode+0x2e0/0x2f0 [btrfs]
> [ 925.549921] RSP: 0018:88004f229be8 EFLAGS: 00010286
> [ 925.549921] RAX: ffe4 RBX: 880048392c70 RCX:
> 00018000
> [ 925.549921] RDX: 1b1a RSI: 0001 RDI:
> 88007a6f8420
> [ 925.549921] RBP: 88004f229c28 R08: 0004 R09:
>
> [ 925.549921] R10: R11: R12:
> 880048393bf8
> [ 925.549921] R13: 880048392cb8 R14: 880050ff3540 R15:
> 88005294
> [ 925.549921] FS: 7fbf18b23700() GS:88007dc0()
> knlGS:
> [ 925.549921] CS: 0010 DS: ES: CR0: 8005003b
> [ 925.549921] CR2: 7fbcc68ba000 CR3: 4b4a8000 CR4:
> 06f0
> [ 925.549921] DR0: DR1: DR2:
>
> [ 925.549921] DR3: DR6: 0ff0 DR7:
> 0400
> [ 925.549921] Process clone-file (pid: 31167, threadinfo 88004f228000,
> task 88004b4e5140)
> [ 925.549921] Stack:
> [ 925.549921] 880048f7ddc0 00018000 88004f229c38
> 880048393bf8
> [ 925.549921] 880050ff3540 880048393bf8 880051a900a0
> 88005294
> [ 925.549921] 88004f229c78 a0034633 88004f229c58
> a005f08b
> [ 925.549921] Call Trace:
> [ 925.549921] [] btrfs_update_inode+0x53/0x160 [btrfs]
> [ 925.549921] [] ? btrfs_tree_unlock+0x6b/0xa0 [btrfs]
> [ 925.549921] [] btrfs_ioctl_clone+0xa0a/0xcc0 [btrfs]
> [ 925.549921] [] ? __do_fault+0x4a1/0x590
> [ 925.549921] [] ? lock_release_holdtime+0x3d/0x1c0
> [ 925.549921] [] ? do_page_fault+0x2d0/0x580
> [ 925.549921] [] btrfs_ioctl+0x2db/0xda0 [btrfs]
> [ 925.549921] [] ? do_page_fault+0x2d0/0x580
> [ 925.549921] [] ? debug_check_no_locks_freed+0x177/0x180
> [ 925.549921] [] ? kmem_cache_free+0xb5/0x1b0
> [ 925.549921] [] do_vfs_ioctl+0x98/0x570
> [ 925.549921] [] ? fget_light+0x2fd/0x3c0
> [ 925.549921] [] sys_ioctl+0x4f/0x80
> [ 925.549921] [] system_call_fastpath+0x16/0x1b
> [ 925.549921] Code: e8 06 00 00 8d 0c 49 48 89 ca 48 89 4d c8 e8 c8 0f fa ff
> 85 c0 48 8b 4d c8 75 10 48 89 4b 08 e9 8e fd ff ff 0f 1f 80 00 00 00 00 <0f>
> 0b 66 66 66 66 66 2e 0f 1f 84 00 00 00 00 00 55 48 89 e5 53
> [ 925.549921] RIP []
> btrfs_delayed_update_inode+0x2e0/0x2f0 [btrfs]
> [ 925.549921] RSP
> [ 925.876182] ---[ end trace 8b4c2031e1394913 ]---
>
> the patch has been applied on top of current linus which contains patches from
> both pull requests (ed8f37370d83).
>
> The filesystem consists of 5 devices 23G each, about 100G of usable space,
> mkfs.btrfs with defaults. The kernel tree has about 6G:
>
> $ btrfs fi df .
> Data, RAID0: total=10.00GB, used=5.55GB
> Data: total=8.00MB, used=0.00
> System, RAID1: total=8.00MB, used=4.00KB
> System: total=4.00MB, used=0.00
> Metadata, RAID1: total=1.50GB, used=121.75MB
> Metadata: total=8.00MB, used=0.00
>
> $ df -h .
> FilesystemSize Used Avail Use% Mounted on
> /dev/sda5 110G 5.8G 82G 7% /mnt/sda5
>
> ie. plenty of free space.
>
> It's possible that I've omitted some important bits in the patch itself, or
> this exposes a bug of ENOSPC or delayed-inode.
>
> david
> ---
>
> From: David Sterba
>
> Lift the EXDEV condition and allow different root trees for files being
> cloned, then pass source inode's root when searching for extents.
>
> Signed-off-by: David Sterba
> ---
> fs/btrfs/ioctl.c |7 ---
> 1 files changed, 4 insertions(+), 3 deletions(-)
>
> diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c
> index 0b980af..58eb0ef 100644
