Re: Btrfs bug with g38867a2 and a question
On 09/23/2011 08:55 AM, Mathieu Chouquet-Stringer wrote: On Thu, Sep 22, 2011 at 10:32:13PM +0200, Mathieu Chouquet-Stringer wrote: On Thu, Sep 22, 2011 at 09:30:07PM +0200, Mathieu Chouquet-Stringer wrote: On Thu, Sep 22, 2011 at 03:00:03PM -0400, Josef Bacik wrote: Oh wow sorry I sent you the completely wrong patch, I wish I had caught your reply earlier. Can you run with this patch, which is the one I meant to give you :). Thanks, No worries, I've applied your patch (seems your thunderbird mangled line returns) and I'm rebooting... There: http://mathieu.csetco.com/btrfs/btrfs-screenshots.tar Just realised I truncated the first line! It should have been: Couldn't find in bitmap offset=20737413120, bytes=57344, search=20765691904, search_bytes=65536, ret=0 Ok I have no idea how this could happen. Can you mount -o clear_cache and see if it's just the cache that's bad? Thanks, Josef -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Btrfs bug with g38867a2 and a question
On Fri, Sep 23, 2011 at 10:49:22AM -0400, Josef Bacik wrote: Ok I have no idea how this could happen. Can you mount -o clear_cache and see if it's just the cache that's bad? Thanks, Did that and got this (it's a never ending story, this is from a F16 alpha boot cd hence stack trace could be different): [ 512.455253] [ cut here ] [ 512.455464] kernel BUG at fs/btrfs/inode.c:4586! [ 512.455662] invalid opcode: [#1] SMP [ 512.455874] CPU 1 [ 512.455879] Modules linked in: btrfs zlib_deflate libcrc32c xts lrw gf128mul sha256_generic dm_crypt dm_round_robin dm_multipath linear raid10 raid456 async_raid6_recov async_pq raid6_pq async_xor xor async_memcpy async_tx raid1 raid0 iscsi_ibft iscsi_boot_sysfs pcspkr edd iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi cramfs arc4 firewire_ohci firewire_core sdhci_pci sdhci yenta_socket crc_itu_t mmc_core iwl4965 iwl_legacy mac80211 cfg80211 rfkill nouveau ttm drm_kms_helper drm i2c_algo_bit i2c_core mxm_wmi wmi video e1000e squashfs [ 512.456018] [ 512.456018] Pid: 1349, comm: mount Not tainted 3.0.0-1.fc16.x86_64 #1 LENOVO 6458V5C/6458V5C [ 512.456018] RIP: 0010:[a036132f] [a036132f] btrfs_add_link+0x123/0x17c [btrfs] [ 512.456018] RSP: 0018:880123e09848 EFLAGS: 00010282 [ 512.456018] RAX: ffef RBX: 8801194c9d78 RCX: 0046 [ 512.456018] RDX: 0127 RSI: 88012aa7aaf0 RDI: 0282 [ 512.456018] RBP: 880123e098b8 R08: 8801228ac000 R09: 005a [ 512.456018] R10: 880123e096d8 R11: 88013b4023c0 R12: 8801194ca610 [ 512.456018] R13: 8801165d2090 R14: 000b R15: 8801181fd630 [ 512.456018] FS: 7f7bb600a820() GS:88013bc0() knlGS: [ 512.456018] CS: 0010 DS: ES: CR0: 8005003b [ 512.456018] CR2: 7f7235313768 CR3: 000113f09000 CR4: 06e0 [ 512.456018] DR0: DR1: DR2: [ 512.456018] DR3: DR6: 0ff0 DR7: 0400 [ 512.456018] Process mount (pid: 1349, threadinfo 880123e08000, task 88012aa7a3e0) [ 512.456018] Stack: [ 512.456018] 88010001 00018028 880123e09888 8801194bb000 [ 512.456018] 5aff8801165b3000 01001537 [ 512.456018] 1000 8801194ba1b0 8801194ca610 880123e099d7 [ 512.456018] Call Trace: [ 512.456018] [a0382862] add_inode_ref+0x2e6/0x37c [btrfs] [ 512.456018] [a037547a] ? read_extent_buffer+0xc3/0xe3 [btrfs] [ 512.456018] [a0383208] replay_one_buffer+0x197/0x212 [btrfs] [ 512.456018] [a0381068] walk_up_log_tree+0xe4/0x1aa [btrfs] [ 512.456018] [a0383071] ? replay_one_dir_item+0xbd/0xbd [btrfs] [ 512.456018] [a038148d] walk_log_tree+0x9e/0x19e [btrfs] [ 512.456018] [a0384562] btrfs_recover_log_trees+0x28b/0x298 [btrfs] [ 512.456018] [a0383071] ? replay_one_dir_item+0xbd/0xbd [btrfs] [ 512.456018] [a0355c42] open_ctree+0x11aa/0x14b8 [btrfs] [ 512.456018] [a033b908] btrfs_mount+0x233/0x498 [btrfs] [ 512.456018] [810f3ef3] ? free_pages+0x47/0x4c [ 512.456018] [8113ddac] mount_fs+0x69/0x155 [ 512.456018] [81107e68] ? __alloc_percpu+0x10/0x12 [ 512.456018] [81152ee6] vfs_kern_mount+0x63/0xa0 [ 512.456018] [81153bba] do_kern_mount+0x4d/0xdf [ 512.456018] [81155250] do_mount+0x63c/0x69f [ 512.456018] [81155534] sys_mount+0x88/0xc2 [ 512.456018] [814e4fc2] system_call_fastpath+0x16/0x1b [ 512.456018] Code: 89 f1 4c 89 fa 4c 89 ee 48 89 44 24 08 41 8b 04 24 66 c1 e8 0c 83 e0 0f 0f b6 80 b8 bd 39 a0 89 04 24 e8 db cf fe ff 85 c0 74 02 0f 0b 45 01 f6 4d 63 f6 4c 03 b3 a0 01 00 00 4c 89 b3 a0 01 00 [ 512.456018] RIP [a036132f] btrfs_add_link+0x123/0x17c [btrfs] [ 512.456018] RSP 880123e09848 [ 512.485056] ---[ end trace cea880cef8a5d83b ]--- -- Mathieu Chouquet-Stringer math...@csetco.com The sun itself sees not till heaven clears. -- William Shakespeare -- -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Btrfs bug with g38867a2 and a question
On 09/23/2011 11:31 AM, Mathieu Chouquet-Stringer wrote: On Fri, Sep 23, 2011 at 10:49:22AM -0400, Josef Bacik wrote: Ok I have no idea how this could happen. Can you mount -o clear_cache and see if it's just the cache that's bad? Thanks, Did that and got this (it's a never ending story, this is from a F16 alpha boot cd hence stack trace could be different): Yeah this is a different problem that's fixed upstream, so reboot into your other newer kernel with -o clear_cache. Thanks, Josef -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Btrfs bug with g38867a2 and a question
On Fri, Sep 23, 2011 at 11:34:40AM -0400, Josef Bacik wrote: Yeah this is a different problem that's fixed upstream, so reboot into your other newer kernel with -o clear_cache. Thanks, Ok I'm back in business now, thanks... Now I'll try to understand why my pc sometimes hangs doing write IOs... -- Mathieu Chouquet-Stringer math...@csetco.com The sun itself sees not till heaven clears. -- William Shakespeare -- -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Btrfs bug with g38867a2 and a question
On Thu, Sep 22, 2011 at 01:10:55AM +0200, David Sterba wrote: please prefix printk messages with btrfs: Well my computer crashed before I could reboot with the newly compiled kernel. And now it bugs while it tries to mount the kernel meaning my computer is fscked! :-) Bug at fs/btrfs/free-space-cache.c:1327 (I have a screenshot if needed) function remove_from_bitmap /* * XXX - this can go away after a few releases. * * since the only user of btrfs_remove_free_space is the tree logging * stuff, and the only way to test that is under crash conditions, we * want to have this debug stuff here just in case somethings not * working. Search the bitmap for the space we are trying to use to * make sure its actually there. If its not there then we need to stop * because something has gone wrong. */ search_start = *offset; search_bytes = *bytes; search_bytes = min(search_bytes, end - search_start + 1); ret = search_bitmap(ctl, bitmap_info, search_start, search_bytes); ===BUG_ON(ret 0 || search_start != *offset); So question is now: how do I recover from this? :-) -- Mathieu Chouquet-Stringer math...@csetco.com The sun itself sees not till heaven clears. -- William Shakespeare -- -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Btrfs bug with g38867a2 and a question
On Thu, Sep 22, 2011 at 01:05:28PM +0200, David Sterba wrote: On Thu, Sep 22, 2011 at 12:13:44PM +0200, Mathieu Chouquet-Stringer wrote: On Thu, Sep 22, 2011 at 01:10:55AM +0200, David Sterba wrote: please prefix printk messages with btrfs: Well my computer crashed before I could reboot with the newly compiled kernel. And now it bugs while it tries to mount the kernel meaning my computer is fscked! :-) Bug at fs/btrfs/free-space-cache.c:1327 (I have a screenshot if needed) Yes please. There you go: http://mathieu.csetco.com/btrfs-crash3.jpg -- Mathieu Chouquet-Stringer math...@csetco.com The sun itself sees not till heaven clears. -- William Shakespeare -- -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Btrfs bug with g38867a2 and a question
On 09/22/2011 02:05 AM, Mathieu Chouquet-Stringer wrote: On Sep 22, 2011 1:11 AM, David Sterba d...@jikos.cz mailto:d...@jikos.cz wrote: On Wed, Sep 21, 2011 at 04:18:29PM -0400, Josef Bacik wrote: diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index b128fa0..66ba0a7 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -782,7 +782,12 @@ static noinline int cow_file_range(struct inode *inode, struct extent_map_tree *em_tree = BTRFS_I(inode)-extent_tree; int ret = 0; - BUG_ON(btrfs_is_free_space_inode(root, inode)); + if (btrfs_is_free_space_inode(root, inode)) { + printk(KERN_ERR trying to write to free space inode, +isize=%Lu, start=%Lu, end=%Lu\n, i_size_read(inode), +start, end); please prefix printk messages with btrfs: Well this time it doesn't boot anymore... Bug at fs/btrfs/free-space-cache.c:1327 I had another crash before being able to reboot... Is there a way to recover from that? How do I disable space cache? Well that is from the tree logging code, so give this a whirl. It's going to dump a lot of info so make sure you capture everything before the --- cut here --- line. Thanks, Josef diff --git a/fs/btrfs/free-space-cache.c b/fs/btrfs/free-space-cache.c index 1bcbe76..236a938 100644 --- a/fs/btrfs/free-space-cache.c +++ b/fs/btrfs/free-space-cache.c @@ -417,6 +417,8 @@ int load_free_space_cache(struct btrfs_fs_info *fs_info, } if (entry-type == BTRFS_FREE_SPACE_EXTENT) { + printk(KERN_ERR adding extent [%llu-%llu]\n, + e-offset, e-bytes); spin_lock(block_group-tree_lock); ret = link_free_space(block_group, e); spin_unlock(block_group-tree_lock); @@ -431,6 +433,8 @@ int load_free_space_cache(struct btrfs_fs_info *fs_info, page_cache_release(page); goto free_cache; } + printk(KERN_ERR adding bitmap [%llu-%llu]\n, + e-offset, e-bytes); spin_lock(block_group-tree_lock); ret = link_free_space(block_group, e); block_group-total_bitmaps++; -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Btrfs bug with g38867a2 and a question
On Thu, Sep 22, 2011 at 10:12:12AM -0400, Josef Bacik wrote: Well that is from the tree logging code, so give this a whirl. It's going to dump a lot of info so make sure you capture everything before the --- cut here --- line. Thanks, The patch doesn't apply cleanly to the latest git (since my laptop doesn't boot anymore, I'm using a different host to compile this) but it was simple enough to patch manually. Right now I'm on v3.1-rc7-d93dc5c. I'm compiling at the moment, so bear with me! :-) My patch looks like this: --- fs/btrfs/free-space-cache.c.orig2011-09-22 11:51:46.442129324 +0200 +++ fs/btrfs/free-space-cache.c 2011-09-22 17:11:46.715162968 +0200 @@ -381,6 +381,8 @@ int __load_free_space_cache(struct btrfs } if (entry-type == BTRFS_FREE_SPACE_EXTENT) { + printk(KERN_ERR adding extent [%llu-%llu]\n, + e-offset, e-bytes); spin_lock(ctl-tree_lock); ret = link_free_space(ctl, e); spin_unlock(ctl-tree_lock); @@ -402,6 +404,8 @@ int __load_free_space_cache(struct btrfs page_cache_release(page); goto free_cache; } + printk(KERN_ERR adding bitmap [%llu-%llu]\n, + e-offset, e-bytes); spin_lock(ctl-tree_lock); ret = link_free_space(ctl, e); ctl-total_bitmaps++; -- Mathieu Chouquet-Stringer math...@csetco.com The sun itself sees not till heaven clears. -- William Shakespeare -- -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Btrfs bug with g38867a2 and a question
On Thu, Sep 22, 2011 at 10:12:12AM -0400, Josef Bacik wrote: Well that is from the tree logging code, so give this a whirl. It's going to dump a lot of info so make sure you capture everything before the --- cut here --- line. Thanks, Here's the output of this patch: http://mathieu.csetco.com/btrfs/IMG_20110922_200713.jpg http://mathieu.csetco.com/btrfs/IMG_20110922_200817.jpg http://mathieu.csetco.com/btrfs/IMG_20110922_200829.jpg http://mathieu.csetco.com/btrfs/IMG_20110922_200841.jpg http://mathieu.csetco.com/btrfs/IMG_20110922_200849.jpg http://mathieu.csetco.com/btrfs/IMG_20110922_200900.jpg http://mathieu.csetco.com/btrfs/IMG_20110922_200909.jpg http://mathieu.csetco.com/btrfs/IMG_20110922_200922.jpg -- Mathieu Chouquet-Stringer math...@csetco.com The sun itself sees not till heaven clears. -- William Shakespeare -- -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Btrfs bug with g38867a2 and a question
On Thu, Sep 22, 2011 at 03:00:03PM -0400, Josef Bacik wrote: Oh wow sorry I sent you the completely wrong patch, I wish I had caught your reply earlier. Can you run with this patch, which is the one I meant to give you :). Thanks, No worries, I've applied your patch (seems your thunderbird mangled line returns) and I'm rebooting... -- Mathieu Chouquet-Stringer math...@csetco.com The sun itself sees not till heaven clears. -- William Shakespeare -- -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Btrfs bug with g38867a2 and a question
On Thu, Sep 22, 2011 at 09:30:07PM +0200, Mathieu Chouquet-Stringer wrote: On Thu, Sep 22, 2011 at 03:00:03PM -0400, Josef Bacik wrote: Oh wow sorry I sent you the completely wrong patch, I wish I had caught your reply earlier. Can you run with this patch, which is the one I meant to give you :). Thanks, No worries, I've applied your patch (seems your thunderbird mangled line returns) and I'm rebooting... There: http://mathieu.csetco.com/btrfs/btrfs-screenshots.tar -- Mathieu Chouquet-Stringer math...@csetco.com The sun itself sees not till heaven clears. -- William Shakespeare -- -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Btrfs bug with g38867a2 and a question
Hello all, I've been using BTRFS for quite some time on this laptop and I just recompiled the latest kernel from git (3.1.0-rc6-00247-g38867a2). After a couple minutes, I hit this bug twice (this a hand written transcript, pics here [1]) kernel BUG at fs/btrfs/inode.c:785! stack being: _raw_spin_unlock_irqrestore __wake_up btrfs_tree_read_unlock_blocking free_extent_buffer run_delalloc_nocow run_dellaloc_range free_extent_state find_lock_delalloc_range.constrpop __extent_writepage ... Note the stack trace is exactly the same in these two crashes. Since google found nothing, I'm asking for your help! Looking at the source, line 785 is function cow_file_range, code: BUG_ON(btrfs_is_free_space_inode(root, inode)); My fs (a simple partition on a OCZ-VERTEX2 ssd) is mounted with the following options: ssd,discard,autodefrag with disk space caching as it had been added earlier: btrfs: disk space caching is enabled Btrfs detected SSD devices, enabling SSD mode Anything I could do to help debug this one? Moreover, I've been experiencing long lags when my computer seems to be just busy doing writes (right now I'm on 3.1.0-rc6-00067-gf1fcd9f): it's not hung or anything except all IOs are blocked (hence I can start anything to see what's going on but I have a gauge on my xfce panel that goes 100% with writes). After some point, it just recovers and everything is back to normal. I've tried capturing something with sysrq-t but I haven't been able to find anything striking. Should I just submit something similar to this mailing list? [1] http://mathieu.csetco.com/btrfs-crash1.jpg http://mathieu.csetco.com/btrfs-crash2.jpg -- Mathieu Chouquet-Stringer math...@csetco.com The sun itself sees not till heaven clears. -- William Shakespeare -- -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Btrfs bug with g38867a2 and a question
On 09/21/2011 02:53 PM, Mathieu Chouquet-Stringer wrote: Hello all, I've been using BTRFS for quite some time on this laptop and I just recompiled the latest kernel from git (3.1.0-rc6-00247-g38867a2). After a couple minutes, I hit this bug twice (this a hand written transcript, pics here [1]) kernel BUG at fs/btrfs/inode.c:785! stack being: _raw_spin_unlock_irqrestore __wake_up btrfs_tree_read_unlock_blocking free_extent_buffer run_delalloc_nocow run_dellaloc_range free_extent_state find_lock_delalloc_range.constrpop __extent_writepage ... Note the stack trace is exactly the same in these two crashes. Since google found nothing, I'm asking for your help! Looking at the source, line 785 is function cow_file_range, code: BUG_ON(btrfs_is_free_space_inode(root, inode)); My fs (a simple partition on a OCZ-VERTEX2 ssd) is mounted with the following options: ssd,discard,autodefrag with disk space caching as it had been added earlier: btrfs: disk space caching is enabled Btrfs detected SSD devices, enabling SSD mode Anything I could do to help debug this one? Yup, can you apply this patch and reproduce? It will print out some debug info before the --- cut here --- line, which is what I need. Thanks, Josef diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index b128fa0..66ba0a7 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -782,7 +782,12 @@ static noinline int cow_file_range(struct inode *inode, struct extent_map_tree *em_tree = BTRFS_I(inode)-extent_tree; int ret = 0; - BUG_ON(btrfs_is_free_space_inode(root, inode)); + if (btrfs_is_free_space_inode(root, inode)) { + printk(KERN_ERR trying to write to free space inode, + isize=%Lu, start=%Lu, end=%Lu\n, i_size_read(inode), + start, end); + BUG_ON(btrfs_is_free_space_inode(root, inode)); + } trans = btrfs_join_transaction(root); BUG_ON(IS_ERR(trans)); trans-block_rsv = root-fs_info-delalloc_block_rsv; -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Btrfs bug with g38867a2 and a question
On Wed, Sep 21, 2011 at 04:18:29PM -0400, Josef Bacik wrote: Yup, can you apply this patch and reproduce? It will print out some debug info before the --- cut here --- line, which is what I need. Thanks, Compiling right now. That said, I had the same exact trace under rc6-00067-gf1fcd9f (right after rebooting to this older kernel) so it doesn't seem to be related to the latest git... Not sure how to trigger the bug though, I'll keep you posted! -- Mathieu Chouquet-Stringer math...@csetco.com The sun itself sees not till heaven clears. -- William Shakespeare -- -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Btrfs bug with g38867a2 and a question
On Wed, Sep 21, 2011 at 04:18:29PM -0400, Josef Bacik wrote: diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index b128fa0..66ba0a7 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -782,7 +782,12 @@ static noinline int cow_file_range(struct inode *inode, struct extent_map_tree *em_tree = BTRFS_I(inode)-extent_tree; int ret = 0; - BUG_ON(btrfs_is_free_space_inode(root, inode)); + if (btrfs_is_free_space_inode(root, inode)) { + printk(KERN_ERR trying to write to free space inode, +isize=%Lu, start=%Lu, end=%Lu\n, i_size_read(inode), +start, end); please prefix printk messages with btrfs: + BUG_ON(btrfs_is_free_space_inode(root, inode)); + } trans = btrfs_join_transaction(root); BUG_ON(IS_ERR(trans)); trans-block_rsv = root-fs_info-delalloc_block_rsv; -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html