Re: hitting BUG_ON on troublesome FS
On 02/05/2014 01:34 AM, Remco Hosman - Yerf-it.com wrote: How can i tell? Label: data uuid: a8626d67-4684-4b23-99b3-8d5fa8e7fd69 Total devices 5 FS bytes used 820.00KiB devid2 size 1.82TiB used 1.00GiB path /dev/sdb2 devid3 size 1.82TiB used 1.00GiB path /dev/sdf2 devid5 size 2.73TiB used 3.00GiB path /dev/sdd2 devid10 size 2.73TiB used 2.03GiB path /dev/sde2 devid11 size 3.64TiB used 1.03GiB path /dev/sdc1 Data, RAID10: total=2.00GiB, used=768.00KiB Data, RAID1: total=1.00GiB, used=12.00KiB System, RAID1: total=32.00MiB, used=4.00KiB Metadata, RAID1: total=1.00GiB, used=36.00KiB i made a image with `btrfs-image`, when i do -c 9, the file size is 7k, so eazy enough to mail if it would be of any use. Yup that would be perfect. THanks, Josef -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: hitting BUG_ON on troublesome FS
How can i tell? Label: data uuid: a8626d67-4684-4b23-99b3-8d5fa8e7fd69 Total devices 5 FS bytes used 820.00KiB devid2 size 1.82TiB used 1.00GiB path /dev/sdb2 devid3 size 1.82TiB used 1.00GiB path /dev/sdf2 devid5 size 2.73TiB used 3.00GiB path /dev/sdd2 devid10 size 2.73TiB used 2.03GiB path /dev/sde2 devid11 size 3.64TiB used 1.03GiB path /dev/sdc1 Data, RAID10: total=3D2.00GiB, used=3D768.00KiB Data, RAID1: total=3D1.00GiB, used=3D12.00KiB System, RAID1: total=3D32.00MiB, used=3D4.00KiB Metadata, RAID1: total=3D1.00GiB, used=3D36.00KiB i made a image with `btrfs-image`, when i do -c 9, the file size is 7k, = so eazy enough to mail if it would be of any use. Remco On 04 Feb 2014, at 22:48, Josef Bacik wrote: > > On 02/03/2014 03:51 PM, Remco Hosman - Yerf-it.com wrote: >> FIrst, a bit of history of the filesystem: >> used to be 6 disks, now 5. partially raid1 / raid10. been migrating back and >> forth a few times. >> As some point, a balance would not complete and would end with 164 >> ENOSPC’ses, while there was plenty of unallocated space on each disk. >> >> i scanned for extends larger then 1gig and found a few, so ran a recursive >> balance of the entire FS. >> >> I deceided to empty the filesystem and format it. >> >> i pulled most files off it some via btrfs send/receive, some via rsync. but >> 1 subvol wouldn’t send. i don’t remember the exact error, but it was that a >> extend could not be found on 1 of the disks. >> >> with only a few 100gig of data left, i decided to balance some remaining >> empty space before doing a `btrfs dev del`, so have another disk to store >> more data on. >> but im hitting a snag, i hit a BUG_ON when doing a `btrfs bal start >> -dusage=2 /mountpoint` : >> >> [ 3327.678329] btrfs: found 198 extents >> [ 3328.117274] btrfs: relocating block group 84473084968960 flags 17 >> [ 3329.278521] btrfs: found 103 extents >> [ 3331.907931] btrfs: found 103 extents >> [ 3332.386172] btrfs: relocating block group 84466642518016 flags 17 >> [ .536595] btrfs: found 86 extents >> [ 3335.982967] btrfs: found 86 extents >> [ 3336.599555] btrfs (4746) used greatest stack depth: 2744 bytes left >> [ 3379.073464] btrfs: relocating block group 89878368419840 flags 17 >> [ 3381.608948] btrfs: found 499 extents >> [ 3383.884696] [ cut here ] >> [ 3383.884720] kernel BUG at fs/btrfs/relocation.c:3405! >> [ 3383.884731] invalid opcode: [#1] SMP >> [ 3383.884742] Modules linked in: >> [ 3383.884753] CPU: 0 PID: 5663 Comm: btrfs Not tainted 3.13.0 #1 >> [ 3383.884763] Hardware name: System manufacturer System Product >> Name/E45M1-I DELUXE, BIOS 0405 08/08/2012 >> [ 3383.884778] task: 8802360eae80 ti: 88010dcaa000 task.ti: >> 88010dcaa000 >> [ 3383.884790] RIP: 0010:[] [] >> __add_tree_block+0x1c5/0x1e0 >> [ 3383.884811] RSP: 0018:88010dcaba38 EFLAGS: 00010202 >> [ 3383.884821] RAX: 0001 RBX: 880039f18000 RCX: >> >> [ 3383.884832] RDX: RSI: RDI: >> >> [ 3383.884843] RBP: 88010dcaba90 R08: 88010dcab9f4 R09: >> 88010dcab930 >> [ 3383.884854] R10: R11: 047f R12: >> 1000 >> [ 3383.884865] R13: 88023489c630 R14: R15: >> 528d112e4000 >> [ 3383.884876] FS: 7f8e27e74880() GS:88023ec0() >> knlGS: >> [ 3383.884888] CS: 0010 DS: ES: CR0: 8005003b >> [ 3383.884897] CR2: 7f60d89f35a8 CR3: 0001b5ada000 CR4: >> 07f0 >> [ 3383.884907] Stack: >> [ 3383.884941] 88010dcabb28 4000812bde34 00a8528d112e >> 0010 >> [ 3383.885012] 1000 1000 0f3a >> 8802348d6990 >> [ 3383.885082] 88001cbf5a00 880039f18000 00b8 >> 88010dcabb00 >> [ 3383.885153] Call Trace: >> [ 3383.885192] [] add_data_references+0x244/0x2e0 >> [ 3383.885232] [] relocate_block_group+0x56b/0x640 >> [ 3383.885272] [] btrfs_relocate_block_group+0x1a2/0x2f0 >> [ 3383.885313] [] btrfs_relocate_chunk.isra.27+0x6a/0x740 >> [ 3383.885355] [] ? btrfs_set_path_blocking+0x31/0x70 >> [ 3383.885432] [] ? btrfs_search_slot+0x386/0x960 >> [ 3383.885473] [] ? free_extent_buffer+0x47/0xa0 >> [ 3383.885513] [] btrfs_balance+0x90b/0xea0 >> [ 3383.885553] [] btrfs_ioctl_balance+0x162/0x520 >> [ 3383.885592] [] btrfs_ioctl+0xcbd/0x25c0 >> [ 3383.885632] [] ? __do_page_fault+0x1dc/0x520 >> [ 3383.885673] [] do_vfs_ioctl+0x2c8/0x490 >> [ 3383.885712] [] SyS_ioctl+0x81/0xa0 >> [ 3383.885752] [] tracesys+0xdd/0xe2 >> [ 3383.885787] Code: ff 48 8b 4d a8 48 8d 75 b6 4c 89 ea 48 89 df e8 42 e7 >> ff ff 4c 89 ef 89 45 a8 e8 c7 0f f9 ff 8b 45 a8 e9 69 ff ff ff 85 c0 74 d6 >> <0f> 0b 66 0f 1f 84 00 00 00 00 00 b8 f4 ff ff ff e9 50 ff ff ff >> [ 3383.886001] RIP [] __add_tree_block+0x1c5/0x1e0 >>
Re: hitting BUG_ON on troublesome FS
On 02/03/2014 03:51 PM, Remco Hosman - Yerf-it.com wrote: FIrst, a bit of history of the filesystem: used to be 6 disks, now 5. partially raid1 / raid10. been migrating back and forth a few times. As some point, a balance would not complete and would end with 164 ENOSPC’ses, while there was plenty of unallocated space on each disk. i scanned for extends larger then 1gig and found a few, so ran a recursive balance of the entire FS. I deceided to empty the filesystem and format it. i pulled most files off it some via btrfs send/receive, some via rsync. but 1 subvol wouldn’t send. i don’t remember the exact error, but it was that a extend could not be found on 1 of the disks. with only a few 100gig of data left, i decided to balance some remaining empty space before doing a `btrfs dev del`, so have another disk to store more data on. but im hitting a snag, i hit a BUG_ON when doing a `btrfs bal start -dusage=2 /mountpoint` : [ 3327.678329] btrfs: found 198 extents [ 3328.117274] btrfs: relocating block group 84473084968960 flags 17 [ 3329.278521] btrfs: found 103 extents [ 3331.907931] btrfs: found 103 extents [ 3332.386172] btrfs: relocating block group 84466642518016 flags 17 [ .536595] btrfs: found 86 extents [ 3335.982967] btrfs: found 86 extents [ 3336.599555] btrfs (4746) used greatest stack depth: 2744 bytes left [ 3379.073464] btrfs: relocating block group 89878368419840 flags 17 [ 3381.608948] btrfs: found 499 extents [ 3383.884696] [ cut here ] [ 3383.884720] kernel BUG at fs/btrfs/relocation.c:3405! [ 3383.884731] invalid opcode: [#1] SMP [ 3383.884742] Modules linked in: [ 3383.884753] CPU: 0 PID: 5663 Comm: btrfs Not tainted 3.13.0 #1 [ 3383.884763] Hardware name: System manufacturer System Product Name/E45M1-I DELUXE, BIOS 0405 08/08/2012 [ 3383.884778] task: 8802360eae80 ti: 88010dcaa000 task.ti: 88010dcaa000 [ 3383.884790] RIP: 0010:[] [] __add_tree_block+0x1c5/0x1e0 [ 3383.884811] RSP: 0018:88010dcaba38 EFLAGS: 00010202 [ 3383.884821] RAX: 0001 RBX: 880039f18000 RCX: [ 3383.884832] RDX: RSI: RDI: [ 3383.884843] RBP: 88010dcaba90 R08: 88010dcab9f4 R09: 88010dcab930 [ 3383.884854] R10: R11: 047f R12: 1000 [ 3383.884865] R13: 88023489c630 R14: R15: 528d112e4000 [ 3383.884876] FS: 7f8e27e74880() GS:88023ec0() knlGS: [ 3383.884888] CS: 0010 DS: ES: CR0: 8005003b [ 3383.884897] CR2: 7f60d89f35a8 CR3: 0001b5ada000 CR4: 07f0 [ 3383.884907] Stack: [ 3383.884941] 88010dcabb28 4000812bde34 00a8528d112e 0010 [ 3383.885012] 1000 1000 0f3a 8802348d6990 [ 3383.885082] 88001cbf5a00 880039f18000 00b8 88010dcabb00 [ 3383.885153] Call Trace: [ 3383.885192] [] add_data_references+0x244/0x2e0 [ 3383.885232] [] relocate_block_group+0x56b/0x640 [ 3383.885272] [] btrfs_relocate_block_group+0x1a2/0x2f0 [ 3383.885313] [] btrfs_relocate_chunk.isra.27+0x6a/0x740 [ 3383.885355] [] ? btrfs_set_path_blocking+0x31/0x70 [ 3383.885432] [] ? btrfs_search_slot+0x386/0x960 [ 3383.885473] [] ? free_extent_buffer+0x47/0xa0 [ 3383.885513] [] btrfs_balance+0x90b/0xea0 [ 3383.885553] [] btrfs_ioctl_balance+0x162/0x520 [ 3383.885592] [] btrfs_ioctl+0xcbd/0x25c0 [ 3383.885632] [] ? __do_page_fault+0x1dc/0x520 [ 3383.885673] [] do_vfs_ioctl+0x2c8/0x490 [ 3383.885712] [] SyS_ioctl+0x81/0xa0 [ 3383.885752] [] tracesys+0xdd/0xe2 [ 3383.885787] Code: ff 48 8b 4d a8 48 8d 75 b6 4c 89 ea 48 89 df e8 42 e7 ff ff 4c 89 ef 89 45 a8 e8 c7 0f f9 ff 8b 45 a8 e9 69 ff ff ff 85 c0 74 d6 <0f> 0b 66 0f 1f 84 00 00 00 00 00 b8 f4 ff ff ff e9 50 ff ff ff [ 3383.886001] RIP [] __add_tree_block+0x1c5/0x1e0 [ 3383.886042] RSP [ 3383.886359] ---[ end trace 075209044ce10da3 ]--- Anything i can do to resolve / debug the issue? Are you using skinny extents at all? Thanks, Josef -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: hitting BUG_ON on troublesome FS
to reply to my own thread: i managed to empty the filesystem, but it still segfaults in the same way when i try to balance the last few blocks. a `btrfs bal start -dusage=0 /mountpoint` finishes, but leaves a few blocks allocated. when i skip the usage=0, it segfaults. What can i do to provide useful information? Remco On 03 Feb 2014, at 21:51, Remco Hosman - Yerf-it.com wrote: > FIrst, a bit of history of the filesystem: > used to be 6 disks, now 5. partially raid1 / raid10. been migrating back and > forth a few times. > As some point, a balance would not complete and would end with 164 > ENOSPC’ses, while there was plenty of unallocated space on each disk. > > i scanned for extends larger then 1gig and found a few, so ran a recursive > balance of the entire FS. > > I deceided to empty the filesystem and format it. > > i pulled most files off it some via btrfs send/receive, some via rsync. but 1 > subvol wouldn’t send. i don’t remember the exact error, but it was that a > extend could not be found on 1 of the disks. > > with only a few 100gig of data left, i decided to balance some remaining > empty space before doing a `btrfs dev del`, so have another disk to store > more data on. > but im hitting a snag, i hit a BUG_ON when doing a `btrfs bal start -dusage=2 > /mountpoint` : > > [ 3327.678329] btrfs: found 198 extents > [ 3328.117274] btrfs: relocating block group 84473084968960 flags 17 > [ 3329.278521] btrfs: found 103 extents > [ 3331.907931] btrfs: found 103 extents > [ 3332.386172] btrfs: relocating block group 84466642518016 flags 17 > [ .536595] btrfs: found 86 extents > [ 3335.982967] btrfs: found 86 extents > [ 3336.599555] btrfs (4746) used greatest stack depth: 2744 bytes left > [ 3379.073464] btrfs: relocating block group 89878368419840 flags 17 > [ 3381.608948] btrfs: found 499 extents > [ 3383.884696] [ cut here ] > [ 3383.884720] kernel BUG at fs/btrfs/relocation.c:3405! > [ 3383.884731] invalid opcode: [#1] SMP > [ 3383.884742] Modules linked in: > [ 3383.884753] CPU: 0 PID: 5663 Comm: btrfs Not tainted 3.13.0 #1 > [ 3383.884763] Hardware name: System manufacturer System Product Name/E45M1-I > DELUXE, BIOS 0405 08/08/2012 > [ 3383.884778] task: 8802360eae80 ti: 88010dcaa000 task.ti: > 88010dcaa000 > [ 3383.884790] RIP: 0010:[] [] > __add_tree_block+0x1c5/0x1e0 > [ 3383.884811] RSP: 0018:88010dcaba38 EFLAGS: 00010202 > [ 3383.884821] RAX: 0001 RBX: 880039f18000 RCX: > > [ 3383.884832] RDX: RSI: RDI: > > [ 3383.884843] RBP: 88010dcaba90 R08: 88010dcab9f4 R09: > 88010dcab930 > [ 3383.884854] R10: R11: 047f R12: > 1000 > [ 3383.884865] R13: 88023489c630 R14: R15: > 528d112e4000 > [ 3383.884876] FS: 7f8e27e74880() GS:88023ec0() > knlGS: > [ 3383.884888] CS: 0010 DS: ES: CR0: 8005003b > [ 3383.884897] CR2: 7f60d89f35a8 CR3: 0001b5ada000 CR4: > 07f0 > [ 3383.884907] Stack: > [ 3383.884941] 88010dcabb28 4000812bde34 00a8528d112e > 0010 > [ 3383.885012] 1000 1000 0f3a > 8802348d6990 > [ 3383.885082] 88001cbf5a00 880039f18000 00b8 > 88010dcabb00 > [ 3383.885153] Call Trace: > [ 3383.885192] [] add_data_references+0x244/0x2e0 > [ 3383.885232] [] relocate_block_group+0x56b/0x640 > [ 3383.885272] [] btrfs_relocate_block_group+0x1a2/0x2f0 > [ 3383.885313] [] btrfs_relocate_chunk.isra.27+0x6a/0x740 > [ 3383.885355] [] ? btrfs_set_path_blocking+0x31/0x70 > [ 3383.885432] [] ? btrfs_search_slot+0x386/0x960 > [ 3383.885473] [] ? free_extent_buffer+0x47/0xa0 > [ 3383.885513] [] btrfs_balance+0x90b/0xea0 > [ 3383.885553] [] btrfs_ioctl_balance+0x162/0x520 > [ 3383.885592] [] btrfs_ioctl+0xcbd/0x25c0 > [ 3383.885632] [] ? __do_page_fault+0x1dc/0x520 > [ 3383.885673] [] do_vfs_ioctl+0x2c8/0x490 > [ 3383.885712] [] SyS_ioctl+0x81/0xa0 > [ 3383.885752] [] tracesys+0xdd/0xe2 > [ 3383.885787] Code: ff 48 8b 4d a8 48 8d 75 b6 4c 89 ea 48 89 df e8 42 e7 ff > ff 4c 89 ef 89 45 a8 e8 c7 0f f9 ff 8b 45 a8 e9 69 ff ff ff 85 c0 74 d6 <0f> > 0b 66 0f 1f 84 00 00 00 00 00 b8 f4 ff ff ff e9 50 ff ff ff > [ 3383.886001] RIP [] __add_tree_block+0x1c5/0x1e0 > [ 3383.886042] RSP > [ 3383.886359] ---[ end trace 075209044ce10da3 ]--- > Anything i can do to resolve / debug the issue? > > Remco-- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: hitting BUG_ON on troublesome FS
Remco Hosman - Yerf-it.com posted on Mon, 03 Feb 2014 21:51:26 +0100 as excerpted: > Anything i can do to resolve / debug the issue? I see from the trace you're running kernel 3.13.0. (FWIW 3.13.1 is out, but there weren't any btrfs commits therein, as they weren't upstream in 3.14-pre yet at that time.) You might try kernel 3.14-rc1 now that it's out. There's a big btrfs git pull in that, including a number of btrfs send/receive fixes. FWIW, btrfs send/receive seems to be a big focus right now, and I see a lot of patches floating by on the list even after that pull, so there's more where those came from. I'm not a dev (just another btrfs user and list regular for several kernel cycles now) and haven't followed the patches closely enough to know if your particular balance problem is covered in one of them, but as I said, since 3.14-rc1 is out now, you might as well try it... -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
hitting BUG_ON on troublesome FS
FIrst, a bit of history of the filesystem: used to be 6 disks, now 5. partially raid1 / raid10. been migrating back and forth a few times. As some point, a balance would not complete and would end with 164 ENOSPC’ses, while there was plenty of unallocated space on each disk. i scanned for extends larger then 1gig and found a few, so ran a recursive balance of the entire FS. I deceided to empty the filesystem and format it. i pulled most files off it some via btrfs send/receive, some via rsync. but 1 subvol wouldn’t send. i don’t remember the exact error, but it was that a extend could not be found on 1 of the disks. with only a few 100gig of data left, i decided to balance some remaining empty space before doing a `btrfs dev del`, so have another disk to store more data on. but im hitting a snag, i hit a BUG_ON when doing a `btrfs bal start -dusage=2 /mountpoint` : [ 3327.678329] btrfs: found 198 extents [ 3328.117274] btrfs: relocating block group 84473084968960 flags 17 [ 3329.278521] btrfs: found 103 extents [ 3331.907931] btrfs: found 103 extents [ 3332.386172] btrfs: relocating block group 84466642518016 flags 17 [ .536595] btrfs: found 86 extents [ 3335.982967] btrfs: found 86 extents [ 3336.599555] btrfs (4746) used greatest stack depth: 2744 bytes left [ 3379.073464] btrfs: relocating block group 89878368419840 flags 17 [ 3381.608948] btrfs: found 499 extents [ 3383.884696] [ cut here ] [ 3383.884720] kernel BUG at fs/btrfs/relocation.c:3405! [ 3383.884731] invalid opcode: [#1] SMP [ 3383.884742] Modules linked in: [ 3383.884753] CPU: 0 PID: 5663 Comm: btrfs Not tainted 3.13.0 #1 [ 3383.884763] Hardware name: System manufacturer System Product Name/E45M1-I DELUXE, BIOS 0405 08/08/2012 [ 3383.884778] task: 8802360eae80 ti: 88010dcaa000 task.ti: 88010dcaa000 [ 3383.884790] RIP: 0010:[] [] __add_tree_block+0x1c5/0x1e0 [ 3383.884811] RSP: 0018:88010dcaba38 EFLAGS: 00010202 [ 3383.884821] RAX: 0001 RBX: 880039f18000 RCX: [ 3383.884832] RDX: RSI: RDI: [ 3383.884843] RBP: 88010dcaba90 R08: 88010dcab9f4 R09: 88010dcab930 [ 3383.884854] R10: R11: 047f R12: 1000 [ 3383.884865] R13: 88023489c630 R14: R15: 528d112e4000 [ 3383.884876] FS: 7f8e27e74880() GS:88023ec0() knlGS: [ 3383.884888] CS: 0010 DS: ES: CR0: 8005003b [ 3383.884897] CR2: 7f60d89f35a8 CR3: 0001b5ada000 CR4: 07f0 [ 3383.884907] Stack: [ 3383.884941] 88010dcabb28 4000812bde34 00a8528d112e 0010 [ 3383.885012] 1000 1000 0f3a 8802348d6990 [ 3383.885082] 88001cbf5a00 880039f18000 00b8 88010dcabb00 [ 3383.885153] Call Trace: [ 3383.885192] [] add_data_references+0x244/0x2e0 [ 3383.885232] [] relocate_block_group+0x56b/0x640 [ 3383.885272] [] btrfs_relocate_block_group+0x1a2/0x2f0 [ 3383.885313] [] btrfs_relocate_chunk.isra.27+0x6a/0x740 [ 3383.885355] [] ? btrfs_set_path_blocking+0x31/0x70 [ 3383.885432] [] ? btrfs_search_slot+0x386/0x960 [ 3383.885473] [] ? free_extent_buffer+0x47/0xa0 [ 3383.885513] [] btrfs_balance+0x90b/0xea0 [ 3383.885553] [] btrfs_ioctl_balance+0x162/0x520 [ 3383.885592] [] btrfs_ioctl+0xcbd/0x25c0 [ 3383.885632] [] ? __do_page_fault+0x1dc/0x520 [ 3383.885673] [] do_vfs_ioctl+0x2c8/0x490 [ 3383.885712] [] SyS_ioctl+0x81/0xa0 [ 3383.885752] [] tracesys+0xdd/0xe2 [ 3383.885787] Code: ff 48 8b 4d a8 48 8d 75 b6 4c 89 ea 48 89 df e8 42 e7 ff ff 4c 89 ef 89 45 a8 e8 c7 0f f9 ff 8b 45 a8 e9 69 ff ff ff 85 c0 74 d6 <0f> 0b 66 0f 1f 84 00 00 00 00 00 b8 f4 ff ff ff e9 50 ff ff ff [ 3383.886001] RIP [] __add_tree_block+0x1c5/0x1e0 [ 3383.886042] RSP [ 3383.886359] ---[ end trace 075209044ce10da3 ]--- Anything i can do to resolve / debug the issue? Remco-- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
