Re: hitting BUG_ON on troublesome FS
to reply to my own thread: i managed to empty the filesystem, but it still segfaults in the same way when i try to balance the last few blocks. a `btrfs bal start -dusage=0 /mountpoint` finishes, but leaves a few blocks allocated. when i skip the usage=0, it segfaults. What can i do to provide useful information? Remco On 03 Feb 2014, at 21:51, Remco Hosman - Yerf-it.com re...@yerf-it.com wrote: FIrst, a bit of history of the filesystem: used to be 6 disks, now 5. partially raid1 / raid10. been migrating back and forth a few times. As some point, a balance would not complete and would end with 164 ENOSPC’ses, while there was plenty of unallocated space on each disk. i scanned for extends larger then 1gig and found a few, so ran a recursive balance of the entire FS. I deceided to empty the filesystem and format it. i pulled most files off it some via btrfs send/receive, some via rsync. but 1 subvol wouldn’t send. i don’t remember the exact error, but it was that a extend could not be found on 1 of the disks. with only a few 100gig of data left, i decided to balance some remaining empty space before doing a `btrfs dev del`, so have another disk to store more data on. but im hitting a snag, i hit a BUG_ON when doing a `btrfs bal start -dusage=2 /mountpoint` : [ 3327.678329] btrfs: found 198 extents [ 3328.117274] btrfs: relocating block group 84473084968960 flags 17 [ 3329.278521] btrfs: found 103 extents [ 3331.907931] btrfs: found 103 extents [ 3332.386172] btrfs: relocating block group 84466642518016 flags 17 [ .536595] btrfs: found 86 extents [ 3335.982967] btrfs: found 86 extents [ 3336.599555] btrfs (4746) used greatest stack depth: 2744 bytes left [ 3379.073464] btrfs: relocating block group 89878368419840 flags 17 [ 3381.608948] btrfs: found 499 extents [ 3383.884696] [ cut here ] [ 3383.884720] kernel BUG at fs/btrfs/relocation.c:3405! [ 3383.884731] invalid opcode: [#1] SMP [ 3383.884742] Modules linked in: [ 3383.884753] CPU: 0 PID: 5663 Comm: btrfs Not tainted 3.13.0 #1 [ 3383.884763] Hardware name: System manufacturer System Product Name/E45M1-I DELUXE, BIOS 0405 08/08/2012 [ 3383.884778] task: 8802360eae80 ti: 88010dcaa000 task.ti: 88010dcaa000 [ 3383.884790] RIP: 0010:[812f0bd5] [812f0bd5] __add_tree_block+0x1c5/0x1e0 [ 3383.884811] RSP: 0018:88010dcaba38 EFLAGS: 00010202 [ 3383.884821] RAX: 0001 RBX: 880039f18000 RCX: [ 3383.884832] RDX: RSI: RDI: [ 3383.884843] RBP: 88010dcaba90 R08: 88010dcab9f4 R09: 88010dcab930 [ 3383.884854] R10: R11: 047f R12: 1000 [ 3383.884865] R13: 88023489c630 R14: R15: 528d112e4000 [ 3383.884876] FS: 7f8e27e74880() GS:88023ec0() knlGS: [ 3383.884888] CS: 0010 DS: ES: CR0: 8005003b [ 3383.884897] CR2: 7f60d89f35a8 CR3: 0001b5ada000 CR4: 07f0 [ 3383.884907] Stack: [ 3383.884941] 88010dcabb28 4000812bde34 00a8528d112e 0010 [ 3383.885012] 1000 1000 0f3a 8802348d6990 [ 3383.885082] 88001cbf5a00 880039f18000 00b8 88010dcabb00 [ 3383.885153] Call Trace: [ 3383.885192] [812f1a54] add_data_references+0x244/0x2e0 [ 3383.885232] [812f2a2b] relocate_block_group+0x56b/0x640 [ 3383.885272] [812f2ca2] btrfs_relocate_block_group+0x1a2/0x2f0 [ 3383.885313] [812cbcca] btrfs_relocate_chunk.isra.27+0x6a/0x740 [ 3383.885355] [81281a31] ? btrfs_set_path_blocking+0x31/0x70 [ 3383.885432] [81286816] ? btrfs_search_slot+0x386/0x960 [ 3383.885473] [812c6f07] ? free_extent_buffer+0x47/0xa0 [ 3383.885513] [812ceedb] btrfs_balance+0x90b/0xea0 [ 3383.885553] [812d5ec2] btrfs_ioctl_balance+0x162/0x520 [ 3383.885592] [812d9aed] btrfs_ioctl+0xcbd/0x25c0 [ 3383.885632] [818c094c] ? __do_page_fault+0x1dc/0x520 [ 3383.885673] [81136868] do_vfs_ioctl+0x2c8/0x490 [ 3383.885712] [81136ab1] SyS_ioctl+0x81/0xa0 [ 3383.885752] [818c4f5b] tracesys+0xdd/0xe2 [ 3383.885787] Code: ff 48 8b 4d a8 48 8d 75 b6 4c 89 ea 48 89 df e8 42 e7 ff ff 4c 89 ef 89 45 a8 e8 c7 0f f9 ff 8b 45 a8 e9 69 ff ff ff 85 c0 74 d6 0f 0b 66 0f 1f 84 00 00 00 00 00 b8 f4 ff ff ff e9 50 ff ff ff [ 3383.886001] RIP [812f0bd5] __add_tree_block+0x1c5/0x1e0 [ 3383.886042] RSP 88010dcaba38 [ 3383.886359] ---[ end trace 075209044ce10da3 ]--- Anything i can do to resolve / debug the issue? Remco-- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: hitting BUG_ON on troublesome FS
How can i tell? Label: data uuid: a8626d67-4684-4b23-99b3-8d5fa8e7fd69 Total devices 5 FS bytes used 820.00KiB devid2 size 1.82TiB used 1.00GiB path /dev/sdb2 devid3 size 1.82TiB used 1.00GiB path /dev/sdf2 devid5 size 2.73TiB used 3.00GiB path /dev/sdd2 devid10 size 2.73TiB used 2.03GiB path /dev/sde2 devid11 size 3.64TiB used 1.03GiB path /dev/sdc1 Data, RAID10: total=3D2.00GiB, used=3D768.00KiB Data, RAID1: total=3D1.00GiB, used=3D12.00KiB System, RAID1: total=3D32.00MiB, used=3D4.00KiB Metadata, RAID1: total=3D1.00GiB, used=3D36.00KiB i made a image with `btrfs-image`, when i do -c 9, the file size is 7k, = so eazy enough to mail if it would be of any use. Remco On 04 Feb 2014, at 22:48, Josef Bacik jba...@fb.com wrote: On 02/03/2014 03:51 PM, Remco Hosman - Yerf-it.com wrote: FIrst, a bit of history of the filesystem: used to be 6 disks, now 5. partially raid1 / raid10. been migrating back and forth a few times. As some point, a balance would not complete and would end with 164 ENOSPC’ses, while there was plenty of unallocated space on each disk. i scanned for extends larger then 1gig and found a few, so ran a recursive balance of the entire FS. I deceided to empty the filesystem and format it. i pulled most files off it some via btrfs send/receive, some via rsync. but 1 subvol wouldn’t send. i don’t remember the exact error, but it was that a extend could not be found on 1 of the disks. with only a few 100gig of data left, i decided to balance some remaining empty space before doing a `btrfs dev del`, so have another disk to store more data on. but im hitting a snag, i hit a BUG_ON when doing a `btrfs bal start -dusage=2 /mountpoint` : [ 3327.678329] btrfs: found 198 extents [ 3328.117274] btrfs: relocating block group 84473084968960 flags 17 [ 3329.278521] btrfs: found 103 extents [ 3331.907931] btrfs: found 103 extents [ 3332.386172] btrfs: relocating block group 84466642518016 flags 17 [ .536595] btrfs: found 86 extents [ 3335.982967] btrfs: found 86 extents [ 3336.599555] btrfs (4746) used greatest stack depth: 2744 bytes left [ 3379.073464] btrfs: relocating block group 89878368419840 flags 17 [ 3381.608948] btrfs: found 499 extents [ 3383.884696] [ cut here ] [ 3383.884720] kernel BUG at fs/btrfs/relocation.c:3405! [ 3383.884731] invalid opcode: [#1] SMP [ 3383.884742] Modules linked in: [ 3383.884753] CPU: 0 PID: 5663 Comm: btrfs Not tainted 3.13.0 #1 [ 3383.884763] Hardware name: System manufacturer System Product Name/E45M1-I DELUXE, BIOS 0405 08/08/2012 [ 3383.884778] task: 8802360eae80 ti: 88010dcaa000 task.ti: 88010dcaa000 [ 3383.884790] RIP: 0010:[812f0bd5] [812f0bd5] __add_tree_block+0x1c5/0x1e0 [ 3383.884811] RSP: 0018:88010dcaba38 EFLAGS: 00010202 [ 3383.884821] RAX: 0001 RBX: 880039f18000 RCX: [ 3383.884832] RDX: RSI: RDI: [ 3383.884843] RBP: 88010dcaba90 R08: 88010dcab9f4 R09: 88010dcab930 [ 3383.884854] R10: R11: 047f R12: 1000 [ 3383.884865] R13: 88023489c630 R14: R15: 528d112e4000 [ 3383.884876] FS: 7f8e27e74880() GS:88023ec0() knlGS: [ 3383.884888] CS: 0010 DS: ES: CR0: 8005003b [ 3383.884897] CR2: 7f60d89f35a8 CR3: 0001b5ada000 CR4: 07f0 [ 3383.884907] Stack: [ 3383.884941] 88010dcabb28 4000812bde34 00a8528d112e 0010 [ 3383.885012] 1000 1000 0f3a 8802348d6990 [ 3383.885082] 88001cbf5a00 880039f18000 00b8 88010dcabb00 [ 3383.885153] Call Trace: [ 3383.885192] [812f1a54] add_data_references+0x244/0x2e0 [ 3383.885232] [812f2a2b] relocate_block_group+0x56b/0x640 [ 3383.885272] [812f2ca2] btrfs_relocate_block_group+0x1a2/0x2f0 [ 3383.885313] [812cbcca] btrfs_relocate_chunk.isra.27+0x6a/0x740 [ 3383.885355] [81281a31] ? btrfs_set_path_blocking+0x31/0x70 [ 3383.885432] [81286816] ? btrfs_search_slot+0x386/0x960 [ 3383.885473] [812c6f07] ? free_extent_buffer+0x47/0xa0 [ 3383.885513] [812ceedb] btrfs_balance+0x90b/0xea0 [ 3383.885553] [812d5ec2] btrfs_ioctl_balance+0x162/0x520 [ 3383.885592] [812d9aed] btrfs_ioctl+0xcbd/0x25c0 [ 3383.885632] [818c094c] ? __do_page_fault+0x1dc/0x520 [ 3383.885673] [81136868] do_vfs_ioctl+0x2c8/0x490 [ 3383.885712] [81136ab1] SyS_ioctl+0x81/0xa0 [ 3383.885752] [818c4f5b] tracesys+0xdd/0xe2 [ 3383.885787] Code: ff 48 8b 4d a8 48 8d 75 b6 4c 89 ea 48 89 df e8 42 e7 ff ff 4c 89 ef 89 45 a8 e8 c7 0f f9 ff 8b 45 a8 e9 69 ff ff ff 85 c0 74 d6 0f 0b 66 0f 1f 84 00
hitting BUG_ON on troublesome FS
FIrst, a bit of history of the filesystem: used to be 6 disks, now 5. partially raid1 / raid10. been migrating back and forth a few times. As some point, a balance would not complete and would end with 164 ENOSPC’ses, while there was plenty of unallocated space on each disk. i scanned for extends larger then 1gig and found a few, so ran a recursive balance of the entire FS. I deceided to empty the filesystem and format it. i pulled most files off it some via btrfs send/receive, some via rsync. but 1 subvol wouldn’t send. i don’t remember the exact error, but it was that a extend could not be found on 1 of the disks. with only a few 100gig of data left, i decided to balance some remaining empty space before doing a `btrfs dev del`, so have another disk to store more data on. but im hitting a snag, i hit a BUG_ON when doing a `btrfs bal start -dusage=2 /mountpoint` : [ 3327.678329] btrfs: found 198 extents [ 3328.117274] btrfs: relocating block group 84473084968960 flags 17 [ 3329.278521] btrfs: found 103 extents [ 3331.907931] btrfs: found 103 extents [ 3332.386172] btrfs: relocating block group 84466642518016 flags 17 [ .536595] btrfs: found 86 extents [ 3335.982967] btrfs: found 86 extents [ 3336.599555] btrfs (4746) used greatest stack depth: 2744 bytes left [ 3379.073464] btrfs: relocating block group 89878368419840 flags 17 [ 3381.608948] btrfs: found 499 extents [ 3383.884696] [ cut here ] [ 3383.884720] kernel BUG at fs/btrfs/relocation.c:3405! [ 3383.884731] invalid opcode: [#1] SMP [ 3383.884742] Modules linked in: [ 3383.884753] CPU: 0 PID: 5663 Comm: btrfs Not tainted 3.13.0 #1 [ 3383.884763] Hardware name: System manufacturer System Product Name/E45M1-I DELUXE, BIOS 0405 08/08/2012 [ 3383.884778] task: 8802360eae80 ti: 88010dcaa000 task.ti: 88010dcaa000 [ 3383.884790] RIP: 0010:[812f0bd5] [812f0bd5] __add_tree_block+0x1c5/0x1e0 [ 3383.884811] RSP: 0018:88010dcaba38 EFLAGS: 00010202 [ 3383.884821] RAX: 0001 RBX: 880039f18000 RCX: [ 3383.884832] RDX: RSI: RDI: [ 3383.884843] RBP: 88010dcaba90 R08: 88010dcab9f4 R09: 88010dcab930 [ 3383.884854] R10: R11: 047f R12: 1000 [ 3383.884865] R13: 88023489c630 R14: R15: 528d112e4000 [ 3383.884876] FS: 7f8e27e74880() GS:88023ec0() knlGS: [ 3383.884888] CS: 0010 DS: ES: CR0: 8005003b [ 3383.884897] CR2: 7f60d89f35a8 CR3: 0001b5ada000 CR4: 07f0 [ 3383.884907] Stack: [ 3383.884941] 88010dcabb28 4000812bde34 00a8528d112e 0010 [ 3383.885012] 1000 1000 0f3a 8802348d6990 [ 3383.885082] 88001cbf5a00 880039f18000 00b8 88010dcabb00 [ 3383.885153] Call Trace: [ 3383.885192] [812f1a54] add_data_references+0x244/0x2e0 [ 3383.885232] [812f2a2b] relocate_block_group+0x56b/0x640 [ 3383.885272] [812f2ca2] btrfs_relocate_block_group+0x1a2/0x2f0 [ 3383.885313] [812cbcca] btrfs_relocate_chunk.isra.27+0x6a/0x740 [ 3383.885355] [81281a31] ? btrfs_set_path_blocking+0x31/0x70 [ 3383.885432] [81286816] ? btrfs_search_slot+0x386/0x960 [ 3383.885473] [812c6f07] ? free_extent_buffer+0x47/0xa0 [ 3383.885513] [812ceedb] btrfs_balance+0x90b/0xea0 [ 3383.885553] [812d5ec2] btrfs_ioctl_balance+0x162/0x520 [ 3383.885592] [812d9aed] btrfs_ioctl+0xcbd/0x25c0 [ 3383.885632] [818c094c] ? __do_page_fault+0x1dc/0x520 [ 3383.885673] [81136868] do_vfs_ioctl+0x2c8/0x490 [ 3383.885712] [81136ab1] SyS_ioctl+0x81/0xa0 [ 3383.885752] [818c4f5b] tracesys+0xdd/0xe2 [ 3383.885787] Code: ff 48 8b 4d a8 48 8d 75 b6 4c 89 ea 48 89 df e8 42 e7 ff ff 4c 89 ef 89 45 a8 e8 c7 0f f9 ff 8b 45 a8 e9 69 ff ff ff 85 c0 74 d6 0f 0b 66 0f 1f 84 00 00 00 00 00 b8 f4 ff ff ff e9 50 ff ff ff [ 3383.886001] RIP [812f0bd5] __add_tree_block+0x1c5/0x1e0 [ 3383.886042] RSP 88010dcaba38 [ 3383.886359] ---[ end trace 075209044ce10da3 ]--- Anything i can do to resolve / debug the issue? Remco-- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
