BUG ON during btrfs check
Hi! I'v hit a BUG ON during btrfs check: - server:~# btrfs check --progress --repair /dev/sde enabling repair mode Opening filesystem to check... Checking filesystem on /dev/sde UUID: d5fa971b-6546-424d-87c1-dcd688eacdac [1/7] checking root items (0:59:38 elapsed, 104205418 items checked) Fixed 0 roots. [2/7] checking extents (3:12:27 elapsed, 5[2/7] checking extents (3:12:28 elapsed, 5234ref mismatch on [30539776 16384] extent item 0, found 1 elapsed, 7917478 items checked) tree backref 30539776 parent 2 root 2 not found in extent tree backpointer mismatch on [30539776 16384] adding new tree backref on start 30539776 len 16384 parent 0 root 217548 items checked) btrfs unable to find ref byte nr 693215772672 parent 0 root 2 owner 1 offset 0checked) transaction.c:168: btrfs_commit_transaction: BUG_ON `ret` triggered, value -5 btrfs(+0x3b748)[0x562bc6163748] btrfs(btrfs_commit_transaction+0x53)[0x562bc6163af5] btrfs(+0x554bc)[0x562bc617d4bc] btrfs(cmd_check+0x1288)[0x562bc617ee75] btrfs(main+0x1f3)[0x562bc613be63] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0x7fc93c23bb45] btrfs(_start+0x2a)[0x562bc613beaa] Abgebrochen - Kernel: 4.19.0-2-amd64 #1 SMP Debian 4.19.16-1 (2019-01-17) x86_64 GNU/Linux btrfs-progs v4.20.1 After that I mounted the fs and got this: - [54682.472317] BTRFS info (device sdf): enabling inode map caching [54682.472321] BTRFS info (device sdf): disk space caching is enabled [54682.472324] BTRFS info (device sdf): has skinny extents [54710.634379] BTRFS info (device sdf): checking UUID tree [54776.365908] BTRFS error (device sdf): parent transid verify failed on 30539776 wanted 695635 found 695646 [54776.365916] BTRFS error (device sdf): parent transid verify failed on 30539776 wanted 695635 found 695646 [54780.325562] BTRFS error (device sdf): parent transid verify failed on 30539776 wanted 695635 found 695646 [54780.325567] BTRFS error (device sdf): parent transid verify failed on 30539776 wanted 695635 found 695646 - /dev/sde, ID: 2 Device size: 3.64TiB Device slack: 0.00B Data,RAID1: 2.29TiB Metadata,single: 800.00MiB Metadata,RAID1:124.00GiB System,RAID1:8.00MiB Unallocated: 1.23TiB /dev/sdf, ID: 1 Device size: 3.64TiB Device slack: 0.00B Data,single: 8.00MiB Data,RAID1: 2.29TiB Metadata,single: 816.00MiB Metadata,RAID1:124.00GiB System,single: 4.00MiB System,RAID1:8.00MiB Unallocated: 1.23TiB [54682.472317] BTRFS info (device sdf): enabling inode map caching [54682.472321] BTRFS info (device sdf): disk space caching is enabled [54682.472324] BTRFS info (device sdf): has skinny extents [54710.634379] BTRFS info (device sdf): checking UUID tree [54776.365908] BTRFS error (device sdf): parent transid verify failed on 30539776 wanted 695635 found 695646 [54776.365916] BTRFS error (device sdf): parent transid verify failed on 30539776 wanted 695635 found 695646 [54780.325562] BTRFS error (device sdf): parent transid verify failed on 30539776 wanted 695635 found 695646 [54780.325567] BTRFS error (device sdf): parent transid verify failed on 30539776 wanted 695635 found 695646 [54980.930943] INFO: task btrfs-transacti:23138 blocked for more than 120 seconds. [54980.930945] Tainted: GE 4.19.0-2-amd64 #1 Debian 4.19.16-1 [54980.930946] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [54980.930947] btrfs-transacti D0 23138 2 0x8000 [54980.930950] Call Trace: [54980.930957] ? __schedule+0x2a2/0x870 [54980.930959] ? kmem_cache_alloc_trace+0x155/0x1d0 [54980.930961] schedule+0x28/0x80 [54980.930987] wait_block_group_cache_progress+0xa0/0xd0 [btrfs] [54980.930989] ? finish_wait+0x80/0x80 [54980.931008] find_free_extent+0x50c/0x1070 [btrfs] [54980.931026] btrfs_reserve_extent+0x9b/0x180 [btrfs] [54980.931046] __btrfs_prealloc_file_range+0x121/0x450 [btrfs] [54980.931064] cache_save_setup+0x1d4/0x3c0 [btrfs] [54980.931081] btrfs_setup_space_cache+0x97/0xc0 [btrfs] [54980.931099] commit_cowonly_roots+0xde/0x2f0 [btrfs] [54980.931120] ? btrfs_qgroup_account_extents+0xba/0x1d0 [btrfs] [54980.931140] btrfs_commit_transaction+0x310/0x870 [btrfs] [54980.931160] ? start_transaction+0x9b/0x3e0 [btrfs] [54980.931179] transaction_kthread+0x147/0x180 [btrfs] [54980.931198] ? btrfs_cleanup_transaction+0x520/0x520 [btrfs] [54980.931200] kthread+0x112/0x130 [54980.931202] ? kthread_bind+0x30/0x30 [54980.931204] ret_from_fork+0x35/0x40 [54997.668292] WARNING: CPU: 2 PID: 23138 at fs/btrfs/extent-tree.c:679
Re: Debian 3.7.1 BTRFS crash
Am 13.03.2013, 12:31 Uhr, schrieb Swâmi Petaramesh : Le 13/03/2013 11:56, Bart Noordervliet a écrit : USB flash drives are rubbish for any filesystem except FAT32 and then still only gracefully accept large sequential writes. A few years ago I thought it would be a good idea to put the root partition of a few of my small Debian servers on USB flash, so that the harddisks could spin down at night and I could easily prepare and switch a new Debian-version. However, each and every USB stick got trashed within a year I have an ARM box that runs a little Debian server (typically an advanced NAS), it uses an USB key as an ext2 root filesystem. Everything but big storage is there, and it's been up and running 24/7 for 3+ years without any USB key incident... The difference is the fs. Ext3 uses a journal which uses always the same physical sectors on disc. If the disc is a hard disk, it does not matter, rewrites are no problem for platters. If it is an modern SSD, the SSD- controller takes care and redirects the writes to different physical sectors. USB-sticks have no smart controller and so the writes hit always the same physical sector, it's like burning a hole in the flash chip. If the commit time is standard for desktops set to 5 seconds, then a whole year means a lot of writes to the same sector on an USB-stick. Regards Norbert -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Snapshot Cleaner not Working with inode_cache
Am 20.02.2013, 02:14 Uhr, schrieb Liu Bo : I think I know why inode_cache keeps us from freeing space, inode_cache adds a cache_inode in each btrfs root, and this cache_inode will be iput at the very last of stage during umount, ie. after we do cleanup work on old snapshot/subvols, where we free the space. A remount will force btrfs to do cleanup work on old snapshots during mount. This may explain the situation. thanks, liubo I don't know how long the code behaves that way, but this is exactly what I see here on debian kernel 3.2.35. Norbert -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cross-subvolume cp --reflink
Am 29.04.2012, 01:53 Uhr, schrieb Hubert Kario : On Sunday 01 of April 2012 11:42:23 Jérôme Poulin wrote: On Sun, Apr 1, 2012 at 11:27 AM, Norbert Scheibner wrote: > Some users tested this patch successfully for week,s or months in 2 or 3 > kernel versions since then, true? If this feature must be implented in VFS in another patch, why not just activate what works and make the future patch disable it again? Why would (should) it be impleemented in VFS? reflink copy is completely different from normal copy and hard link. I wouldn't make a VFS issue out of that. That should be another discussion. But: Subvolumes in btrfs are barriers *only* in btrfs and not visible in VFS. That is just a bug in my opinion, so it should work anyway, but to look at it from VFS point of view is strengthening me in wanting the outstanding patches integrated, as this feature could be supported by VFS in the future. IMHO it's strictly btrfs business and not supporting reflink copy between arbitrary directories is a bug. I don't know exactly, but I think ZFS is another candidate for "cp --reflink". For some of the log-structured filesystems this could be usefull too, but I don't know if some of them already supports this or plan to support this in the future. Greetings Norbert -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cross-subvolume cp --reflink
> On: Sun, 01 Apr 2012 19:22:42 +0200"Klaus A. Kreil" wrote > I am just an interested reader on the btrfs list and so far have never > posted or sent a message to the list, but I do have a dedup bash script > that searches for duplicates underneath a directory (provided as an > argument) and hard links identical files. > > It works very well for an ext3 filesystem, but I guess the basics should > be the same for a btrfs filesystem. Everyone feel free to correct me here, but: At the moment there is a little problem with the maximum number of hard links in a directory. So I wouldn't use them wherever possible to avoid any thinkable problems in the near future. Plus to hard link 2 files means, that change one file You change the other one. It's something You either don't want to happen or something, which could be done in better ways. The cp --reflink method on a COW-fs is a much smarter method. Plus hard links across subvolumes do match the case of hard links across devices on a traditional fs, which is forbidden. Plus hard links In my opinion should really be substituted by soft links, because hard links are not transparent at the first sight and can not be copied as it. So no, I'd rather want the patch to allow cross-subvolume cp --reflink in the kernel and I will wait for that to happen. Greetings Norbert -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cross-subvolume cp --reflink
> On: Sun, 01 Apr 2012 20:19:24 +0300 Konstantinos Skarlatos wrote > > I use btrfs for my backups. Ones a day I rsync --delete --inplace the > complete system to a subvolume, snapshot it, delete some tempfiles in the > snapshot. > > In my setup I rsync --inplace many servers and workstations, 4-6 times > a day into a 12TB btrfs volume, each one in its own subvolume. After > every backup a new ro snapshot is created. > > I have many cross-subvolume duplicate files (OS files, programs, many > huge media files that are copied locally from the servers to the > workstations etc), so a good "dedupe" script could save lots of space, > and allow me to keep snapshots for much longer. So the script should be optimized not to try to deduplicate the whole fs everytime but the newly written ones. You could take such a file list out of the rsync output or the btrfs subvolume find-new command. Albeit the reflink patch, You could use such a bash-script inside one subvolume, after the rsync and before the snapshot. I don't know how much space it saves for You in this situation, but it's worth a try and a good way to develop such a script, because before You write anything to disc You can see how many duplicates are there and how much space could be freed. MfG Norbert -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cross-subvolume cp --reflink
> On: Sun, 01 Apr 2012 19:45:13 +0300 Konstantinos Skarlatos wrote > > That's my point. This poor man's dedupe would solve my problems here > very well. I don't need a zfs-variant of dedupe. I can implement such a > file-based dedupe with userland tools and would be happy. > > do you have any scripts that can search a btrfs filesystem for dupes > and replace them with cp --reflink? Nothing really working and tested very well. After I get to known the missing cp --reflink feature I stopped to develop the script any further. I use btrfs for my backups. Ones a day I rsync --delete --inplace the complete system to a subvolume, snapshot it, delete some tempfiles in the snapshot. In addition to that I wanted to shrink file-duplicates. What the script should do: 1. I md5sum every file 2. If the checksums are identical, I compare the files 3. If 2 or more files are really identical: - move one to a temp-dir - cp --reflink the second to the position and name of the first - do a chown --reference, chmod --reference and touch --reference to copy owner, file mode bits and time from the orginal to the reflink-copy and then delete the original in temp-dir Everything could be done with bash. Thinkable is the use of a database for the md5sums, which could be used for other purposes in the future. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: cross-subvolume cp --reflink
> On: Sun, 01 Apr 2012 18:30:13 +0300 Konstantinos Skarlatos wrote > +1 from me too, i would save enormous amounts of space with a patch > like that, at least until dedupe is implemented. We could call it "poor > man's dedupe" That's my point. This poor man's dedupe would solve my problems here very well. I don't need a zfs-variant of dedupe. I can implement such a file-based dedupe with userland tools and would be happy. It's there, it's tested, it doesn't break another thing. Use it! Greetings Norbert -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
cross-subvolume cp --reflink
Glück Auf! I know its been discussed more then ones, but as a user I really would like to see the patch for allowing this in the kernel. Some users tested this patch successfully for weeks or months in 2 or 3 kernel versions since then, true? I'd say by creating a snapshot, it's nothing else in the end. More then one file or tree sharing the same data on disc, or am I wrong? So why not? Norbert -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
btrsfck shows errors...
Glück Auf! On kernel 3.2 with btrfs-progs-dangerdonteveruse-62b7993. I have these subvolumes on a working fs on 1 TB hdd: ID 257 top level 5 path system-backup-20110803-182701-monthly ID 481 top level 5 path system-backup-20110901-040001-monthly ID 809 top level 5 path system-backup-20111001-040001-monthly ID 864 top level 5 path system-backup-2001-040001-monthly ID 928 top level 5 path system-backup-20111201-040001-monthly ID 1404 top level 5 path system-backup-20120101-155108-monthly ID 1431 top level 5 path system-backup-20120201-040001-monthly ID 1432 top level 5 path system-backup-20120202-040001 ID 1433 top level 5 path system-backup-20120203-040001 ID 1723 top level 5 path system-backup ID 2373 top level 5 path system-backup-20120210-040001 ID 2378 top level 5 path system-backup-20120211-040001 ID 2383 top level 5 path system-backup-20120212-040002 ID 2391 top level 5 path system-backup-20120213-040001 ID 2398 top level 5 path system-backup-20120214-040001 ID 2401 top level 5 path system-backup-20120215-040001 btrsfsck shows: checking extents checking fs roots root 5 inode 18446744073709551604 errors 2000 root 5 inode 18446744073709551605 errors 1 root 257 inode 18446744073709551604 errors 2000 root 257 inode 18446744073709551605 errors 1 root 481 inode 610772 errors 400 root 481 inode 18446744073709551604 errors 2000 root 481 inode 18446744073709551605 errors 1 root 809 inode 610772 errors 400 root 809 inode 18446744073709551604 errors 2000 root 809 inode 18446744073709551605 errors 1 root 864 inode 610772 errors 400 root 864 inode 18446744073709551604 errors 2000 root 864 inode 18446744073709551605 errors 1 root 928 inode 610772 errors 400 root 928 inode 18446744073709551604 errors 2000 root 928 inode 18446744073709551605 errors 1 root 1404 inode 610772 errors 400 root 1404 inode 18446744073709551604 errors 2000 root 1404 inode 18446744073709551605 errors 1 root 1431 inode 610772 errors 400 root 1431 inode 18446744073709551604 errors 2000 root 1431 inode 18446744073709551605 errors 1 root 1432 inode 610772 errors 400 root 1432 inode 18446744073709551604 errors 2000 root 1432 inode 18446744073709551605 errors 1 root 1433 inode 610772 errors 400 root 1433 inode 18446744073709551604 errors 2000 root 1433 inode 18446744073709551605 errors 1 root 1723 inode 610772 errors 400 root 1723 inode 18446744073709551604 errors 2000 root 1723 inode 18446744073709551605 errors 1 root 2373 inode 610772 errors 400 root 2373 inode 18446744073709551604 errors 2000 root 2373 inode 18446744073709551605 errors 1 root 2378 inode 610772 errors 400 root 2378 inode 18446744073709551604 errors 2000 root 2378 inode 18446744073709551605 errors 1 root 2383 inode 610772 errors 400 root 2383 inode 18446744073709551604 errors 2000 root 2383 inode 18446744073709551605 errors 1 root 2391 inode 610772 errors 400 root 2391 inode 18446744073709551604 errors 2000 root 2391 inode 18446744073709551605 errors 1 root 2398 inode 610772 errors 400 root 2398 inode 18446744073709551604 errors 2000 root 2398 inode 18446744073709551605 errors 1 root 2401 inode 610772 errors 400 root 2401 inode 18446744073709551604 errors 2000 root 2401 inode 18446744073709551605 errors 1 found 683579826176 bytes used err is 1 total csum bytes: 664064404 total tree bytes: 3535802368 total fs tree bytes: 2500636672 btree space waste bytes: 958368184 file data blocks allocated: 763722764288 referenced 763722764288 I already tried scrub and balance, both worked, but the errors are still there. It's the same fs I mentioned in my post from Thu, 09 Feb 2012 18:42:32 +0100. Is there anything I can do to repair that fs now or do I have to wait for a really working btrfsck? Cheers Norbert -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Freeing space over reboot question
On Sat, 11 Feb 2012 19:56:32 +0600 Roman Mamedov wrote: > > Have You tried it Yourself? I think the problem was the remount > > before the space has been completely freed in background. It > > left a valid and working fs, with still work to do. > > Yes, after some snapshot deletions the umount takes a really long time. Ok, then the question is, why did this not happen here? > > In my opinion the kernel should either stall the umount till the > > space is given free > > In my experience it seemed to do just that. Hm, I did the umount under kernel 3.1 and after the reboot I used kernel 3.2. Is that the reason? Regards Norbert -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Freeing space over reboot question
On: Fri, 10 Feb 2012 00:20:55 +0600 Roman Mamedov wrote > AFAIK the only reliable way currently to ensure the space after a > subvolume > deletion is freed, is to remount the FS. Have You tried it Yourself? I think the problem was the remount before the space has been completely freed in background. It left a valid and working fs, with still work to do. In my opinion the kernel should either stall the umount till the space is given free or restart the cleaner after the next mount. To stall the umount could be annoying and make the user believe something is broken, because the cleaner could take a while and it feels to classic for modern fs. Is there a way to leave an entry in the log to replay after the next mount? Pardon me if I use the wrong terms here, I'm just an interested user and not a fs-crack or even a programmer. Regards Norbert -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Freeing space over reboot question
On Thu, 9 Feb 2012 12:11:19 -0600 Chester wrote: > A similar thing has happened to me recently. The snapshot deletion > happens asynchronously and should continue after a reboot (in my > case). If you boot up your system and leave it idle, take a look at > iotop. You might see a [btrfs-cleaner] doing some work. No that didn't help. The backup did not start immediately after the reboot and 5 days later - still no change. Norbert -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Freeing space over reboot question
Glück Auf! I use now kernel 3.2. The filesystem was originally created under 2.6.39 on 1 whole hdd, mounted with "noatime,nodiratime,inode_cache". I use it for backups: rsync the whole system to a subvolume, snapshot it and then delete some tempfiles in the snapshot, which are 90% of the full-backup, all once a day. In figures: on this 1 TB hdd is the full-backup with around 600 GiB and 10 to 20 snapshots with around 30 GiB each, all together using around 700 GiB on disc. What I did: - I deleted (by accident) the big subvolume with the full-backup with "btrfs subvolume delete" and recreated it with the same name with a snapshot of the latest snapshot. - During the deletion of this big subvolume in background I changed the kernel from 3.1 to 3.2 and did a reboot. - After that, the fs was operational, but the space was still used and the next system-backup onto this fs failed with no space left errors. "btrfs filesystem df" showed me that the fs used the whole hdd and that there were only some kB free, which fits to the errors from rsync during backup. So the used space of subvolume I deleted, was not freed. How to get the space back which should have been freed? A balance did not help. What worked was the deletion of that half-filled subvolume, which I use for the full backup. After that the space got freed and the next balance run shrinked the fs again, so that it uses only a part of the hdd. What I wonder is: Couldn't this be a little bit more user-friendly? If there is a background process running like this here, freeing some space, should the umount take as long as the background process or should the background process stop immediately and restart after the next mount (if possible, especially with a kernel change in between or the possibility that the fs gets mounted read-only)? ... Or is this all nonsense and it happened here because I rebooted and after that used another kernel. My best wishes Norbert -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: panic after remove of device during rebalance
Another reboot - another try - only this time I ran "btrfsctl -a" first. Now the first process to stay in uninterruptable sleep was the umount after theses tests. modprobe btrfs ./btrfsctl -a Scanning for Btrfs filesystems ./btrfs-show Label: none uuid: ca5e7037-a65c-45d8-b954-f64ab0799964 Total devices 2 FS bytes used 6.01GB devid5 size 623.25GB used 7.32GB path /dev/md15 devid1 size 9.31GB used 7.32GB path /dev/md11 ./btrfsck /dev/md11 fs tree 256 refs 1 not found unresolved ref root 257 dir 256 index 8 namelen 8 name subvol00 error 600 found 6449324032 bytes used err is 1 total csum bytes: 6291456 total tree bytes: 6873088 total fs tree bytes: 36864 btree space waste bytes: 159776 file data blocks allocated: 10737418240 referenced 10737418240 ls -la /mnt/btrfs-tests/ insgesamt 6291468 dr-xr-xr-x 1 root root 240 2. Feb 09:46 . drwxr-xr-x 9 root root4096 3. Feb 15:07 .. -rwxrwxr-x 1 samba samba 1073741824 31. Jan 17:53 large-file-test-000 -rwxrwxr-x 1 samba samba 1073741824 31. Jan 18:05 large-file-test-001 -rwxrwxr-x 1 samba samba 1073741824 31. Jan 18:12 large-file-test-002 -rwxrwxr-x 1 samba samba 1073741824 31. Jan 18:17 large-file-test-003 -rwxrwxr-x 1 samba samba 1073741824 31. Jan 18:24 large-file-test-004 -rwxrwxr-x 1 samba samba 1073741824 31. Jan 18:13 large-file-test-100 drwxrwxr-x 1 samba samba168 1. Feb 16:52 snap00 ./btrfs-vol -b /mnt/btrfs-tests/ Speicherzugriffsfehler Feb 3 15:04:04 server kernel: [ 402.415500] Btrfs loaded Feb 3 15:04:11 server kernel: [ 410.152115] device fsid d8455ca637705eca-649979b04af654b9 devid 1 transid 681 /dev/md11 Feb 3 15:04:11 server kernel: [ 410.154554] device fsid d8455ca637705eca-649979b04af654b9 devid 5 transid 681 /dev/md15 Feb 3 15:04:11 server kernel: [ 410.157135] device fsid d8455ca637705eca-649979b04af654b9 devid 1 transid 681 /dev/md/11 Feb 3 15:04:11 server kernel: [ 410.160300] device fsid d8455ca637705eca-649979b04af654b9 devid 5 transid 681 /dev/md/15 Feb 3 15:05:37 server kernel: [ 495.302414] device fsid d8455ca637705eca-649979b04af654b9 devid 1 transid 681 /dev/md11 Feb 3 15:05:37 server kernel: [ 495.304530] device fsid d8455ca637705eca-649979b04af654b9 devid 5 transid 681 /dev/md15 Feb 3 15:05:37 server kernel: [ 495.306343] device fsid d8455ca637705eca-649979b04af654b9 devid 1 transid 681 /dev/md/11 Feb 3 15:05:37 server kernel: [ 495.309194] device fsid d8455ca637705eca-649979b04af654b9 devid 5 transid 681 /dev/md/15 Feb 3 15:06:08 server kernel: [ 526.808238] device fsid d8455ca637705eca-649979b04af654b9 devid 1 transid 681 /dev/md11 Feb 3 15:06:08 server kernel: [ 526.811494] device fsid d8455ca637705eca-649979b04af654b9 devid 5 transid 681 /dev/md15 Feb 3 15:06:08 server kernel: [ 526.832216] device fsid d8455ca637705eca-649979b04af654b9 devid 1 transid 681 /dev/md/11 Feb 3 15:06:08 server kernel: [ 526.837340] device fsid d8455ca637705eca-649979b04af654b9 devid 5 transid 681 /dev/md/15 Feb 3 15:07:36 server kernel: [ 614.725114] device fsid d8455ca637705eca-649979b04af654b9 devid 1 transid 681 /dev/md11 Feb 3 15:07:36 server kernel: [ 614.757610] btrfs: unlinked 1 orphans Feb 3 15:07:36 server kernel: [ 614.757610] btrfs: unlinked 1 orphans Feb 3 15:14:26 server kernel: [ 1024.486214] btrfs: relocating block group 73774923776 flags 17 Feb 3 15:14:27 server kernel: [ 1026.148298] btrfs allocation failed flags 17, wanted 4096 Feb 3 15:14:27 server kernel: [ 1026.148302] space_info has 260046848 free, is full Feb 3 15:14:27 server kernel: [ 1026.148306] space_info total=6774456320, pinned=0, delalloc=81199104, may_use=0, used=6442450944, root=0, super=65536, reserved=71892992 Feb 3 15:14:27 server kernel: [ 1026.148309] block group 61291364352 has 1073741824 bytes, 1058013184 used 0 pinned 15663104 reserved Feb 3 15:14:27 server kernel: [ 1026.148311] block group has cluster?: no Feb 3 15:14:27 server kernel: [ 1026.148312] 0 blocks of free space at or bigger than bytes is Feb 3 15:14:27 server kernel: [ 1026.148315] block group 62365106176 has 1073741824 bytes, 1069547520 used 0 pinned 4194304 reserved Feb 3 15:14:27 server kernel: [ 1026.148317] block group has cluster?: no Feb 3 15:14:27 server kernel: [ 1026.148318] 0 blocks of free space at or bigger than bytes is Feb 3 15:14:27 server kernel: [ 1026.148320] block group 69139562496 has 1073741824 bytes, 1063256064 used 0 pinned 10485760 reserved Feb 3 15:14:27 server kernel: [ 1026.148321] block group has cluster?: no Feb 3 15:14:27 server kernel: [ 1026.148323] 0 blocks of free space at or bigger than bytes is Feb 3 15:14:27 server kernel: [ 1026.148325] block group 70213304320 has 1073741824 bytes, 1062207488 used 0 pinned 11534336 reserved Feb 3 15:14:27 server kernel: [ 1026.148326] block group has cluster?: no Feb 3 15:14:27 server kernel: [ 1026.148327] 0 blocks of free space at or bigger than bytes is Feb 3 15:14:27 serve
Re: panic after remove of device during rebalance
This happened after reboot: ./btrfs-show Label: none uuid: ca5e7037-a65c-45d8-b954-f64ab0799964 Total devices 2 FS bytes used 6.01GB devid5 size 623.25GB used 7.32GB path /dev/md15 devid1 size 9.31GB used 7.32GB path /dev/md11 ./btrfsck /dev/md11 failed to open /dev/btrfs-control skipping device registration failed to open /dev/btrfs-control skipping device registration failed to open /dev/btrfs-control skipping device registration failed to open /dev/btrfs-control skipping device registration failed to open /dev/btrfs-control skipping device registration failed to open /dev/btrfs-control skipping device registration fs tree 256 refs 1 not found unresolved ref root 257 dir 256 index 8 namelen 8 name subvol00 error 600 found 6449324032 bytes used err is 1 total csum bytes: 6291456 total tree bytes: 6873088 total fs tree bytes: 36864 btree space waste bytes: 159776 file data blocks allocated: 10737418240 referenced 10737418240 mount -t btrfs /dev/md15 /home/samba/temp/btrfs-tests/ mount: wrong fs type, bad option, bad superblock on /dev/md15, missing codepage or helper program, or other error Manchmal liefert das Syslog wertvolle Informationen versuchen Sie dmesg | tail oder so ./btrfsck /dev/md11 fs tree 256 refs 1 not found unresolved ref root 257 dir 256 index 8 namelen 8 name subvol00 error 600 found 6449324032 bytes used err is 1 total csum bytes: 6291456 total tree bytes: 6873088 total fs tree bytes: 36864 btree space waste bytes: 159776 file data blocks allocated: 10737418240 referenced 10737418240 And the logs: Feb 2 14:10:11 server kernel: [ 4188.478576] Btrfs loaded Feb 2 14:10:11 server kernel: [ 4188.480370] device fsid d8455ca637705eca-649979b04af654b9 devid 5 transid 679 /dev/md15 Feb 2 14:10:11 server kernel: [ 4188.481053] btrfs: failed to read the system array on md15 Feb 2 14:10:11 server kernel: [ 4188.496590] btrfs: open_ctree failed Feb 2 14:10:17 server kernel: [ 4194.622045] device fsid d8455ca637705eca-649979b04af654b9 devid 1 transid 679 /dev/md11 Feb 2 14:10:17 server kernel: [ 4194.667871] device fsid d8455ca637705eca-649979b04af654b9 devid 5 transid 679 /dev/md15 Feb 2 14:10:17 server kernel: [ 4194.695295] device fsid d8455ca637705eca-649979b04af654b9 devid 1 transid 679 /dev/md/11 Feb 2 14:10:17 server kernel: [ 4194.699507] device fsid d8455ca637705eca-649979b04af654b9 devid 5 transid 679 /dev/md/15 Feb 2 14:10:20 server kernel: [ 4197.287320] device fsid d8455ca637705eca-649979b04af654b9 devid 1 transid 679 /dev/md11 Feb 2 14:10:20 server kernel: [ 4197.290500] device fsid d8455ca637705eca-649979b04af654b9 devid 5 transid 679 /dev/md15 Feb 2 14:10:20 server kernel: [ 4197.307757] device fsid d8455ca637705eca-649979b04af654b9 devid 1 transid 679 /dev/md/11 Feb 2 14:10:20 server kernel: [ 4197.312969] device fsid d8455ca637705eca-649979b04af654b9 devid 5 transid 679 /dev/md/15 Feb 2 14:10:31 server kernel: [ 4208.291488] device fsid d8455ca637705eca-649979b04af654b9 devid 5 transid 679 /dev/md15 Feb 2 14:10:31 server kernel: [ 4208.309594] btrfs: unlinked 1 orphans Feb 2 14:15:25 server kernel: [ 4502.850336] btrfs: relocating block group 73774923776 flags 17 Feb 2 14:15:27 server kernel: [ 4504.572962] btrfs allocation failed flags 17, wanted 4096 Feb 2 14:15:27 server kernel: [ 4504.572966] space_info has 260046848 free, is full Feb 2 14:15:27 server kernel: [ 4504.572969] space_info total=6774456320, pinned=0, delalloc=81199104, may_use=0, used=6442450944, root=0, super=65536, reserved=71892992 Feb 2 14:15:27 server kernel: [ 4504.572973] block group 61291364352 has 1073741824 bytes, 1058013184 used 0 pinned 15663104 reserved Feb 2 14:15:27 server kernel: [ 4504.572975] block group has cluster?: no Feb 2 14:15:27 server kernel: [ 4504.572976] 0 blocks of free space at or bigger than bytes is Feb 2 14:15:27 server kernel: [ 4504.572979] block group 62365106176 has 1073741824 bytes, 1069547520 used 0 pinned 4194304 reserved Feb 2 14:15:27 server kernel: [ 4504.572980] block group has cluster?: no Feb 2 14:15:27 server kernel: [ 4504.572982] 0 blocks of free space at or bigger than bytes is Feb 2 14:15:27 server kernel: [ 4504.572984] block group 69139562496 has 1073741824 bytes, 1063256064 used 0 pinned 10485760 reserved Feb 2 14:15:27 server kernel: [ 4504.572986] block group has cluster?: no Feb 2 14:15:27 server kernel: [ 4504.572987] 0 blocks of free space at or bigger than bytes is Feb 2 14:15:27 server kernel: [ 4504.572989] block group 70213304320 has 1073741824 bytes, 1062207488 used 0 pinned 11534336 reserved Feb 2 14:15:27 server kernel: [ 4504.572990] block group has cluster?: no Feb 2 14:15:27 server kernel: [ 4504.572992] 0 blocks of free space at or bigger than bytes is Feb 2 14:15:27 server kernel: [ 4504.572994] block group 71287046144 has 332005376 bytes, 312475648 used 0 pinned 19529728 reserved Feb 2 14:15:27 server kernel
panic after remove of device during rebalance
Hi, During some btrfs-tests for my own on a btrfs-volume started with 5 devices of different size, some snapshots and subvolumes and a few large files, I removed one device after another (always rebalancing after remove) til I ended up with 3. I use the latest btrfs-tools snapshot and the 2.6.32 kernel with debian patches for sid. btrfs-show then said: Label: none uuid: ca5e7037-a65c-45d8-b954-f64ab0799964 Total devices 3 FS bytes used 6.01GB devid5 size 623.25GB used 0.00 path /dev/md15 devid3 size 93.13GB used 9.01GB path /dev/md13 devid1 size 9.31GB used 9.01GB path /dev/md11 Then I removed number 3. ./btrfs-vol -r /dev/md13 /home/samba/temp/btrfs-tests/ ioctl returns 0 ./btrfs-show Label: none uuid: ca5e7037-a65c-45d8-b954-f64ab0799964 Total devices 3 FS bytes used 6.01GB devid3 size 93.13GB used 9.01GB path /dev/sdc4 devid5 size 623.25GB used 8.31GB path /dev/md15 devid1 size 9.31GB used 8.31GB path /dev/md11 (/dev/sdc4 is the underlying device under /dev/md13, which I removed, I don't know why it still shows up as /dev/sdc4, but that happened before with the other devices I removed, so I didn't bother) Now I startet to rebalance. After 30 minutes or so ps ax still said: 17995 pts/3S+ 0:16 ./btrfs-vol -b /home/samba/temp/btrfs-tests/ After an hour ps ax said 17995 pts/3R+68:31 ./btrfs-vol -b /home/samba/temp/btrfs-tests/ and btrfs-vol consumes 100% of 1 CPU and can not be killed. And thats what ./btrfsck /dev/md11 produced fs tree 256 refs 1 not found unresolved ref root 257 dir 256 index 8 namelen 8 name subvol00 error 600 found 6449324032 bytes used err is 1 total csum bytes: 6291456 total tree bytes: 6873088 total fs tree bytes: 36864 btree space waste bytes: 159776 file data blocks allocated: 10737418240 referenced 10737418240 subvol00 is a subvolume I created and deleted before. The error 600 was there before I started removing devices. Thats what I found in the logs: Feb 2 10:40:27 server kernel: [250931.124172] [ cut here ] Feb 2 10:40:27 server kernel: [250931.124239] kernel BUG at fs/btrfs/inode.c:788! Feb 2 10:40:27 server kernel: [250931.124304] invalid opcode: [#1] SMP Feb 2 10:40:27 server kernel: [250931.124371] last sysfs file: /sys/class/hwmon/hwmon0/temp1_input Feb 2 10:40:27 server kernel: [250931.124440] Modules linked in: btrfs zlib_deflate crc32c libcrc32c autofs4 cpufreq_powersave cpufreq_ondemand cpufreq_stats ipt_REJECT ipt_MASQUERADE xt_TCPMSS xt_mac ipt_REDIRECT xt_DSCP xt_tcpudp xt_state xt_length ipt_LOG xt_limit iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 iptable_mangle iptable_filter ip_tables x_tables nf_conntrack_tftp nf_conntrack_sip nf_conntrack_proto_udplite nf_conntrack_pptp nf_conntrack_proto_gre nf_conntrack_netlink nfnetlink nf_conntrack_irc nf_conntrack_h323 nf_conntrack_ftp nf_conntrack ppp_async crc_ccitt ppp_generic slhc ipv6 nls_utf8 isofs loop powernow_k8 freq_table cpufreq_userspace video backlight ftdi_sio pl2303 asus_atk0110 output wmi usbserial snd_pcm snd_timer snd soundcore snd_page_alloc processor edac_core button i2c_nforce2 pcspkr i2c_core evdev ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod raid1 md_mod ata_generic pata_amd sd_mod amd74xx ahci libata forcedeth firewire_ohci firewire_core crc_itu_t ide_pci_generic ohci_hcd sky2 scsi_mod ehci_hcd ide_core thermal fan thermal_sys hwmon [last unloaded: scsi_wait_scan] Feb 2 10:40:27 server kernel: [250931.125004] Feb 2 10:40:27 server kernel: [250931.125004] Pid: 17936, comm: flush-btrfs-6 Not tainted (2.6.32 #1) System Product Name Feb 2 10:40:27 server kernel: [250931.125004] EIP: 0060:[] EFLAGS: 00010286 CPU: 2 Feb 2 10:40:27 server kernel: [250931.125004] EIP is at cow_file_range+0x5f8/0x610 [btrfs] Feb 2 10:40:27 server kernel: [250931.125004] EAX: ffe4 EBX: ECX: 8989 EDX: 0001 Feb 2 10:40:27 server kernel: [250931.125004] ESI: 000e EDI: 1000 EBP: ESP: d3c0dc18 Feb 2 10:40:27 server kernel: [250931.125004] DS: 007b ES: 007b FS: 00d8 GS: SS: 0068 Feb 2 10:40:27 server kernel: [250931.125004] Process flush-btrfs-6 (pid: 17936, ti=d3c0c000 task=c431e070 task.ti=d3c0c000) Feb 2 10:40:27 server kernel: [250931.125004] Stack: Feb 2 10:40:27 server kernel: [250931.125004] 0277 1000 8540 000e Feb 2 10:40:27 server kernel: [250931.125004] <0> d3c0dc8b 0001 c8e6dab0 c243dea0 c8e6dbcc Feb 2 10:40:27 server kernel: [250931.125004] <0> 1000 c8e6dab4 ce603800 d8593db4 0277 1000 Feb 2 10:40:27 server kernel: [250931.125004] Call Trace: Feb 2 10:40:27 server kernel: [250931.125004] [] ? run_delalloc_range+0x3d6/0x440 [btrfs] Feb 2 10:40:27 server kernel: [250931.125004] [] ? __extent_writepage+0x938/0xae0 [btrfs] Feb
